Skip to content

SFTP: Stream file transfers instead of buffering entire file in memory #239

Description

@BrianLeishman

Summary

SFTP file operations (download_file_from_sftp, upload_file_to_sftp, copy) read entire files into a Vec<u8>, which will OOM on large files.

Current State

  • src-tauri/src/locations/sftp/mod.rs:385sftp.read(remote_path) loads full file
  • src-tauri/src/locations/sftp/mod.rs:408tokio::fs::read(local_path) loads full file
  • src-tauri/src/locations/sftp/mod.rs:268copy() reads full file into memory
  • Same pattern exists in SMB provider (via sidecar)

Proposed Changes

Use russh-sftp's open() + chunked read()/write() instead of the convenience read()/write() methods:

let file = sftp.open(path).await?;
let mut buf = vec![0u8; 64 * 1024]; // 64KB chunks
loop {
    let n = file.read(&mut buf).await?;
    if n == 0 { break; }
    dest.write_all(&buf[..n]).await?;
}

Acceptance Criteria

  • File transfers use constant memory regardless of file size
  • Downloads stream to disk in chunks
  • Uploads stream from disk in chunks
  • Copy streams through a temp file without full buffering
  • Progress reporting for large transfers (nice-to-have)

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions