Skip to content

Replace gRPC download call in proxy with local download_task using task manager#1915

Draft
gaius-qi with Copilot wants to merge 6 commits into
mainfrom
copilot/implement-download-task-in-proxy
Draft

Replace gRPC download call in proxy with local download_task using task manager#1915
gaius-qi with Copilot wants to merge 6 commits into
mainfrom
copilot/implement-download-task-in-proxy

Conversation

Copilot AI commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

The proxy downloaded tasks through the dfdaemon download gRPC client over a unix socket (dfdaemon_download_client.download_task in proxy/mod.rs), adding unnecessary gRPC transport overhead to every proxied download. This implements the same download_task logic locally in the proxy using resource::task::Task functions directly, removing the gRPC hop.

Local download_task in proxy/mod.rs

Mirrors the gRPC download_task handler in grpc/dfdaemon_download.rs, but calls the task manager directly:

  • Generates task/host/peer IDs via task.id_generator (content, blob-digest-based, and URL-based TaskIDParameter).
  • Calls task.download_started, resolves the range from the request header via get_range, then spawns task.download writing into a local mpsc channel (DOWNLOAD_STREAM_BUFFER_SIZE), returning the receiver as the response stream.
  • Preserves the success path (download_finished, output-path hard-link/copy checks, digest verification), failure path (download_failed, backend error details), and all download task metrics.
  • Drops gRPC-only concerns: tonic Request/Response wrappers, OpenTelemetry context extraction, and blocklist checks.

Local prefetch_task

Mirrors grpc::prefetch_task but invokes the local download_task instead of creating a gRPC client. download_task returns a boxed future to break the recursive future type between download_task and prefetch_task.

proxy_via_dfdaemon rework

// Before: gRPC stream over unix socket
let response = dfdaemon_download_client.download_task(request).await?;
let mut out_stream = response.into_inner();

// After: local mpsc stream from the task manager
let mut out_stream = download_task(config.clone(), task.clone(), request).await?;
  • Consumes mpsc::Receiver<Result<DownloadTaskResponse, Status>> instead of a tonic stream.
  • Handles ClientError::BackendError directly (status code + headers) instead of decoding tonic Status details JSON.
  • Removes ResourceExhausted/PermissionDenied mappings that only originated from gRPC server middleware.

Cleanup

Removed DfdaemonDownloadClient construction from Proxy::run and its parameter from all proxy handlers (handler, http_handler, https_handler, upgraded_tunnel, upgraded_handler, registry mirror handlers).

…task.rs

- Move `download_task` and `prefetch_task` functions into new
  `proxy/task.rs`
- Rename them to `task::download` and `task::prefetch` with public
  visibility
- Clean up unused imports in `proxy/mod.rs` after the extraction
- Declare `task` as a public submodule in `proxy/mod.rs

Signed-off-by: Gaius <gaius.qi@gmail.com>
@gaius-qi gaius-qi added the enhancement New feature or request label Jul 10, 2026
@gaius-qi gaius-qi added this to the v2.6.0 milestone Jul 10, 2026
@codecov

codecov Bot commented Jul 10, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 155 lines in your changes missing coverage. Please review.
✅ Project coverage is 45.85%. Comparing base (72f989f) to head (0ea9c8b).

Files with missing lines Patch % Lines
dragonfly-client/src/proxy/task.rs 0.00% 136 Missing ⚠️
dragonfly-client/src/proxy/mod.rs 0.00% 19 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1915      +/-   ##
==========================================
- Coverage   46.11%   45.85%   -0.27%     
==========================================
  Files          93       94       +1     
  Lines       23526    23662     +136     
==========================================
  Hits        10850    10850              
- Misses      12676    12812     +136     
Files with missing lines Coverage Δ
dragonfly-client/src/proxy/mod.rs 0.00% <0.00%> (ø)
dragonfly-client/src/proxy/task.rs 0.00% <0.00%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants