feat(backend): add configurable range compliance check for HTTP backend#1881
feat(backend): add configurable range compliance check for HTTP backend#1881yxxhero wants to merge 4 commits into
Conversation
Add enableRangeComplianceCheck config option (default: true) to the dfdaemon backend configuration. When enabled, the HTTP backend validates that origin servers honor Range requests for non-zero-offset pieces by checking for 206 Partial Content status and matching Content-Range header. This prevents silent data corruption when registry mirrors or other origins ignore Range headers and return the full body with 200 OK. The validation logic is extracted into a dedicated validate_range_response method for readability, consistent with existing helper methods in the HTTP backend implementation. Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1881 +/- ##
==========================================
+ Coverage 45.98% 46.75% +0.76%
==========================================
Files 93 93
Lines 23437 23778 +341
==========================================
+ Hits 10778 11117 +339
- Misses 12659 12661 +2
🚀 New features to boost your workflow:
|
Signed-off-by: yxxhero <aiopsclub@163.com>
|
this is a solid safety net — without it a broken origin that ignores Range headers would silently serve the full file starting from byte 0, and pieces would get corrupted without any error. seen this happen with CDNs that strip Range on cache misses. the check being opt-in via one thought — when |
Description
Add
enableRangeComplianceCheckconfig option (default:false) to the dfdaemon backend configuration. When enabled, the HTTP backend validates that origin servers honor Range requests for non-zero-offset pieces by checking for206 Partial Contentstatus and matchingContent-Rangeheader.This prevents silent data corruption when registry mirrors or other origins ignore Range headers and return the full body with
200 OK— causingreader.take(piece_length)to write the blob's leading bytes at the wrong offset.Motivation
When downloading from registry mirrors (e.g. Harbor, Nexus) that proxy requests to upstream registries, some origins ignore the
Rangeheader and return the full blob with200 OK. Dragonfly splits downloads into pieces and usesreader.take(piece_length)to read each piece. For non-zero-offset pieces, this silently reads the blob's leading bytes and writes them at the wrong offset, corrupting the reassembled file with a mismatched sha256.This feature adds an opt-in validation to detect and reject such responses.
Changes
dragonfly-client-config/src/dfdaemon.rs: Addenable_range_compliance_checkfield toBackendstruct with default valuefalsedragonfly-client-backend/src/http.rs: Addenable_range_compliance_checktoHTTPstruct and constructor; extract validation logic intovalidate_range_response()methoddragonfly-client-backend/src/lib.rs: Passconfig.backend.enable_range_compliance_checktoHTTP::new()Configuration
Test Coverage
should_get_partial_content_for_nonzero_range— 206 with matching Content-Range succeedsshould_reject_when_origin_ignores_range_and_returns_200— 200 for non-zero offset is rejectedshould_reject_when_content_range_start_mismatches— 206 with wrong Content-Range start is rejectedshould_allow_full_body_for_zero_offset_range— 200 for zero-offset (first piece) is allowedSigned-off-by: yxxhero aiopsclub@163.com