feat: Endpoint Persistence using Network Volume (phase 2) - #31
Merged
Conversation
Add core download acceleration modules with aria2c integration: - download_accelerator.py: Main acceleration classes with multi-connection downloads - huggingface_accelerator.py: Specialized HF model acceleration - constants.py: Download acceleration configuration constants - __init__.py: Package structure for src module
Enhanced dependency installation with intelligent acceleration: - Auto-detects large packages for acceleration (torch, transformers, etc.) - Integrates with remote executor for acceleration control - Maintains backward compatibility with existing workflows - Provides graceful fallback when aria2c unavailable
Enhanced workspace manager with HuggingFace model pre-caching: - Pre-cache specified HF models before function execution - Integrates with volume-aware caching system - Optimizes cold start times for ML workloads
Comprehensive test suite for download acceleration: - Integration tests for aria2 detection and fallback behavior - HF model acceleration testing with authentication - Volume-aware acceleration scenarios - Error handling and performance validation
- Update test files moved to src/ directory - Enhanced test coverage for acceleration features - Updated dependencies and documentation - Submodule updates for tetra-rp
- Added nala accelerated installation for large system packages - Enhanced DependencyInstaller with automatic nala fallback to apt-get - Updated Docker images to include nala package manager - Added comprehensive system package acceleration tests - Improved acceleration logging with system package status
Simplify dependency installation by removing aria2c acceleration for Python packages. UV's built-in parallel downloading and caching is superior and eliminates the need for additional complexity. Changes: - Remove LARGE_PACKAGE_PATTERNS from constants.py - Simplify DependencyInstaller.install_dependencies() to single parameter - Remove Python package acceleration logic and related methods - Update RemoteExecutor to use simplified API - Update tests to match new simplified interface System package acceleration (nala) and HuggingFace model acceleration remain intact as they provide meaningful performance benefits over standard tools. Core functionality verified: - All handler tests pass (8/8) - All unit tests pass (98/98) - Code quality checks pass (format, lint, typecheck)
Add conditional acceleration logic - passes accelerate_downloads to installers, HF model caching only when accelerated + models specified
…bled Implement _install_with_pip() method and route between UV (accelerated) vs pip (standard) based on accelerate_downloads parameter
Add HfXetDownloader for subsequent downloads, implement smart strategy: hf_xet for cached files → hf_transfer for fresh downloads → fallback
Add tests for both acceleration enabled/disabled scenarios, verify UV vs pip routing, update existing test assertions
Update test expectations to handle accelerate_downloads parameter in integration scenarios
Update build files and dependency locks to support new acceleration functionality
Always use UV for Python package installation regardless of acceleration setting. The _install_with_pip method has been removed as UV provides more reliable virtual environment handling and package management. - Remove _install_with_pip() method (70 lines) - Simplify install_dependencies() to always use UV - Maintain differential installation when acceleration is enabled
Update dependency installer tests to reflect the removal of pip support: - Fix test_install_dependencies_with_acceleration_disabled to expect UV - Rename test_install_dependencies_pip_failure to test_install_dependencies_uv_failure - Update assertions to check for "uv pip" commands - Update test descriptions and expected error messages All tests now correctly validate UV-only package installation behavior.
Rename test_pip_no_acceleration.json to test_uv_no_acceleration.json and update content to reflect UV-only package installation: - Update function name from test_pip_installation_without_acceleration to test_uv_installation_without_acceleration - Update success message to reference UV instead of pip - Maintain same test logic for package import validation This test validates that packages installed with accelerate_downloads=False are properly available using UV package manager.
Add parallel installation of dependencies when acceleration is enabled: - Add async wrappers for dependency and model download methods - Implement _install_dependencies_parallel() using asyncio.gather() - Add _install_dependencies_sequential() for non-accelerated path - Add _process_parallel_results() for error handling - Route between parallel/sequential execution based on accelerate_downloads flag When accelerate_downloads=True, system packages, Python packages, and HF model downloads execute concurrently for improved performance.
Add accelerate_model_download_async() method to WorkspaceManager to support parallel execution of model downloads when acceleration is enabled. This async wrapper allows HF model downloads to run concurrently with dependency installations for improved performance.
Update test mocks and expectations for parallel execution implementation: - Fix AsyncMock setup for async dependency installation methods - Update test_dependency_management.py for async method calls - Update test_download_acceleration_integration.py for parallel execution - Update test_remote_executor.py with proper AsyncMock usage All tests now properly mock async methods and validate parallel execution behavior when acceleration is enabled.
- Remove 4 obsolete test files (debug logging, subprocess debug, vLLM symlink, redundant HF) - Add 6 new comprehensive test files covering advanced functionality: * test_system_dependencies.json - System package installation * test_class_persistence.json - Instance reuse with instance_id * test_function_args.json - Serialized arguments/kwargs testing * test_mixed_dependencies.json - Combined system + Python dependencies * test_class_custom_method.json - Custom method execution * test_error_scenarios.json - Error handling and edge cases - Update CLAUDE.md to fix test file location references Total test coverage: 11 files (was 5) covering all handler functionality
- Remove custom HfXetDownloader class (~160 lines) - now redundant - Update huggingface_hub requirement to >=0.32.0 for automatic hf_xet - Leverage HF Hub's native snapshot_download() with transparent acceleration - Simplify HuggingFaceAccelerator to use HF's built-in caching and Xet support - Update workspace_manager to trust HF's cache hierarchy (HF_HOME only) - Remove manual Xet detection and file-by-file download logic - Update tests to reflect native HF Hub integration approach - Add documentation for automatic HF acceleration features Benefits: - Automatic chunk-level deduplication via native hf_xet integration - Simplified codebase with 332 fewer lines of redundant code - Better performance using HF's battle-tested acceleration - Future-proof - automatically works with new Xet-enabled repos - Transparent operation - no code changes needed for acceleration
- Add strategy pattern for HF model downloads with tetra and native implementations - Implement model pattern matching for selective acceleration - Add comprehensive test coverage for download strategies - Integrate with existing workspace and cache management systems
1. Cache Sync Manager (src/cache_sync_manager.py) - Major Refactoring - Timestamp-based tracking: Changed from temporary baseline files to timestamp-based change detection - Uses _baseline_time (float) instead of _baseline_path (string) - On first run: uses current time as baseline - On subsequent runs: uses existing tarball's mtime as baseline - Improved file finding: Changed from find -newer <file> to find -newermt <timestamp> with filters to exclude refs/, .no_exist/, and hydration markers - Better tar performance: Changed from append (tar -r) to concatenate (tar -A) for combining tarballs - Cleaner temp file handling: Uses tempfile.NamedTemporaryFile and centralized cleanup via _cleanup_temp_file() helper - Properties for paths: Added _tarball_path and _hydration_marker_path properties for consistent path access - Hydration tracking: Calls mark_last_hydrated() after successful sync 2. HuggingFace Cache (src/huggingface_cache.py) - Added info log when starting model download 3. Remote Executor (src/remote_executor.py) - Simplified HF model cache logging to log stdout/error directly without wrapper text - Updated comment for baseline marking 4. Subprocess Utils (src/subprocess_utils.py) - Removed stdin_input parameter (unused) - Added suppress_output parameter to optionally silence subprocess output logging 5. Tests (tests/unit/test_cache_sync_manager.py) - Updated all tests to work with timestamp-based baseline instead of file-based - Mocks datetime.now() instead of Path.touch() - Updated test scenarios for tar concatenation flow - Fixed test method names
This was referenced Oct 6, 2025
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull Request Overview
Adds endpoint cache persistence via a network volume (phase 2) by introducing a CacheSyncManager to hydrate and sync cache tarballs before and after dependency/model installations. Key changes integrate cache hydration/sync into function execution flow, adjust dependency installation behavior, and refine HuggingFace model caching and subprocess logging.
- Introduces CacheSyncManager with sync and hydration logic and extensive unit tests.
- Adds cache hydration/sync calls into RemoteExecutor workflow and modifies HF model caching log behavior.
- Adds suppress_output flag to subprocess utility and removes prior UV cache disabling in dependency installation.
Reviewed Changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/unit/test_remote_executor.py | Adds tests verifying hydration decision logic in executor before dependency/model operations. |
| tests/unit/test_cache_sync_manager.py | Comprehensive tests for new cache sync and hydration manager behaviors. |
| src/subprocess_utils.py | Adds suppress_output parameter to optionally silence subprocess output logging. |
| src/remote_executor.py | Integrates cache hydration/baseline/sync lifecycle around installations; adjusts HF cache logging. |
| src/huggingface_cache.py | Adds start-download log and expands revision matching logic/documentation. |
| src/dependency_installer.py | Removes --no-cache flag and UV cache disabling; always passes environment copy. |
| src/constants.py | Adds cache directory and volume path constants. |
| src/cache_sync_manager.py | New implementation handling cache baseline, delta tar creation, hydration, and markers. |
| docs/System_Python_Runtime_Architecture.md | Updates documented accelerated install command to match code change. |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
deanq
marked this pull request as ready for review
October 6, 2025 03:03
Contributor
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
This was referenced Oct 10, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.