Abhishek/enable large reads support#4864
Conversation
|
Hey there and thank you for opening this pull request! 👋🏼 We require pull request titles to follow the Conventional Commits specification and it looks like your proposed title needs to be adjusted. Details: |
There was a problem hiding this comment.
Code Review
This pull request introduces 'rapid' and 'non-rapid' bucket type groups to streamline GCSFuse optimizations, updating default parameters like max background requests, congestion thresholds, and FUSE max pages limits accordingly. It also implements a sync.Pool for read buffers to optimize memory usage during large reads on regional buckets when the kernel reader is enabled. The reviewer pointed out a critical issue in cmd/mount.go where post-mount optimized parameters (such as read-ahead, max background, and congestion threshold) are never applied in non-GKE environments because ApplyNonGKE is not called after fuse.Mount succeeds.
| // Apply pre mount kernel settings in non-GKE environments for non dynamic mounts when kernel reader is enabled. | ||
| if !isDynamicMount(bucketName) && !cfg.IsGKEEnvironment(mountPoint) && newConfig.FileSystem.EnableKernelReader { | ||
| kernelparamsManager := kernelparams.NewKernelParamsManager() | ||
| if kernelparams.ShouldUpdateMaxPagesLimit(int(newConfig.FileSystem.FuseMaxPagesLimit)) { | ||
| kernelparamsManager.SetMaxPagesLimit(int(newConfig.FileSystem.FuseMaxPagesLimit)) | ||
| kernelparamsManager.ApplyNonGKE(mountPoint) | ||
| } | ||
| } |
There was a problem hiding this comment.
While MaxPagesLimit is a system-wide pre-mount setting, other optimized parameters such as MaxReadAheadKb, MaxBackgroundRequests, and CongestionWindowThreshold are per-connection post-mount settings. They can only be applied after the FUSE mount is established and the device major/minor numbers are created. Currently, there is no call to ApplyNonGKE after fuse.Mount in cmd/mount.go, which means these optimized parameters are never applied in non-GKE environments. Please ensure ApplyNonGKE is called with these parameters after fuse.Mount succeeds.
Please ensure your PR title follows the format:
Example:
feat(api): add user login endpointAvailable types:
feat: A new featurefix: A bug fixdocs: Documentation only changesstyle: Changes that do not affect the meaning of the code (white-space, formatting, missing semi-colons, etc)refactor: A code change that neither fixes a bug nor adds a featureperf: A code change that improves performancetest: Adding missing tests or correcting existing testsbuild: Changes that affect the build system or external dependencies (example scopes: gulp, broccoli, npm)ci: Changes to our CI configuration files and scripts (example scopes: Travis, Circle, BrowserStack, SauceLabs)chore: Other changes that don't modify src or test filesrevert: Reverts a previous commitDescription
Link to the issue in case of a bug fix.
Testing details
Any backward incompatible change? If so, please explain.