feat: allow replicas on full disks reclaim the space and squash all data into the heads

[apoorvajagtap]

Which issue(s) this PR fixes:

Issue longhorn/longhorn#13636

What this PR does / why we need it:

This PR so far:

• Scan all snap & head files to build a sector location list/map that indicates which snap file contains the latest data chunk at a specific offset.
• For each sector [offset, length], if there is the latest data in a snap based on the above sector location map, directly and blindly punches holes for all snaps older than the snap.
• moves the data from the snap file to the head then punches hole for those specific sectors in the snap file.

Special notes for your reviewer:

Additional documentation or context

[codecov]

Codecov Report

❌ Patch coverage is 0% with 488 lines in your changes missing coverage. Please review.
✅ Project coverage is 2.96%. Comparing base (6e546b4) to head (b05bb44).

Files with missing lines	Patch %	Lines
pkg/local/replica/recoverer/sectormap/chain.go	0.00%	140 Missing ⚠️
pkg/local/replica/recoverer/prune/punch.go	0.00%	96 Missing ⚠️
.../local/replica/recoverer/sectormap/location_map.go	0.00%	53 Missing ⚠️
pkg/local/replica/recoverer/coalesce/coalesce.go	0.00%	48 Missing ⚠️
pkg/local/replica/recoverer_linux.go	0.00%	41 Missing ⚠️
cmd/local/subcmd/recover.go	0.00%	39 Missing ⚠️
pkg/local/replica/recoverer/sectormap/output.go	0.00%	39 Missing ⚠️
pkg/local/replica/recoverer/sectormap/load.go	0.00%	25 Missing ⚠️
pkg/local/replica/recoverer/common/common.go	0.00%	6 Missing ⚠️
cmd/local/longhornctl-local.go	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff            @@
##           master    #572      +/-   ##
=========================================
- Coverage    3.32%   2.96%   -0.36%     
=========================================
  Files          43      52       +9     
  Lines        4064    4552     +488     
=========================================
  Hits          135     135              
- Misses       3926    4414     +488     
  Partials        3       3              

Flag	Coverage Δ
unittests	2.96% <0.00%> (-0.36%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[apoorvajagtap]

@derekbit @shuo-wu I also had to update the vendored packages. I've kept those changes in separate commits to make the review easier. Let me know if there's a preferred approach for handling vendored package updates. Thanks!

[derekbit]

@derekbit @shuo-wu I also had to update the vendored packages. I've kept those changes in separate commits to make the review easier. Let me know if there's a preferred approach for handling vendored package updates. Thanks!

We usually put a commit for updating vendor packages in the same PR. The commit title can be chore(vendor): update dependencies.

[mergify]

This pull request is now in conflict. Could you fix it @apoorvajagtap? 🙏

[apoorvajagtap]

The following are still WIP, I'll update the comment once all the comments are addressed.
#572 (comment)
#572 (comment)

If b/w permits in the meantime, please feel free to look through the latest commits with rest of the changes. Thanks!

The latest commits have all the reviews addressed. PTAL!

[apoorvajagtap]

@shuo-wu The latest commit incorporates the suggestions.

I have added a temporary logic as per mentioned here. Let me know your thoughts (I see a scope for efficiency a little)

I just realised I missed addressing the naming for OwnerFiles, shall send that out in the final fix.
PS: At present I have kept the commits separate for easier review & backtracking. I shall squash those once we are positive. Thanks!

[shuo-wu]

LGTM now

[mergify]

This pull request is now in conflict. Could you fix it @apoorvajagtap? 🙏