ci(windows): bust vcpkg cache on overlay/triplet/requirements changes#5934
Open
ci(windows): bust vcpkg cache on overlay/triplet/requirements changes#5934
Conversation
The Windows vcpkg cache key was only keyed by matrix.vcpkg-version, so changes to: - thirdparty/vcpkg/ports/** (MeshLib's overlay ports) - thirdparty/vcpkg/triplets/** - requirements/windows.txt - thirdparty/install.bat didn't invalidate the cache. A pre-built C:\vcpkg\installed\ tree was restored verbatim and install.bat's "vcpkg install ..." saw every port as "already installed" (total install time: 3.57 ms in the last PR run), so overlay ports never got consulted. Concrete symptom observed on PR #5932 (zlib → zlib-ng-compat overlay): - Linux-vcpkg (which rebuilds vcpkg inside a Docker image every time `prepare-image / linux-vcpkg-build-upload` runs) picked up zlib-ng; compression tests showed -0.3% zip size and -41 to -45% wall time on Release builds. - Windows vcpkg served the cached pre-zlib-ng tree; zip sizes and test times were indistinguishable from master. Fix: include a hashFiles() of the overlay/triplet/requirements/ install-script content in the cache key. First run after merging this takes the full ~20 min vcpkg install; subsequent runs hit the cache again as before.
Fedr
added a commit
that referenced
this pull request
Apr 20, 2026
Observed in run 24638998286: on Windows, a fresh install of our zlib overlay (triggered by PR #5934 busting the vcpkg cache key) failed with: error: building zlib:x64-windows-meshlib failed with: BUILD_FAILED Missing msys2-mingw-w64-x86_64-pkgconf-1~2.2.0-1-any.pkg.tar.zst ... 404 from every msys2 mirror Root cause: our portfile called vcpkg_fixup_pkgconfig(), which on Windows invokes vcpkg_acquire_msys to download the pinned mingw-w64-pkgconf package from msys2 mirrors. That particular version has been pruned from the live mirrors; every one of the six fallback URLs 404s. Deterministic failure that re-runs won't fix. The call is not needed by our consumers. Every MeshLib/vcpkg consumer that depends on zlib finds it via the CMake target `ZLIB::ZLIB` exported from lib/cmake/ZLIB/ZLIBConfig.cmake; nobody reads lib/pkgconfig/zlib.pc via pkg-config. The .pc file is still generated by zlib-ng's own CMake, with a correct `Libs: -lz` entry in compat mode - it just doesn't get post-processed. Removing the call sidesteps the msys2 dependency entirely. zlib-ng-compat build steps (cmake configure, cmake install, copy_pdbs, cmake_config_fixup for ZLIB, copyright install) are unaffected.
Fedr
added a commit
that referenced
this pull request
Apr 20, 2026
One-line change to .github/workflows/build-test-windows.yml: append a unique suffix to the vcpkg cache key so the existing cached C:\vcpkg\installed\ tree is not restored. vcpkg reinstalls every port from scratch; upstream blosc's vcpkg_fixup_pkgconfig() tries to fetch mingw-w64-x86_64-pkgconf-1~2.2.0-1-any.pkg.tar.zst from msys2 mirrors, gets 404 on all six, and fails the Windows build. This is the same failure first seen on PR #5932 after PR #5934 (which adds the path-based cache busting in a principled way) exposed it. Master itself doesn't fail because its Windows vcpkg cache is a hot hit every run. Purely a diagnostic PR. Do not merge.
MaxRayskiy
pushed a commit
that referenced
this pull request
Apr 21, 2026
Observed in run 24638998286: on Windows, a fresh install of our zlib overlay (triggered by PR #5934 busting the vcpkg cache key) failed with: error: building zlib:x64-windows-meshlib failed with: BUILD_FAILED Missing msys2-mingw-w64-x86_64-pkgconf-1~2.2.0-1-any.pkg.tar.zst ... 404 from every msys2 mirror Root cause: our portfile called vcpkg_fixup_pkgconfig(), which on Windows invokes vcpkg_acquire_msys to download the pinned mingw-w64-pkgconf package from msys2 mirrors. That particular version has been pruned from the live mirrors; every one of the six fallback URLs 404s. Deterministic failure that re-runs won't fix. The call is not needed by our consumers. Every MeshLib/vcpkg consumer that depends on zlib finds it via the CMake target `ZLIB::ZLIB` exported from lib/cmake/ZLIB/ZLIBConfig.cmake; nobody reads lib/pkgconfig/zlib.pc via pkg-config. The .pc file is still generated by zlib-ng's own CMake, with a correct `Libs: -lz` entry in compat mode - it just doesn't get post-processed. Removing the call sidesteps the msys2 dependency entirely. zlib-ng-compat build steps (cmake configure, cmake install, copy_pdbs, cmake_config_fixup for ZLIB, copyright install) are unaffected.
MaxRayskiy
pushed a commit
that referenced
this pull request
Apr 21, 2026
Observed in run 24638998286: on Windows, a fresh install of our zlib overlay (triggered by PR #5934 busting the vcpkg cache key) failed with: error: building zlib:x64-windows-meshlib failed with: BUILD_FAILED Missing msys2-mingw-w64-x86_64-pkgconf-1~2.2.0-1-any.pkg.tar.zst ... 404 from every msys2 mirror Root cause: our portfile called vcpkg_fixup_pkgconfig(), which on Windows invokes vcpkg_acquire_msys to download the pinned mingw-w64-pkgconf package from msys2 mirrors. That particular version has been pruned from the live mirrors; every one of the six fallback URLs 404s. Deterministic failure that re-runs won't fix. The call is not needed by our consumers. Every MeshLib/vcpkg consumer that depends on zlib finds it via the CMake target `ZLIB::ZLIB` exported from lib/cmake/ZLIB/ZLIBConfig.cmake; nobody reads lib/pkgconfig/zlib.pc via pkg-config. The .pc file is still generated by zlib-ng's own CMake, with a correct `Libs: -lz` entry in compat mode - it just doesn't get post-processed. Removing the call sidesteps the msys2 dependency entirely. zlib-ng-compat build steps (cmake configure, cmake install, copy_pdbs, cmake_config_fixup for ZLIB, copyright install) are unaffected.
Fedr
added a commit
that referenced
this pull request
Apr 23, 2026
Resolves conflicts from master's recent work (MRZlib move MRIOExtras→ MRMesh, #5948 adds MRZlibTests.cpp, #5933+#5960 overhaul CompressZip tests, #5957 CUDA cache-key fix, #5722 triplet-keyed vcpkg cache). Conflict resolutions: - .github/workflows/build-test-windows.yml — combined both sides' improvements in the cache key: master-side per-triplet partition (for the x64-windows-meshlib-iterator-debug triplet from #5722) AND branch-side hashFiles(overlay ports / triplets / requirements / install.bat) (from #5934, needed so overlay port changes on this PR bust the cache). Key is now vcpkg-cache-<VERSION>-<TRIPLET>-<HASH>. - source/MRMesh/MRZlib.h — accept master's version (master re-homed the file in #5944; branch had deleted it when it lived briefly in MRIOExtras). - source/MRIOExtras/MRZlib.{h,cpp} — follow master's layout: keep MRIOExtras/MRZlib.h as a forwarding header (#include <MRMesh/MRZlib.h>) and delete MRIOExtras/MRZlib.cpp (the old implementation body, now in MRMesh). - source/MRTest/MRTest.vcxproj — rename ClCompile entry from MRZlib.cpp to MRZlibTests.cpp to match master's #5948 test-file rename. Also drops the duplicate MRZlib.cpp entry. - source/MRTest/MRZlib.cpp — deleted, master renamed to MRZlibTests.cpp in #5948. - source/MRTest/MRZipCompressTests.cpp — accept master's version, which includes the level-0..9 loop (#5960) and the round-trip decompress check (#5958) on top of the branch's earlier simpler version. Branch-side novelty was just the sphere test itself, which is already on master. - source/MRTest/MRZlibTests.cpp — accept master's version, engine-agnostic assertions (tolerance-based compressedSize checks that I relaxed in the zlib-ng PR work). Branch's unique contributions — the actual point of this PR — are preserved: - thirdparty/vcpkg/ports/zlib/{portfile.cmake,vcpkg.json} -- vcpkg overlay port that builds zlib-ng in ZLIB_COMPAT=ON as a drop-in zlib replacement for every port that depends on zlib. - thirdparty/install.bat -- --overlay-ports flag added alongside the existing --overlay-triplets. - thirdparty/vcpkg/downloads/msys2-*.pkg.tar.zst -- pre-staged msys2 packages so vcpkg's pinned 2024.10.21 index doesn't hit stale msys2 mirrors during vcpkg_fixup_pkgconfig().
Fedr
added a commit
that referenced
this pull request
Apr 23, 2026
The master-merge earlier on this branch brought in PR #5722's cache-key format (vcpkg-cache-<VERSION>-<TRIPLET>) but dropped the hashFiles() appendix from #5934. On this PR that matters: the last run (24830925083) had a cache hit on the older key and reused a pre- overlay vcpkg installed tree, so "vcpkg install" saw every port already-installed and built zero of this PR's overlay zlib port. Effect: MRMesh.dll and libzip ended up linked against stock zlib, and the CompressSphereToZip numbers on Windows matched master's stock-zlib baseline (1172-1355 ms Release) instead of the zlib-ng speedup we see on Linux-vcpkg legs in the same run (~450 ms Release). Re-add the hashFiles() over thirdparty/vcpkg/{ports,triplets}/, requirements/windows.txt, and thirdparty/install.bat so a change to any of them busts the cache and forces a fresh install. On this PR specifically, that ensures the new thirdparty/vcpkg/ports/zlib overlay actually gets built and replaces stock zlib with zlib-ng- compat in the cached tree. The fix for the unit-test failure (MRMesh.ZlibCompressStats pinned stats.compressedSize against stock-zlib reference byte counts) came in automatically with the master merge -- master's version of source/MRTest/MRZlibTests.cpp uses engine-agnostic assertions.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The Windows vcpkg cache in
.github/workflows/build-test-windows.ymlis keyed only bymatrix.vcpkg-version:Changes to any of these don't invalidate the cache:
thirdparty/vcpkg/ports/**— MeshLib's overlay ports (e.g. customopencascade-minimal,laz-perf,openctm,clip, and as of #5932 azlib→ zlib-ng-compat override)thirdparty/vcpkg/triplets/**— custom triplets passed via--overlay-tripletsrequirements/windows.txt— the package listinstall.batfeeds to vcpkgthirdparty/install.bat— changes to how vcpkg is invokedA pre-built
C:\vcpkg\installed\tree gets restored verbatim;install.bat'svcpkg install ...step sees every transitively-required port as already installed and completes in a few milliseconds without consulting overlay ports.Evidence from PR #5932's Windows run
From
windows-build-test (msvc-2022, Release, CMake, …)on run 24629386911:zlibdoesn't even appear in the already-installed list (it's a transitive dep). vcpkg saw every port satisfied by the cached tree — which had been populated on master before PR #5932's overlay existed — and declared the install plan done. The overlay was detected (Using overlay: D:\a\MeshLib\MeshLib\thirdparty\vcpkg\portsappears in the log) but never used because nothing needed installing.Compression benchmark numbers, for reference (from the
MRMesh.CompressSphereToZiptest in PR #5933):Linux-vcpkg rebuilds vcpkg from scratch inside the
prepare-image / linux-vcpkg-build-uploadDocker image each time, so it doesn't hit this cache-key problem. Windows did.Fix
Expand the cache key to hash the relevant paths:
Any change to overlay ports, triplets, the package list, or the install script busts the cache and forces a fresh install. Steady-state unchanged: same branch twice in a row still hits the cache.
Cost
First run after this lands does the full Windows vcpkg install (~20 min, one-time). Subsequent same-branch runs hit the cache again as before. Every future overlay-affecting PR pays a one-time rebuild, which is the correct behavior.
Not in this PR
Rollback
Single-file revert of
.github/workflows/build-test-windows.ymlrestores the old cache key. No lingering state — the cache entries under the new key just become orphans that expire naturally.