test: verify mesa builds on bzlmod Windows with tool staging#1531
Draft
test: verify mesa builds on bzlmod Windows with tool staging#1531
Conversation
novas0x2a
force-pushed
the
test-mesa-windows
branch
9 times, most recently
from
cdfe406 to
b6b51c1
Compare
The framework was staging tools by symlinking from the exec root and guessing sidecar paths (.runfiles, .runfiles_manifest). On Windows with bzlmod, the exec-root paths to tool runfiles exceed MAX_PATH — for example, meson's Python runtime can't load its own DLLs because the path is too long. Fix this by: - Adding staged_path, runtime_files, and runfiles metadata fields to ToolInfo so toolchains can declare that a tool's runtime closure should be staged into EXT_BUILD_DEPS (which uses short paths on Windows) - Copying the declared runtime closure and recreating the runfiles tree under EXT_BUILD_DEPS instead of symlinking from the exec root - Setting staged_path on meson and cmake toolchains so their runtimes are staged when used by foreign rules Based on 0198230 and 177a45af from fix-examples-windows.
When a tool is staged into EXT_BUILD_DEPS, its runfiles tree is copied alongside it, but the Python runfiles library (used by the meson wrapper) had no way to discover the staged tree. Fix this by exporting RUNFILES_DIR in _copy_deps_and_tools, after the short-path aliases have been applied. This ensures that on Windows RUNFILES_DIR uses the short temp path (avoiding MAX_PATH), while on Linux/RBE it uses the standard EXT_BUILD_DEPS path. Do NOT set RUNFILES_MANIFEST_FILE or copy the manifest separately: the manifest contains absolute paths from the original build machine (local Bazel cache paths on RBE, long exec-root paths on Windows) that are not valid in the staged tree. _repo_mapping is inside the .runfiles tree and is already covered by the bulk directory copy.
The find+touch that flattens timestamps after a directory copy is the dominant cost of tool-runfiles staging (~60s on macOS, ~115s on Windows), but it only matters for autotools source trees where timestamp ordering prevents regeneration. Add a flatten_timestamps parameter so callers can opt out. Existing callers pass True (preserving current behavior); the runfiles staging copy passes False since nothing in a Python runfiles tree is timestamp-sensitive.
Verify that cross-platform tool staging fixes the MAX_PATH failure when building mesa on bzlmod Windows.
Temporary instrumentation to measure the actual cost of the bulk runfiles copy on each CI platform. Emits TOOL_STAGING_START/END with nanosecond timestamps to the build log, then greps them to stderr in cleanup_on_success so they appear in Bazel action output. Remove after measurement.
novas0x2a
force-pushed
the
test-mesa-windows
branch
from
b6b51c1 to
d02b940
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This is a test branch — not intended to be merged independently. Depends on #1529.
Test plan
Examples/8.x/bzlmod/default on Windowspasses (mesa builds)Examples/8.x/mixed/default on Windowspasses