web: navigate from unresolved to resolved CA build steps#1682
Draft
amaanq wants to merge 3 commits intoNixOS:masterfrom
Draft
web: navigate from unresolved to resolved CA build steps#1682amaanq wants to merge 3 commits intoNixOS:masterfrom
amaanq wants to merge 3 commits intoNixOS:masterfrom
Conversation
… closure Port `tests/functional/dyn-drv/non-trivial.nix` from the Nix test suite as a Hydra integration test. This exercises recursive-nix: a single derivation dynamically creates a DAG of inner derivations (a-e) via `nix derivation add`, outputs the final `.drv`, and a wrapper depends on building that `.drv` through `builtins.outputOf`. Porting this test uncovered a bug in `upload_nars_regular`: it only uploaded the direct build outputs without computing their closure first. When the output is a `.drv` file whose references include other dynamically-created store paths (from recursive-nix), the queue runner's import would fail because those referenced paths were never uploaded. `upload_nars_presigned` already handled this correctly by calling `query_requisites` before uploading — this commit brings the regular path in line. Supporting changes: - Add `nixBinDir` to test `config.nix.in` so job expressions can find the `nix` CLI for recursive-nix builds - Generalize `replace_variable_in_file` to accept multiple substitutions in a single pass (needed because it reads from `.in` each time) - Add `extra-system-features = recursive-nix` to the test's nix.conf so the builder advertises the feature for scheduling
Instead of resolving at StepInfo construction and carrying two drv identities through the gRPC layer, resolve in realise_drv_on_valid_machine once all deps are built. If resolution yields a different drv, the original step is marked Resolved and a new DB step is created for the resolved drv with a resolvedTo FK linking them. The builder only ever sees one drv. We create a new Step for that resolution and bunt it back to the scheduler. This grants us more flexibility in execution and the method can be used in the future for dynamic derivations, which won't map 1:1 with the original derivations. We also now only create database steps when we are sure they are necessary, reducing the number of duplicate `BuildStep` rows. In order to make tests more consistent, CA derivations will fail if they cannot be fully resolved. Otherwise, there could be inconsistent successes depending on which builder a step was performed on. As part of this, add local outputs to resolution table With the current queue-runner design, all dependency outputs of a CAFloating derivation must be recorded in the hydra database. This is true for things built or substituted by hydra, but until now not by things found on the local nix store. This may occur for outputs that are part of the system configuration. Therefore, add all local outputs that are not already in the database to the resolution table upon creating a step. This makes it possible to build derivations from `contentAddressedByDefault` nixpkgs. Co-Authored-By: Artemis Tosini <artemis.tosini@obsidian.systems> Co-Authored-By: John Ericson <John.Ericson@Obsidian.Systems> Co-Authored-By: Amaan Qureshi <git@amaanq.com>
Resolved CA steps do not own the log users are trying to inspect, so leaving them as terminal pages makes the build view look stuck or failed in the wrong place. The log page now follows the recorded resolution chain to the step that actually performed the work, while preserving a pending page for unresolved, running, or malformed chains. The redirect banner is reconstructed from BuildSteps instead of session flash so copied log URLs and refreshes keep the same context. The JSON summary exposes the same durable resolution data for API clients that need to follow CA build steps without scraping HTML.
Member
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
This depends on #1629
Resolved CA steps do not own the log users are trying to inspect, so
leaving them as terminal pages makes the build view look stuck or failed
in the wrong place. The log page now follows the recorded resolution chain
to the step that actually performed the work, while preserving a pending
page for unresolved, running, or malformed chains.
The redirect banner is reconstructed from BuildSteps instead of session
flash so copied log URLs and refreshes keep the same context. The JSON
summary exposes the same durable resolution data for API clients that
need to follow CA build steps without scraping HTML.