Adding L1 tests for host node selection#5456
Open
rishabhmalikMS wants to merge 7 commits intomasterfrom
Open
Conversation
Contributor
Author
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Contributor
Author
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
surajitshil-03
approved these changes
Feb 3, 2026
Contributor
Author
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
sanjuyadav24
reviewed
Feb 4, 2026
…ives - Remove AssertTaskResult method that incorrectly allowed strategy mode failures - Update both legacy and strategy modes to expect success with proper error handling - Simplify assertions by removing unnecessary TaskResult.Succeeded conditions - Make test validation straightforward without conditional logic weaknesses - Address PR feedback on log-based testing approach and assertion reliability Both modes now properly assert success, making tests meaningful validation of actual node selection behavior in L1 integration scenarios.
…soft/azure-pipelines-agent into users/rishabhmalik/L1Tests
Contributor
Author
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Contributor
Author
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
| if (useStrategy) | ||
| { | ||
| // Strategy mode: Verify orchestrator was invoked | ||
| hasNodeSelection = log.Any(x => x.Contains("[Host] Selected Node version:") && x.Contains("(Strategy:")) || |
Contributor
There was a problem hiding this comment.
this approach relies on log based testing, for new strategy patterns we are sending telemetry and that is already mocked in L1
we can check moving strategy based test cases to telemetry testing as logs are brittle and can break test in future
| hasNodeSelection = log.Any(x => x.Contains(NODE_SELECTION_LOG_PATTERN)); | ||
| } | ||
|
|
||
| Assert.True(hasNodeSelection, $"Node selection process should be initiated: {modeDescription}"); |
Contributor
There was a problem hiding this comment.
The strategy branch accepts 4 patterns with || — if the orchestrator emits only the legacy pattern "Using node path:", the test still passes, silently masking a regression where strategy mode fell back to legacy
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
This PR addresses L1 testing for Azure Pipelines Agent host-based node version selection. The existing test infrastructure was lacked to validate both strategy mode (new) and legacy mode node selection behaviors for Node.js task execution.
Description
NodeSelectionL1Tests.csto support both strategy mode and legacy mode validation with proper logging pattern detectionAGENT_USE_NODE_STRATEGY=true(strategy mode) andfalse(legacy mode) behaviorsRisk Assessment (Low)
Low Risk - This PR only adds and enhances test infrastructure without modifying production agent logic. The changes are isolated to the test suite (
src/Test/L1/Worker/) and improve validation coverage of existing node selection functionality. No runtime behavior changes for end users.Unit Tests Added or Updated (Yes / No)
Unit tests (L0 tests) were already added in previous PR.
Additional Testing Performed
List manual or automated tests performed beyond unit tests (e.g., integration, scenario, regression).
Change Behind Feature Flag (Yes / No)
No - These are test infrastructure improvements only.
Tech Design / Approach
Documentation Changes Required (Yes/No)
Indicate whether related documentation needs to be updated.
Logging Added/Updated (Yes/No)
Telemetry Added/Updated (Yes/No)
Rollback Scenario and Process (Yes/No)
Dependency Impact Assessed and Regression Tested (Yes/No)