Convert dotnet-format integration tests from pipeline YAML to MSTest/Helix

[MichaelSimons]

Summary

Converts the dotnet-format integration test pipeline legs (YAML + PowerShell) into MSTest integration tests that run on Helix. This eliminates redundant SDK builds and reduces PR validation cost.

Compute Savings

Measured from build 1519902 (main, 2026-07-21):

Each old leg builds its own full SDK copy before running format. The new approach eliminates this by running as Helix work items on the existing Linux x64 build leg.

Leg	Old Leg Duration	SDK Build Overhead	Format Work	After this PR
sdk	20m 10s	~14m	~6m	✅ Kept (saves ~14m build)
Roslyn	31m	~13m	~18m	✅ Kept (saves ~13m build)
msbuild	18m 23s	~11m	~7m	✅ Kept (saves ~11m build)
project-system	13m 54s	~10m	~4m	✅ Kept (saves ~10m build)
efcore	17m 16s	~10m	~7m	❌ Dropped (saves ~17m)
aspnetcore	26m 39s	~16m	~11m	❌ Dropped (saves ~27m)

Scenario	Total agent-minutes saved
CI builds	~92m (redundant SDK builds eliminated + efcore/aspnetcore dropped)
PR builds (format-related changes)	~92m (same as CI)
PR builds (unrelated changes)	~127m (all legs skipped via conditional test filtering)

What changed

• New test project: test/dotnet-format.IntegrationTests/ using MSTest.Sdk with one test class per repo (SDK, Roslyn, MSBuild, ProjectSystem). Inherits from the MSTest SdkTest base class for standard test infrastructure (logging, binlog arguments).
• Base class pattern: FormatIntegrationTestBase handles clone, Arcade restore, global.json manipulation, solution restore, and format execution with stdout validation. Adding or removing a repo requires only adding or deleting a ~15-line class.
• Dynamic Helix sharding: PartitionByClass=true metadata causes AssemblyScheduler to discover test classes at scheduling time and create one Helix work item per class automatically.
• Conditional test filtering: Uses the ConditionalTests.props DotnetFormat scope with trigger paths. Tests run in PR validation only when relevant files change; they always run in CI.
• Platform targeting: Tests run only on Linux x64 (enforced at the UnitTests.proj scheduling level, not in test code).
• Removed old infrastructure: Deleted eng/dotnet-format/ (YAML template, PowerShell verifier, CMD wrapper, RSP file) and removed pipeline references from .vsts-ci.yml / .vsts-pr.yml.

Design

Each repo is a separate test class inheriting from FormatIntegrationTestBase:

[TestClass]
public class SdkFormatTests : FormatIntegrationTestBase
{
    protected override string RepoUrl => "https://github.com/dotnet/sdk";
    protected override string Sha => "e6bc966cc3d1348265b0831c6daca23267169d8f";
    protected override string TargetSolution => "sdk.slnx";
    protected override string RepoName => "sdk";
}

This one-class-per-repo pattern enables:

1. Helix sharding — each class becomes its own work item (parallel execution)
2. Zero maintenance — adding or removing a repo only requires adding or deleting a class file; sharding adapts automatically

Test lifecycle (mirrors the old format-verifier.ps1)

1. Clone — shallow single-commit fetch of the target SHA
2. Arcade restore — runs eng/Build.ps1 -restore (or eng/build.sh --restore) while global.json is intact to install Arcade SDK tooling
3. Strip global.json sdk section — preserves msbuild-sdks for NuGet SDK resolver but forces the SDK under test to be used
4. Solution restore — restores with the parent SDK's dotnet and repo-local nuget.config
5. Format workspace — dotnet format <solution> --no-restore --verify-no-changes --verbosity detailed
6. Format folder — dotnet format whitespace <path> --folder --verify-no-changes --verbosity detailed
7. Stdout validation — verifies "Formatted X of Y files" appears with Y > 0 (catches silent no-op exits)

Repos dropped

AspNetCore and EfCore: These repos are restore-intensive and not feasible to run on Helix. The ROI is questionable for a format validation check.

Bug fixes included

• HelixTasks loop variable mutation (CreateHelixTestWorkItems.cs): The arguments variable was being mutated inside the partition loop, causing "-- " prefix to accumulate on each iteration. Fixed by computing formattedArguments once before the loop.
• Conditional test scope wildcard-in-directory-name validation (EvaluateConditionalTestScopes.cs): The ValidatePatternBaseDir method extracted the base directory by taking the prefix before the first wildcard and trimming trailing slashes. For patterns like test/dotnet-format.*/**, this produced test/dotnet-format. as the base directory (which doesn't exist), causing validation to fail. Fixed by truncating to the last / before the first wildcard character, so the base directory is test (which exists). Added a unit test for this case.

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).
2 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.