Microsoft.DotNet.Watch.UnitTests.LoggingTests.BinaryLogs: intermittent hang (test host hang-dump timeout, exit code 7)

[MichaelSimons]

Build Information

Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1499678
Leg Name: TestBuild: linux (x64) - ubuntu.2204.amd64.open — Helix work item dotnet-watch.Tests.dll.5
Helix job: 5707f132-4437-408b-845d-e5f187551ea9
Pull request where encountered: #55196 (failed on first attempt, passed after a manual retry)

Summary

Microsoft.DotNet.Watch.UnitTests.LoggingTests.BinaryLogs (Hot Reload test that launches the WatchHotReloadApp test asset) intermittently hangs. The test host never completes, the blame hang-dump timeout of 00:48:00 fires, hang dumps are collected for the dotnet/WatchHotReloadApp process tree, and the work item is killed. The Helix work item finishes with exit code 7 and fails the Monitor Helix Jobs step, turning the whole build red.

The rest of the assembly finishes quickly ("42 completed, 0 failed") and BinaryLogs is the only test still running when the dump is taken, so it is the culprit. The failure is intermittent — the same build passed on retry — so this is tracked with BuildRetry: true.

Error Details

running... 42 completed, 0 failed | active: BinaryLogs
[slow] still running after 32m 00s: BinaryLogs (dotnet-watch.Tests.dll|net11.0|x64)
...
Hang dump timeout of '00:48:00' expired
Dumping this process tree (from bottom):
 >  47548 - dotnet
 + - 47580 - dotnet
 + -- 47597 - dotnet
 + --- 47870 - dotnet
 + ---- 47895 - WatchHotReloadApp
The following tests were still running when dump was taken (format: [<time-elapsed-since-start>] <name>):
[00:50:16] BinaryLogs
Creating dump file '.../WatchHotReloadApp_47895_hang.dmp'
...
Test host crashed; recovered 42 test result(s) from the TRX streaming sidecar
Test application process didn't exit gracefully, exit code is '137'
+ export _commandExitCode=7
+ exit 7
['dotnet-watch.Tests.dll.5' END OF WORK ITEM LOG: Command exited with 7]

AzDO reported:

Work item 'dotnet-watch.Tests.dll.5' in job 'TestBuild: linux (x64) - ubuntu.2204.amd64.open' failed (Finished, exit code 7).

Error Message

{
    "ErrorMessage": [
        "Hang dump timeout of",
        "] BinaryLogs",
        "END OF WORK ITEM LOG: Command exited with 7"
    ],
    "BuildRetry": true,
    "ExcludeConsoleLog": false
}

The multi-line pattern is intentionally shard-agnostic (does not pin the .dll.5 shard number). It requires, in order: the blame hang-dump timeout message, BinaryLogs reported as still-running when the dump was taken, and the work item exiting with code 7 — which together uniquely identify a hang caused by this test without matching unrelated failures. Validated with Test-KnownIssuePattern.ps1 against the failing work item console log (RESULT: PASS).

ExcludeConsoleLog is false because these lines appear only in the Helix work-item console log, not the AzDO build log.

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=1499678
Error message validated: [Hang dump timeout of ] BinaryLogs END OF WORK ITEM LOG: Command exited with 7]
Result validation: ❌ Known issue did not match with the provided build.
Validation performed at: 7/13/2026 1:09:15 PM UTC

Report

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	0

[github-actions]

This Known Build Error has had 0 hits in the past month and has been labeled stale. It will be auto-closed in 7 days if the hit count remains at 0. Add the keep-open label to prevent auto-closure.

[MichaelSimons]

Pattern gap: this KBE misses non-BinaryLogs variants of the dotnet-watch hang

While auditing Known Build Error coverage for PRs merged in the last 24h, I found two more occurrences of this same dotnet-watch test-host hang (48-min blame hang-dump → work item exit code 7 → red Monitor Helix Jobs), but on different tests, which the current ErrorMessage does not match:

PR	Build	Failing work item	Hanging test
#55384	1518122 (same-commit retry, passed on re-run)	dotnet-watch.Tests.dll.1 (TestBuild: linux x64)	BrowserDiagnostics
#55297	1517459 (/ba-g)	dotnet-watch.Tests.dll.3 (FullFramework: windows x64)	MobileHotReloadTests.HotReload_WithWebSocketCapability

The current pattern requires the line ] BinaryLogs:

["Hang dump timeout of", "] BinaryLogs", "END OF WORK ITEM LOG: Command exited with 7"]

So it only matches the BinaryLogs variant. The hang actually reproduces across multiple Hot Reload / browser tests (BinaryLogs, BrowserDiagnostics, HotReload_WithWebSocketCapability, …), all with the identical signature. This is consistent with this issue's own validation block reporting 0 hits (24h/7d/1mo) and "Known issue did not match" — the pattern is too narrow to catch real occurrences.

Suggested fix: make the pattern test-agnostic (but still dotnet-watch-specific + shard-agnostic)

{
    "ErrorMessage": [
        "dotnet-watch.Tests.dll",
        "Hang dump timeout of '00:48:00' expired",
        "END OF WORK ITEM LOG: Command exited with 7"
    ],
    "BuildRetry": true,
    "ExcludeConsoleLog": false
}

This drops the per-test ] BinaryLogs line and instead anchors on the dotnet-watch.Tests.dll assembly + the blame hang-dump timeout + the exit-7 work-item terminator. It stays specific to this failure mode (won't match other assemblies that hang) and does not pin the shard number.

Validated against the BrowserDiagnostics hang from build 1518122 (#55384) with Test-KnownIssuePattern.ps1 → RESULT: PASS (matched lines: assembly banner, Hang dump timeout of '00:48:00' expired @ line 300, ['dotnet-watch.Tests.dll.1' END OF WORK ITEM LOG: Command exited with 7] @ line 369). The current ] BinaryLogs pattern returns RESULT: FAIL on that same log.

Filed as part of a KBE gap-detection sweep — no separate/duplicate issue was created since this is the canonical dotnet-watch-hang tracker; broadening the pattern here should restore coverage.