Fix two flaky FB test failures: /proc race and an N/A legend read as no data - #1176
Merged
Conversation
docker exec ... find / -name pmm-agent.log walks /proc, so a process that exits mid-walk makes find warn and exit 1. verifyCommand asserts a zero exit, so the scenario fails at random. Prune /proc from the walk. Signed-off-by: Claude <noreply@anthropic.com>
The MySQL User Details 'Users Activity' state timeline maps its null state to the text N/A, so Grafana renders an N/A legend entry whenever the range contains nulls -- normal on a freshly provisioned server, where the series starts partway into the last hour. The no-data locator matched that legend label and reported the panel as having no data even though it was drawing Active and Passive states, failing PMM-T324 at random. Signed-off-by: Claude <noreply@anthropic.com>
yurkovychv
approved these changes
Aug 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Failures fixed (investigator)
codeceptjs-e2e/tests/qa-integration/pmm_pgsm_integration_test.js:253/@pgsm-pmm-integration— Verify Postgresql Dashboard Instance Summary has Data with socket based service and Agent loge2e_tests/tests/dashboards/mysql/mysqlDashboards.test.ts:77/@pmm-ps-integration— PMM-T324 Verify MySQL - MySQL User Details dashboardTwo unrelated failures, both in pmm-qa's own test code. Neither is a PMM regression — the FB build under test (PMM-15310, gRPC keepalive) is not implicated in either.
1.
@pgsm-pmm-integration—find /racing/procfind /descends into/proc. When a process exits mid-walk its/proc/<pid>directorydisappears,
findwrites a warning to stderr and exits 1, andverifyCommandassertscode === 0— so the scenario fails whenever a short-lived process happens to die duringthe walk. Nothing to do with PMM; the log itself is always at
/var/log/pmm-agent.log.Fix: prune
/procfrom the walk, in both places that do this lookup (lines 253 and 644).2.
@pmm-ps-integrationPMM-T324 — a legend entry read as "no data"The panel had data. From the ARIA/DOM snapshot Playwright recorded in the failing run's
own trace artifact, the element the locator matched is a viz legend label:
Users Activityis a state timeline whose value mapping renders the null state as thetext
N/A(MySQL_User_Details.json, panel 1058). Grafana therefore adds anN/Alegend entry whenever the visible range contains nulls — routine on a freshly provisioned
server, where the series only starts partway into
now-1h, or after any brief scrape gap.noDataPanelNamematches any element whose text is exactly"N/A"and reports theenclosing panel's title, so that legend entry was read as "this panel has no data" while
the panel was plainly drawing its
ActiveandPassivestates.Fix: exclude Grafana viz-legend labels from the no-data marker match. Genuine indicators
still match — a panel rendering
No data, a stat whose value isN/Aor-, anddata-testid Panel data error message.This is deliberately not handled by adding
Users Activityto the dashboard'snoDataMetrics: the data is there, so suppressing the panel would switch off a real"MySQL User Details shows nothing" signal for good. The bug is in the detection.
Verification
One throwaway Linode VM running the FB build under test — server
perconalab/pmm-server-fb:PR-4520-8bebd04(digest
sha256:d21bf9e1…), FB client tarball, each job's ownWIZARD_ARGS(
--database pdpgsql, and--database ps,SETUP_TYPE=replication,MY_ROCKS=true --database ps,SETUP_TYPE=gr,QUERY_SOURCE=slowlog).PGSM — the failing command, run against the real
pdpgsql_pmm_17_1container:find / -name pmm-agent.log(current)find / -path /proc -prune -o -name pmm-agent.log -print(this PR)Same box, same container, same load; the A/B ran the two alternately so they saw identical
/procchurn. Failures reproduced the CI signature exactly (find: '/proc/<pid>': No such file or directory).stdout is unchanged (
/var/log/pmm-agent.log).PMM-T324 — reproduced by putting a real 3.5-minute scrape gap inside the panel's window
(
docker pause/unpauseof the monitored node), which is what makes Grafana add theN/Alegend entry. Both locators evaluated against that live DOM:
main["Users Activity"]— the CI failure[]with the panel's legend reading
N/A,Passive,Activeand the timeline fully drawn.Without an induced gap the test passes on the same VM at
main(as it does in most FB runs —this failed in 1 of the last 9). The same locator change was also checked against the exact
legend markup from the failing run's trace, alongside a panel rendering
No data, a statwhose value is
N/A, and a panel error message — only the legend match disappears.npx eslintclean on both changed files.What only a real CI run can confirm: that the rest of the
@pgsm-pmm-integrationand@pmm-ps-integrationsuites stay green — the PGSM scenario and PMM-T324 were verified atthe command and locator level rather than by a full 22-minute suite pass.
Not in scope
mysqlDashboards.test.ts:77, but for adifferent failure (panel titles trimmed by PMM-15308 Trim whitespace from dashboard titles pmm#5767). Different files, no overlap.
codeceptjs-e2e/tests/pages/dashboardPage.jscarries the same marker list inverifyThereAreNoGraphsWithoutData, so it has the same latent blind spot — left alone heresince it did not fail and could not be verified against a real failure.
Generated by Claude Code