Skip to content

fix(e2e): exclude test fixtures from @parity content crawls + robust ephemeral loop timing (#1771 follow-up) - #1809

Merged
Adam-S-Daniel merged 1 commit into
mainfrom
fix/1771-ephemeral-unpoison
May 28, 2026
Merged

fix(e2e): exclude test fixtures from @parity content crawls + robust ephemeral loop timing (#1771 follow-up)#1809
Adam-S-Daniel merged 1 commit into
mainfrom
fix/1771-ephemeral-unpoison

Conversation

@Adam-S-Daniel

Copy link
Copy Markdown
Owner

Why the loops stayed red (the vicious cycle + #1723 Cat-2 regression)

The #1771 step-4 ephemeral prod loops (cms-publish-loop-prod-mutate, cms-media-roundtrip) were still failing in real prod after #1797/#1801/#1804. The proven root cause, reproduced locally:

The ephemeral posts are created through the genuinely-UI-driven Decap "+ New Post" leg. Decap writes only the fields the posts collection declares (admin/config*.yml), which has no sitemap/robots field and whose test_fixture is a widget: hidden, default: false the editor can't toggle. So the post that actually lands on main carries published: true + a future date + test_fixture: false and no sitemap/robots keys — verified against the real Create Post commit:

---
title: E2E Media Roundtrip 1779924060949
slug: e2e-media-roundtrip-1779924060949
date: 2099-12-31 00:00:00 +0000
featured_image: /assets/images/uploads/e2e-media-roundtrip-1779924060949.png
published: true
test_fixture: false      # ← NOT true
---

It therefore looks exactly like real public content. The @parity content-crawl specs enumerate every published /blog/ post:

  • console-clean.spec.js runs in BOTH the required e2e-admin (@admin-read) and parity checks;
  • image-alt-text.spec.js crawls every sitemap URL (the post lands in the sitemap — no sitemap: false).

When a crashed run leaves an orphan on main (a post whose uploaded image was already deleted), /blog/<slug>/ 404s a resource → console-clean/image-alt-text red-fail → every cms PR is BLOCKED, including the loop's OWN create PR (verified: create PR #1808 BLOCKED on e2e-admin/parity/finalize failing on /blog/e2e-media-roundtrip-…/). The create PR never merges → the post never serves → the loop's serve leg times out → the run fails → leaves another orphan → repeat. This is the exact #1723 Category-2 class (transient main state poisoning a shared required check), reintroduced by the ephemeral redesign.

Crucially, the design's intended test_fixture: true / sitemap: false markers exist only in the composePost afterAll fallback — they were never on the UI-created posts — so a test_fixture-only exclusion would not have worked.

Fix A (primary — breaks the cycle): public-content crawls exclude test fixtures

A single shared, well-named enumeration predicate, e2e/public-content.js#isTestFixturePost, is now the source of truth used by console-clean, image-alt-text, and sitemap. It excludes a post on ANY of:

  1. test_fixture: true (the documented marker — _e2e/unpublish canaries, composePost fallback),
  2. sitemap: false (fixtures opt out of the sitemap),
  3. the structural e2e- slug signature (/^\d{4}-\d{2}-\d{2}-e2e-/ filename, ^e2e- URL slug — already the codebase-wide fixture detector in admin/posts-list-enhance.js).

Signal 3 is load-bearing: it catches the UI-created posts that carry neither flag, because the slug:/date: the spec types are reliably present. Real public posts (even one titled "E2E Testing Guide" with a non-e2e- slug) are not excluded.

Locked by e2e/public-content.test.js, including the exact UI-shaped no-flag post and a console-clean-style end-to-end enumeration.

Local proof (negative control): with the exclusion temporarily disabled, both console-clean (a 5th test for the canary URL → console.error: Failed to load resource … 404) and image-alt-text (<img> with no alt on the canary) FAIL on a UI-shaped canary that is in the built sitemap and renders at /blog/…/. With the exclusion on, the canary URL is never enumerated and both stay green.

Fix B: robust ephemeral create→serve→delete→404 timing

  • reopenForPublishedDelete 6 → 13 min (it timed out waiting for Decap to re-sync to the published file after the create PR's squash auto-merge + branch removal; under contention 6 min is too tight).
  • create-PR waitForMerge 5 → 10 min (absorbs GitHub API lag between deploy completing and the PR flipping merged:true; the merge has normally already landed by the URL-serve gate).
  • TEST_TIMEOUT_MS raised to fit the worst-case leg sum within the job timeouts: prod 40 → 46 min (50-min job), media 55 → 80 min (95-min job).

The loop still genuinely exercises the real Decap UI create→publish→serve→UI-delete→404 chain; the afterAll existence-only API safety-net is unchanged.

Fix C: harden orphan sweeping

sweep-stale-cms-prs.yml gains a dedicated ephemeral_threshold_hours (default 3h) for the ephemeral prod-loop orphan tier (vs 6h for human-hostable tiers) plus a 3-hourly cron, so a killed run's orphan is gone within ~3-6h. 3h is deliberately above the longest run's 95-min job timeout so the sweep can never reap a post still under active test. With Fix A this is defence-in-depth (orphans no longer poison checks) + tidiness.

Docs / lint plumbing

  • Reconciled the inaccurate "born with test_fixture: true" claims in AGENTS.md, prod-mutate-fixture.js, and cms-posts-list-enhance.spec.js to reflect what the UI actually writes; documented the Cat-2 regression + the public-content crawl scope.
  • select-specs.js wires e2e/public-content.js as a dependency of the three crawl specs (and the parity-preview selector) + maps the new unit test.

Validation

  • npm run lint (0 errors) + npm run format:check clean.
  • All constraint lints green: silent-catch-lint, select-specs, cms-recursion-churn, cms-editor-ui, fixture-baseline, canary-content, workflow-prod-loop-serialized, plus the new public-content + cms-posts-list-enhance + prod-mutate-fixture.
  • Both ephemeral specs parse/list under --project chromium-desktop-3k --list.
  • Local TARGET=local runs of console-clean / image-alt-text / sitemap / draft-isolation pass with AND without a UI-shaped canary present.

🤖 Generated with Claude Code

…ephemeral loop timing (#1771 follow-up)

The #1771 step-4 ephemeral prod loops stayed red in real prod due to a
vicious cycle + a #1723 Cat-2 regression.

Root cause (proven locally): the ephemeral prod-loop posts are created
through the genuinely-UI-driven Decap "+ New Post" leg. Decap writes only
the posts-collection fields, which declare NO sitemap/robots and whose
test_fixture is a hidden default:false widget the editor can't toggle.
So the post that lands on main carries published:true + a future date +
test_fixture:false and NO sitemap/robots keys (verified against the real
Create Post commits) — it looks exactly like real public content. The
@parity content-crawl specs (console-clean in required e2e-admin AND
parity; image-alt-text; sitemap) enumerated it, and a transient orphan a
crashed run left on main (deleted image now 404s) red-failed those
required checks on EVERY cms PR — including the loop's own create PR, so
the loop could never go green. The design's intended test_fixture:true /
sitemap:false markers (only the composePost afterAll fallback sets them)
were never actually on the UI-created posts, so a test_fixture-only
exclusion would not have worked.

Fix A (primary — breaks the cycle): a single shared, well-named
enumeration predicate e2e/public-content.js#isTestFixturePost, used by
console-clean, image-alt-text, and sitemap. It excludes a post on ANY of:
test_fixture:true OR sitemap:false (the _e2e/unpublish canaries) OR the
structural e2e- slug signature (/^\d{4}-\d{2}-\d{2}-e2e-/ filename, ^e2e-
URL slug — the codebase-wide fixture detector). The slug signature is the
load-bearing one: it catches the UI-created posts that carry neither flag.
Locked by e2e/public-content.test.js (incl. the UI-shaped no-flag post +
a console-clean-style end-to-end enumeration). Local negative-control
proof: with the exclusion disabled both console-clean and image-alt-text
FAIL on a UI-shaped canary's 404'd resource / no-alt img; with it on,
they never enumerate it.

Fix B (robust timing): reopenForPublishedDelete 6 -> 13 min (it timed out
waiting for Decap to re-sync to published after the create PR's squash
auto-merge + branch removal); create-PR waitForMerge 5 -> 10 min (API lag
absorption). TEST_TIMEOUT_MS raised to fit the job timeouts (prod 40->46
min within the 50-min job; media 55->80 min within the 95-min job).

Fix C (orphan sweeping): sweep-stale-cms-prs.yml gains a dedicated
ephemeral_threshold_hours (default 3h, vs 6h human-hostable tiers; 3h is
deliberately above the longest run's 95-min job timeout so the sweep
cannot reap a post still under active test) and a 3-hourly cron, so a
killed run's orphan is gone within ~3-6h.

Docs: reconciled the inaccurate "born with test_fixture:true" claims in
AGENTS.md, prod-mutate-fixture.js, and cms-posts-list-enhance.spec.js to
reflect what the UI actually writes, and documented the Cat-2 regression
+ public-content crawl scope. select-specs.js wires public-content.js as a
dependency of the three crawl specs + the new unit test.

All constraint lints green (silent-catch, select-specs, cms-recursion-churn,
cms-editor-ui, fixture-baseline, canary-content, workflow-prod-loop-serialized);
npm run lint + format:check clean.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@github-actions github-actions Bot added not-decap-created PR was not created by Decap CMS (head branch is not cms/<col>/<slug>) cms/draft Content draft — not ready for publish labels May 28, 2026
@github-actions

github-actions Bot commented May 28, 2026

Copy link
Copy Markdown
Contributor

🗑️ Preview environment cleaned up.

@Adam-S-Daniel
Adam-S-Daniel merged commit dcabe5f into main May 28, 2026
30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cms/draft Content draft — not ready for publish not-decap-created PR was not created by Decap CMS (head branch is not cms/<col>/<slug>)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant