docs: GB10 deployment guide + coverage-enforcing release gate

[tbraun96]

What this is

Two contributor-facing deliverables — a GB10 deployment guide for users and a
release-pipeline skill for maintainers — plus a real serve-matrix release
gate and the low-risk CI/CLA/doc convergence the work exposed. Docs + tooling
only; no engine code changes (so nothing here can regress a model).

Why

Two gaps kept coming up:

1. New users had no single "what do I run?" page. Recipes assume you've already
   picked a model and quant that fit your box. docs/GB10_DEPLOYMENT_GUIDE.md is
   that missing first-read.
2. "Verified" wasn't enforceable. tests/run_all_models.py writes per-model
   JSON but never fails — and a model that crashed at boot writes no JSON at all,
   so a glob-only scorer silently reports the survivors green. That's a false ship.

Changes

Release gate (tests/) — run_all_models.py now writes _manifest.json (the
roster it planned to cover, from planned_specs()). gate_results.py reads that
manifest and fails any planned model with no passing result — coverage is
enforced, not assumed. Exit 0 ships / 1 blocks. test_gate_results.py (stdlib
unittest, no GPU) locks in the per-model bars and the planned-but-absent = FAIL
property. 14/14 green.

Deployment guide (docs/GB10_DEPLOYMENT_GUIDE.md) — model×quant matrix
(single-node + scale-out), goal→recipe picker, weight/KV quant + OOM ladder,
consolidated known-issues, EP=2 troubleshooting, and what "verified" means. Routes
to the recipes / QUICKSTART / DEPLOYMENT instead of duplicating them; the recipe
defaults stay the serve-config SSOT. Linked from the README as the welcoming
first-read.

atlas-release skill (.claude/skills/atlas-release/) — the build → verify →
image → publish pipeline + upstream-sync PR automation, grounded in the real
commands, containers, tags, and gates already in-tree. De-personalized so any
contributor can run it. ⚠️ .claude/ is gitignored — force-added so reviewers can
see it; drop this path if skills should stay out of the tree.

Convergence (applied where unambiguous):

• scripts/check-license-headers.sh — the script AGENTS.md / the PR template /
  the book all referenced but that never existed; thin wrapper over the same
  skywalking-eyes engine CI runs.
• docker/gb10/Dockerfile — ARG ATLAS_GIT_SHA + revision LABEL so an image
  records its commit (release.sh already passed the build-arg into a no-op).
• .github/workflows/cla.yml — allowlist the maintainer identities so
  AI-authored maintainer PRs skip the CLA ceremony; external contributors stay
  covered.
• AGENTS.md — file-size cap 250→500 to match the CI SSOT; clippy/license lines
  aligned to CI; model-matrix pointer → the new guide.
• Dead docs/EP2-TROUBLESHOOTING.md links (DEPLOYMENT.md, book, adr/0007)
  repointed to the guide §7; CONTRIBUTING.md notes CI-green ≠ shippable;
  docs/releases/ shipped-SHA stub.

Testing

• python3 -m unittest tests.test_gate_results → 14/14 pass.
• tests/run_all_models.py planner produces a 21-config roster for a full run;
  the gate enforces every one of them.
• No Rust touched → the CUDA-free CI lanes (fmt/clippy/test/file-size) don't apply
  to these files; new .py/.sh are outside the .licenserc.yaml SPDX paths.

Authorship

AI-generated by Atlas's maintainer tooling, per CONTRIBUTING.md §Authorship.

Not included / follow-ups

• Two files being edited by a concurrent workstream (ssm_qwen35.rs,
  scripts/package-distro.sh) are deliberately excluded — not part of this
  change.
• Recommended next (not in this PR): a self-hosted push:[main] workflow that
  runs the gate and docker saves a "ready to publish <sha>" artifact, leaving
  the docker push human — closing the main → :latest drift. Shape is in
  .claude/skills/atlas-release/references/pipeline.md §Closing the gap.

[SeedSource]

Codeowner review — APPROVE. Reviewed the load-bearing surface at head 85544ca (gate logic, CLA workflow, license script, Dockerfile, AGENTS.md); docs skimmed. Docs + tooling only, no engine code, all 20 checks green. For a no-engine-code PR that's the right bar. The gate is well-scoped and the tests lock the property that matters (planned-but-absent = FAIL). Three non-blocking notes.

What's good

• gate_results.py closes a real false-green. The manifest-vs-results coverage check (a planned model that never booted writes no <label>.json → FAIL, not invisible) is exactly right, and test_gate_results.py::Coverage::test_missing_model_is_a_failure pins it. Known-gap parsers scoring all-N/A correctly pass; a wired parser going all-WARN/FAIL correctly regresses. Clean separation of gate_manifest (enforced) vs gate_glob (loud, opt-in only).
• Dockerfile org.opencontainers.image.revision LABEL — this directly answers the :latest provenance question that's bitten us downstream (an image with no git-SHA is unshippable to verify against). release.sh passing ATLAS_GIT_SHA + the inspect one-liner is the whole fix. 👍

Non-blocking notes

1. gate_results.py — an empty section is scored PASS, the same false-green class this PR targets. verdict() guards every bar with if coh and … / if fib and …, and _is_model_result() only checks that the coherence key exists, not that it's non-empty. So a result file like {"model":"x","coherence":[]} returns [] → verified. If single_gpu_suite.py can ever emit an empty coherence list (all probes errored before scoring), it slips the gate. Cheap hardening: treat empty/absent coherence as a FAIL (if not coh or coh_pass < COHERENCE_MIN_PASS) — a verified model must have actually run probes. Only matters if that shape is reachable; flagging since it's the exact bug class the gate exists to catch.
2. cla.yml allowlist omits two owners. allowlist: …,AzeezIsh,tbraun96 covers 2 of the human owners but not rsafier (3rd owner) nor @SeedSource (4th global CODEOWNER). Same rationale (already signed, own the copyright) applies to both — either add them or confirm the omission is intentional.
3. .claude/skills/atlas-release/ is force-added past .gitignore. You already flagged this in the PR body. Codeowner call: keeping contributor/release tooling in-tree is useful, but a force-added path under a gitignored dir will silently drop future edits and reads as inconsistent. If it stays, prefer un-ignoring .claude/skills/ explicitly (a scoped !.claude/skills/ un-ignore) so it's tracked on purpose rather than by -f.

Nice cleanup on the check-license-headers.sh wrapper (same skywalking-eyes engine as CI — no drift) and the AGENTS.md 250→500 LoC alignment to the CI SSOT. None of the above blocks; approving.

🤖 Generated with Claude Code

[Sujimoshi]

Nice — this closes a real gap (the "planned but never booted = silent green" failure mode is exactly right to fix). A few things worth flagging on the gate design while it's still easy to adjust the bars, based on tests/gate_results.py:

1. tps(0) is a floor, not a regression check. verdict() only fails a model when sum(tps)/len(tps) <= 0 — i.e. it catches "server hung / crashed", not "model got 5x slower." A model that regresses from, say, 80 tok/s to 15 tok/s ships green. We hit this exact shape of gap ourselves this week: a DFlash config we were about to publish turned out ~40% slower than plain greedy on a realistic prompt, and nothing about a tps>0 check would have caught that — it only becomes visible with a comparison against a stored baseline (even a generous "no more than X% below last-known-good" would do it).

2. No speculative-decode-aware bars. coherence/fibonacci/tool_calls/tps are good general bars, but for K=γ/DFlash-style paths none of them touch acceptance rate (tau) or T=0 determinism-vs-baseline parity. A model could pass every existing bar while its draft acceptance rate silently collapsed to near-zero (all the cost of speculative decoding, none of the benefit) — a config-specific bar here would have caught the case above directly.

3. Manifest and gate share one source of truth. _manifest.json comes from run_all_models.py::planned_specs(), and gate_results.py trusts that roster as-is. If the planner itself silently drops a model (a planner bug, not a boot failure), the gate has no independent signal to catch it — coverage is enforced against "what the planner intended," not against an external support-matrix listing. Might be worth a lightweight cross-check against docs/GB10_DEPLOYMENT_GUIDE.md's model matrix (or wherever the canonical support list ends up) so the two can't drift/degrade together silently.

4. No trend/history tracking. Each run is judged in isolation — there's no comparison to a previous green run's numbers, so a model that quietly loses 20-30% tok/s release-over-release stays green indefinitely as long as it stays above 0. Even just persisting the last N all_models_results/ snapshots and flagging "N% below last green" would turn this into an actual regression gate rather than a crash gate.

None of these block what's here — the coverage-enforcement mechanism itself is the important fix and it's solid. Flagging in case the bars in verdict() get revisited before this becomes the standard release gate.

[Sujimoshi]

One more gap, orthogonal to the gate itself but adjacent to this PR's scope (deployment guide + release tooling): there's no shared, canonical prompt set for manual/ad-hoc dev testing.

Concretely, in the DFlash work this week we ended up with three different prompts in play with no agreed relationship between them: the short factual bench prompt ("What is the capital of France?", max_tokens<=30, used for the README throughput number), an internal code-generation prompt (min-heap implementation, used for profiling), and a generic-prose prompt a reviewer asked for separately ("write a detailed history of X") as a more realistic end-user case. None of these were declared as "the" representative prompt(s) for a given testing purpose, so results measured on one don't transfer to conclusions about another — we hit this directly: a config's speedup on the code prompt did not hold on the prose prompt (very different acceptance-rate/tau behavior), and it took real back-and-forth to even notice the two weren't comparable.

This is a different problem from the automated matrix in tests/run_all_models.py (which is about model×quant coverage, not prompt diversity) — this is about giving contributors 1-2 blessed, checked-in prompts (e.g. one short-factual, one long-form-prose, maybe one code-gen) with agreed max_tokens/temperature, so that "I tested it on X" means the same thing across people and PRs, and speedup/regression claims are comparable by construction instead of by coincidence.

Might fit naturally as a short section in docs/GB10_DEPLOYMENT_GUIDE.md or as a small tests/prompts/ fixture referenced from CONTRIBUTING.md — happy to be wrong about where it belongs, just flagging that it doesn't exist anywhere yet and would have saved real time this week.

[Sujimoshi]

Tying the last two comments together with a longer-term framing: right now this whole pipeline — run_all_models.py → _manifest.json → gate_results.py, plus the standard-prompts idea above — is a set of scripts a human has to remember to run and interpret. The PR body itself says as much (the "not included" section: a push:[main] workflow that runs the gate is a recommended follow-up, not yet done).

Worth naming explicitly as the target shape: this should end up as a real e2e/integration test suite wired into CI — run_all_models.py + gate_results.py (+ the standard prompts, if those land) as an actual CI job with a pass/fail check on the PR, not a manually-invoked release-time ritual. The gate's own design already has the right property for this (pure verdict logic, explicit exit codes) — it just isn't triggered by anything yet. Concretely that likely means: a scheduled or on-demand GPU-lane workflow (can't run per-PR given the hardware cost, but nightly/pre-release on a self-hosted GB10 runner) that produces the manifest+results and runs the gate as a required check, with the standard prompts as fixed fixtures it exercises rather than ad-hoc contributor invocations.

Not asking for that in this PR — just flagging it as the natural next step so it's tracked somewhere (an issue, or a line in .claude/skills/atlas-release/references/pipeline.md §Closing the gap) rather than left implicit. Right now "did we run the gate before this release" is still a matter of someone remembering to; the tooling this PR adds is necessary but not sufficient for that to stop being true.

[Sujimoshi]

Follow-through on the three comments above: opened #253 as a concrete draft of the "pipeline as pytest" idea — a deployed_atlas/deployed_atlas_ep2 fixture pair + snapshot-based tps regression check (replacing the tps > 0 floor) + GPU-topology-aware xdist_groups + point debugging via pytest -k. It's explicitly a proposal/RFC (draft PR, no real baseline data yet — that needs a live GB10 run), not asking to block this PR on it.

One doc suggestion while it's still easy to adjust: references/verify-matrix.md §Gate is a thorough writeup of the current run_all_models.py + single_gpu_suite.py + gate_results.py trio (including the exact bars and the tps>0 floor this comment thread flagged). Once #253's direction is settled one way or the other, might be worth a line there (or in references/pipeline.md §Closing the gap) pointing at it, so a future reader of the skill doesn't have to discover the proposal by searching PRs.

[AzeezIsh]

looks good