docs: fix stale workflow, job, and script references across three guides - #2533
docs: fix stale workflow, job, and script references across three guides#2533jkim323 wants to merge 4 commits into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2533 +/- ##
==========================================
- Coverage 82.75% 82.74% -0.01%
==========================================
Files 155 155
Lines 20960 20960
Branches 13 13
==========================================
- Hits 17345 17344 -1
- Misses 3613 3614 +1
Partials 2 2
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
Eval Execution |
katriendg
left a comment
There was a problem hiding this comment.
Nice cleanup — the main.yml → release-stable.yml job references, the vally lint --eval-spec flag fix, and the new evals-ci/SC-9 sections all check out against the current repo state.
One inline correction on the "Upgrading gh-aw-actions" sequence (see comment) — actions-lock.json is compiler-managed, not something to hand-edit.
Also: docs/architecture/agentic-workflows.md, docs/contributing/evals-ci.md, and docs/security/security-model.md all have ms.date frontmatter but none is bumped in this diff. Per repo convention, please update ms.date to today's date on all three touched files.
Approving — good to merge once the gh-aw-actions section is fixed, ms.date is bumped, and the branch is rebased onto main (currently behind).
| 1. Bump the `gh-aw-actions` version in `.github/aw/actions-lock.json`. | ||
| 2. Run `gh aw compile` to regenerate every `*.lock.yml` file against the new version. | ||
| 3. Commit the updated `actions-lock.json` and the regenerated lock files together. |
There was a problem hiding this comment.
This sequence has the causality backwards. .github/aw/actions-lock.json is a compiler-maintained cache of resolved action@version → SHA pairs, not something meant to be hand-edited — the repo's own .github/dependabot.yml even carries the comment # Managed by gh aw compile. Version-locked to the gh-aw compiler; do not bump. on this dependency, and upstream gh-aw docs say explicitly: "Never edit these references by hand — run gh aw compile or gh aw update-actions to regenerate them."
Suggested correction:
1. Upgrade the `gh aw` CLI/compiler (the resolved `gh-aw-actions` version is tied to the compiler release, not set independently).
2. Run `gh aw compile` (or `gh aw update-actions`) with API access to re-resolve actions and regenerate `actions-lock.json` plus every `*.lock.yml` file.
3. Commit `actions-lock.json` and the regenerated lock files together.
Pull Request
Description
Six documentation-drift issues shared only three target files, so this change remediates all of them in a single pass.
docs/security/security-model.mdcarried supply-chain control entries that pointed at a workflow file which no longer exists and a release job that was renamed. SC-7 and SC-8 now referencerelease-stable.yml, and SC-9 now names thevex-attestjob. SC-9 was also the only detailed control in that table without a matching implementation block, so this adds one following the existing SC-8 pattern, covering detection, drafting, release attestation, permissions, and the human review gate.docs/architecture/agentic-workflows.mdstill described an issue-triage decomposition step that was removed from the agent and its workflow. The Mermaid diagram nodes for that branch are gone, the flow now routes directly to the agent-ready evaluation, and the Key Actions cell and tip block match current behavior. This page also gained a section explaining that*.lock.ymlfiles are generated bygh aw compile, thatgh-aw-actionsis pinned in.github/aw/actions-lock.jsonrather than managed by Dependabot, and the bump-then-recompile upgrade procedure.docs/contributing/evals-ci.mdunderstated whatci:eval:lint:schemaenforces. It now documents all three checks that lane performs, adds an exit-code table forTest-EvalSpec.ps1, and explains the generated agent inventory and its subagent enrollment rule. A new table documents six eval scripts that had no coverage in any guide. A seventh drift item surfaced while verifying that page: it documented the lint flag asvally lint --eval, butpackage.jsonandInvoke-BaselineEquivalence.ps1have both used--eval-specsince June. All three occurrences in this file are corrected.Related Issue(s)
Fixes #2528
Fixes #2251
Fixes #2400
Fixes #2403
Fixes #2427
Fixes #2288
Type of Change
Select all that apply:
Code & Documentation:
Infrastructure & Configuration:
AI Artifacts:
hve-builderand addressed all actionable findings.github/instructions/*.instructions.md).github/prompts/*.prompt.md).github/agents/*.agent.md).github/skills/*/SKILL.md).github/hooks/*/*.json)evals/)Other:
.ps1,.sh,.py)Testing
Staleness sweep
Every claim in all six issues was re-verified against current
mainbefore any edit, rather than trusted as written. The issues were filed between April and July, and several had drifted. The sweep confirmed all six gaps are still real, and it caught six stale details that would have introduced incorrect content if the issue text had been copied literally:Test-VallyTestSafety.ps1listed as undocumentedeval:lint:schema, running inlint:allci:eval:lint:schema;lint:allno longer existsattest-and-upload-vexvex-attestactions: readcontents: readandissues: readThe sweep also confirmed the parts that were accurate, including the absence of
main.yml, the presence ofTest-OrphanedStimulusTaginTest-EvalSpec.ps1, the Dependabot exclusions, theactions-lock.jsonpin file, the detection schedule and scanner, and both attestation types produced at release.Issue #2288 depended on #2427: its proposed content named the same non-existent job that #2427 exists to correct. The security-model edits were sequenced so the renames landed first and the new SC-9 block was authored against corrected names.
Line numbers cited in #2251 and #2288 had drifted, so edits were anchored to headings and content rather than the cited positions.
Checklist
Required Checks
AI Artifact Contributions
hve-builderreview mode to review contributionhve-builderreviewRequired Local Checks
The following local-safe validation commands must pass before merging:
npm run validate:localnpm run validate:docsnpm run spell-checknpm run lint:md-linksSecurity Considerations
Additional Notes