Skip to content

docs: fix stale workflow, job, and script references across three guides - #2533

Open
jkim323 wants to merge 4 commits into
mainfrom
docs/stale-workflow-job-script-refs
Open

docs: fix stale workflow, job, and script references across three guides#2533
jkim323 wants to merge 4 commits into
mainfrom
docs/stale-workflow-job-script-refs

Conversation

@jkim323

@jkim323 jkim323 commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

Pull Request

Description

Six documentation-drift issues shared only three target files, so this change remediates all of them in a single pass.

docs/security/security-model.md carried supply-chain control entries that pointed at a workflow file which no longer exists and a release job that was renamed. SC-7 and SC-8 now reference release-stable.yml, and SC-9 now names the vex-attest job. SC-9 was also the only detailed control in that table without a matching implementation block, so this adds one following the existing SC-8 pattern, covering detection, drafting, release attestation, permissions, and the human review gate.

docs/architecture/agentic-workflows.md still described an issue-triage decomposition step that was removed from the agent and its workflow. The Mermaid diagram nodes for that branch are gone, the flow now routes directly to the agent-ready evaluation, and the Key Actions cell and tip block match current behavior. This page also gained a section explaining that *.lock.yml files are generated by gh aw compile, that gh-aw-actions is pinned in .github/aw/actions-lock.json rather than managed by Dependabot, and the bump-then-recompile upgrade procedure.

docs/contributing/evals-ci.md understated what ci:eval:lint:schema enforces. It now documents all three checks that lane performs, adds an exit-code table for Test-EvalSpec.ps1, and explains the generated agent inventory and its subagent enrollment rule. A new table documents six eval scripts that had no coverage in any guide. A seventh drift item surfaced while verifying that page: it documented the lint flag as vally lint --eval, but package.json and Invoke-BaselineEquivalence.ps1 have both used --eval-spec since June. All three occurrences in this file are corrected.

Related Issue(s)

Fixes #2528
Fixes #2251
Fixes #2400
Fixes #2403
Fixes #2427
Fixes #2288

Type of Change

Select all that apply:

Code & Documentation:

  • Bug fix (non-breaking change fixing an issue)
  • New feature (non-breaking change adding functionality)
  • Breaking change (fix or feature causing existing functionality to change)
  • Documentation update

Infrastructure & Configuration:

  • GitHub Actions workflow
  • Linting configuration (markdown, PowerShell, etc.)
  • Security configuration
  • DevContainer configuration
  • Dependency update

AI Artifacts:

  • Reviewed contribution with hve-builder and addressed all actionable findings
  • Copilot instructions (.github/instructions/*.instructions.md)
  • Copilot prompt (.github/prompts/*.prompt.md)
  • Copilot agent (.github/agents/*.agent.md)
  • Copilot skill (.github/skills/*/SKILL.md)
  • Copilot hook (.github/hooks/*/*.json)
  • Eval spec added/updated for changed AI artifacts (evals/)

Note for AI Artifact Contributors:

  • Agents: Research, indexing/referencing other project (using standard VS Code GitHub Copilot/MCP tools), planning, and general implementation agents likely already exist. Review .github/agents/ before creating new ones.
  • Skills: Must include both bash and PowerShell scripts. See Skills.
  • Model Versions: Only contributions targeting the latest Anthropic and OpenAI models will be accepted. Older model versions (e.g., GPT-3.5, Claude 3) will be rejected.
  • See Agents Not Accepted and Model Version Requirements.

Other:

  • Script/automation (.ps1, .sh, .py)
  • Other (please describe):

Testing

Staleness sweep

Every claim in all six issues was re-verified against current main before any edit, rather than trusted as written. The issues were filed between April and July, and several had drifted. The sweep confirmed all six gaps are still real, and it caught six stale details that would have introduced incorrect content if the issue text had been copied literally:

Source issue Stale in the issue Verified current state
#2403 Test-VallyTestSafety.ps1 listed as undocumented Already documented, so scope dropped from seven scripts to six
#2400 Command named eval:lint:schema, running in lint:all Command is ci:eval:lint:schema; lint:all no longer exists
#2400 Agent inventory stated as 73 slugs 62 slugs today, so the count was deliberately left out of the docs
#2288 Release job attest-and-upload-vex Job is vex-attest
#2288 Drafting credited to a "VEX Generator" agent The workflow imports the SSSC Reviewer agent
#2288 Drafting permissions include actions: read Frontmatter grants only contents: read and issues: read

The sweep also confirmed the parts that were accurate, including the absence of main.yml, the presence of Test-OrphanedStimulusTag in Test-EvalSpec.ps1, the Dependabot exclusions, the actions-lock.json pin file, the detection schedule and scanner, and both attestation types produced at release.

Issue #2288 depended on #2427: its proposed content named the same non-existent job that #2427 exists to correct. The security-model edits were sequenced so the renames landed first and the new SC-9 block was authored against corrected names.

Line numbers cited in #2251 and #2288 had drifted, so edits were anchored to headings and content rather than the cited positions.

Checklist

Required Checks

  • Documentation is updated (if applicable)
  • Files follow existing naming conventions
  • Changes are backwards compatible (if applicable)
  • Tests added for new functionality (if applicable) (N/A — documentation-only change adds no functionality)

AI Artifact Contributions

  • Used hve-builder review mode to review contribution
  • Addressed all actionable findings from the hve-builder review
  • Verified contribution follows common standards and type-specific requirements

Required Local Checks

The following local-safe validation commands must pass before merging:

  • Local validation aggregate: npm run validate:local
  • Documentation validation (if docs changed): npm run validate:docs
  • Spell checking: npm run spell-check
  • Link validation: npm run lint:md-links

Security Considerations

  • This PR does not contain any sensitive or NDA information
  • Any new dependencies have been reviewed for security issues (N/A — no dependency changes)
  • Security-related scripts follow the principle of least privilege (N/A — no scripts modified)

Additional Notes

@jkim323
jkim323 requested a review from a team as a code owner July 28, 2026 19:09
@codecov-commenter

codecov-commenter commented Jul 28, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.74%. Comparing base (cf29fb4) to head (457c38c).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #2533      +/-   ##
==========================================
- Coverage   82.75%   82.74%   -0.01%     
==========================================
  Files         155      155              
  Lines       20960    20960              
  Branches       13       13              
==========================================
- Hits        17345    17344       -1     
- Misses       3613     3614       +1     
  Partials        2        2              
Flag Coverage Δ
docusaurus 94.44% <ø> (ø)
pester 86.18% <ø> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

Copy link
Copy Markdown
Contributor

Eval Execution

⚠️ No eval summary was produced.

@katriendg katriendg left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice cleanup — the main.ymlrelease-stable.yml job references, the vally lint --eval-spec flag fix, and the new evals-ci/SC-9 sections all check out against the current repo state.

One inline correction on the "Upgrading gh-aw-actions" sequence (see comment) — actions-lock.json is compiler-managed, not something to hand-edit.

Also: docs/architecture/agentic-workflows.md, docs/contributing/evals-ci.md, and docs/security/security-model.md all have ms.date frontmatter but none is bumped in this diff. Per repo convention, please update ms.date to today's date on all three touched files.

Approving — good to merge once the gh-aw-actions section is fixed, ms.date is bumped, and the branch is rebased onto main (currently behind).

Comment on lines +139 to +141
1. Bump the `gh-aw-actions` version in `.github/aw/actions-lock.json`.
2. Run `gh aw compile` to regenerate every `*.lock.yml` file against the new version.
3. Commit the updated `actions-lock.json` and the regenerated lock files together.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sequence has the causality backwards. .github/aw/actions-lock.json is a compiler-maintained cache of resolved action@version → SHA pairs, not something meant to be hand-edited — the repo's own .github/dependabot.yml even carries the comment # Managed by gh aw compile. Version-locked to the gh-aw compiler; do not bump. on this dependency, and upstream gh-aw docs say explicitly: "Never edit these references by hand — run gh aw compile or gh aw update-actions to regenerate them."

Suggested correction:

1. Upgrade the `gh aw` CLI/compiler (the resolved `gh-aw-actions` version is tied to the compiler release, not set independently).
2. Run `gh aw compile` (or `gh aw update-actions`) with API access to re-resolve actions and regenerate `actions-lock.json` plus every `*.lock.yml` file.
3. Commit `actions-lock.json` and the regenerated lock files together.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment