diff --git a/.github/instructions/hve-core/hve-builder.instructions.md b/.github/instructions/hve-core/hve-builder.instructions.md index e7f52f0ab..6295a464a 100644 --- a/.github/instructions/hve-core/hve-builder.instructions.md +++ b/.github/instructions/hve-core/hve-builder.instructions.md @@ -49,6 +49,25 @@ Treat delegation as a first-class architecture decision, not an afterthought. Be * Design the loop explicitly: define dispatch inputs, owned evidence, return schema, stage gate, and which later step consumes the result. Parallelize only independent work. * Reuse before authoring. Prefer reusing an existing subagent, skill, or instruction file as it stands; when it almost fits, prefer adjusting or extending it; create a new artifact only when no existing one can be reasonably adapted. +## Skill-Forward Orchestrator Architecture + +Reusable behavior that must work both directly and under a parent belongs in a user-invocable skill by default. Author the skill first, then add a wrapper only when a user needs an entry point the skill alone does not provide. Do not build one large agent that carries the workflow and then expose a second path into the same logic. + +| Responsibility | Artifact | Control | +|--------------------------------------------------------------------------|------------------------------|---------------------------------------------------------------| +| Reusable workflow and domain capability, invoked directly or by a parent | Skill | `user-invocable: true` | +| Thin user-facing coordination over one or more skills | Agent or prompt wrapper | `disable-model-invocation: true` when it must not be a worker | +| Convention that applies whenever matching paths are edited | Path-scoped instruction file | `applyTo` glob | +| Isolated, parallel, fresh-context, tool-specific, or model-specific work | Subagent | `user-invocable: false` when background-only | + +Keep the wrapper thin: it sequences skills, holds user-facing state, and reports. It does not restate the phase protocols, gates, or evidence rules the skill already owns. + +Standalone-versus-parent differences belong in the shared skill contract, not in a duplicate workflow. A standalone run advises the next action; a parent consumes the same skill return and owns continuation. Optional orchestrator inputs may narrow delegated work, but they must not turn one artifact into two materially different workflows. + +Treat an agent that is both user-invocable and dispatchable as a subagent as an exception. It qualifies only when both contexts share all six of purpose, inputs, outputs, interaction model, write authority, and safety contract. When any one diverges, split the artifact: put the shared behavior in a skill and give each context its own thin entry point. + +The current RPI structure is the reference example. RPI Agent coordinates the lifecycle without duplicating phase protocols. The `rpi-research`, `rpi-plan`, `rpi-implement`, and `rpi-review` skills carry behavior that works the same way directly and under that parent. RPI Researcher and RPI Planner stay bounded background-only workers. + ## Load-Timing and Authority Routing For every rule or fact an artifact would carry, place it where it loads at the right time and binds with the right force. This keeps always-loaded surfaces short and moves enforcement off advisory prose. @@ -227,6 +246,7 @@ Every item applies to the whole file. Mark an item not applicable when it does n * [ ] File structure and frontmatter follow the File Types and Frontmatter Requirements for the artifact type. * [ ] Each fact sits at the right load timing and authority; always-loaded surfaces stay short and non-inferable. * [ ] Delegation is used where it isolates or right-sizes work, and existing subagents, skills, and instructions are reused before new ones are created. +* [ ] Behavior needed both directly and under a parent lives in one user-invocable skill; wrappers stay thin and no workflow logic is duplicated across a standalone and an orchestrated path. * [ ] Connected artifacts agree on modes, stage gates, result vocabulary, and terminal outcomes. * [ ] Existing non-tool capability-bearing frontmatter is preserved unless approved, verified evidence supports a Major, behavior-tested change. * [ ] Each model declaration uses the exact ordered list for its responsibility-selected profile; any override or proxy run is narrow and disclosed. @@ -252,6 +272,7 @@ Remove these on sight when improving or replacing an artifact. The `hve-builder` * Hand-injecting tool descriptions into prompt text and parsing the output; use the native tools field. * Response prefilling on model families that no longer support it, and JSON mode where schema-constrained structured outputs exist. * Kitchen-sink instruction files, copied style guides, copied templates, and exhaustive edge-case lists. Prefer scoped, referenced, evaluation-informed artifacts. +* Workflow logic duplicated between a standalone path and an orchestrated path, or a single agent carrying two materially different contracts for direct and parent-dispatched use. Move the shared behavior into one user-invocable skill and keep each entry point thin. * Singular AGENT.md where AGENTS.md is the current format; keep a compatibility link where needed. * Unsourced length ceilings, fixed iteration counts used as quality theater, and model fallback lists chosen without first selecting a responsibility-based profile. * Calling simulation or emulation native runtime validation. diff --git a/docs/contributing/ai-artifacts-common.md b/docs/contributing/ai-artifacts-common.md index fffe21e0d..0f7e0fada 100644 --- a/docs/contributing/ai-artifacts-common.md +++ b/docs/contributing/ai-artifacts-common.md @@ -657,7 +657,7 @@ See [official documentation](https://example.com) for details. * Starts with frontmatter (YAML between `---` delimiters) * Followed by markdown content -* Omits attribution suffix from the `description` field (added automatically at distribution) +* Omits any attribution suffix from the `description` field * Single newline at EOF ## RFC 2119 Directive Language diff --git a/docs/contributing/custom-agents.md b/docs/contributing/custom-agents.md index e1cdc4b9d..2796f4147 100644 --- a/docs/contributing/custom-agents.md +++ b/docs/contributing/custom-agents.md @@ -535,7 +535,7 @@ Before submitting your agent, verify: * [ ] Core directives with RFC 2119 keywords * [ ] Examples wrapped in XML-style blocks * [ ] Success criteria defined -* [ ] Attribution footer present +* [ ] Attribution footer absent ### Common Standards diff --git a/docs/contributing/instructions.md b/docs/contributing/instructions.md index 3ac960f5f..c69a74fe0 100644 --- a/docs/contributing/instructions.md +++ b/docs/contributing/instructions.md @@ -575,7 +575,7 @@ Before submitting your instructions file, verify: * [ ] Code examples wrapped in XML-style blocks * [ ] Anti-patterns section with alternatives * [ ] Validation/testing requirements -* [ ] Attribution footer present +* [ ] Attribution footer absent ### Code Examples diff --git a/docs/contributing/prompts.md b/docs/contributing/prompts.md index 06968d28a..18e8b9b2f 100644 --- a/docs/contributing/prompts.md +++ b/docs/contributing/prompts.md @@ -537,7 +537,7 @@ Before submitting your prompt, verify: * [ ] Workflow steps with clear sequence * [ ] Success criteria defined * [ ] Error handling documented -* [ ] Attribution footer present +* [ ] Attribution footer absent ### Workflow Definition @@ -568,6 +568,61 @@ Before submitting your prompt, verify: * [ ] Compatible with existing prompts/workflows * [ ] Does not duplicate existing prompt functionality +## Authoring with the HVE Builder Skill + +The `hve-builder` skill is the lifecycle entrypoint for prompts, instruction files, +agents, subagents, and skills. It applies the standards on this page, dispatches +independent review, runs behavior testing when a change warrants it, and resolves a +single overall outcome. Prefer it over hand-editing when you are creating a new prompt +or making a behavior-bearing change to an existing one. + +Activate it by asking for the work in natural language, optionally naming the mode. +There is no slash command; `hve-builder` is a skill, not a prompt. + +### Modes + +| Mode | Write authority | Use when | +|------------|------------------------------|--------------------------------------------------------------| +| `create` | Creates new source artifacts | The target prompt does not exist yet | +| `improve` | Edits existing source | An existing prompt needs new or corrected behavior | +| `refactor` | Edits existing source | Cleanup must preserve current behavior | +| `replace` | Rewrites existing source | The artifact needs wholesale replacement | +| `review` | Read-only; writes evidence | You want static and behavior findings without source changes | +| `validate` | Read-only; writes evidence | You want host validation results only | + +The skill infers the narrowest safe mode when you do not name one, and asks only when +plausible modes would grant materially different write authority. + +### Compatibility aliases + +Three alias skills preserve legacy activation phrasing and route straight to +`hve-builder`. They add no second author, test, or evaluation loop. + +| Alias skill | Routes to | +|-------------------|----------------------------------------| +| `prompt-builder` | `hve-builder` in `create` or `improve` | +| `prompt-analyze` | `hve-builder` in read-only `review` | +| `prompt-refactor` | `hve-builder` in `refactor` | + +Each alias translates its legacy `promptFiles` input to the `hve-builder` `targets` +input. New work should name `hve-builder` and its mode directly. + +### Behavior testing + +`hve-builder` delegates behavior testing to `hve-builder-tester`, which is the sole +behavior-testing entrypoint. Behavior testing runs for major mutations and for +behavior-bearing review targets, and is legitimately skipped for eligible minor and +medium changes. + +### Evidence + +Runs write author, review, behavior-test, and validation evidence under +`.copilot-tracking/hve-builder/{{YYYY-MM-DD}}/` unless you supply a different evidence +root. Read-only modes change nothing else. + +Authoring standards for all artifact kinds live in +`.github/instructions/hve-core/hve-builder.instructions.md`. + ## Testing Your Prompt See [AI Artifacts Common Standards - Common Testing Practices](ai-artifacts-common.md#common-testing-practices) for testing guidelines. For prompts specifically: diff --git a/docs/contributing/skills.md b/docs/contributing/skills.md index 66d275faa..6fa375683 100644 --- a/docs/contributing/skills.md +++ b/docs/contributing/skills.md @@ -101,11 +101,11 @@ The `scripts/` directory is **optional**. When present, it **MUST** contain at l **`description`** (string, MANDATORY) -| Property | Value | -|----------|---------------------------------------------------------------------------------------------------| -| Purpose | Concise explanation of skill functionality | -| Format | Single sentence describing the skill; no attribution suffix (added automatically at distribution) | -| Example | `'Video-to-GIF conversion skill with FFmpeg two-pass optimization'` | +| Property | Value | +|----------|---------------------------------------------------------------------| +| Purpose | Concise explanation of skill functionality | +| Format | Single sentence describing the skill; no attribution suffix | +| Example | `'Video-to-GIF conversion skill with FFmpeg two-pass optimization'` | ### Frontmatter Example diff --git a/docs/customization/build-system.md b/docs/customization/build-system.md index b43327afb..a5ee08aa6 100644 --- a/docs/customization/build-system.md +++ b/docs/customization/build-system.md @@ -62,6 +62,33 @@ The `scripts/linting/schemas/schema-mapping.json` file defines the glob-to-schem Patterns are evaluated from most specific to least specific, and the first match determines the schema. When no pattern matches, `base-frontmatter.schema.json` applies as the default. +### Planner State Schemas + +The same `scripts/linting/schemas/` directory also holds JSON Schemas for planner session +state. These validate the `state.json` files that phase-based planning agents persist under +`.copilot-tracking/`. They are not frontmatter schemas, so they are not listed in +`schema-mapping.json` and are not exercised by `npm run lint:frontmatter`. + +| Schema | Validates | +|-----------------------------------|--------------------------------------------------------------| +| `accessibility-state.schema.json` | `.copilot-tracking/accessibility/{project-slug}/state.json` | +| `rai-state.schema.json` | `.copilot-tracking/rai-plans/{project-slug}/state.json` | +| `security-state.schema.json` | `.copilot-tracking/security-plans/{project-slug}/state.json` | +| `sssc-state.schema.json` | `.copilot-tracking/sssc-plans/{project-slug}/state.json` | + +Each schema is the source of truth for its planner's required keys, field types, enum +values, and defaults. Agent and instruction files show illustrative state snippets with +JSON-literal defaults; when a snippet and its schema disagree, the schema wins. + +Two enforcement paths cover these schemas: + +* Editor validation through the `json.schemas` entries in `.vscode/settings.json`, which + bind the RAI and accessibility state paths to their schemas as you edit. +* Pester coverage in `scripts/tests/linting/Test-PlannerStateSchemas.Tests.ps1` and + `scripts/tests/linting/Test-AccessibilityStateSchema.Tests.ps1`, which validate schema + fixtures and guard the inline state examples in agent and instruction files against + drift. Run them with `npm run test:ps -- -TestPath "scripts/tests/linting/"`. + ### Adding Custom Schemas To add validation for a new file type: diff --git a/scripts/README.md b/scripts/README.md index c2fb9d3cf..c93b2920c 100644 --- a/scripts/README.md +++ b/scripts/README.md @@ -2,7 +2,7 @@ title: Scripts description: PowerShell scripts for linting, validation, and security automation author: HVE Core Team -ms.date: 2026-07-16 +ms.date: 2026-07-31 ms.topic: reference keywords: - powershell @@ -88,6 +88,7 @@ The `linting/` directory contains scripts for validating code quality and docume | `Invoke-PythonLint.ps1` | Python linting via ruff | | `Invoke-PythonTests.ps1` | Python tests via pytest | | `Validate-AdrConsistency.ps1` | Validate ADR structure and Govern-phase consistency rules | +| `Validate-AssetDocs.ps1` | Validate asset documentation coverage and sync | See [linting/README.md](linting/README.md) for detailed documentation. @@ -95,17 +96,38 @@ See [linting/README.md](linting/README.md) for detailed documentation. The `evals/` directory contains PowerShell entry points for agent-behavior, baseline-equivalence, moderation, and other eval automation. -| Script | Purpose | -|----------------------------------|--------------------------------------------------------------------------| -| `Build-AgentBehaviorSpec.ps1` | Regenerate the agent-behavior eval spec from per-agent stimulus partials | -| `Build-AgentInventory.ps1` | Generate the authoritative agent inventory used by eval suites | -| `Invoke-AgentMatrix.ps1` | Run the agent-behavior matrix and aggregate per-agent summaries | -| `Invoke-BaselineEquivalence.ps1` | Run baseline-vs-customized equivalence evals for a target agent | -| `Invoke-ContentModeration.ps1` | Invoke the content moderation CLI over prompt or output content | -| `Invoke-CorpusModeration.ps1` | Moderate changed AI corpus content from the changed-artifact manifest | -| `Invoke-VallyEvals.ps1` | Execute vally evals for changed AI artifacts | - -See [../evals/README.md](../evals/README.md) for the broader eval framework documentation. +| Script | Purpose | +|-------------------------------------------|-------------------------------------------------------------------------------------| +| `Build-AgentBehaviorSpec.ps1` | Regenerate the agent-behavior eval spec from per-agent stimulus partials | +| `Build-AgentInventory.ps1` | Generate the authoritative agent inventory used by eval suites | +| `Get-AgentDependencyMap.ps1` | Build a JSON map of agent dependencies for the baseline-equivalence dispatcher | +| `Get-ChangedAIArtifact.ps1` | Emit a JSON manifest of AI customization artifacts changed between two git refs | +| `Get-ChangedSpecStimulus.ps1` | Emit a JSON manifest of synthetic artifacts derived from changed eval specs | +| `Invoke-AgentMatrix.ps1` | Run the agent-behavior matrix and aggregate per-agent summaries | +| `Invoke-ArtifactModeration.ps1` | Moderate all eval specs plus changed AI artifacts as a pre-job gate | +| `Invoke-BaselineEquivalence.ps1` | Run baseline-vs-customized equivalence evals for a target agent | +| `Invoke-ContentModeration.ps1` | Invoke the content moderation CLI over prompt or output content | +| `Invoke-CorpusModeration.ps1` | Moderate changed AI corpus content from the changed-artifact manifest | +| `Invoke-VallyEvals.ps1` | Execute vally evals for changed AI artifacts | +| `New-AgentMatrixDashboard.ps1` | Render a self-contained HTML dashboard for the per-agent behavior matrix | +| `New-AgentSurfaceSignatures.ps1` | Generate a per-agent surface signature YAML for baseline equivalence runs | +| `New-EquivalenceDashboard.ps1` | Render a self-contained HTML dashboard for a local baseline-equivalence run | +| `Test-CopilotToken.ps1` | Pre-flight probe for the `COPILOT_GITHUB_TOKEN` secret used by vally evals | +| `Test-EvalSpec.ps1` | Validate vally eval spec files against the embedded schema | +| `Test-EvalSpecText.ps1` | Run alex.js and retext-profanities against the AI-artifact markdown corpus | +| `Test-StimulusPresence.ps1` | Verify every changed AI artifact has a matching eval-spec stimulus backlink | +| `Test-VallyTestSafety.ps1` | Repo-wide safety lint flagging eval stimuli and corpora that need refusal coverage | +| `Update-AgentMatrixSummariesFromLogs.ps1` | Rebuild per-agent matrix JSON summaries from existing vally logs without re-running | + +Most of these run through CI-owned `ci:eval:*` package scripts. See +[../docs/contributing/evals-ci.md](../docs/contributing/evals-ci.md) for the command +taxonomy and prerequisites, and [../evals/README.md](../evals/README.md) for the broader +eval framework documentation. + +`Get-AgentDependencyMap.ps1`, `Get-ChangedAIArtifact.ps1`, `Get-ChangedSpecStimulus.ps1`, +`New-AgentSurfaceSignatures.ps1`, `Test-CopilotToken.ps1`, and +`Update-AgentMatrixSummariesFromLogs.ps1` have no package-script wrapper and are invoked +directly by workflows or run ad hoc with `pwsh -NoProfile -File`. ## Devcontainer Scripts @@ -149,12 +171,38 @@ fails on orphan pages, missing current pages, and generated-region drift. The `security/` directory contains scripts for security scanning and dependency management: -| Script | Purpose | -|-------------------------------------|-------------------------------------------| -| `Test-DependencyPinning.ps1` | Validate dependency pinning compliance | -| `Test-SHAStaleness.ps1` | Check for outdated SHA pins | -| `Update-ActionSHAPinning.ps1` | Automate updating GitHub Actions SHA pins | -| `Test-ActionVersionConsistency.ps1` | Validate action version consistency | +| Script | Purpose | +|-------------------------------------|----------------------------------------------------------------------| +| `Install-PSModules.ps1` | Install pinned PowerShell modules for local and CI environments | +| `Invoke-PipAudit.ps1` | Audit Python dependencies for known vulnerabilities | +| `Sign-PlannerArtifacts.ps1` | Generate a SHA-256 manifest for planner artifacts and sign it | +| `Test-ActionVersionConsistency.ps1` | Validate action version consistency | +| `Test-DangerousWorkflow.ps1` | Detect template-injection patterns in GitHub Actions workflows | +| `Test-DependencyPinning.ps1` | Validate dependency pinning compliance | +| `Test-PrValidationGate.ps1` | Validate that the PR-validation gate job depends on every other job | +| `Test-PSModulePins.ps1` | Validate PowerShell module version pins against the canonical config | +| `Test-PublicDependencyFeeds.ps1` | Validate that committed dependency metadata uses public feeds | +| `Test-SHAStaleness.ps1` | Check for outdated SHA pins | +| `Test-WorkflowPermissions.ps1` | Validate least-privilege permissions in workflows | +| `Update-ActionSHAPinning.ps1` | Automate updating GitHub Actions SHA pins | + +Run locally: + +```bash +npm run lint:dangerous-workflow +npm run lint:pr-gate +npm run lint:ps-module-pins +npm run lint:public-dependency-feeds +npm run security:sign +``` + +`Install-PSModules.ps1`, `Test-SHAStaleness.ps1`, and `Update-ActionSHAPinning.ps1` have no +package-script wrapper and are invoked directly by CI: `Install-PSModules.ps1` by the +`setup-ps-modules` composite action and `copilot-setup-steps.yml`, `Test-SHAStaleness.ps1` +by `sha-staleness-check.yml`, and `Update-ActionSHAPinning.ps1` by +`weekly-security-maintenance.yml`. Run them ad hoc with `pwsh -NoProfile -File`. + +See [security/README.md](security/README.md) for detailed documentation. ## Plugins diff --git a/scripts/linting/README.md b/scripts/linting/README.md index 4466a3a08..503f02e46 100644 --- a/scripts/linting/README.md +++ b/scripts/linting/README.md @@ -2,7 +2,7 @@ title: Linting Scripts description: PowerShell scripts for code quality validation and documentation checks author: HVE Core Team -ms.date: 2026-07-08 +ms.date: 2026-07-28 ms.topic: reference keywords: - powershell @@ -531,11 +531,57 @@ The linting directory also contains these scripts that are not yet covered in th | `Test-ExtensionArtifactNaming.ps1` | Validate extension-vsix artifact producer and consumer naming across the extension release workflows | | `Update-ModelCatalog.ps1` | Refresh the model catalog from GitHub docs data | | `Format-MarkdownTables.ps1` | Normalize markdown tables to the repository formatting convention | +| `Validate-AssetDocs.ps1` | Validate asset documentation coverage, orphans, sync, structure, and authored completeness | + +#### `Validate-AssetDocs.ps1` + +Enforces the documentation-as-a-required-artifact contract for every documentable +GenAI asset (agent, prompt, instruction, skill) against the `docs/reference` tree. +It runs five checks and writes a JSON summary, exiting non-zero when any +error-level finding is present: + +| Check | Behavior | +|-----------|-------------------------------------------------------------------------------------------------| +| Coverage | Every asset has a docs page; an error under `-FailOnMissing`, otherwise a warning | +| Orphans | Every `docs/reference` page maps to an existing asset | +| Sync | Generated regions match a fresh render; reported under `-CheckSync` | +| Structure | Required H2 sections and generated-region markers are present | +| Authored | Human sections differ from stubs; an error under `-RequireAuthoredContent`, otherwise a warning | + +Reference index pages (`README.md`) are excluded from the coverage, sync, +structure, and authored checks and are never treated as orphans. The +`How to use it` section is required only for interactive assets. + +##### Parameters + +* `-RepoRoot` - Repository root (default: the git top level) +* `-FailOnMissing` (switch) - Treat missing documentation pages as errors +* `-CheckSync` (switch) - Compare generated regions against a fresh render and report drift as errors +* `-RequireAuthoredContent` (switch) - Treat remaining stub placeholders as errors +* `-ChangedFilesOnly` (switch) - Validate only assets and pages affected by changed files +* `-BaseBranch` - Git reference for changed-file detection (default: `origin/main`) +* `-OutputPath` - JSON results path (default: `logs/asset-docs-validation-results.json`) + +##### Usage + +```powershell +# Warning-level report +./scripts/linting/Validate-AssetDocs.ps1 + +# Enforce coverage and generated-region sync +./scripts/linting/Validate-AssetDocs.ps1 -FailOnMissing -CheckSync +``` + +##### GitHub Actions Integration + +* npm script: `npm run lint:asset-docs` +* Regenerate pages first with `npm run docs:generate`; preview drift with `npm run docs:generate:check` ## npm Scripts | npm Script | Description | |----------------------------------|------------------------------------------------------------------------------------------------------------------------| +| `lint:asset-docs` | Run `pwsh -NoProfile -File scripts/linting/Validate-AssetDocs.ps1 -FailOnMissing -CheckSync` to enforce asset docs | | `lint:extension-artifact-naming` | Run `pwsh -NoProfile -File scripts/linting/Test-ExtensionArtifactNaming.ps1` to validate extension VSIX artifact names | ## Shared Module diff --git a/scripts/security/README.md b/scripts/security/README.md index ef646a918..eb53d69d0 100644 --- a/scripts/security/README.md +++ b/scripts/security/README.md @@ -37,8 +37,13 @@ The security scripts share common modules and follow a consistent pattern: * [`Test-ActionVersionConsistency.ps1`](#test-actionversionconsistencyps1): action version alignment * [`Update-ActionSHAPinning.ps1`](#update-actionshapinningps1): auto-remediation of SHA pins * [`Invoke-PipAudit.ps1`](#invoke-pipauditps1): Python dependency audit +* [`Test-PublicDependencyFeeds.ps1`](#test-publicdependencyfeedsps1): public dependency feed enforcement * [`Test-WorkflowPermissions.ps1`](#test-workflowpermissionsps1): workflow permissions validation +* [`Test-DangerousWorkflow.ps1`](#test-dangerousworkflowps1): workflow template-injection detection +* [`Test-PrValidationGate.ps1`](#test-prvalidationgateps1): PR-validation gate completeness * [`Install-PSModules.ps1`](#install-psmodulesps1): centralized PS module install with retry +* [`Test-PSModulePins.ps1`](#test-psmodulepinsps1): PS module version pin enforcement +* [`Sign-PlannerArtifacts.ps1`](#sign-plannerartifactsps1): planner artifact manifest and signing * [`Modules/SecurityClasses.psm1`](#modulessecurityclassespsm1): shared data types * [`Modules/SecurityHelpers.psm1`](#modulessecurityhelperspsm1): shared utilities @@ -228,6 +233,38 @@ reach production. ./scripts/security/Invoke-PipAudit.ps1 -Path ".github/skills/experimental/powerpoint" ``` +### `Test-PublicDependencyFeeds.ps1` + +Validates that committed dependency metadata uses canonical public feeds. + +Purpose: Prevent dependency-confusion and credential-leak vectors by rejecting +non-public hosts, plain HTTP, embedded credentials, and non-literal npm registry +values in committed manifests and lockfiles. + +#### Features + +* Scans npm, Python, and uv dependency manifests and lockfiles for source URLs +* Requires npm registry declarations to use the canonical public npm registry +* Flags plain HTTP sources and URLs carrying embedded credentials +* Writes structured results to `logs/public-dependency-feeds-results.json` +* Integrates with `npm run lint:public-dependency-feeds` + +#### Parameters + +* `-RepoRoot` - Repository root to scan (default: the repository containing this script) +* `-OutputPath` - JSON results path (default: `logs/public-dependency-feeds-results.json`) +* `-FailOnViolation` (switch) - Exit with non-zero code when a prohibited source is found + +#### Usage + +```powershell +# Report only +./scripts/security/Test-PublicDependencyFeeds.ps1 + +# Enforce in CI +./scripts/security/Test-PublicDependencyFeeds.ps1 -FailOnViolation +``` + ### `Test-WorkflowPermissions.ps1` Validates that GitHub Actions workflow files declare permissions at the workflow @@ -283,6 +320,75 @@ workflow-level set rather than being capped by it. ./scripts/security/Test-WorkflowPermissions.ps1 -Format sarif -FailOnViolation ``` +### `Test-DangerousWorkflow.ps1` + +Detects template-injection patterns in GitHub Actions workflows. + +Purpose: Catch direct interpolation of attacker-controllable GitHub event values +into `run` or script execution contexts. This is a narrowed single-rule gate that +emits the rule ID `dangerous-workflow/template-injection`. Broader +dangerous-workflow coverage, including untrusted checkout and unpinned actions, is +provided advisory-only by the Poutine scanner in CI. + +#### Features + +* Scans workflow YAML files for injected event-context expressions +* Outputs results in `console`, `json`, or `sarif` format +* Integrates with `npm run lint:dangerous-workflow` + +#### Parameters + +* `-Path` - Directory containing workflow YAML files (default: `.github/workflows`) +* `-Format` - Output format: `console`, `json`, or `sarif` (default: `console`) +* `-OutputPath` - Path for result output file (default: `logs/dangerous-workflow-results.json`, or the `.sarif` variant for SARIF output) +* `-FailOnViolation` (switch) - Exit with non-zero code when in-scope findings remain + +#### Usage + +```powershell +# Console report +./scripts/security/Test-DangerousWorkflow.ps1 + +# Enforce and export SARIF +./scripts/security/Test-DangerousWorkflow.ps1 -FailOnViolation -Format sarif +``` + +### `Test-PrValidationGate.ps1` + +Validates that the PR-validation aggregator gate job depends on every other job. + +Purpose: Prevent a job from silently bypassing required-check enforcement because +the gate job never waits on it. + +#### Features + +* Parses the workflow structurally with `ConvertFrom-Yaml` rather than regex +* Detects missing jobs, where a job exists but is absent from the gate's `needs:` list +* Detects stale needs, where `needs:` references a job ID that no longer exists +* Treats an absent gate job as a violation +* Writes JSON results under `logs/` and a readable summary to the console +* Integrates with `npm run lint:pr-gate` + +#### Parameters + +* `-WorkflowPath` - Workflow YAML file to validate (default: `.github/workflows/pr-validation.yml`) +* `-GateJobId` - Aggregator gate job ID (default: `pr-validation-success`) +* `-OutputPath` - JSON results path (default: `logs/pr-validation-gate-results.json`) +* `-FailOnViolation` (switch) - Exit with non-zero code and name the offending jobs + +#### Usage + +```powershell +# Report only +./scripts/security/Test-PrValidationGate.ps1 + +# Enforce in CI +./scripts/security/Test-PrValidationGate.ps1 -FailOnViolation +``` + +This validator requires the `PowerShell-Yaml` module at the version pinned in +`ps-module-versions.json`. + ### `Install-PSModules.ps1` Installs PowerShell modules declared in `ps-module-versions.json` with @@ -351,6 +457,79 @@ Parameters take precedence over environment variables. ./scripts/security/Install-PSModules.ps1 -Force -Import ``` +### `Test-PSModulePins.ps1` + +Validates PowerShell module version pins against the canonical pin config. + +Purpose: Keep every module pin in the repository aligned with +`ps-module-versions.json` so a single manifest governs the supply chain. + +#### Features + +* Scans tracked and untracked, non-ignored files for pins expressed as + `Install-Module -RequiredVersion`, `Import-Module -RequiredVersion`, and + `#Requires -Modules @{ ... RequiredVersion='...' }` +* Compares every pin for a managed module against the canonical version +* Allows a small fixture list of files that intentionally carry non-canonical literals +* Writes JSON results to `logs/ps-module-pins-results.json` +* Exits non-zero on violations +* Integrates with `npm run lint:ps-module-pins` + +#### Parameters + +* `-ConfigPath` - Path to the canonical pin config (default: `scripts/security/ps-module-versions.json`) + +#### Usage + +```powershell +./scripts/security/Test-PSModulePins.ps1 +``` + +### `Sign-PlannerArtifacts.ps1` + +Generates a SHA-256 manifest for planner artifacts and optionally signs it with cosign. + +Purpose: Provide cryptographic provenance for RAI, SSSC, and other planner +session outputs. + +#### Features + +* Enumerates every file under a planner session directory and hashes it with SHA-256 +* Resolves RAI sessions from `-ProjectSlug` under `.copilot-tracking/rai-plans/` +* Accepts any other planner session directory through `-SessionPath` +* Excludes the manifest itself and cosign signature files (`.sig`, `.bundle`) from the inventory +* Signs the manifest with Sigstore keyless signing when `-IncludeCosign` is set, + and warns rather than fails when cosign is unavailable + +#### Parameters + +* `-ProjectSlug` - RAI session slug under `.copilot-tracking/rai-plans/` (mutually exclusive with `-SessionPath`) +* `-SessionPath` - Planner session directory, absolute or repository-relative (mutually exclusive with `-ProjectSlug`) +* `-ManifestName` - Manifest file name written inside the session directory (default: `artifact-manifest.json`) +* `-OutputPath` - Full manifest path; overrides `-ManifestName` +* `-IncludeCosign` (switch) - Sign the manifest with cosign keyless signing + +#### Usage + +```powershell +# RAI session manifest +./scripts/security/Sign-PlannerArtifacts.ps1 -ProjectSlug "contoso-ai" + +# RAI session manifest with cosign signing +./scripts/security/Sign-PlannerArtifacts.ps1 -ProjectSlug "contoso-ai" -IncludeCosign + +# SSSC session manifest +./scripts/security/Sign-PlannerArtifacts.ps1 -SessionPath '.copilot-tracking/sssc-plans/contoso-supply-chain' -ManifestName 'sssc-manifest.json' +``` + +The package scripts `npm run rai:sign`, `npm run sssc:sign`, and +`npm run security:sign` all invoke this script; pass parameters after `--`. + +Under the `-SessionPath` parameter set, the manifest's `projectSlug` field is +populated from the session directory leaf. The field name is retained for +back-compatibility with existing RAI manifest consumers; callers that need to +distinguish project slug from session label should rely on `sessionPath`. + ## Modules ### `Modules/SecurityClasses.psm1` @@ -390,13 +569,19 @@ preferred for new entries. Security scripts integrate with these workflows: -| Workflow | Script(s) | Trigger | -|---------------------------------|--------------------------------|--------------| -| `dependency-pinning-scan.yml` | `Test-DependencyPinning.ps1` | PR, schedule | -| `sha-staleness-check.yml` | `Test-SHAStaleness.ps1` | Schedule | -| `pr-validation.yml` | `Test-DependencyPinning.ps1` | Pull request | -| `pip-audit.yml` | `Invoke-PipAudit.ps1` | PR, schedule | -| `workflow-permissions-scan.yml` | `Test-WorkflowPermissions.ps1` | PR, schedule | +| Workflow | Script(s) | Trigger | +|---------------------------------|--------------------------------|-------------------------------| +| `dependency-pinning-scan.yml` | `Test-DependencyPinning.ps1` | PR, schedule | +| `sha-staleness-check.yml` | `Test-SHAStaleness.ps1` | Schedule | +| `pr-validation.yml` | `Test-DependencyPinning.ps1` | Pull request | +| `pip-audit.yml` | `Invoke-PipAudit.ps1` | PR, schedule | +| `workflow-permissions-scan.yml` | `Test-WorkflowPermissions.ps1` | PR, schedule | +| `dangerous-workflow-scan.yml` | `Test-DangerousWorkflow.ps1` | Called by `pr-validation.yml` | + +`dangerous-workflow-scan.yml` is a reusable `workflow_call` workflow rather than a +directly triggered one. It runs `Test-DangerousWorkflow.ps1` as the blocking gate and +the Poutine scanner alongside it, with Poutine advisory by default through the +`poutine-soft-fail` input. ## Related Documentation