docs(security): add Black Hat 2025 misuse playbook and critical routing#22282
docs(security): add Black Hat 2025 misuse playbook and critical routing#22282BrianCLong wants to merge 1 commit intomainfrom
Conversation
BrianCLong
added
the
codex
ChatGPT Codex Connector
|
Warning Rate limit exceeded
Your organization is not enrolled in usage-based pricing. Contact your admin to enable usage-based pricing to continue reviews beyond the rate limit, or try again in 23 minutes and 48 seconds. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a comprehensive security playbook for responding to AI-enabled misuse patterns and updates the prompt injection quick reference guide with formatting improvements and a new section on high-risk routing. The review feedback suggests strengthening the policy for hidden instruction channels by using a hard deny and clarifying the recovery threshold criteria in the incident response workflow.
| | User asks for mass misinformation planning | Critical MA/PI | **Hard deny** + safe alternative guidance | Security on-call + Trust & Safety | | ||
| | Prompt requests sabotage of employer systems or workforce transition | High IN/DP | **Hard deny** + insider-risk warning | Security + HR/legal workflow | | ||
| | Content appears to poison behavior/feedback loops | High DP/GH | Quarantine artifact, block learning path | ML security review | | ||
| | Hidden instruction channel detected | High PI/TI | Strip, sanitize, and re-run policy checks | SOC triage | |
There was a problem hiding this comment.
The action for "Hidden instruction channel detected" is currently "Strip, sanitize, and re-run policy checks". Since a hidden channel (e.g., invisible characters, steganography) is a deliberate attempt to bypass security controls, it should be treated with the same severity as other high-risk misuse. Simply stripping the content and proceeding might allow the rest of the prompt to execute, which could still be part of a multi-stage attack. Consider changing the default action to a Hard deny to align with the "High PI/TI" classification and the handling of other high-risk scenarios in this table.
| | Hidden instruction channel detected | High PI/TI | Strip, sanitize, and re-run policy checks | SOC triage | | |
| | Hidden instruction channel detected | High PI/TI | **Hard deny** + incident logging | SOC triage | |
| 2. **Classify:** map event to MAESTRO layers and STRIDE+AI category. | ||
| 3. **Notify:** trigger Security + Trust response channel with evidence bundle. | ||
| 4. **Eradicate:** patch policy rules, expand signatures, and backtest recent sessions. | ||
| 5. **Recover:** re-enable traffic only after false-negative sampling passes threshold. |
There was a problem hiding this comment.
The phrase "after false-negative sampling passes threshold" is ambiguous. In a security context, you typically want the false-negative rate to be below a maximum allowable threshold. It would be clearer to specify that the rate must fall below the threshold or that the sampling results must meet the required safety criteria.
| 5. **Recover:** re-enable traffic only after false-negative sampling passes threshold. | |
| 5. **Recover:** re-enable traffic only after false-negative sampling falls below the safety threshold. |
## Summary Rebuilds the golden-main merge train from a clean `main` base and converges the currently mergeable PR set into one replacement branch. This branch absorbs: - #22296 - #22279 - #22281 - #22282 - #22283 - #22284 - #22285 - #22295 - #22297 - #22286 - #22291 - #22280 - #22277 - unique non-conflicting surfaces from #22241 - the admissibility/CACert/failure-demo runtime lane from #22314 This branch supersedes: - #22298 as the contaminated/conflicting convergence branch - #22277 as a standalone merge vehicle - #22241 as the broad mixed-purpose convergence vehicle once remaining review is complete - #22314 as the standalone admissibility lane now folded into the golden path This branch intentionally excludes: - #22292 because it targets `merge-surge/staging`, not `main` ## Conflict policy used while absorbing #22241 When merging `#22241` on top of the cleaned train, the following files conflicted and were resolved in favor of the current train versions so the newer focused CI/governance repairs remain authoritative: - `.github/ci/required-checks.json` - `.github/workflows/drift-sentinel.yml` - `.github/workflows/pr-gate.yml` - `docs/ci/REQUIRED_CHECKS_POLICY.yml` - `pnpm-lock.yaml` - `scripts/ci/check_branch_protection_drift.mjs` - `scripts/ci/validate_workflows.mjs` All other `#22241` changes merged on top of the train. ## Mapping Change Summary This convergence branch updates workflow, schema, and governance contracts that control merge eligibility, admissibility evidence, and deterministic trust artifacts. ## Diff - Added admissibility/evidence/CACert surfaces including `packages/evidence/schemas/decision_trace.schema.json` - Tightened golden-lane workflow policy and drift handling in `.github/workflows/_policy-enforcer.yml`, `.github/workflows/execution-graph-reconciliation.yml`, `.github/workflows/post-ga-hardening-enforcement.yml`, `.github/workflows/merge-surge.yml`, `.github/workflows/control-plane-drift.yml` - Realigned governance state in `governance/pilot-ci-policy.json` and `governance/branch-protection.json` - Repaired deterministic reconciliation verification in `scripts/ci/verify_execution_graph_reconciliation.mjs` and `scripts/ci/drift-sentinel.mjs` ## Justification The repo needed one mergeable replacement lane that restores deterministic governance checks, folds the admissibility implementation into the golden path, and suppresses broken optional PR workflows that were blocking convergence without being canonical required checks. ## Impact - Canonical pilot checks remain `pr-gate / gate` and `drift-sentinel / enforce` - Merge-train branches no longer fail ordinary small-PR enforcement gates by construction - Optional broken workflows are narrowed to their owned surfaces so they stop contaminating this convergence lane and the immediate post-merge main push - Execution-graph reconciliation now accepts the repo’s canonical snake_case trust bundle fields ## Rollback Plan Revert commit `ce32b96c0f` from `merge-train/golden-main-20260331-final`, then rerun the prior golden-lane checks and restore the previous PR body. ## Backfill Plan After the lane is green, backfill the same workflow scoping and governance-contract repairs into any surviving PRs that still touch `.github/workflows/**` or governance surfaces, then close superseded PRs against `#22309`. ## Validation Evidence Local validation completed: - `node scripts/ci/drift-sentinel.mjs` - `ruby -e 'require "yaml"; ... YAML.load_file(...)'` over all edited workflow files - `jq . governance/pilot-ci-policy.json` - `jq . governance/branch-protection.json` - merge-marker scan over all edited files returned clean ## Notes - Live GitHub PR checks on the open PR set are being converged through this single branch instead of salvaging each broken lane independently. - I did not run the full local verification matrix in this session; this PR is intended to give the repo one clean convergence lane for CI and human review. - After this PR lands, the absorbed PRs should be closed as superseded. --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: Bot <bot@summit.ai> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Gemini CLI <gemini-cli@google.com>
|
Superseded by #22309, which is now merged into main. |
Pull request was closed
1 participant
Motivation
Description
docs/security/BLACKHAT_2025_MISUSE_RESPONSE_PLAYBOOK.mdwhich defines abuse patterns, required controls (pre-generation classifier, generation guardrails, post-generation validator, learning-loop isolation, evidence requirements), detection signals, and an incident response workflow.docs/security/PROMPT_INJECTION_QUICK_REFERENCE.mdto normalize formatting and code examples, improve tables for readability, and add a new High-Risk Misuse Routing (Critical) section that instructs engineers to apply a critical misuse classifier and route matches to refusal + telemetry + quarantine.Testing
pnpm exec prettier --check docs/security/BLACKHAT_2025_MISUSE_RESPONSE_PLAYBOOK.md docs/security/PROMPT_INJECTION_QUICK_REFERENCE.mdwhich initially reported style issues and then re-ranpnpm exec prettier --writeto fix formatting.pnpm exec prettier --checkand verified all modified files match Prettier style.Codex Task