feat(red-team): govern generated attacks behind curate, review, and a new grant - #55
Open
worldofhacks wants to merge 1 commit into
Open
feat(red-team): govern generated attacks behind curate, review, and a new grant#55worldofhacks wants to merge 1 commit into
worldofhacks wants to merge 1 commit into
Conversation
… new grant Hosted generation was implemented but structurally undispatchable: the coordinator requires every proposal to equal the reviewed corpus byte-for-byte, so a generated attempt falls outside the grant that authorized the run. That check is correct. The missing piece was a governed way to mint a NEW corpus identity and take a NEW grant. Tier 1, stages 1-3 of the two-stage loop. The generator itself is unmodified and still never touches a target. * curate (agents/red_team/curation.py) - total, deterministic, network-free: normalize -> materialize + validate against attack-case v1 -> structurally minimize -> dedupe within batch and against the base corpus -> novelty-score -> content-address. Every candidate becomes a curated case or a typed rejection, so nothing is silently dropped and the reviewer sees what was refused. The untrusted generator authors input_sequence and nothing else. oracle_expectation is forced to kind "none", and severity/exploitability/ground_truth_refs are neutralized rather than inherited: a generated case carrying the template's synthetic_canary_match oracle would assert a deterministic canary hit its own turns may never trip, and would still pass schema validation. ground_truth_refs is deliberately non-resolving so a generated case cannot enter the authored corpus until a human authors its label. Minimization is structural only; it is skipped for denial_of_service, where collapsing repeats would destroy the attack while reporting success. Semantic minimization needs target feedback and belongs to Tier 2. * review gate (agents/red_team/review_gate.py) - the human authorization decision, bound to exact bytes. Refuses a bundle mutated after presentation, self-approval by the generating principal, approval by omission, and a decision transplanted onto different content. Emits records in the provenance shape load_live_100_corpus already validates (source_kind hosted_red_team) rather than a second notion of "approved". * fresh authorization (campaign/generated_profile.py) - mirrors tool_profile.py. Re-derives each approved case hash rather than trusting the field, produces a corpus identity provably distinct from the base, and refuses a scope carrying the base corpus hash, a foreign corpus id, or a spent run nonce. Stage 4 (dispatch) stacks on the 0022 governed four-role path and is NOT wired; prepare_generated_dispatch runs every gate and stops there rather than no-op'ing, because a silent no-op would look like a successful dispatch. Tier 2 (multi-round feedback) is next, not done. No live run, verdict, or confirmed exploit is claimed. No migration: stages 1-3 are pure and content-addressed, following the existing tool_profile.py precedent, so there is no revision number to reconcile. 48 new tests; full suite 1901 passed / 3 skipped; ruff 0.16.0 check + format clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
worldofhacks
changed the base branch from
codex/integration-docs-after-pr48
to
main
July 26, 2026 06:02
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tier 1 of generative red-teaming as a governed two-stage loop — stages 1–3.
Base, please read first
Branched from
b0e641c(codex/integration-b34-reconcile-802), per the task's "reusing 0022'sgoverned path (PR #50, landing on b0e641c)". That branch is not published, so this PR shows
7 commits — 6 are your unpublished integration base and only
0678a7cis mine. Retargetonce
b0e641cis pushed, or review0678a7calone. Nothing outside my 7 new files is touched.What this closes
The 2026-07-25 coverage review recorded that hosted generation was implemented but structurally
undispatchable: the coordinator requires every proposal to equal the reviewed corpus
byte-for-byte, so a generated attempt falls outside the grant that authorized the run.
That check is correct and is not loosened here. An authorization is content-addressed over the
exact corpus hash, so a generated case is by construction outside every prior grant. The missing
piece was a governed way to mint a new corpus identity and take a new grant.
The invariant
agents/red_team/curation.pyagents/red_team/review_gate.pycampaign/generated_profile.pyThe generator (
generate_traced, qwen,target_scope:none) is unmodified and still nevertouches a target.
Two things worth your attention
The generator authors
input_sequenceand nothing else.oracle_expectationis forced tokind: "none", andseverity/exploitability/ground_truth_refsare neutralized ratherthan inherited from the governed template. This is not cosmetic: a generated case inheriting the
template's
synthetic_canary_matchoracle would assert a deterministic canary hit its own turns maynever trip, and would still pass schema validation.
ground_truth_refsis deliberatelynon-resolving, so writing a generated case into the authored corpus fails the corpus validator's
bidirectional case↔label check until a human authors its label.
Minimization is structural only, and skipped for
denial_of_service— collapsing repeats in atoken-exhaustion attack would destroy it while reporting success. Semantic (delta-debugging)
minimization needs target feedback, so it belongs to Tier 2.
Honest limits
prepare_generated_dispatch()runs every gate and stops, rather thanno-op'ing — a silent no-op would look exactly like a successful dispatch.
Coordination
tool_profile.py/corpus.pyprecedent — no revision number to reconcile with A.agents/red_team/__init__.py—campaignalready depends onagents.red_team; re-exporting would invert the layering.load_live_100_corpusalready validates(
source_kind: hosted_red_team), rather than a second notion of "approved".Gates
Pushed identically to GitHub and GitLab at
0678a7c.🤖 Generated with Claude Code