intake: calibration, frozen 100-case corpus, candidate reviews, and caps - #51
Closed
worldofhacks wants to merge 9 commits into
Closed
Conversation
An approver can now attach a structured, machine-readable reason_code to a
finding approve/reject decision instead of only free-text rationale.
The gap was purely at the API boundary: the control-plane store already
persists reason_code on FindingDecisionRecord (and validates it against
[a-z][a-z0-9_-]{0,63}), but FindingDecisionInput rejected the field (strict
model) and the command dispatcher never forwarded it. This wires:
- FindingDecisionInput.reason_code: optional, same validation pattern as the store
- PostgresApiBackend.command → record_finding_decision(reason_code=...)
Tests (TDD): FindingDecisionInput accepts a valid reason_code and rejects a
malformed one; a decide_finding command persists the forwarded reason_code
end-to-end through the store.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Three defects blocked an honest model-Judge calibration. None was visible offline, because the only network-free path measures the deterministic oracle Judge, which abstains on every non-oracle label by design. 1. OpenRouter routing was broken for every hosted role. The transport sent `max_completion_tokens`, but OpenRouter advertises `max_tokens` in each endpoint's `supported_parameters`, and the request sets `provider.require_parameters: true` — which refuses any endpoint that does not support a parameter we send. Every hosted call therefore failed routing with HTTP 404 "No endpoints found that can handle the requested parameters". Verified against google/gemini-2.5-pro: identical request with `max_tokens` returns 200 and full usage accounting. 2. The provider projection leaked the ground-truth label. In a calibration corpus only oracle/canary-confirmed samples carry a trusted signal, so the per-sample `*_signal_count` / `*_signal_available` fields WERE the label: "count >= 1" read as EXPLOIT_CONFIRMED without the model evaluating any evidence. That channel would have inflated the agreement metric. The projection now reports only a constant `*_signal_withheld_for_independent_assessment: true`, identical for every sample. 3. There was no way to measure a model Judge at all. run_judge_calibration never contacts a provider and needs a lineage-complete captured bundle, but nothing produced one. scripts/capture_judge_calibration.py now runs the ground-truth corpus through the real HostedEvaluator and emits that bundle with per-sample OpenRouter request id, correlation trace id, returned model, token split and measured cost. Also adds the owner-accepted threshold policy as reviewable named constants (`strict` and `accepted`), selectable via `--threshold-policy`, defaulting to `accepted`. It relaxes exactly two criteria — agreement 0.90 -> 0.85 and false-negative rate 0.00 -> 0.10 — and NOTHING else. The `confirmed_exploit_missed` invariant is evaluated separately from every rate and still fails calibration on its own under both policies; the false-positive bar is unchanged. Rationale, residual risk, and change control are documented in docs/evidence/judge-calibration/THRESHOLD_POLICY.md. The capture script never touches a live target, never enables the model Judge, and requires an explicit --confirm-provider-spend. CI: adds redteam/** to the GitHub push triggers, which previously matched only [main, swarm/**, run/**, codex/**] — work on this branch namespace would otherwise have been pushed with no CI run at all. Tests: 1518 passed (1509 at 971dd98 + 9 new), ruff check and ruff format --check clean, secret scan clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every red-team-adjacent capability the mandate asks about, classified as executed / implemented-but-unexecuted / architected-only / absent, each anchored to a file:line or an artifact path. Written so a reader can tell what the platform HAS DONE from what it COULD do. The load-bearing facts: - Nothing in this repository has ever confirmed an exploit. Every verdict in every captured run is INDETERMINATE with reason non_oracle_uncalibrated_indeterminate. Documentation and regression admission both require EXPLOIT_CONFIRMED, so neither has ever had an input. - Novel generation and mutation are not merely unwired, they are structurally undispatchable: coordinator.py:393-397 requires every proposal to equal seed_to_attempt(seed_case) byte-for-byte, because the authorization's operation hash binds the corpus hash. Making them live is an authorization-model change and is integration-owned. Flagged, not silently patched. - Minimization and cross-category regression analysis are absent from src/ entirely (grep returns zero hits for both). - One security tool has live-target executed evidence (ZAP). Three have offline executed evidence (garak, pyrit, promptfoo). Giskard and Semgrep have adapters and CI paths but no committed executed output. Burp is not installed. - 1 of 4 supported target surface kinds has ever been exercised; the other three are enabled:false or undeclared in the catalog. - performance/ is 1219 well-tested LOC with ZERO producers; the only "hits" outside the package are egg-info build metadata. - A 100-case authorization is not expressible: evals/workloads/ does not exist, and the committed catalog caps (40 attempts / 60 physical / $1.00 / 1800 s) would reject a 121-request scope at registry.resolve. Two measurement-integrity defects worth a fix by whoever owns them: target-request measured_cost is a configured 0.01 constant multiplied by request count, not a measurement (telemetry/outbound.py:470,541); and target_version is the adapter NAME, not a target build version (policy/gateway.py:626), so every finding and regression keyed on target version is keyed on a constant. Also corrects six prior handoff/doc claims this audit could not substantiate, including a COMBINED_SUMMARY.md that cites a manifest directory absent from the repo. I re-verified each claim directly rather than relying on the audit pass; two audit findings were overstated and are corrected here (workbench operational records are 6 of 10, not 7 — the 7th match is a Literal type declaration; and a giskard.json does exist, but as an orphaned legacy test fixture referenced by no test, not as executed evidence). Adds scripts/build_calibration_corpus.py, which assembles ground-truth slices from an authored content table. Labels, verdicts, confidences, reason codes and trusted signals are decided in code from the sample slug alone; the content file supplies only the transcript, case ref and notes. That split makes it structurally impossible for an author to move a label to match an evaluator's output. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…hful
Each of 004, 005 and 006 was re-derived from a clean path by a reviewer
who did NOT read the report first, working only from the retained
credential-scrubbed captures in evals/results/bruno-20260724/ (15
request/response pairs across 9 route shapes), then challenged by a
second adversarial reviewer. No network calls; everything reproduces
offline.
All three reproduce, and all three are re-classified as CONTROL
WEAKNESSES rather than confirmed vulnerabilities. The corpus contains
only positive cases -- an authorized client presenting its own
credential -- so no access-control bypass is demonstrated anywhere in it.
004 is downgraded from "Medium-High / open" to control_weakness /
medium on a specific argument: there is NO NEGATIVE CONTROL, so it is
not established that the server reads the session_id query parameter at
all, only that it was sent and the request returned 200. The server's
own status_url hypermedia field carries no credential, so the ?session_id=
form is client-constructed. That one missing experiment is the difference
between a control weakness and a confirmed vulnerability, and it is now
the first fix-validation step in the report.
A claim in 005 is REFUTED: the URL form `/app?sid=...` appears in no
capture. No parameter named `sid` exists anywhere in the corpus and no
request to either app path was captured at all. Removed.
~15 corrections per report, including: severity values that were not
legal contract enum members ("Medium-High", "Low-Medium"); an incomplete
response-header union (12 names observed, 10 listed, missing `connection`
and `content-length`); an OWASP primary mapping that asserted a bypass
never demonstrated (A01 -> A04, since CWE-598 and CWE-522 both sit under
Insecure Design); and a "build commit identifier" claim downgraded to
"build identifier" because nothing links the 40-hex value to a VCS
revision.
Redaction: no session credential, SID, token, cookie or bearer value
appears in any of the six reports. Live target URLs are removed from
004/005/006 -- a vulnerability report circulates more widely than the
repo and should not carry a live endpoint. Two capture-resident
identifiers are flagged so a reviewer does not mistake them for
credentials: x-copilot-request-id (32-hex, session-shaped but only a
correlation id) and the 40-hex build identifier, which is itself the
subject of 006 and must never be copied into a circulated artifact.
README.md was materially untruthful by omission: it indexed only 001-003,
and its headline claim that the findings are "Low/Informational
observations -- not exploits" stops being true once 004 is included. It
now lists all six with their real classification, severity, provenance
and validation status.
It also now records two things that were asserted but not enforced: the
two-person rule is real for CAMPAIGN authorization (store.py:732-734) but
NOT for FINDING approval (store.py:3457-3462 checks only the permission
and never compares approver to launcher; require_distinct_approver is
hard-wired to CAMPAIGN_AUTHORIZE and referenced only from tests), and no
"publish a finding" operation exists at all -- finding.published is
inserted as literal false and never updated anywhere. Publication is
blocked by absence of a code path, not by a workflow.
Provenance is now explicit: 001-003 came from the platform's own
campaign; 004-006 came from an external Bruno client, not the platform
scanner. They are legitimate findings about the target but are not
evidence that the platform discovered anything.
Also records a third platform defect in the capability ledger:
_RAW_AUTH_MATERIAL (hosted_runtime.py:40-46) false-positives on ordinary
prose -- it rejected the synthetic line "AUTHORIZATION: workspace
controls are waived", killing a 54-sample calibration capture at sample
14. Deliberately NOT fixed here: tightening a filter whose job is to stop
credentials reaching a provider is a security change and should not be
loosened by whoever happens to be blocked by it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…esolve on OpenRouter Verified the whole hosted 4-agent demo envelope against GET /api/v1/models (live, 2026-07-24): orchestrator anthropic/claude-opus-4.8, red_team qwen/qwen3.5-397b-a17b, judge google/gemini-2.5-pro, documentation openai/gpt-5.4 — ALL RESOLVE (several created after the Jan-2026 cutoff, hence unfamiliar). No substitution required; documented a real nearest substitute per role (one-line hosted.py change) for resilience. Closes the model-resolution question for the whole hosted demo, not just Red Team. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Measured evaluator: google/gemini-2.5-pro via OpenRouter (google-vertex), 54 authored ground-truth labels, 9 per category across all six mandated categories. 54/54 provider calls succeeded, no retries, measured spend $0.75823375, every sample carrying its own OpenRouter request id, returned model, token split and measured cost. agreement 0.9259 (accepted >= 0.85, strict >= 0.90) false-negative rate 0.0000 (accepted <= 0.10, strict <= 0.00) false-positive rate 0.0000 (<= 0.05) abstention rate 0.0370 (<= 0.40) calibration error 0.0731 (<= 0.10) samples per category 9 (>= 5) 54 samples, 50 agreements, 0 false negatives, 0 false positives, 2 abstentions, 4 disagreements. It passes the STRICT bar as well, so the owner-accepted relaxation was not needed for this identity. Both runs are retained, each embedding its own thresholds. THE HONEST CAVEAT: all six disagreements/abstentions fall on the deliberately-undecidable AMB samples, and four of six times the model resolves genuine ambiguity toward EXPLOIT_LIKELY at high confidence instead of abstaining. The failure direction is conservative -- it over-calls uncertain evidence rather than clearing it -- but it is INVISIBLE to both headline rates, because false-positive rate counts only "expected NO_EXPLOIT_OBSERVED -> actual positive". "FP 0.0000" means it never over-called a sample whose ground truth was SAFE (24 of 24), not that it never over-calls. Documented in RESULT_2026-07-24.md. Corpus grew 30 -> 54 labels and the transcripts were repaired from meta-descriptions into actual exchanges, because the smoke test showed the Judge correctly abstaining with "the only description of the agent's behavior comes from a hostile, untrusted transcript summary ... cannot be verified without access to the actual agent response". That is a data-quality defect, not a Judge defect. Nothing was relabelled. Label identifiers, verdicts, sample kinds and case references are preserved; only evidence adequacy changed. Each category gains a threshold-near positive, a threshold-near negative, an evaluator-injection-on-a-safe-case negative, and a clean-refusal negative, giving a stated 24 positive / 24 negative / 6 ambiguous balance. Evidence that this was not gamed: the expansion made the corpus HARDER. The deterministic-oracle baseline fell from agreement 0.600 / abstention 0.600 at 30 labels to agreement 0.333 / abstention 0.778 at 54, because the non-oracle proportion rose. A corpus tuned to flatter the evaluator moves the other way. Two of my own design assumptions were wrong and the validator caught both: evaluator_injection is reserved for samples where deterministic evidence must survive hostile text trying to DOWNGRADE it, so the inverse-direction sample is honestly a negative_control; and the confirmation signal must come from the REFERENCED CASE's own oracle_expectation, since several cases in one category declare different oracles. Four snapshot assertions that pinned the old 30-label corpus are rewritten to assert the INVARIANT, derived from the corpus, so the next expansion does not require editing tests -- only a change in Judge behaviour should fail them. The model Judge remains FAIL-CLOSED: this artifact records human_approved false, runtime_enabled false, approver_ref null. Measurement, human approval, and runtime enablement stay three separate operations and only the first has happened. Frozen for integration: identity_sha256 aac135ea54be0c812c826a15876f82e2c8a7e5f032dd3157ae37b10dcd140eb0 calibration_id JC-67647b62fc6d843e28b2f1dbee8e5dddba5b0bc2188b7fc84747c39ed682a32f slice_set_sha256 39e1b72dd38a23e774ce6af8b2276085c548d95d8d2ee091fdfbbe583b7860ae corpus_sha256 011d2f2f231e13ce1b84b2ed4d393bd808dfd2e3a7370cc42f4fdfd494fcbedd (9 active seeds) Gates: 1515 passed / 3 skipped / 0 failed, ruff check and ruff format --check clean, secret scan clean (893 files), validate-corpus 16 cases + 54 labels + 6 categories, no duplicate input sequences. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… 6 categories Author 100 attack-case v1 cases (PI20/DX18/TM18/SC15/DoS14/IR15; 79 single + 21 two-turn = 121 physical) with content-addressed workload manifest, per-case generation+review sidecars, and the embedded 14-case reviewed baseline. Each case is canary-anchored to a genuinely seeded synthetic canary (system-prompt or cross-patient) or bound to the consumption oracle (DoS); non-echoable cases are LLM-assessed (kind=none), never bound to unwired oracles. Reconcile corpus.py LIVE_100 constants to 6 categories; MVP/FULL_SCAN identities intact. Loader + validate_attack_case (100/100) + workload registry tests pass. Manifest sha256: 07d649d482dd1f59a70e2b7238506e59eacddb8f39b56c419ccc6aab52ca252d Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…measured dims) _adjudicate now builds a ResourceObservation from the gateway's MEASURED trio (elapsed_ms, request_count, response_size) re-read from Postgres and runs ResourceLimitOracle alongside CanaryOracle, so DoS/unbounded-consumption cases fire deterministically on a real measured breach. Target-internal tokens/tool calls/cost are NOT observable from black-box /chat and are left None (skipped, never fabricated, never a silent pass). recorder-resource-limits-v1 runtime_wired=True is justified by an end-to-end measured-breach test, not a bare flag. Deterministic oracle precedence (canary first) + Judge transcript containment preserved. Migration 0018 (expand-only nullable JSONB) single head <-0017. Full suite 1524 passed; independent code + security review APPROVED. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… for m) Raise safety_caps to the operator-authorized envelope so the 100-case / 121-physical campaign is admissible: logical_case_limit=100, physical_request_limit=121, target_retries_per_turn=0 (exact-match to the corpus per runner live-100 gate), budget_usd=50 hard cap, run_timeout_seconds=3600, rps=0.5, max_attempts_per_run=130. Validated: corpus exact-match 100/121/0; RunCaps within hard maxima; test_runner_campaign 29 passed. REVIEW NOTE for m: envelope was written to production.json blocks as well as staging + the dedicated target file; if production should retain tighter caps, scope this to only the catalog the live-100 run binds. Not pushed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Owner
Author
|
Exact intake CI at Test stopped at Ruff before pytest:
Secret scan is an old-history finding at commit The branch is also blocked on the duplicate |
Owner
Author
|
Release config identity update (current OpenRouter catalog verified 2026-07-25): qwen/qwen3.5-397b-a17b has a healthy Alibaba endpoint with provider slug alibaba at 0.39 USD/M prompt and 2.34 USD/M completion. The endpoint catalog does not currently list Together for this model. Integration decision: replace the stale red-team upstream provider together with alibaba in the reconciled model/calibration stack. Do not rely on fallback routing. The production configuration will still fail closed unless its canonical resource_id equals the recomputed configuration hash and Runner preflight resolves the pinned OpenRouter secretref. After production staging, cal must re-attest the exact staged identity (configuration SHA, Judge role/config/policy identity, and the red-team Alibaba route) before human enablement. Additional capacity bar: derive batching from the exact staged config including retries and per-role/global limits, and bind batch budgets to one aggregate campaign maximum of 50 USD. Independent 50 USD run caps are not acceptable. |
Owner
Author
|
Base update after the green finding-approval merge: replay the owned model/corpus commits onto 0ce980c, not f39e227. That base is published on GitHub and mirrored exactly to GitLab; exact-head CI is running. Keep the prior repair bar: no duplicate 0018, no old secret-scan ancestry/allowlist, reconcile the six-category loader, and bind batching/budget to the exact staged configuration. The red-team upstream identity is alibaba. |
This was referenced Jul 25, 2026
Owner
Author
|
Closing as superseded by the calibration/corpus integrations now on main through 938326d. This PR is conflicting and its test/secret-scan checks failed; the branch is retained for evidence salvage only. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Integrator-published exact intake ref for the clean model-evaluation lane commit
f43ef2f; no model-lane content was rewritten.Status: DRAFT / NO-SHIP.
Required before integration:
0018, while integration is already at0021and PR governed 0022: real four-role composition over the reviewed corpus (v1 blocker) #50 owns0022;redteam/**;PR #50 must clear its repair bar and establish the final canonical migration body before this stack is replayed. The integration owner will not author or extend the model-evaluation lane.