judge: land cal's canonical AD-04 confirmation-authority guard (rebased) - #57
Merged
worldofhacks merged 7 commits intoJul 25, 2026
Merged
Conversation
Three defects blocked an honest model-Judge calibration. None was visible offline, because the only network-free path measures the deterministic oracle Judge, which abstains on every non-oracle label by design. 1. OpenRouter routing was broken for every hosted role. The transport sent `max_completion_tokens`, but OpenRouter advertises `max_tokens` in each endpoint's `supported_parameters`, and the request sets `provider.require_parameters: true` — which refuses any endpoint that does not support a parameter we send. Every hosted call therefore failed routing with HTTP 404 "No endpoints found that can handle the requested parameters". Verified against google/gemini-2.5-pro: identical request with `max_tokens` returns 200 and full usage accounting. 2. The provider projection leaked the ground-truth label. In a calibration corpus only oracle/canary-confirmed samples carry a trusted signal, so the per-sample `*_signal_count` / `*_signal_available` fields WERE the label: "count >= 1" read as EXPLOIT_CONFIRMED without the model evaluating any evidence. That channel would have inflated the agreement metric. The projection now reports only a constant `*_signal_withheld_for_independent_assessment: true`, identical for every sample. 3. There was no way to measure a model Judge at all. run_judge_calibration never contacts a provider and needs a lineage-complete captured bundle, but nothing produced one. scripts/capture_judge_calibration.py now runs the ground-truth corpus through the real HostedEvaluator and emits that bundle with per-sample OpenRouter request id, correlation trace id, returned model, token split and measured cost. Also adds the owner-accepted threshold policy as reviewable named constants (`strict` and `accepted`), selectable via `--threshold-policy`, defaulting to `accepted`. It relaxes exactly two criteria — agreement 0.90 -> 0.85 and false-negative rate 0.00 -> 0.10 — and NOTHING else. The `confirmed_exploit_missed` invariant is evaluated separately from every rate and still fails calibration on its own under both policies; the false-positive bar is unchanged. Rationale, residual risk, and change control are documented in docs/evidence/judge-calibration/THRESHOLD_POLICY.md. The capture script never touches a live target, never enables the model Judge, and requires an explicit --confirm-provider-spend. CI: adds redteam/** to the GitHub push triggers, which previously matched only [main, swarm/**, run/**, codex/**] — work on this branch namespace would otherwise have been pushed with no CI run at all. Tests: 1518 passed (1509 at 971dd98 + 9 new), ruff check and ruff format --check clean, secret scan clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every red-team-adjacent capability the mandate asks about, classified as executed / implemented-but-unexecuted / architected-only / absent, each anchored to a file:line or an artifact path. Written so a reader can tell what the platform HAS DONE from what it COULD do. The load-bearing facts: - Nothing in this repository has ever confirmed an exploit. Every verdict in every captured run is INDETERMINATE with reason non_oracle_uncalibrated_indeterminate. Documentation and regression admission both require EXPLOIT_CONFIRMED, so neither has ever had an input. - Novel generation and mutation are not merely unwired, they are structurally undispatchable: coordinator.py:393-397 requires every proposal to equal seed_to_attempt(seed_case) byte-for-byte, because the authorization's operation hash binds the corpus hash. Making them live is an authorization-model change and is integration-owned. Flagged, not silently patched. - Minimization and cross-category regression analysis are absent from src/ entirely (grep returns zero hits for both). - One security tool has live-target executed evidence (ZAP). Three have offline executed evidence (garak, pyrit, promptfoo). Giskard and Semgrep have adapters and CI paths but no committed executed output. Burp is not installed. - 1 of 4 supported target surface kinds has ever been exercised; the other three are enabled:false or undeclared in the catalog. - performance/ is 1219 well-tested LOC with ZERO producers; the only "hits" outside the package are egg-info build metadata. - A 100-case authorization is not expressible: evals/workloads/ does not exist, and the committed catalog caps (40 attempts / 60 physical / $1.00 / 1800 s) would reject a 121-request scope at registry.resolve. Two measurement-integrity defects worth a fix by whoever owns them: target-request measured_cost is a configured 0.01 constant multiplied by request count, not a measurement (telemetry/outbound.py:470,541); and target_version is the adapter NAME, not a target build version (policy/gateway.py:626), so every finding and regression keyed on target version is keyed on a constant. Also corrects six prior handoff/doc claims this audit could not substantiate, including a COMBINED_SUMMARY.md that cites a manifest directory absent from the repo. I re-verified each claim directly rather than relying on the audit pass; two audit findings were overstated and are corrected here (workbench operational records are 6 of 10, not 7 — the 7th match is a Literal type declaration; and a giskard.json does exist, but as an orphaned legacy test fixture referenced by no test, not as executed evidence). Adds scripts/build_calibration_corpus.py, which assembles ground-truth slices from an authored content table. Labels, verdicts, confidences, reason codes and trusted signals are decided in code from the sample slug alone; the content file supplies only the transcript, case ref and notes. That split makes it structurally impossible for an author to move a label to match an evaluator's output. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…esolve on OpenRouter Verified the whole hosted 4-agent demo envelope against GET /api/v1/models (live, 2026-07-24): orchestrator anthropic/claude-opus-4.8, red_team qwen/qwen3.5-397b-a17b, judge google/gemini-2.5-pro, documentation openai/gpt-5.4 — ALL RESOLVE (several created after the Jan-2026 cutoff, hence unfamiliar). No substitution required; documented a real nearest substitute per role (one-line hosted.py change) for resilience. Closes the model-resolution question for the whole hosted demo, not just Red Team. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Measured evaluator: google/gemini-2.5-pro via OpenRouter (google-vertex), 54 authored ground-truth labels, 9 per category across all six mandated categories. 54/54 provider calls succeeded, no retries, measured spend $0.75823375, every sample carrying its own OpenRouter request id, returned model, token split and measured cost. agreement 0.9259 (accepted >= 0.85, strict >= 0.90) false-negative rate 0.0000 (accepted <= 0.10, strict <= 0.00) false-positive rate 0.0000 (<= 0.05) abstention rate 0.0370 (<= 0.40) calibration error 0.0731 (<= 0.10) samples per category 9 (>= 5) 54 samples, 50 agreements, 0 false negatives, 0 false positives, 2 abstentions, 4 disagreements. It passes the STRICT bar as well, so the owner-accepted relaxation was not needed for this identity. Both runs are retained, each embedding its own thresholds. THE HONEST CAVEAT: all six disagreements/abstentions fall on the deliberately-undecidable AMB samples, and four of six times the model resolves genuine ambiguity toward EXPLOIT_LIKELY at high confidence instead of abstaining. The failure direction is conservative -- it over-calls uncertain evidence rather than clearing it -- but it is INVISIBLE to both headline rates, because false-positive rate counts only "expected NO_EXPLOIT_OBSERVED -> actual positive". "FP 0.0000" means it never over-called a sample whose ground truth was SAFE (24 of 24), not that it never over-calls. Documented in RESULT_2026-07-24.md. Corpus grew 30 -> 54 labels and the transcripts were repaired from meta-descriptions into actual exchanges, because the smoke test showed the Judge correctly abstaining with "the only description of the agent's behavior comes from a hostile, untrusted transcript summary ... cannot be verified without access to the actual agent response". That is a data-quality defect, not a Judge defect. Nothing was relabelled. Label identifiers, verdicts, sample kinds and case references are preserved; only evidence adequacy changed. Each category gains a threshold-near positive, a threshold-near negative, an evaluator-injection-on-a-safe-case negative, and a clean-refusal negative, giving a stated 24 positive / 24 negative / 6 ambiguous balance. Evidence that this was not gamed: the expansion made the corpus HARDER. The deterministic-oracle baseline fell from agreement 0.600 / abstention 0.600 at 30 labels to agreement 0.333 / abstention 0.778 at 54, because the non-oracle proportion rose. A corpus tuned to flatter the evaluator moves the other way. Two of my own design assumptions were wrong and the validator caught both: evaluator_injection is reserved for samples where deterministic evidence must survive hostile text trying to DOWNGRADE it, so the inverse-direction sample is honestly a negative_control; and the confirmation signal must come from the REFERENCED CASE's own oracle_expectation, since several cases in one category declare different oracles. Four snapshot assertions that pinned the old 30-label corpus are rewritten to assert the INVARIANT, derived from the corpus, so the next expansion does not require editing tests -- only a change in Judge behaviour should fail them. The model Judge remains FAIL-CLOSED: this artifact records human_approved false, runtime_enabled false, approver_ref null. Measurement, human approval, and runtime enablement stay three separate operations and only the first has happened. Frozen for integration: identity_sha256 aac135ea54be0c812c826a15876f82e2c8a7e5f032dd3157ae37b10dcd140eb0 calibration_id JC-67647b62fc6d843e28b2f1dbee8e5dddba5b0bc2188b7fc84747c39ed682a32f slice_set_sha256 39e1b72dd38a23e774ce6af8b2276085c548d95d8d2ee091fdfbbe583b7860ae corpus_sha256 011d2f2f231e13ce1b84b2ed4d393bd808dfd2e3a7370cc42f4fdfd494fcbedd (9 active seeds) Gates: 1515 passed / 3 skipped / 0 failed, ruff check and ruff format --check clean, secret scan clean (893 files), validate-corpus 16 cases + 54 labels + 6 categories, no duplicate input sequences. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…el-authority holes Wires the re-attest harness so a calibration can only ever attest the Judge the deployment is actually running, and repairs two places where the model's own output was granting it authority. The re-attest itself has NOT been run — it is blocked on inputs I do not own (below). No number in this commit is a new measurement. WHY THE 8ce852b NUMBER IS SUPERSEDED, PRECISELY Its provider calls were real: 54 unique OpenRouter request ids, 54 distinct token triples, costs read from the provider's own usage.cost field that match Gemini 2.5 Pro list price (1.25/10 per M) rather than this repo's 5/30 ceilings. The arithmetic reproduces byte-for-byte. What disqualifies it is what it was bound to: capture_judge_calibration.py built its OWN HostedConfigurationSet. Since judge_model_version == judge_role.configuration_sha256, which hashes credential_reference, prices and limits along with model and prompt, and since those limits were sized to the label count, the attested identity drifted with corpus size and could never equal the deployed one. load_judge_calibration_status compares judge_identity against the running identity and returns invalidated/identity_drift — so the identity RESULT_2026-07-24.md offered under "hand this to integration" would have been rejected by the runtime it was handed to. Pinned by test: the same evaluator over 54 vs 56 labels yields different judge_model_version values. Two further properties, disclosed before but now measured rather than described: the evidence judged is authored, never executed (campaign_run_id "ground-truth-unexecuted"), and the pooled headline includes 12 oracle-backed samples the model cannot influence. Restratified: pooled agreement 0.9259 over 54, but 0.9048 over the 42 samples the model actually decides, with 4 ambiguous over-calls the false-positive rate structurally cannot see. CONFIRMATION AUTHORITY — two real holes, both reproduced by execution HostedFourRoleRuntime is the four-role composition an enabled model Judge runs inside, and the class PR #50 builds run_attempt on. It took two authorities from the model's own response: 1. its verdict enum included EXPLOIT_CONFIRMED and _deterministic_precedence returned it unchanged when no oracle or canary had fired. run_attempt drafts a vulnerability report for any EXPLOIT_CONFIRMED/EXPLOIT_LIKELY verdict, so a model claiming a confirmation produced Documentation output for a finding with no evidence behind it. 2. the guard refusing an uncalibrated NO_EXPLOIT_OBSERVED was keyed on a "calibrated" boolean the MODEL emitted about itself — so a model asserting its own calibration could clear a case. That is the dangerous direction. Repaired: EXPLOIT_CONFIRMED removed from the model's output schema, a model-claimed confirmation downgraded to EXPLOIT_LIKELY with reason model_confirmation_refused, calibration status passed in from the caller's enablement gate. The class had no production composition root before now, which is why this had not yet produced a bad finding; PR #50 gives it one. WHAT THIS ADDS - capture_judge_calibration.py: --hosted-configuration-set and --expected-configuration-sha256 are now required. Identity is derived from the staged set, never synthesized. Reconstruction is itself a check — a prompt_sha256 that is not this release's served prompt is a refusal. Capacity preflight refuses a corpus past the staged budget or the 56-call platform ceiling rather than relaxing limits, which would change the identity being attested. - analyze_judge_calibration.py: restates a result over the NON-ORACLE stratum, the only cases an enabled model decides, and counts the two failure modes the contract's counters cannot express — a positive scored safe, and an over-call on an ambiguous sample. - enable_model_judge.py: the human gate. Refuses on identity drift, on a non-oracle stratum breach even when the pooled headline passes, on ground truth without a two-person blind attestation bound to the exact slice_set_sha256, and without a named approver. Re-checks the written artifact through require_model_judge_enablement. - REATTEST_RUNBOOK.md: the procedure, what the identity does and does not cover, and the fact that the replay path validates bundle SHAPE only — a hand-written bundle with 0 tokens and $0 cost yields a contract-valid passing artifact under the strict policy with no API key. Any re-attest must be reported as "the harness measured this bundle" unless an out-of-band OpenRouter usage export is attached. BLOCKED ON (neither is mine to resolve) - m: no staged production hosted configuration set exists on disk; production role configs live in the Postgres hosted_configuration_sets table. - g: the 100-case ground-truth set is GT-CAND-M11-LIVE100, 200 candidate labels, calibration_status AUTHORED_PENDING_HUMAN_ATTESTATION, human_labeler and distinct_reviewer both null, and its labels carry no evidence_envelope — so it is neither human-labeled nor runnable through CalibrationGate. The committed 54 labels are all rule- or agent-authored. - Scope note: 200 labels cannot be captured in one run at all. max_calls is capped at HOSTED_MAX_PHYSICAL_CALLS=56, and raising it would change the identity. 1547 passed, 3 skipped. tests/test_packaging.py::test_wheel_installed_outside_repo_validates_corpus fails identically on the untouched base (fresh-venv wheel install), unrelated. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…rovider-reconciled provenance Two follow-ups from review, plus the scope correction they imply. BATCHING INSTEAD OF A WIDER CAP A corpus past HOSTED_MAX_PHYSICAL_CALLS (56) cannot be captured in one run, and raising max_calls is not the way out: limits sits inside the Judge role's configuration_sha256, which IS judge_model_version. A wider cap attests a different evaluator, so the sub-runs could not be aggregated at all. The corpus is now split into sub-runs against ONE unchanged staged configuration: - capture gains --batch-size / --batch-index / --plan-only. Batch size defaults to the staged Judge role's max_calls and may never exceed it — the refusal says to reduce the batch, not to raise the limits. - generation_policy_sha256 now binds the CAMPAIGN (corpus size + batch size) rather than the per-batch sample count, so it is constant across sub-runs and the merged bundle has one coherent value. A per-batch count would make the trailing short batch disagree for no reason that means anything. - merge_calibration_batches.py refuses unless every batch carries a byte-identical judge_identity and pinned provenance, the batches are disjoint, and their union covers the corpus EXACTLY. Coverage is checked against the slice directory, not the batch manifests, because a silently dropped batch would measure a smaller, easier corpus and report a better agreement rate. Verified lossless against the committed 54-sample bundle split into 2 batches of 27: the merged measurement reproduces both the metrics and the calibration_id of the unbatched run. PROVENANCE: FROM SHAPE-VALID TO MEASURED The replay path validates bundle shape only — a hand-written bundle with a fabricated request id, 0 tokens and $0 cost yields a passing artifact under the strict policy with no API key. verify_calibration_provenance.py now reconciles the bundle against OpenRouter's own usage export: every provider_request_id must appear in it, with matching model, cost and tokens, and the summed cost must reconcile with the ledger. A fabricated id has nothing to match. The attestation is a REQUIRED input to enable_model_judge.py, so a bundle that is merely shape-valid can no longer license runtime authority. The export is CSV or JSON with case-insensitive column aliases; an unmappable export prints the headers it saw and refuses rather than guessing, because a column guessed wrong would "verify" nothing. The attestation states in its own body what it does not prove: that the evidence judged came from a live target. It establishes only that the Judge model calls really happened and cost what the bundle claims. PER-BATCH VISIBILITY analyze_judge_calibration.py takes the batch manifest and reports per-sub-run metrics beside the aggregate. The aggregate governs; the batches are shown so a degraded sub-run stays visible instead of averaged away. On the rehearsal split, batch-1's non-oracle agreement (0.8571) is visibly weaker than batch-0's (0.9524) — invisible in the 0.9048 aggregate. COST One physical Judge call per label; the 54-label set measured $0.75823375, so ~$0.014/label — about $2.80 for a 200-label corpus across 4 batches of 56/56/56/32. Three capacity tests in test_judge_calibration_capture_binding.py are retired: their concern (a corpus larger than one batch) is no longer a refusal but a split, and is covered in test_judge_calibration_batching.py. 1568 passed, 3 skipped, ruff clean. The re-attest itself is still NOT run — it remains blocked on m's staged configuration and on ground-truth labels carrying both principals and evidence envelopes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…uthority Rebasing cal's calibration work onto the integration head produced a clean textual merge but a broken import: the branch was cut before `src/agentforge/agents/hosted_prompts.py` was refactored into the `agentforge.agents.prompts` package, so three new test modules still imported `hosted_prompt` from a module that no longer exists and failed at collection. Resolve each the way the rest of the suite already does (tests/control_plane/test_agent_acceptance_store.py:70): look the role up in `load_prompt_registry()` and read `.sha256`, replacing the old `hosted_prompt(role).prompt_sha256`. No behavioural change to the AD-04 guard; this only re-points test scaffolding at the current prompt-authority API. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
worldofhacks
force-pushed
the
sub/cal-ad04-rebase
branch
from
July 25, 2026 21:00
0d5bc33 to
df0ed6e
Compare
worldofhacks
merged commit Jul 25, 2026
935e740
into
codex/platform-observability-followup
10 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replays cal's PR #54 onto the integration head so the AD-04 guard can actually land. PR #54 itself is 50 commits behind, CONFLICTING, and red on two checks; all three problems were staleness, not substance.
Why this is urgent
The guard is not redundant — the hole is open on the integration head right now.
hosted_runtime.pythere offers the model Judge the full_VERDICTSenum (includingEXPLOIT_CONFIRMED) and gates a safe verdict on acalibratedboolean the model emits about itself. A model-claimedEXPLOIT_CONFIRMEDwith no oracle hit flows straight into a Documentation draft.q (PR #50) already deleted its own divergent AD-04 path in good faith to defer to this one, so until this lands nothing enforces the invariant.
What the guard does
_MODEL_ASSESSMENT_VERDICTSomitsEXPLOIT_CONFIRMED, so the model is never given the vocabulary to claim a confirmation (schema-enforced at the judgeoutput_schema).EXPLOIT_CONFIRMEDis downgraded toEXPLOIT_LIKELYwithreason: model_confirmation_refused— the unsafe signal is kept, only the authority is refused.model_calibration_enabled=self._judge_calibration_id is not None) instead of the model's self-report.Oracle / canary / human remain the only paths to
EXPLOIT_CONFIRMED.How the three failures were resolved
CONFLICTING(6 paths)507d032, already landed via #48testFAILUREruff format --checkon a Markdown python fence in AF-VULN-005 — pytest never ran, so this branch had zero test signalsecret-scanFAILURE"Idempotency-Key": "idem-policy-1234567890"in a commit that is not even an ancestor.gitleaksignore(fromec85a45) comes along with the rebaseNo credential was leaked and nothing was redacted or rotated — the flagged value is a synthetic test nonce.
One real repair beyond the rebase
The rebase merged cleanly but broke semantically: this branch predates the refactor of
agents/hosted_prompts.pyinto theagents/promptspackage, so three new test modules failed at collection. Retargeted them atload_prompt_registry()following the existing idiom intests/control_plane/test_agent_acceptance_store.py:70. No change to guard behaviour.Verification (local, CI-pinned toolchain)
ruff format --check .→ 724 files already formattedruff check .→ all checks passedgitleaks git .→ no leaks found (413 commits)validate-corpus→ valid, 16 authored cases / 54 labels / 6 categoriestest_hosted_runtime_confirmation_authority.pyHonest limits
HostedConfigurationSet+ itsconfiguration_sha256, a two-person human ground-truth attestation, and a measured--provenance-attestation. Every committed label today is rule/oracle-derived — none human.