Sovereign Evidence Rail + L3 Memory Governor - #69514
Open
drussell23 wants to merge 1 commit into
Open
Conversation
Evidence-driven graduation of the REVIEW/PLAN shadow subagents to
authoritative status, plus a worktree-RAM-budget governor for L3.
Unit D — L3 Memory Governor (l3_memory_governor.py + subagent_scheduler.py):
reuses MemoryPressureGate; caps concurrent worktrees by live RAM,
strictest-wins with the existing fan-out gate. Off-is-inert.
Unit B — Evaluator (shadow_evaluator.py): pure REVIEW binary agreement +
PLAN refinement (coverage ∧ acyclic ∧ disjoint); malformed→block.
Unit A — Telemetry store (shadow_telemetry_store.py): async SQLite,
to_thread non-blocking writer, two-phase upsert, rolling FIFO cap.
Unit C — Gate + breaker (shadow_graduation_gate.py): event-driven 50-soak
→ flips _AUTHORITATIVE via existing persist_flag_to_env; PlanBreaker
(CRITICAL-pressure pre-empt → legacy); AGENT_DEGRADATION SSE.
Live wiring: producers in orchestrator _run_{plan,review}_shadow,
authoritative REVIEW tier-raise, GLS construct/teardown. OFF-inert:
store disabled → orchestrator _shadow_store None → FSM byte-identical.
Rebased onto main (Slice 255); keep-both resolution of the EVENT_TYPE_*
collision with the loop's orthogonal Slice 252 Shadow-Telemetry (spec §14
documents the two unrelated "shadows"). Endorsed-DAG-drives-execution is
deferred (observer-only today). 45 rail/governor tests + 49 SSE green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
[integrity-verified: daaafc3768bc]
💡 PR Description Suggestions
|
📊 PR Size AnalysisSize: Review Checklist
|
github-actions
Bot
added
documentation
💡 PR Description Suggestions
|
🤖 CI/CD Pipeline ResultsStatus: success Pipeline Stages
|
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| self._pressure_fn = pressure_fn or _default_pressure_fn | ||
|
|
||
| def should_use_legacy(self, *, dag) -> BreakerDecision: | ||
| level = "ok" |
| assert gov is None | ||
| # Composition guard: None -> no truncation. | ||
| if gov is not None and gov.n_allowed < len(selected): | ||
| selected = selected[:gov.n_allowed] |
Comment on lines
+1607
to
+1609
| async def _run_review_shadow( | ||
| self, ctx: Any, best_candidate: Any, | ||
| ) -> "Optional[RiskTier]": |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Evidence-driven graduation of the REVIEW/PLAN shadow subagents to authoritative status, plus a worktree-RAM-budget governor for the L3 parallel scheduler. Replaces "shadow forever" with a durable ledger that auto-graduates after a 50-op aligned soak, with a graceful-degradation circuit breaker.
Units
autonomy/l3_memory_governor.py+subagent_scheduler.py): caps concurrent worktrees =floor(avail_mb / JARVIS_L3_WORKTREE_RAM_BUDGET_MB), reusingMemoryPressureGate(no duplicate probe), strictest-wins with the existing Slice 5 Arc B fan-out gate.shadow_evaluator.py): pure, never-raises. REVIEW = binary block-vs-allow agreement; PLAN = refinement (DAG covers 100% of legacy tasks ∧ acyclic ∧ disjoint owned-paths). Malformed → blocks graduation.shadow_telemetry_store.py): async SQLite.jarvis/shadow_telemetry.db, boundedto_threadwriter (never blocks loop / never raises), two-phase upsert keyed(op_id,agent), rolling FIFO cap.shadow_graduation_gate.py): event-driven 50-soak → flipsJARVIS_{PLAN,REVIEW}_SUBAGENT_AUTHORITATIVEvia the existing credential-safepersist_flag_to_env.PlanBreakertrips to legacy on cyclical/empty DAG or CRITICAL memory pressure (pre-emptive); emitsAGENT_DEGRADATIONSSE.Live FSM wiring
Producers in
orchestrator._run_{plan,review}_shadow; authoritative REVIEW tier-raise (strictest-wins into the GATE-localrisk_tier); store constructed/owned inGovernedLoopService._build_components/stop.Safety
OFF-is-inert: with
JARVIS_SHADOW_TELEMETRY_STORE_ENABLED=false, the store is never built →orchestrator._shadow_storestaysNone→ every producer/gate/authoritative path is skipped → FSM byte-identical. All new flags default to today's behavior; graduation only fires after a real 50-op soak.Verification
45rail + governor unit/integration tests pass (incl. end-to-end: 50 aligned → graduate, 1 divergence → block).49SSE broker tests pass against main's broker (keep-both validated).ast.parse+ scope-confirmation (these files can't be imported in the dev sandbox — split-brain guard); guarded so OFF-inert is bulletproof.Notes for reviewers
main(Slice 255). The only conflict was a mechanicalEVENT_TYPE_*collision with the loop's orthogonal Slice 252 Shadow-Telemetry — resolved keep-both. Spec §14 documents that the loop's resilience "Shadow Mode" and this subagent "Shadow Rail" are unrelated systems.ctx, notbest_candidate); the breaker/telemetry run but the DAG does not yet steer the L3 scheduler.semantic_guard_hardis conservativelyFalseat the REVIEW hook (guardian signal out of scope there) — flagged in-code for a follow-up.🤖 Generated with Claude Code
Summary by cubic
Graduates the REVIEW and PLAN shadow subagents based on evidence and adds an L3 memory governor that keeps worktree parallelism within live RAM limits. This makes promotion safe and prevents RAM thrash on busy hosts.
New Features
JARVIS_{PLAN,REVIEW}_SUBAGENT_AUTHORITATIVE.AGENT_DEGRADATIONSSE.JARVIS_SHADOW_TELEMETRY_STORE_ENABLED=false.floor(avail_mb / JARVIS_L3_WORKTREE_RAM_BUDGET_MB); strictest-wins with the existing fan-out gate.Migration
JARVIS_SHADOW_TELEMETRY_STORE_ENABLED=true(optionalJARVIS_SHADOW_GRADUATION_THRESHOLD, default 50).JARVIS_L3_MEMORY_GOVERNOR_ENABLEDandJARVIS_L3_WORKTREE_RAM_BUDGET_MB.Written for commit a580562. Summary will update on new commits.