Skip to content

fix(codex): keep Stop plan review in the current turn - #1169

Open
rNoz wants to merge 1 commit into
backnotprop:mainfrom
rNoz:rnoz/plan-decision-reuse
Open

fix(codex): keep Stop plan review in the current turn#1169
rNoz wants to merge 1 commit into
backnotprop:mainfrom
rNoz:rnoz/plan-decision-reuse

Conversation

@rNoz

@rNoz rNoz commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Summary

Addresses the Codex Stop half of #1075.

The Codex Stop hook can scrape the latest <proposed_plan> from the rollout after a later turn that proposed no plan. That resurfaces a plan from an earlier turn and opens another review.

This change keeps plan candidates inside the turn identified by the current Stop payload.

Approach

  • Require a non-empty Stop turn_id with a matching id-carrying rollout marker before inspecting plan candidates.
  • Track turn markers while collecting candidates, and accept assistant <proposed_plan> blocks only while the active marker matches the Stop turn.
  • Keep the existing turn filter for persisted Plan items.
  • Fail closed when turn identity is unavailable instead of guessing from stale transcript content.
  • Emit a PLANNOTATOR_DEBUG breadcrumb for missing Stop identity or a missing rollout marker.
  • Keep duplicate-plan comparison normalization in a small shared helper.

There is no decision store or replay on this path.

Validation

  • bun test apps/hook/server/codex-session.test.ts: 26 pass
  • bun test apps/hook/server: 231 pass
  • bun run typecheck: pass
  • bun run build:review: pass
  • bun run build:hook: pass

Tests cover missing and blank Stop turn IDs, missing rollout markers, debug-only diagnostics, later-turn stale plan blocks, turn contexts without IDs, and existing Stop re-entry behavior.

Compatibility

No public API, endpoint, config, or payload changes. Claude ExitPlanMode behavior is unchanged.

@backnotprop

Copy link
Copy Markdown
Owner

Deep review done, including end-to-end reproduction from a worktree with an isolated data dir. The store itself is careful work: fail-open is real (missing file, corrupt JSON, empty session_id all fall through to a fresh review, verified), the config resolver matches house convention with garbage values disabling the feature, hashing makes path-shaped session ids safe, and 7-day pruning bounds growth. But the risk lives at the two call sites, not in the store, and that is where this needs changes.

Blocking

  1. An approval replay survives /rewind and /compact. The store keys on session_id, and this repo's own session-log docs state that /rewind writes nothing (entries re-parent, orphaning the rest) and a /compact boundary is a new root in the SAME transcript. Same transcript means same session_id means the store still hits. Reproduced: seed an approval, pipe a byte-identical ExitPlanMode event, and the hook instantly emits behavior allow with no review UI, no browser, no user-visible output. The nightmare sequence is: approve plan P, regret it, /rewind to change course, agent re-proposes identical P, silent auto-approve. The codebase already has machinery for ignoring rewind-orphaned branches (resolveActiveBranchIndices); the store has no equivalent, and no TTL beyond 7-day file retention, so a resumed session can replay a days-old approval.

  2. Content-only keys cannot distinguish a stale re-scrape from a genuine re-proposal. Those are byte-identical inputs with opposite correct answers. And every replay is silent: no systemMessage, nothing on stderr, no plans-archive snapshot. Minimum: emit a systemMessage so the human sees the gate was auto-answered. Better: bind replay to the same submission occurrence, not the same bytes (for Claude, no intervening user turn or a short TTL in minutes; for Codex, record the rollout entry index and skip only candidates at or before it).

  3. The Codex half replays denials with no human in the loop, and targets a case that mostly does not exist. The replayed block carries the original feedback verbatim and unlabeled, so the agent believes the human freshly rejected its current turn; if the agent's position is that P is what it wants, it can never reach a human again that session. Probing getLatestCodexPlan with realistic fixtures: turns with task_started markers already exclude the old plan, and Plan items are already turn-filtered. The one shape that re-scrapes is an assistant proposed_plan block, because getAssistantProposedPlanText candidates are NOT turn-filtered while plan items are (codex-session.ts:270-305). Fixing the turn filter there is tighter than a decision store and carries none of the replay risk.

Fix before merge regardless of direction: plan-decision-store.ts contains a literal NUL byte (the separator at line 44 is a raw 0x00, not an escape). Verified consequences: git shows the file as Binary (the PR diff literally says Bin 0 -> 4151 bytes, so nobody can review it as a diff), grep needs -a, and rg does not list the file's own definitions. Use \u0000 in the template literal.

Should-fix, if the store direction survives: label replayed feedback (e.g. "Previously requested changes (not re-reviewed): ..."); Claude-path denials are recorded with user feedback but never read (write-only prose on disk); a malformed record on the Codex path currently becomes a fabricated "Plan changes requested" denial rather than being treated as absent (validate decision === approved|denied); the 10 new tests only cover the store in isolation, with nothing on approve-then-resubmit, deny-then-resubmit, corrupt-store-reopens, or session boundaries (I hand-verified several of those; they should be CI regression tests); the env-variables reference page is missing the new knob and AGENTS.md does not say it is hook-runtimes-only (OpenCode and Pi are unaffected); the normalizer duplicates codex-session's normalizePlan and the comment claiming it mirrors version-history dedup is inaccurate (saveToHistory compares exact strings).

Honest bottom line: the annoyance in #1075 is real, but the Claude half needs an occurrence/freshness bound plus a visible replay signal before auto-answering a human gate, and the Codex half is better fixed at collectPlanCandidates' turn filtering than by replaying denials. Happy to talk through the occurrence-binding design if useful, and to re-review quickly on push.

rNoz added a commit to rNoz/plannotator that referenced this pull request Jul 31, 2026
Reuse only fresh approvals for the active Claude ExitPlanMode occurrence; never replay Codex or denial decisions.

Filter Codex proposal fallbacks to the requested turn.

Refs backnotprop#1169
@rNoz

rNoz commented Jul 31, 2026

Copy link
Copy Markdown
Contributor Author

Pushed the review fixes. Claude reuse is visible and bound to one active submission occurrence. Codex now filters stale plans by turn. Checks are passing.

@backnotprop

Copy link
Copy Markdown
Owner

Re-verified f2c5bab in depth, including 10 independent probes and end-to-end runs through the real hook binary. This redesign is exactly right and all three blockers are resolved:

  • Approvals only, occurrence-keyed, 5-minute TTL. The store now records only approvals, bound to the specific live ExitPlanMode transcript entry via the repo's existing rewind-aware branch resolver. Reproduced end to end: a same-occurrence retry replays WITH the visible signal (top-level systemMessage sibling of hookSpecificOutput, correct per the hooks schema), and a rewound byte-identical plan opens a fresh review. Compact boundaries, intervening turns, TTL boundary plus-minus 1ms, backward clock skew, stale tool_use_id, garbage transcripts, legacy v1 store records: every probe resolves to a fresh review. Content alone no longer authorizes anything, and denials are never persisted on either path.
  • Codex fixed at the source. The turn-identity gate in collectPlanCandidates kills the stale re-scrape (my original three fixtures now behave correctly) and the Codex block in index.ts is byte-identical to main: no store, no replay. The normalizer is properly shared now.
  • NUL byte gone (file is text to git and rg again), plus a regression test asserting no raw NUL in the source, which is a nice touch. Docs added to the env-variables reference with the hook-runtime-only scoping stated. Malformed and legacy records are ignored cleanly. Temp-file leak fixed. Full suite 2621 pass, typecheck clean.

Two small asks before merge:

  1. Missing turn identity now silently disables Codex plan review. getAssistantProposedPlanText returns null when the Stop payload lacks turn_id or the rollout has no id-carrying turn marker; on a Stop hook, no output means the turn just ends and the user silently loses the review. It is narrow (turn_id has shipped since Add Codex Stop-hook plan review #577) but it lands exactly on the fallback shapes most likely to have messy markers, and there is no debug breadcrumb at any of the Stop-path early exits. Add a PLANNOTATOR_DEBUG log line when a candidate is dropped for lack of turn identity, and if you can, confirm once against a real Codex Stop payload.
  2. One unverified assumption worth a single real-session check: the feature only engages if the ExitPlanMode tool_use entry is already in the transcript when the PermissionRequest hook fires. If it is not, occurrence is null and the feature quietly no-ops (fail-open, correct direction, but the Plan review re-opens for a plan already decided in the same session #1075 annoyance would return). A quick confirmation in a live Claude session that reuse actually triggers would close the loop.

Benign residuals, no action needed: a turn boundary without an id inherits the previous activeTurnId (extra review, safe direction); concurrent hook writes can lose a record (extra review, verified with 8 parallel writers, no temp leaks); future record-shape changes invalidate old records via the strict two-key check (safe, worth a comment).

rNoz added a commit to rNoz/plannotator that referenced this pull request Jul 31, 2026
Reuse only fresh approvals for the active Claude ExitPlanMode occurrence; never replay Codex or denial decisions.

Filter Codex proposal fallbacks to the requested turn.

Refs backnotprop#1169
@rNoz
rNoz force-pushed the rnoz/plan-decision-reuse branch from f2c5bab to 07dbdc0 Compare July 31, 2026 20:19
@rNoz

rNoz commented Jul 31, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto current main and resolved the environment reference conflict. Added debug-only breadcrumbs for unsafe Codex Stop skips. Local rollout markers and Claude occurrence timing were also checked.

@backnotprop

Copy link
Copy Markdown
Owner

Verified the final push directly: the PLANNOTATOR_DEBUG breadcrumb covers both skip shapes (missing Stop payload turn_id, missing id-carrying rollout marker) with distinct messages on stderr and a clean exit, exactly as asked, and the rebase onto current main resolved cleanly. All 208 hook tests pass on the head. This closes out both asks from the re-review; merge-ready from my side once the post-rebase CI finishes.

@rNoz
rNoz marked this pull request as ready for review July 31, 2026 20:24
@jpvarbed

jpvarbed commented Aug 1, 2026

Copy link
Copy Markdown

Thanks for fixing this! This looks like it addresses both cases from my issue without skipping reviews for genuinely new plans.
Appreciate it!

@backnotprop

Copy link
Copy Markdown
Owner

Thanks for confirming both cases, that closes the loop on the report. This is queued to merge.

rNoz added a commit to rNoz/plannotator that referenced this pull request Aug 4, 2026
Reuse only fresh approvals for the active Claude ExitPlanMode occurrence; never replay Codex or denial decisions.

Filter Codex proposal fallbacks to the requested turn.

Refs backnotprop#1169
@rNoz
rNoz force-pushed the rnoz/plan-decision-reuse branch from 07dbdc0 to 2e0078d Compare August 4, 2026 23:15
@rNoz

rNoz commented Aug 4, 2026

Copy link
Copy Markdown
Contributor Author

@backnotprop rebased onto current main (7682628) — the only conflict was the @plannotator/shared/config import line in apps/hook/server/index.ts (main dropped the now-unused resolveUseJina import; kept resolvePlanDecisionReuse). All 225 hook tests pass and typecheck is clean on the rebased head (2e0078d). CI is running.

Just pinging because you mentioned it was queued to merge, but it didn't happen after 3 days.

@rNoz

rNoz commented Aug 9, 2026

Copy link
Copy Markdown
Contributor Author

@backnotprop reminder for this and the other #1154 fix, both ready

rNoz added a commit to rNoz/plannotator that referenced this pull request Aug 25, 2026
Reuse only fresh approvals for the active Claude ExitPlanMode occurrence; never replay Codex or denial decisions.

Filter Codex proposal fallbacks to the requested turn.

Refs backnotprop#1169
@rNoz
rNoz force-pushed the rnoz/plan-decision-reuse branch from 77b0944 to e9e7f3e Compare August 25, 2026 12:07
@rNoz

rNoz commented Aug 25, 2026

Copy link
Copy Markdown
Contributor Author

CI passing again, ready for you @backnotprop

@backnotprop

Copy link
Copy Markdown
Owner

Following up here after my comment on #1075. I have walked the user-visible sequences again and here is where I land.

The Codex Stop fix I understand and want. The transcript scraper is our logic, and re-surfacing an already-decided plan across a turn boundary is our bug. The turn-identity boundary and the fail-closed handling look like the right shape.

The ExitPlanMode half I am going to pass on. The sequences that produce a duplicate review on Claude are: the model re-submitting the same plan (which your fix deliberately still reviews, and I agree with that call), the harness re-asking for the same tool call, or duplicate hook registration in local config. In all of those Plannotator is doing its job by asking. A decision store in our hook would be compensating for harness behavior in the wrong layer, and it is most of this diff. If there is a real-world Claude sequence I am missing, the video repro I asked for on the issue is the best way to show me.

If you trim this PR to the Codex changes I will review it for merge. Sorry for the runaround, the Aug 1 "queued to merge" came before I had pulled the two cases apart.

AI-assisted (Claude) under maintainer direction.

@rNoz
rNoz force-pushed the rnoz/plan-decision-reuse branch from e9e7f3e to 404397b Compare August 28, 2026 18:41
@rNoz rNoz changed the title fix(hook): skip re-review of a plan already decided this session fix(codex): keep Stop plan review in the current turn Aug 28, 2026
@rNoz

rNoz commented Aug 28, 2026

Copy link
Copy Markdown
Contributor Author

Trimmed this to the Codex-only Stop fix requested in the 2026-08-25 review.

The Claude occurrence-bound approval reuse, its config, storage, transcript handling, tests, and docs are withdrawn from this PR. The Claude half of #1075 remains open for a separate contribution if a real-world reproduction supports it.

The new head is one commit on current main and changes only the Codex rollout parser, its tests, the Stop-path gate, and a three-line plan normalizer.

Fresh local gates:

  • Codex session tests: 26 pass
  • Hook tests: 231 pass
  • Typecheck: pass
  • Review build: pass
  • Hook build: pass

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants