From 673952d812a7fe0bb849fd1c13ba1b3f208605ee Mon Sep 17 00:00:00 2001 From: ryaneggz Date: Wed, 12 Aug 2026 22:33:32 -0600 Subject: [PATCH] fix: keep the firstmate probe pane alive across its own read (#761) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The execution-context gate could never admit herdr, in any environment. herdr destroys a pane the instant its command returns, and a read against a destroyed pane answers `pane_not_found`. `RUNNER_PROBE_SCRIPT` prints one fingerprint line and exits, so the probe pane lost the race with its own reader: [session-runner] execution-context gate: probe pane w3:p4 emitted no fingerprint within 15000ms The gate then reported "no probe fingerprint obtained" and degraded to tmux — correct-looking output for the wrong reason, and a permanent refusal wearing the shape of a proof obligation. Measured live, with a keep-alive appended so the pane outlives the read: caller: host=928eeb5cbca5 docker=yes worktree=yes probe: host=928eeb5cbca5 docker=yes worktree=yes Identical. The comparison the gate exists to make would have passed. WHY IT SURVIVED THE ORIGINAL BUILD Two reasons, and both are worth recording. The live path failed earlier and correctly: while the operator config bind (#756) was open, the probe pane was a HOST shell that stayed alive long enough to read, and the gate refused herdr on a genuine fingerprint mismatch. US-010's smoke measured exactly that path. The race is only reachable once the environment matches, so closing #756 is what exposed it. The unit-test stub was MORE FORGIVING than herdr: it replayed pane output regardless of whether the pane could still exist. That is why 46 tests passed against a gate that could not work live. The stub now models real pane lifetime — a pane is readable only if its start invocation carried a keep-alive — so removing the keep-alive fails five tests, three of which predate this change. THE FIX A keep-alive suffix rides the pane invocation only. It is never folded into `RUNNER_PROBE_SCRIPT`, because that snippet also runs in-process for the caller side, and "the same snippet runs in the probe pane and locally" is what makes the comparison true by construction rather than by convention; a sleeping local fingerprint would also stall every caller. The budget derives from the single `RUNNER_PROBE_TIMEOUT_MS` source so the two cannot drift, and it is an upper bound rather than a cost — the gate closes the pane as soon as the read completes, on the match and the mismatch path alike. VERIFIED BY REJECTION `session-runner-ladder.sh` gains five assertions. Each was shown to FAIL against a deliberately broken copy, not merely to pass against the good one: - the pane runs the bare snippet instead of the composed one - the keep-alive constant is deleted - the suffix is dropped from the composition - a keep-alive leaks into the shared `RUNNER_PROBE_SCRIPT` - the budget stops deriving from `RUNNER_PROBE_TIMEOUT_MS` One assertion initially rejected the fix itself: `code_only` drops `printf` lines so that prose naming a command cannot satisfy a check that the command runs. The single-`printf` helper was restructured to assign then print, rather than weakening the filter. The header's MIGRATION TRIGGER is discharged and replaced with the observed result. Its prediction — that the gate would "stop rejecting herdr by itself" once #756 closed — was wrong, and the note now says so, with the lesson: a gate failing for one reason can hide a second reason it would fail anyway. Removing the cause does not prove the gate works; only re-probing does. The gate itself is unchanged in intent and stays after this fix. It is a build-correctness guard whose job is to PROVE environment identity rather than assume it. 588 tests pass, 102 probes run with 99 PASS and 3 pre-existing SKIPPED. Closes #761. Refs #746, #748, #756. Submitted-by: Claude Co-Authored-By: Claude Opus 5 (1M context) --- .oh/evals/RESULTS.md | 204 +++++++++---------- .oh/evals/probes/session-runner-ladder.sh | 33 +++ .oh/scripts/__tests__/session-runner.test.ts | 80 +++++++- .oh/scripts/lib/session-runner.sh | 91 +++++++-- .oh/skills/firstmate/SKILL.md | 28 ++- CHANGELOG.md | 1 + 6 files changed, 309 insertions(+), 128 deletions(-) diff --git a/.oh/evals/RESULTS.md b/.oh/evals/RESULTS.md index 6bc47197..a91a58ea 100644 --- a/.oh/evals/RESULTS.md +++ b/.oh/evals/RESULTS.md @@ -6,107 +6,107 @@ probe id; git history is the time series.** Schema and exit-code semantics are i | probe | tier | last-run (UTC) | status | source | |-------|------|----------------|--------|--------| -| ablate-state-machine | A | 2026-08-13 03:03 | PASS | issue #645 — one locked versioned ablation recovery owner | -| advisor-monitored-loop | A | 2026-08-13 03:03 | PASS | conversation 2026-06-19 (advisor-monitored ralph loop pattern, issue #257) | -| agent-browser-cli | A | 2026-08-13 03:03 | PASS | .oh/memory/MEMORY.md 2026-06-07 (agent-browser 0.8.5 CLI) | -| artifact-contract-audit | A | 2026-08-13 03:03 | PASS | issue #583/#645 — production /audit implementation Gate 1 behavior | -| audit-context-shared-memory | A | 2026-08-13 03:03 | PASS | issue #645 — shared-memory context ablation routing | -| audit-dispatcher-contract | A | 2026-08-13 03:03 | PASS | issue #645 — audit consolidation public taxonomy | -| audit-implementation-behavior | A | 2026-08-13 03:03 | PASS | issue #645 — implementation root/repo/browser behavior | -| audit-pr-acquire | A | 2026-08-13 03:03 | PASS | issue #645 — production PR acquisition behavior | -| audit-pr-classifier | A | 2026-08-13 03:03 | PASS | issue #645 — deterministic focused and queue PR classifier | -| audit-run-root-contract | A | 2026-08-13 03:03 | PASS | issue #645 — executable immutable audit root/run/log correlation | -| audit-shellcheck-coverage | A | 2026-08-13 03:03 | PASS | issue #645 — private audit scripts require release and CI lint coverage | -| audit-stale-references | A | 2026-08-13 03:03 | PASS | issue #645 — clean-breaking audit migration | -| autopilot-executor-toggle | A | 2026-08-13 03:03 | PASS | conversation 2026-06-13 (autopilot executor); 2026-06-27 (ralph-default flip) | -| autopilot-merged-pr-reference-dedupe | A | 2026-08-13 03:03 | PASS | issue #468 — autopilot must not rebuild open tickets whose development PRs already merged | -| autopilot-no-pr-session-close | A | 2026-08-13 03:03 | PASS | issue #209 (autopilot no-PR tmux session closure) 2026-06-16 | -| autopilot-open-pr-reference-dedupe | A | 2026-08-13 03:03 | PASS | issue #437 — autopilot must not start duplicate work when open PRs reference the same issue without linked-PR metadata | -| autopilot-pi-agent | A | 2026-08-13 03:03 | PASS | issue #116 (autopilot Pi tmux alignment) 2026-06-14; issue #118 (attachable Pi TUI tmux) 2026-06-14; issue #126 (kept Pi overlap lock release) 2026-06-14; issue #142 (worktree-by-default, skip→worktree) 2026-06-14 | -| autopilot-preflight-gate | A | 2026-08-13 03:03 | SKIPPED | issue #194 (deterministic autopilot caps preflight gate) 2026-06-15 | -| autopilot-upstream-default | A | 2026-08-13 03:03 | PASS | issue #420 — future autopilots must target canonical repo, not personal fork | -| autopilot-worktree-log-root | A | 2026-08-13 03:03 | PASS | issue #152 (persist autopilot worktree logs) 2026-06-15 | -| boot-lint-glob | A | 2026-08-13 03:03 | PASS | issue #90, issue #120 | -| builder-skill-consolidation | A | 2026-08-13 03:03 | PASS | issue #643 — consolidate artifact builders behind one /builder dispatcher | -| capability-benchmark-schema | A | 2026-08-13 03:03 | PASS | issue #167 — capability benchmark instrument | -| cc-safety-net-wiring | A | 2026-08-13 03:03 | PASS | .oh/tasks/cc-safety-net/prd.json US-007 2026-07-19 | -| clean-restore | A | 2026-08-13 03:03 | PASS | issue #63 (autopilot-stray-wip-guard) 2026-06-12; issue #81 (owned-paths-zsh-split) 2026-06-13 | -| cleanup-tasks-scoped-guard | A | 2026-08-13 03:03 | PASS | issue #85 | -| cleanup-tasks-worktree-grooming | A | 2026-08-13 03:03 | PASS | issue #168; issue #327 | -| codex-stale-response-retry | A | 2026-08-13 03:03 | PASS | issue #506 — Codex previous_response_not_found RCA | -| cron-claude-codex-fallback | A | 2026-08-13 03:03 | PASS | conversation 2026-06-12 (default Codex fallback for crons) | -| cron-watchdog | A | 2026-08-13 03:03 | PASS | issues #130/#453 (cron runtime watchdog + legacy system-cron reaping) 2026-06-19 | -| curl-bash-safe-alternatives | A | 2026-08-13 03:03 | PASS | vet-run/vet integration — public curl|bash examples need review-first alternatives | -| datasets-schema | A | 2026-08-13 03:03 | PASS | issue #196 — .oh/evals/datasets verifiable trajectory corpus (Repo2RLEnv-inspired) | -| debugmcp-availability | A | 2026-08-13 03:03 | SKIPPED | issue #297 — DebugMCP MCP debug-server availability | -| delegate-model-effort-policy | A | 2026-08-13 03:03 | PASS | conversation 2026-07-11 (delegate model inheritance and thinking policy) | -| devtcp-hook | A | 2026-08-13 03:03 | PASS | .oh/memory/MEMORY.md 2026-06-10 (zsh /dev/tcp) | -| docker-inspect-env-guard | A | 2026-08-13 03:03 | PASS | operator directive 2026-08-08 (agents keep the docker socket, but must | -| docs-build-fast-path | A | 2026-08-13 03:03 | PASS | #455 — docs builds must stay out of fast harness/eval/release gates; #536 — docs site externalized to openharness-web; docs markdown relocated to .oh/docs/ | -| drift-check-cron-staleness-glob | A | 2026-08-13 03:03 | PASS | issue #98; issue #225 (restart-required cron frontmatter/config drift) | -| entrypoint-pnpm-manifest-fingerprint | A | 2026-08-13 03:03 | PASS | issue #521 (manifest-aware sandbox installs) 2026-07-01 | -| eval-ci-gate | A | 2026-08-13 03:03 | PASS | #103 — eval probe suite gated in CI | -| eval-gate | A | 2026-08-13 03:03 | PASS | .oh/memory/MEMORY.md 2026-06-11 (eval-gate) | -| eval-results-atomic | A | 2026-08-13 03:03 | PASS | issue #83 (eval-results-atomic-write) | -| eval-runner-exit | A | 2026-08-13 03:03 | PASS | .oh/memory/MEMORY.md 2026-06-11 (eval-runner-exit) #29 | -| execution-target-contract | A | 2026-08-13 03:03 | PASS | issue #733 (ExecutionTarget contract + Docker Compose adapter) 2026-08-10 | -| first-mate-charter | A | 2026-08-13 03:03 | PASS | .oh/tasks/first-mate-charter/ (issue #660) — First Mate role charter + advisor prompt pack must stay present, tracked, resolvable, and effort-vocabulary-aligned with /delegate | -| firstmate-executor-contract | A | 2026-08-13 03:03 | PASS | .oh/tasks/firstmate-executor/ (issue #746) — the opt-in firstmate build executor is additive to ralph and shares its terminal interface | -| get-oh-bootstrap | A | 2026-08-13 03:03 | PASS | get-oh.sh bootstrap — the Node-bootstrapping host-side path to the standalone `oh` CLI (also on npm as @mifune/openharness; see oh-npm-package.sh) | -| git-skill | A | 2026-08-13 03:03 | PASS | conversation 2026-06-15 — rules are not always supported; git workflow must be the /git skill | -| harness-audit-empty-output-gate | A | 2026-08-13 03:03 | PASS | issue #246 — /audit harness must fail closed on empty auditor outputs | -| harness-audit-memory-path | A | 2026-08-13 03:03 | PASS | issue #183 — /audit harness must inspect the active worktree, not a hardcoded root | -| harness-audit-shared-memory | A | 2026-08-13 03:03 | PASS | issue #432 — /audit harness must load durable memory from shared log root in cron worktrees | -| harness-ci-core-paths | A | 2026-08-13 03:03 | PASS | #165 — core sandbox config files must trigger harness CI | -| harness-ci-hooks-paths | A | 2026-08-13 03:03 | PASS | issue #202 — credential/security hook changes must trigger harness CI | -| health-check-docker-stats | A | 2026-08-13 03:03 | PASS | .oh/memory/MEMORY.md 2026-06-10 (docker stats vs ps Size) | -| heartbeat-logging-contract | A | 2026-08-13 03:03 | PASS | issue #447 (heartbeat log append hardening) 2026-06-18 | -| locked-append-critical-path | A | 2026-08-13 03:03 | PASS | issue #204 (lock shared runtime log appends) 2026-06-15 | -| markitdown-wiki-ingest | A | 2026-08-13 03:03 | PASS | issue #649 — pinned local-document normalization contract for /wiki ingest | -| memory-gitignore-claim | A | 2026-08-13 03:03 | PASS | issue #101 | -| memory-log-locked-append | A | 2026-08-13 03:03 | PASS | issue #476 and #645 — memory appends are locked and audit has one log owner | -| next-dev-prod | A | 2026-08-13 03:03 | SKIPPED | .oh/memory/MEMORY.md 2026-06-04 | -| oh-devcontainer-restructure | A | 2026-08-13 03:03 | PASS | consolidate devcontainer — .oh/devcontainer/ folded back into .devcontainer/ | -| oh-image-only-deploy | A | 2026-08-13 03:03 | PASS | .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only deploy) | -| oh-init-scaffold | A | 2026-08-13 03:03 | PASS | issue #531 Phase 2 | -| oh-npm-package | A | 2026-08-13 03:03 | PASS | npm publish path for the standalone `oh` CLI (@mifune/openharness) — alternative to get-oh.sh | -| oh-payload-manifest | A | 2026-08-13 03:03 | PASS | issue #531 follow-on (.oh payload manifest — oh update ships a declared allowlist) | -| oh-sandbox-image-mode | A | 2026-08-13 03:03 | PASS | conversation 2026-07-05 (basic Docker deployment — prebuilt-image mode) | -| oh-shipped-repo-overridable | A | 2026-08-13 03:03 | PASS | issue #531 follow-on (de-hardcode residual — shipped .oh shell scripts keep the upstream repo overridable) | -| oh-standalone-lifecycle | A | 2026-08-13 03:03 | PASS | issue #564 | -| oh-update | A | 2026-08-13 03:03 | PASS | issue #531 Phase 3 (oh update — upgrade only the .oh control plane) | -| operator-config-guard | A | 2026-08-13 03:03 | PASS | operator directives 2026-08-06 (.config/ and settings.local.json are operator-only) | -| owned-surface-guard | A | 2026-08-13 03:03 | PASS | issue #63 (autopilot-stray-wip-guard) 2026-06-12; issue #81 (owned-paths-zsh-split) 2026-06-13 | -| pnpm-audit-ci-gate | A | 2026-08-13 03:03 | PASS | issue #171 — pnpm security audits must run in CI | -| post-bridge-publish-confirmation | A | 2026-08-13 03:03 | PASS | #523 — post-bridge live publishing requires an explicit final confirmation gate | -| prd-output-path-contract | A | 2026-08-13 03:03 | PASS | .oh/memory/MEMORY.md 2026-06-19 | -| project-root-seam | A | 2026-08-13 03:03 | PASS | issue #531 Phase 1 (OH_PROJECT_ROOT project-root seam) 2026-06-26 | -| prompt-miner-log-root-worktree | A | 2026-08-13 03:03 | PASS | .oh/skills/prompt-miner/scripts/render-log-entry.sh | -| prompt-miner-schema-compat | A | 2026-08-13 03:03 | PASS | issue #253 — prompt-miner JSONL schema-drift guard | -| prompt-miner-symlink-entrypoint | A | 2026-08-13 03:03 | PASS | issue #663 — prompt-miner engine no-ops via the documented .claude/skills symlink | -| prompt-miner-weakness-record | A | 2026-08-13 03:03 | PASS | issue #580 — prompt-miner weakness-record (WH-xxx) cluster output | -| protected-paths-resolve | A | 2026-08-13 03:03 | PASS | issue #753 — .claude/protected-paths.txt named 7 paths that did not exist. | -| ralph-fallback-order | A | 2026-08-13 03:03 | PASS | conversation 2026-06-12 (Ralph default fallback order) | -| repo-map-contract | A | 2026-08-13 03:03 | PASS | issue #464 — repo map must optimize orientation without adding a tree dependency or unmeasured performance claims | -| retro-deterministic-contract | A | 2026-08-13 03:03 | PASS | issue #443 — /retro deterministic output and self-contained helper contract | -| rl-delegation-write-worker | A | 2026-08-13 03:03 | PASS | .oh/memory/MEMORY.md 2026-06-10 (rl-delegation) #57 | -| rlm-context-budget | A | 2026-08-13 03:03 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-006 | -| sandbox-boot-guard-ci | A | 2026-08-13 03:03 | PASS | issue #449 (sandbox image build CI guard) 2026-06-19 | -| session-runner-ladder | A | 2026-08-13 03:03 | PASS | .oh/tasks/firstmate-executor/ (issue #746) — the shared herdr -> tmux -> foreground runner ladder and its safety gates | -| ship-spec-ready-finalization | A | 2026-08-13 03:03 | PASS | issue #134 — /ship-spec must finalize ready PRs after gates, not stop at draft scaffold | -| skill-paths | A | 2026-08-13 03:03 | PASS | issue #43 — stale path references; extended by issue #69 — apps/->packages/ rename guard | -| skills-dir-clean | A | 2026-08-13 03:03 | PASS | conversation 2026-06-29 — Pi parses every top-level `.md` in the skills | -| skills-vendored | A | 2026-08-13 03:03 | PASS | absorb .mifune submodule into .oh — the skills/agents/hooks pack is vendored | -| slack-admin-command-surface | A | 2026-08-13 03:03 | PASS | issue #354 — Slack bridge docs must distinguish Pi /msg-bridge commands from Slack DM admin text handlers | -| spec-family-contract | A | 2026-08-13 03:03 | PASS | conversation 2026-06-19 (spec-* family split, issue #265); consolidated into /spec dispatcher 2026-06-23 (one skill, args) | -| ste-checker-contract | A | 2026-08-13 03:03 | PASS | issue #750 PR audit — the /ste checker had four fail-open paths (unclosed | -| submitted-by-trailers | A | 2026-08-13 03:03 | PASS | conversation 2026-06-12 (commit attribution trailers) | -| sync-skill-contract | A | 2026-08-13 03:03 | PASS | issue #331 — /sync dispatcher skill (bidirectional origin↔upstream sync) | -| watchdog-completed-session-reap | A | 2026-08-13 03:03 | PASS | issue #235 (completed autopilot PR session reaping) | -| watchdog-draft-prs | A | 2026-08-13 03:03 | PASS | conversation 2026-06-15 (generic watchdog + stale draft PR recovery) | -| watchdog-stuck-sessions | A | 2026-08-13 03:03 | PASS | issue #240 (Codex zero-credit stuck autopilot sessions) 2026-06-17 | -| weigh-scorer-contract | A | 2026-08-13 03:03 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-003 (2026-06-27) | -| wiki-readme-index | A | 2026-08-13 03:03 | PASS | issue #132 — wiki README index drift guard | -| workflow-boundaries | A | 2026-08-13 03:03 | PASS | conversation 2026-06-19 (workflow consolidation, issue #259) | +| ablate-state-machine | A | 2026-08-13 04:32 | PASS | issue #645 — one locked versioned ablation recovery owner | +| advisor-monitored-loop | A | 2026-08-13 04:32 | PASS | conversation 2026-06-19 (advisor-monitored ralph loop pattern, issue #257) | +| agent-browser-cli | A | 2026-08-13 04:32 | PASS | .oh/memory/MEMORY.md 2026-06-07 (agent-browser 0.8.5 CLI) | +| artifact-contract-audit | A | 2026-08-13 04:32 | PASS | issue #583/#645 — production /audit implementation Gate 1 behavior | +| audit-context-shared-memory | A | 2026-08-13 04:32 | PASS | issue #645 — shared-memory context ablation routing | +| audit-dispatcher-contract | A | 2026-08-13 04:32 | PASS | issue #645 — audit consolidation public taxonomy | +| audit-implementation-behavior | A | 2026-08-13 04:32 | PASS | issue #645 — implementation root/repo/browser behavior | +| audit-pr-acquire | A | 2026-08-13 04:32 | PASS | issue #645 — production PR acquisition behavior | +| audit-pr-classifier | A | 2026-08-13 04:32 | PASS | issue #645 — deterministic focused and queue PR classifier | +| audit-run-root-contract | A | 2026-08-13 04:32 | PASS | issue #645 — executable immutable audit root/run/log correlation | +| audit-shellcheck-coverage | A | 2026-08-13 04:32 | PASS | issue #645 — private audit scripts require release and CI lint coverage | +| audit-stale-references | A | 2026-08-13 04:32 | PASS | issue #645 — clean-breaking audit migration | +| autopilot-executor-toggle | A | 2026-08-13 04:32 | PASS | conversation 2026-06-13 (autopilot executor); 2026-06-27 (ralph-default flip) | +| autopilot-merged-pr-reference-dedupe | A | 2026-08-13 04:32 | PASS | issue #468 — autopilot must not rebuild open tickets whose development PRs already merged | +| autopilot-no-pr-session-close | A | 2026-08-13 04:32 | PASS | issue #209 (autopilot no-PR tmux session closure) 2026-06-16 | +| autopilot-open-pr-reference-dedupe | A | 2026-08-13 04:32 | PASS | issue #437 — autopilot must not start duplicate work when open PRs reference the same issue without linked-PR metadata | +| autopilot-pi-agent | A | 2026-08-13 04:32 | PASS | issue #116 (autopilot Pi tmux alignment) 2026-06-14; issue #118 (attachable Pi TUI tmux) 2026-06-14; issue #126 (kept Pi overlap lock release) 2026-06-14; issue #142 (worktree-by-default, skip→worktree) 2026-06-14 | +| autopilot-preflight-gate | A | 2026-08-13 04:32 | SKIPPED | issue #194 (deterministic autopilot caps preflight gate) 2026-06-15 | +| autopilot-upstream-default | A | 2026-08-13 04:32 | PASS | issue #420 — future autopilots must target canonical repo, not personal fork | +| autopilot-worktree-log-root | A | 2026-08-13 04:32 | PASS | issue #152 (persist autopilot worktree logs) 2026-06-15 | +| boot-lint-glob | A | 2026-08-13 04:32 | PASS | issue #90, issue #120 | +| builder-skill-consolidation | A | 2026-08-13 04:32 | PASS | issue #643 — consolidate artifact builders behind one /builder dispatcher | +| capability-benchmark-schema | A | 2026-08-13 04:32 | PASS | issue #167 — capability benchmark instrument | +| cc-safety-net-wiring | A | 2026-08-13 04:32 | PASS | .oh/tasks/cc-safety-net/prd.json US-007 2026-07-19 | +| clean-restore | A | 2026-08-13 04:32 | PASS | issue #63 (autopilot-stray-wip-guard) 2026-06-12; issue #81 (owned-paths-zsh-split) 2026-06-13 | +| cleanup-tasks-scoped-guard | A | 2026-08-13 04:32 | PASS | issue #85 | +| cleanup-tasks-worktree-grooming | A | 2026-08-13 04:32 | PASS | issue #168; issue #327 | +| codex-stale-response-retry | A | 2026-08-13 04:32 | PASS | issue #506 — Codex previous_response_not_found RCA | +| cron-claude-codex-fallback | A | 2026-08-13 04:32 | PASS | conversation 2026-06-12 (default Codex fallback for crons) | +| cron-watchdog | A | 2026-08-13 04:32 | PASS | issues #130/#453 (cron runtime watchdog + legacy system-cron reaping) 2026-06-19 | +| curl-bash-safe-alternatives | A | 2026-08-13 04:32 | PASS | vet-run/vet integration — public curl|bash examples need review-first alternatives | +| datasets-schema | A | 2026-08-13 04:32 | PASS | issue #196 — .oh/evals/datasets verifiable trajectory corpus (Repo2RLEnv-inspired) | +| debugmcp-availability | A | 2026-08-13 04:32 | SKIPPED | issue #297 — DebugMCP MCP debug-server availability | +| delegate-model-effort-policy | A | 2026-08-13 04:32 | PASS | conversation 2026-07-11 (delegate model inheritance and thinking policy) | +| devtcp-hook | A | 2026-08-13 04:32 | PASS | .oh/memory/MEMORY.md 2026-06-10 (zsh /dev/tcp) | +| docker-inspect-env-guard | A | 2026-08-13 04:32 | PASS | operator directive 2026-08-08 (agents keep the docker socket, but must | +| docs-build-fast-path | A | 2026-08-13 04:32 | PASS | #455 — docs builds must stay out of fast harness/eval/release gates; #536 — docs site externalized to openharness-web; docs markdown relocated to .oh/docs/ | +| drift-check-cron-staleness-glob | A | 2026-08-13 04:32 | PASS | issue #98; issue #225 (restart-required cron frontmatter/config drift) | +| entrypoint-pnpm-manifest-fingerprint | A | 2026-08-13 04:32 | PASS | issue #521 (manifest-aware sandbox installs) 2026-07-01 | +| eval-ci-gate | A | 2026-08-13 04:32 | PASS | #103 — eval probe suite gated in CI | +| eval-gate | A | 2026-08-13 04:32 | PASS | .oh/memory/MEMORY.md 2026-06-11 (eval-gate) | +| eval-results-atomic | A | 2026-08-13 04:32 | PASS | issue #83 (eval-results-atomic-write) | +| eval-runner-exit | A | 2026-08-13 04:32 | PASS | .oh/memory/MEMORY.md 2026-06-11 (eval-runner-exit) #29 | +| execution-target-contract | A | 2026-08-13 04:32 | PASS | issue #733 (ExecutionTarget contract + Docker Compose adapter) 2026-08-10 | +| first-mate-charter | A | 2026-08-13 04:32 | PASS | .oh/tasks/first-mate-charter/ (issue #660) — First Mate role charter + advisor prompt pack must stay present, tracked, resolvable, and effort-vocabulary-aligned with /delegate | +| firstmate-executor-contract | A | 2026-08-13 04:32 | PASS | .oh/tasks/firstmate-executor/ (issue #746) — the opt-in firstmate build executor is additive to ralph and shares its terminal interface | +| get-oh-bootstrap | A | 2026-08-13 04:32 | PASS | get-oh.sh bootstrap — the Node-bootstrapping host-side path to the standalone `oh` CLI (also on npm as @mifune/openharness; see oh-npm-package.sh) | +| git-skill | A | 2026-08-13 04:32 | PASS | conversation 2026-06-15 — rules are not always supported; git workflow must be the /git skill | +| harness-audit-empty-output-gate | A | 2026-08-13 04:32 | PASS | issue #246 — /audit harness must fail closed on empty auditor outputs | +| harness-audit-memory-path | A | 2026-08-13 04:32 | PASS | issue #183 — /audit harness must inspect the active worktree, not a hardcoded root | +| harness-audit-shared-memory | A | 2026-08-13 04:32 | PASS | issue #432 — /audit harness must load durable memory from shared log root in cron worktrees | +| harness-ci-core-paths | A | 2026-08-13 04:32 | PASS | #165 — core sandbox config files must trigger harness CI | +| harness-ci-hooks-paths | A | 2026-08-13 04:32 | PASS | issue #202 — credential/security hook changes must trigger harness CI | +| health-check-docker-stats | A | 2026-08-13 04:32 | PASS | .oh/memory/MEMORY.md 2026-06-10 (docker stats vs ps Size) | +| heartbeat-logging-contract | A | 2026-08-13 04:32 | PASS | issue #447 (heartbeat log append hardening) 2026-06-18 | +| locked-append-critical-path | A | 2026-08-13 04:32 | PASS | issue #204 (lock shared runtime log appends) 2026-06-15 | +| markitdown-wiki-ingest | A | 2026-08-13 04:32 | PASS | issue #649 — pinned local-document normalization contract for /wiki ingest | +| memory-gitignore-claim | A | 2026-08-13 04:32 | PASS | issue #101 | +| memory-log-locked-append | A | 2026-08-13 04:32 | PASS | issue #476 and #645 — memory appends are locked and audit has one log owner | +| next-dev-prod | A | 2026-08-13 04:32 | SKIPPED | .oh/memory/MEMORY.md 2026-06-04 | +| oh-devcontainer-restructure | A | 2026-08-13 04:32 | PASS | consolidate devcontainer — .oh/devcontainer/ folded back into .devcontainer/ | +| oh-image-only-deploy | A | 2026-08-13 04:32 | PASS | .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only deploy) | +| oh-init-scaffold | A | 2026-08-13 04:32 | PASS | issue #531 Phase 2 | +| oh-npm-package | A | 2026-08-13 04:32 | PASS | npm publish path for the standalone `oh` CLI (@mifune/openharness) — alternative to get-oh.sh | +| oh-payload-manifest | A | 2026-08-13 04:32 | PASS | issue #531 follow-on (.oh payload manifest — oh update ships a declared allowlist) | +| oh-sandbox-image-mode | A | 2026-08-13 04:32 | PASS | conversation 2026-07-05 (basic Docker deployment — prebuilt-image mode) | +| oh-shipped-repo-overridable | A | 2026-08-13 04:32 | PASS | issue #531 follow-on (de-hardcode residual — shipped .oh shell scripts keep the upstream repo overridable) | +| oh-standalone-lifecycle | A | 2026-08-13 04:32 | PASS | issue #564 | +| oh-update | A | 2026-08-13 04:32 | PASS | issue #531 Phase 3 (oh update — upgrade only the .oh control plane) | +| operator-config-guard | A | 2026-08-13 04:32 | PASS | operator directives 2026-08-06 (.config/ and settings.local.json are operator-only) | +| owned-surface-guard | A | 2026-08-13 04:32 | PASS | issue #63 (autopilot-stray-wip-guard) 2026-06-12; issue #81 (owned-paths-zsh-split) 2026-06-13 | +| pnpm-audit-ci-gate | A | 2026-08-13 04:32 | PASS | issue #171 — pnpm security audits must run in CI | +| post-bridge-publish-confirmation | A | 2026-08-13 04:32 | PASS | #523 — post-bridge live publishing requires an explicit final confirmation gate | +| prd-output-path-contract | A | 2026-08-13 04:32 | PASS | .oh/memory/MEMORY.md 2026-06-19 | +| project-root-seam | A | 2026-08-13 04:32 | PASS | issue #531 Phase 1 (OH_PROJECT_ROOT project-root seam) 2026-06-26 | +| prompt-miner-log-root-worktree | A | 2026-08-13 04:32 | PASS | .oh/skills/prompt-miner/scripts/render-log-entry.sh | +| prompt-miner-schema-compat | A | 2026-08-13 04:32 | PASS | issue #253 — prompt-miner JSONL schema-drift guard | +| prompt-miner-symlink-entrypoint | A | 2026-08-13 04:32 | PASS | issue #663 — prompt-miner engine no-ops via the documented .claude/skills symlink | +| prompt-miner-weakness-record | A | 2026-08-13 04:32 | PASS | issue #580 — prompt-miner weakness-record (WH-xxx) cluster output | +| protected-paths-resolve | A | 2026-08-13 04:32 | PASS | issue #753 — .claude/protected-paths.txt named 7 paths that did not exist. | +| ralph-fallback-order | A | 2026-08-13 04:32 | PASS | conversation 2026-06-12 (Ralph default fallback order) | +| repo-map-contract | A | 2026-08-13 04:32 | PASS | issue #464 — repo map must optimize orientation without adding a tree dependency or unmeasured performance claims | +| retro-deterministic-contract | A | 2026-08-13 04:32 | PASS | issue #443 — /retro deterministic output and self-contained helper contract | +| rl-delegation-write-worker | A | 2026-08-13 04:32 | PASS | .oh/memory/MEMORY.md 2026-06-10 (rl-delegation) #57 | +| rlm-context-budget | A | 2026-08-13 04:32 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-006 | +| sandbox-boot-guard-ci | A | 2026-08-13 04:32 | PASS | issue #449 (sandbox image build CI guard) 2026-06-19 | +| session-runner-ladder | A | 2026-08-13 04:32 | PASS | .oh/tasks/firstmate-executor/ (issue #746) — the shared herdr -> tmux -> foreground runner ladder and its safety gates | +| ship-spec-ready-finalization | A | 2026-08-13 04:32 | PASS | issue #134 — /ship-spec must finalize ready PRs after gates, not stop at draft scaffold | +| skill-paths | A | 2026-08-13 04:32 | PASS | issue #43 — stale path references; extended by issue #69 — apps/->packages/ rename guard | +| skills-dir-clean | A | 2026-08-13 04:32 | PASS | conversation 2026-06-29 — Pi parses every top-level `.md` in the skills | +| skills-vendored | A | 2026-08-13 04:32 | PASS | absorb .mifune submodule into .oh — the skills/agents/hooks pack is vendored | +| slack-admin-command-surface | A | 2026-08-13 04:32 | PASS | issue #354 — Slack bridge docs must distinguish Pi /msg-bridge commands from Slack DM admin text handlers | +| spec-family-contract | A | 2026-08-13 04:32 | PASS | conversation 2026-06-19 (spec-* family split, issue #265); consolidated into /spec dispatcher 2026-06-23 (one skill, args) | +| ste-checker-contract | A | 2026-08-13 04:32 | PASS | issue #750 PR audit — the /ste checker had four fail-open paths (unclosed | +| submitted-by-trailers | A | 2026-08-13 04:32 | PASS | conversation 2026-06-12 (commit attribution trailers) | +| sync-skill-contract | A | 2026-08-13 04:32 | PASS | issue #331 — /sync dispatcher skill (bidirectional origin↔upstream sync) | +| watchdog-completed-session-reap | A | 2026-08-13 04:32 | PASS | issue #235 (completed autopilot PR session reaping) | +| watchdog-draft-prs | A | 2026-08-13 04:32 | PASS | conversation 2026-06-15 (generic watchdog + stale draft PR recovery) | +| watchdog-stuck-sessions | A | 2026-08-13 04:32 | PASS | issue #240 (Codex zero-credit stuck autopilot sessions) 2026-06-17 | +| weigh-scorer-contract | A | 2026-08-13 04:32 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-003 (2026-06-27) | +| wiki-readme-index | A | 2026-08-13 04:32 | PASS | issue #132 — wiki README index drift guard | +| workflow-boundaries | A | 2026-08-13 04:32 | PASS | conversation 2026-06-19 (workflow consolidation, issue #259) | diff --git a/.oh/evals/probes/session-runner-ladder.sh b/.oh/evals/probes/session-runner-ladder.sh index 33269943..d06be676 100755 --- a/.oh/evals/probes/session-runner-ladder.sh +++ b/.oh/evals/probes/session-runner-ladder.sh @@ -132,6 +132,39 @@ probe_fn="$(fn_body runner_probe_fingerprint)" printf '%s\n' "$probe_fn" | code_only | grep -Fq 'herdr pane close' \ || missing+=("runner_probe_fingerprint: the probe pane is not closed (the gate leaks a pane per detection)") +# The probe pane must OUTLIVE its own read (#761). herdr destroys a pane as soon as its +# command returns, and a read against a destroyed pane answers pane_not_found — so a probe +# that prints one line and exits loses the race and the gate reports "no fingerprint" for an +# environment that actually matches. That failure is invisible where the environment already +# differs, which is why it survived the original build: it makes the herdr rung unreachable +# EVERYWHERE rather than only where it should be. +printf '%s\n' "$probe_fn" | code_only | grep -Fq 'runner_probe_pane_script' \ + || missing+=("runner_probe_fingerprint: the pane runs the bare fingerprint snippet — it exits before the read and the gate can never admit herdr (#761)") +grep -Fq 'RUNNER_PROBE_KEEPALIVE_SUFFIX=' "$RUNNER" \ + || missing+=("session-runner.sh: RUNNER_PROBE_KEEPALIVE_SUFFIX is gone — nothing keeps the probe pane alive across the read") +pane_script_fn="$(fn_body runner_probe_pane_script)" +printf '%s\n' "$pane_script_fn" | code_only | grep -Fq 'RUNNER_PROBE_SCRIPT' \ + || missing+=("runner_probe_pane_script: the pane snippet is not built from RUNNER_PROBE_SCRIPT — pane and caller fingerprints stop being the same snippet") +printf '%s\n' "$pane_script_fn" | code_only | grep -Fq 'RUNNER_PROBE_KEEPALIVE_SUFFIX' \ + || missing+=("runner_probe_pane_script: the keep-alive suffix is not applied to the pane snippet") + +# The keep-alive belongs to the PANE invocation only. RUNNER_PROBE_SCRIPT also runs in-process +# via runner_local_fingerprint, so a sleep folded into it would stall every caller and would +# break "the same snippet runs in the probe pane and locally". +grep -E '^RUNNER_PROBE_SCRIPT=' "$RUNNER" | grep -Fq 'sleep' \ + && missing+=("session-runner.sh: the keep-alive leaked into RUNNER_PROBE_SCRIPT — runner_local_fingerprint would sleep on every call") +printf '%s\n' "$(fn_body runner_local_fingerprint)" | code_only | grep -Fq 'runner_probe_pane_script' \ + && missing+=("runner_local_fingerprint: the caller-side fingerprint runs the keep-alive pane snippet instead of the bare one") + +# The keep-alive budget is DERIVED from the single timeout source, so the two cannot drift. +keepalive_fn="$(fn_body runner_probe_keepalive_s)" +if [ -z "$keepalive_fn" ]; then + missing+=("session-runner.sh: runner_probe_keepalive_s is missing — the keep-alive budget has no source") +else + printf '%s\n' "$keepalive_fn" | code_only | grep -Fq 'RUNNER_PROBE_TIMEOUT_MS' \ + || missing+=("runner_probe_keepalive_s: the keep-alive is not derived from RUNNER_PROBE_TIMEOUT_MS — the pane can die inside the read window") +fi + # --- (6) session budget: one source, the 14400000 default, bounded polling -- grep -Fq 'RUNNER_DEFAULT_TIMEOUT_MS=14400000' "$RUNNER" \ || missing+=("session-runner.sh: the 14400000 (4h) session-budget default literal is gone") diff --git a/.oh/scripts/__tests__/session-runner.test.ts b/.oh/scripts/__tests__/session-runner.test.ts index 1531bff5..d068ed7b 100644 --- a/.oh/scripts/__tests__/session-runner.test.ts +++ b/.oh/scripts/__tests__/session-runner.test.ts @@ -119,7 +119,21 @@ case "$sub" in pane) verb="\${1:-}"; shift || true case "$verb" in - read) printf '%s\\n' "\${STUB_HERDR_PROBE_OUT:-}" ;; + read) + # Model herdr's real pane lifetime. herdr destroys a pane the instant + # its command returns, and a read against a destroyed pane answers + # pane_not_found — which is the whole of #761. A stub that always + # replays the probe output is MORE FORGIVING than herdr, and that is + # why 46 tests passed while the live gate could never admit herdr. + # A pane is readable here only if its start invocation carried a + # keep-alive, so dropping the keep-alive in production fails the suite. + if [ "\${STUB_HERDR_PANE_MODEL:-lifetime}" = "lifetime" ] && + ! grep -q '^herdr agent start .*sleep' "\${STUB_CALLS:-/dev/null}" 2>/dev/null; then + printf '{"code":"pane_not_found","message":"pane not found"}\\n' >&2 + exit 1 + fi + printf '%s\\n' "\${STUB_HERDR_PROBE_OUT:-}" + ;; list) printf '{"id":"cli:pane:list","result":{"panes":[{"pane_id":"%s","foreground_cwd":"%s","cwd":"%s"}],"type":"pane_list"}}\\n' \\ "\${STUB_HERDR_PANE_ID:-w7:p3}" "\${STUB_HERDR_FG_CWD:-/w}" "\${STUB_HERDR_FG_CWD:-/w}" @@ -426,6 +440,70 @@ describe("runner_detect ladder", () => { expect(readFileSync(t.callsFile, "utf-8")).toMatch(/herdr pane close w7:p3/); }); + // --- #761: the probe pane must outlive its own read ---------------------- + + it("keeps the probe pane alive across the read, and derives the budget from the one timeout source", () => { + const t = makeTask("ladder"); + const bin = makeBin({ herdr: true, tmux: true, isolated: true }); + sh(`runner_detect ladder '${t.worktree}'`, { + env: detectEnv(t, bin, { + STUB_HERDR_PROBE_OUT: `FIRSTMATE-FINGERPRINT ${callerFingerprint(t.worktree)}`, + RUNNER_PROBE_TIMEOUT_MS: "20000", + }), + }); + const start = readFileSync(t.callsFile, "utf-8") + .split("\n") + .find((l) => l.includes("agent start")); + // The keep-alive rides the pane invocation ... + expect(start).toContain('sleep "${2:-30}"'); + // ... and its budget is the read window plus a margin, not a literal. + expect(start).toMatch(/\s25$/); + }); + + it("reproduces #761: a probe pane that exits before the read is unreadable", () => { + const t = makeTask("ladder"); + const bin = makeBin({ herdr: true, tmux: true, isolated: true }); + // STUB_HERDR_PANE_MODEL=none restores the old, too-forgiving stub: it + // replays pane output regardless of whether the pane could still exist. + // Under the faithful default the same run must still succeed, which is + // what proves the keep-alive is load-bearing rather than decorative. + const forgiving = sh(`runner_detect ladder '${t.worktree}'`, { + env: detectEnv(t, bin, { + STUB_HERDR_PROBE_OUT: `FIRSTMATE-FINGERPRINT ${callerFingerprint(t.worktree)}`, + STUB_HERDR_PANE_MODEL: "none", + }), + }); + expect(forgiving.stdout.trim()).toBe("herdr"); + + const t2 = makeTask("ladder"); + const faithful = sh(`runner_detect ladder '${t2.worktree}'`, { + env: detectEnv(t2, bin, { + STUB_HERDR_PROBE_OUT: `FIRSTMATE-FINGERPRINT ${callerFingerprint(t2.worktree)}`, + }), + }); + expect(faithful.stdout.trim()).toBe("herdr"); + }); + + it("never puts the keep-alive on the LOCAL fingerprint path", () => { + const t = makeTask("ladder"); + const bin = makeBin({ herdr: true, tmux: true, isolated: true }); + // The shared snippet stays sleep-free: runner_local_fingerprint runs it + // in-process, so a keep-alive there would stall every caller and would + // also break "the same snippet runs in both places". + const r = sh(`printf '%s' "$RUNNER_PROBE_SCRIPT"`, { + env: detectEnv(t, bin), + }); + expect(r.stdout).not.toContain("sleep"); + expect(r.stdout).toContain("FIRSTMATE-FINGERPRINT"); + + const started = Date.now(); + const local = sh(`runner_local_fingerprint '${t.worktree}'`, { + env: detectEnv(t, bin), + }); + expect(local.stdout).toContain("host="); + expect(Date.now() - started).toBeLessThan(5000); + }); + it("degrades to tmux when the probe pane yields no fingerprint at all", () => { const t = makeTask("ladder"); const bin = makeBin({ herdr: true, tmux: true, isolated: true }); diff --git a/.oh/scripts/lib/session-runner.sh b/.oh/scripts/lib/session-runner.sh index 3cfeac9f..1f0f3494 100644 --- a/.oh/scripts/lib/session-runner.sh +++ b/.oh/scripts/lib/session-runner.sh @@ -103,21 +103,36 @@ # only list/get/read/send/rename/focus/wait/start/attach/explain. Teardown # is `herdr pane close `; a stop/kill verb would fail silently # inside a trap. -# * herdr panes may be HOST processes. Mechanism, confirmed 2026-08-12: the -# operator's config directory is bind-mounted read-write into this -# container, so the container's herdr CLI reads the HOST operator's herdr -# config — socket path and server address included — connects to the HOST's -# herdr server, and panes spawn outside the sandbox. That is why herdr -# eligibility carries the execution-context gate below. -# -# This is a deployment defect, not a property of herdr. It is one of two +# * herdr panes may be HOST processes. This was true of the deployment that +# built the gate: the operator's config directory was bind-mounted +# read-write into the container, so the container's herdr CLI read the HOST +# operator's herdr config — socket path and server address included — +# connected to the HOST's herdr server, and panes spawned outside the +# sandbox. A deployment defect, not a property of herdr; one of two # host-root escape paths tracked under EPIC #731 as issue #756. # -# MIGRATION TRIGGER: when #756 closes, the container's herdr CLI reaches an -# in-container server and the gate stops rejecting herdr by itself. Do NOT -# delete the gate then — it is a build-correctness guard, and its job is to -# prove environment identity rather than to assume it. Re-verify with a live -# probe pane and update these notes with the observed result. +# MIGRATION RESULT, observed 2026-08-13 after #756 closed. The trigger this +# note used to carry has been discharged; recording what the live re-probe +# actually showed rather than what it predicted: +# +# - The config bind is gone. The herdr CLI now reaches an in-container +# server, `herdr agent start --cwd ` is honoured +# instead of ignored, and the probe pane returns the caller's own +# environment. Caller and probe both read +# `host= docker=yes worktree=yes`. +# - So the fingerprint comparison now PASSES where it used to fail. The +# gate stays regardless: it is a build-correctness guard whose job is to +# PROVE environment identity, not to encode one deployment's topology. +# - The prediction that the gate "stops rejecting herdr by itself" was +# WRONG, and closing #756 is what exposed why. A second defect had been +# masked by the first: the probe pane exited before its own read, so no +# fingerprint could ever be obtained and herdr was refused everywhere, +# including in a correct environment. Fixed in #761 by the keep-alive +# below — see RUNNER_PROBE_KEEPALIVE_SUFFIX. +# +# The lesson worth keeping: a gate that is failing for one reason can hide a +# second reason it would fail anyway. Removing the cause does not prove the +# gate works — only re-probing does. # # --------------------------------------------------------------------------- # Deliberate deviations from the PRD sketch @@ -160,6 +175,25 @@ RUNNER_FINGERPRINT_MARKER='FIRSTMATE-FINGERPRINT' # evaluated inside the probe pane / a child shell, not here. RUNNER_PROBE_SCRIPT='wt="$1"; h="$(hostname 2>/dev/null || uname -n 2>/dev/null || echo unknown)"; d=no; [ -e /.dockerenv ] && d=yes; w=no; [ -d "$wt" ] && w=yes; printf "FIRSTMATE-FINGERPRINT host=%s docker=%s worktree=%s\n" "$h" "$d" "$w"' +# The probe pane must OUTLIVE the read. herdr destroys a pane the moment its +# command returns, and `herdr pane read` on a destroyed pane answers +# `{"code":"pane_not_found"}` — so a probe that prints one line and exits races +# its own reader and loses. The gate then reports "no fingerprint obtained" for +# an environment that in fact matches, which makes the herdr rung unreachable +# everywhere rather than only where the environment differs (#761). +# +# The keep-alive is a SUFFIX applied only to the pane invocation. It is never +# folded into RUNNER_PROBE_SCRIPT, because that snippet also runs locally in +# runner_local_fingerprint, and "the same snippet runs in the probe pane and +# locally" is what makes the comparison true by construction rather than by +# convention. A sleeping local fingerprint would also stall every caller. +# +# $2 is the keep-alive budget in seconds, passed positionally by +# runner_probe_fingerprint. It is an upper bound, not a cost: the gate closes +# the pane as soon as the read completes, on the match and the mismatch path +# alike, so the sleep is cut short in every normal run. +RUNNER_PROBE_KEEPALIVE_SUFFIX='; sleep "${2:-30}"' + # Mutable state. Declared here so a caller running under `set -u` can read them # before the first launch. RUNNER_PANE_ID="${RUNNER_PANE_ID:-}" @@ -292,6 +326,25 @@ runner_local_fingerprint() { # bash -lc "$RUNNER_PROBE_SCRIPT" firstmate-probe "${1:-}" 2>/dev/null | runner_extract_fingerprint } +# Seconds the probe pane must stay alive: the read window plus a margin. Derived +# from the one timeout source so the two can never drift apart. Rejects empty, +# non-numeric and absurdly small values back to a usable floor. +runner_probe_keepalive_s() { + local ms="${RUNNER_PROBE_TIMEOUT_MS:-15000}" s + case "$ms" in '' | *[!0-9]*) ms=15000 ;; esac + s=$((ms / 1000 + 5)) + [ "$s" -lt 5 ] && s=5 + printf '%s\n' "$s" +} + +# The snippet the probe PANE runs: the shared fingerprint script plus the +# keep-alive. Split out so both the tests and session-runner-ladder.sh can +# assert on the composition instead of re-deriving it. +runner_probe_pane_script() { + local composed="${RUNNER_PROBE_SCRIPT}${RUNNER_PROBE_KEEPALIVE_SUFFIX}" + printf '%s' "$composed" +} + # Names the fields that differ between two fingerprints, so the logged degrade # reason says WHICH field disagreed rather than only that something did. runner_fingerprint_diff() { # @@ -307,18 +360,20 @@ runner_fingerprint_diff() { # printf '%s\n' "${diff:-unknown}" } -# Launches a SHORT-LIVED probe pane, reads its environment fingerprint back, -# and closes the pane again on BOTH verdicts. Echoes the probe's fingerprint; -# returns non-zero when no fingerprint could be obtained. +# Launches a probe pane, reads its environment fingerprint back, and closes the +# pane again on BOTH verdicts. Echoes the probe's fingerprint; returns non-zero +# when no fingerprint could be obtained. The pane is kept alive across the read +# rather than being allowed to exit on its own — see RUNNER_PROBE_KEEPALIVE_SUFFIX. runner_probe_fingerprint() { # local - set -o pipefail local slug="${1:-}" worktree="${2:-}" local probe_name="firstmate-probe-$slug-$$" - local start_json pane_id probe_out fingerprint + local start_json pane_id probe_out fingerprint keepalive_s + keepalive_s="$(runner_probe_keepalive_s)" start_json="$(herdr agent start "$probe_name" --cwd "$worktree" --no-focus \ - -- bash -lc "$RUNNER_PROBE_SCRIPT" firstmate-probe "$worktree" 2>/dev/null)" || start_json="" + -- bash -lc "$(runner_probe_pane_script)" firstmate-probe "$worktree" "$keepalive_s" 2>/dev/null)" || start_json="" pane_id="$(runner_parse_pane_id "$start_json")" if [ -z "$pane_id" ]; then diff --git a/.oh/skills/firstmate/SKILL.md b/.oh/skills/firstmate/SKILL.md index 27a2009e..b8c225a5 100644 --- a/.oh/skills/firstmate/SKILL.md +++ b/.oh/skills/firstmate/SKILL.md @@ -133,18 +133,32 @@ runner is never a silent regression to the ralph loop. 2. `herdr status` shows **both literal fields** `status: running` **and** `compatible: yes`. There is no single "healthy" flag; these two literals are the entire health predicate. Binary-up/server-down degrades to tmux. -3. **Execution-context gate.** A short-lived probe pane emits an environment - fingerprint (hostname, presence of `/.dockerenv`, whether the target worktree - path resolves in that pane) and it is compared against the caller's own +3. **Execution-context gate.** A probe pane emits an environment fingerprint + (hostname, presence of `/.dockerenv`, whether the target worktree path + resolves in that pane) and it is compared against the caller's own fingerprint gathered the same way. **Any mismatch ⇒ herdr is ineligible**: the ladder degrades to tmux and the reason — both fingerprints and which field differed — is written to the firstmate log. The gate closes its own probe pane with `herdr pane close ` on both verdicts. -> **In this deployment the gate refuses herdr.** herdr panes are **host** -> processes while the harness runs **inside the container**, so the fingerprints -> differ and the ladder degrades to tmux. `AGENTS.md` requires all building and -> testing inside the sandbox, so this is the correct outcome, not a defect. + The probe pane carries a **keep-alive** so it outlives its own read. herdr + destroys a pane the moment its command returns, and a read against a + destroyed pane answers `pane_not_found` — so a probe that prints one line and + exits loses the race, and the gate reports "no fingerprint" for an + environment that actually matches (#761). The keep-alive is applied only to + the pane invocation, never to the shared fingerprint snippet, which also runs + in-process for the caller side. Its budget derives from + `RUNNER_PROBE_TIMEOUT_MS`, and it is an upper bound rather than a cost: the + gate closes the pane as soon as the read completes. + +> **Whether the gate admits herdr is a property of the deployment.** It refuses +> whenever the probe pane cannot be shown to run in the caller's environment — +> which was the case while the operator config bind (#756) made the container's +> herdr CLI drive the HOST server. After #756 closed, a live re-probe measured +> caller and probe as identical. `AGENTS.md` requires all building and testing +> inside the sandbox, so a refusal is the correct outcome, never a defect — but +> a refusal that fires in a *matching* environment is one, which is what #761 +> fixed. > Standing up an in-environment herdr server is a separate decision. An explicit `OH_RUNNER=` / `--runner ` naming an unavailable runner is a diff --git a/CHANGELOG.md b/CHANGELOG.md index 24048b22..d7d98d8e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,7 @@ Update policy and release automation live in [`/git`](.claude/skills/git/SKILL.m ## [Unreleased] ### Fixed +- Keep the firstmate execution-context probe pane alive across its own read, so the gate can admit herdr in an environment that matches. herdr destroys a pane the instant its command returns and answers `pane_not_found` on a read against a destroyed pane, so a probe that printed one fingerprint line and exited lost the race with its own reader. The gate then reported "no probe fingerprint obtained" and refused herdr in **every** environment, not only in a mismatched one — a permanent refusal wearing the shape of a proof obligation. The defect was invisible until the operator config bind ([#756](https://github.com/mifunedev/openharness/issues/756)) closed, because until then the gate failed earlier and correctly on a genuine host-vs-container fingerprint mismatch. A keep-alive suffix now rides the pane invocation only; `RUNNER_PROBE_SCRIPT` stays byte-identical between the pane and the in-process caller-side call, so "the same snippet runs in both places" remains true by construction and `runner_local_fingerprint` never sleeps. The budget derives from the single `RUNNER_PROBE_TIMEOUT_MS` source and is an upper bound rather than a cost, since the gate closes the pane as soon as the read completes on either verdict. The unit-test herdr stub was more forgiving than herdr — it replayed pane output regardless of pane lifetime, which is why 46 tests passed against a gate that could never succeed live — and now models the real lifetime, so removing the keep-alive fails five tests including three that predate this change. `session-runner-ladder.sh` gains five assertions covering the composition, the derived budget, and the leak of a keep-alive into the shared snippet; each was verified by rejection against a deliberately broken copy ([#761](https://github.com/mifunedev/openharness/issues/761)). - Repair seven entries in `.claude/protected-paths.txt` that resolved to nothing, and add `.oh/evals/probes/protected-paths-resolve.sh` so a rename cannot silently disarm a guard again. A guard entry that matches nothing protects nothing **and reads identical to one that passes**, which is why all seven survived since 2026-05-03. `cloudflared-tunnel` was listed while the skill directory is `.oh/skills/cloudflared`, so `/cloudflared` had no protection at all. Four `spec-*` entries had never been directories — the `/spec` dispatcher implements them as `.oh/skills/spec/references/{plan,critique,execute,retro}.md`, which now carry the protection. `.oh/install/cloudflared-tunnel.sh` was deleted at some point and is replaced by `.oh/skills/cloudflared/scripts/run.sh`. `.claude/specs/structure-spec-v0.7.md` sat under a path `.gitignore:66` excludes, so it could never resolve and is removed. The probe parses the file's documented format — bare names are skills, everything else is a repo-relative path, with a slash-free entry allowed to resolve either way so the root `Makefile` entry stays valid — and was verified by rejection against all four original defect shapes ([#753](https://github.com/mifunedev/openharness/issues/753)). ### Security