From 2720cd54ca1e65465751d0d9a08b0aee48743c40 Mon Sep 17 00:00:00 2001 From: Ryan Eggleston Date: Sat, 29 Aug 2026 14:56:47 -0600 Subject: [PATCH 01/23] fix(evals): record the source lesson on five headerless probes (#892) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The runner extracts `# tier:`, `# source:`, and `# desc:` with a fixed grep, so a probe missing `# source:` leaves that column empty in RESULTS.md and loses the only record of which lesson it closes. Rebased across the one-door migration, which moved the target set: the header written for `env-schema-parity.sh` now belongs to its successor `config-schema-parity.sh`, `oh-init-headless-config.sh` picks up the retarget to `oh.json`, and `oh-config-surfaces.sh` — added by #887 after this branch opened — was the fifth probe still missing one. Closes #889 --- .oh/evals/probes/compose-config-path-parity.sh | 1 + .oh/evals/probes/config-schema-parity.sh | 1 + .oh/evals/probes/harness-yaml-migration.sh | 1 + .oh/evals/probes/oh-config-surfaces.sh | 1 + .oh/evals/probes/oh-init-headless-config.sh | 1 + CHANGELOG.md | 3 +++ 6 files changed, 8 insertions(+) diff --git a/.oh/evals/probes/compose-config-path-parity.sh b/.oh/evals/probes/compose-config-path-parity.sh index 729defb5..5190495d 100755 --- a/.oh/evals/probes/compose-config-path-parity.sh +++ b/.oh/evals/probes/compose-config-path-parity.sh @@ -1,5 +1,6 @@ #!/usr/bin/env bash # tier: A +# source: PR #833 (remove harness.yaml — the wrapper and VS Code "Reopen in Container" paths must resolve the same service) 2026-08-26 # desc: the wrapper path and the VS Code "Reopen in Container" path resolve the same service — the parity harness.yaml made impossible set -euo pipefail diff --git a/.oh/evals/probes/config-schema-parity.sh b/.oh/evals/probes/config-schema-parity.sh index 53557fb9..2808512e 100755 --- a/.oh/evals/probes/config-schema-parity.sh +++ b/.oh/evals/probes/config-schema-parity.sh @@ -1,5 +1,6 @@ #!/usr/bin/env bash # tier: A +# source: PR #833 (one schema file — DOCKER_SOCKET, SANDBOX_SSH, OH_SANDBOX_IMAGE, OH_PULL_POLICY, SKIP_PNPM_INSTALL were consumed but undocumented); rewritten for the oh.json/secrets split by PR #887 # desc: the oh.json/.env split loses no variable — every compose-interpolated var is either a documented oh.json field or an allow-listed secret, and neither surface holds the other's keys set -euo pipefail diff --git a/.oh/evals/probes/harness-yaml-migration.sh b/.oh/evals/probes/harness-yaml-migration.sh index f3a53991..e8f209fc 100755 --- a/.oh/evals/probes/harness-yaml-migration.sh +++ b/.oh/evals/probes/harness-yaml-migration.sh @@ -1,5 +1,6 @@ #!/usr/bin/env bash # tier: A +# source: PR #833 (migrate-harness-yaml.sh — append / uncomment-in-place / preserve / overwrite, plus a silent no-op second run) 2026-08-26 # desc: migrate-harness-yaml.sh carries a live harness.yaml into the oh.json/root-dotenv split — non-secret settings land as oh.json fields (never .devcontainer/.env or the retired .oh/config.json), the file is renamed, and the second run is a no-op set -euo pipefail diff --git a/.oh/evals/probes/oh-config-surfaces.sh b/.oh/evals/probes/oh-config-surfaces.sh index 630be9e7..3dbf9ae4 100755 --- a/.oh/evals/probes/oh-config-surfaces.sh +++ b/.oh/evals/probes/oh-config-surfaces.sh @@ -1,5 +1,6 @@ #!/usr/bin/env bash # tier: A +# source: PR #887 (config split across two authored surfaces — a tracked oh.json and a secrets-only root dotenv — with nothing left under $HOME) # desc: the two authored config surfaces stay honest — tracked oh.json holds no allow-listed secret, the root dotenv is gitignored/0600 and holds nothing but allow-listed secrets, .devcontainer/.env is a symlink to ../.env, no live file still depends on the retired .devcontainer/.example.env, and no CLI source relocates config into $HOME set -euo pipefail diff --git a/.oh/evals/probes/oh-init-headless-config.sh b/.oh/evals/probes/oh-init-headless-config.sh index b813db9b..4a37e4a6 100755 --- a/.oh/evals/probes/oh-init-headless-config.sh +++ b/.oh/evals/probes/oh-init-headless-config.sh @@ -1,5 +1,6 @@ #!/usr/bin/env bash # tier: A +# source: PR #827 (installer answers landed in the losing config file); retargeted to the .example.env template by PR #833, then to oh.json by PR #887 # desc: `oh init --yes` provisions headlessly — zero prompts, a default oh.json, a root .env.example byte-identical to the tracked template, and no secrets dotenv set -euo pipefail diff --git a/CHANGELOG.md b/CHANGELOG.md index 1637306f..9a3cee6b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,9 @@ Update policy and release automation live in [`/git`](.claude/skills/git/SKILL.m ## [Unreleased] +### Fixed +- Give the five probes that shipped without one a `# source:` header, so every probe records the lesson it closes and the `source` column in `RESULTS.md` is fully populated ([#889](https://github.com/mifunedev/openharness/issues/889)). + ## [0.5.1] - 2026-08-29 ### Fixed From ad4d0350b888f110a7772a93b5a04d98622f9d86 Mon Sep 17 00:00:00 2001 From: Ryan Eggleston Date: Sat, 29 Aug 2026 15:00:24 -0600 Subject: [PATCH 02/23] fix(delegate): track the wave graph in the .oh/tasks ledger (#888) /delegate steps 4 and 5b instructed TaskCreate/TaskUpdate from .oh/skills/, the canonical pack symlinked into .claude, .codex and .pi. Codex and Pi never had those tools, and Claude Code 2.1.233 stopped providing them by default on current models, so the step was a silent no-op: the wave graph degraded to prose held in a context window that /spec execute later compacts. Steps 4 and 5b now write delegate-graph.json and delegate-log.txt under .oh/tasks//, beside but never overwriting the Advisor-owned prd.json and progress.txt, and a resumed run re-runs only unfinished tasks. The fix is provider-neutral and needs no environment flag. Add skills-task-tool-coupling.sh, a two-sided tier-A invariant: a canonical skill may instruct the task tools only while the sandbox enables them, and the sandbox may enable them only while a skill needs them. Both failure directions were exercised before commit. Also drops a dangling 'skip to Step 9' in the rewritten block; the skill has eight steps. Closes #886 --- .oh/evals/probes/skills-task-tool-coupling.sh | 57 +++++++++++++++++++ .oh/skills/delegate/SKILL.md | 40 +++++++++++-- CHANGELOG.md | 4 ++ 3 files changed, 96 insertions(+), 5 deletions(-) create mode 100755 .oh/evals/probes/skills-task-tool-coupling.sh diff --git a/.oh/evals/probes/skills-task-tool-coupling.sh b/.oh/evals/probes/skills-task-tool-coupling.sh new file mode 100755 index 00000000..ee3158fd --- /dev/null +++ b/.oh/evals/probes/skills-task-tool-coupling.sh @@ -0,0 +1,57 @@ +#!/usr/bin/env bash +# tier: A +# source: council review 2026-08-29 (issue #886) — /delegate instructed Claude-Code-only +# TaskCreate/TaskUpdate from .oh/skills/, the canonical pack symlinked into +# .claude, .codex and .pi. Codex and Pi never had those tools, and Claude Code +# 2.1.233 stopped providing them by default, so the step became a silent no-op. +# desc: the canonical skill pack and the sandbox agree about the Claude-Code-only task +# tools — a skill may instruct them only while the sandbox enables them, and the +# sandbox may enable them only while some skill needs them +set -euo pipefail + +PROBE_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +ROOT="$(cd "$PROBE_DIR" && git rev-parse --show-toplevel 2>/dev/null)" \ + || ROOT="$(cd "$PROBE_DIR/../../.." && pwd)" + +SKILLS="$ROOT/.oh/skills" +TOOLS='TodoWrite|TaskCreate|TaskGet|TaskUpdate|TaskList' + +if [[ ! -d "$SKILLS" ]]; then + echo "SKIPPED: .oh/skills/ absent on this branch" >&2 + exit 2 +fi + +mapfile -t consumers < <( + grep -rlE "\`($TOOLS)\`" "$SKILLS" 2>/dev/null | sed "s|^$ROOT/||" | sort +) + +enabled_in=() +for f in "$ROOT/.devcontainer/docker-compose.yml" \ + "$ROOT/.devcontainer/docker-compose.image-only.yml" \ + "$ROOT/.claude/settings.json" \ + "$ROOT/.oh/templates/full/.claude/settings.json"; do + [[ -f "$f" ]] || continue + grep -qE 'CLAUDE_CODE_ENABLE_TODO_TOOLS' "$f" && enabled_in+=("${f#"$ROOT"/}") +done + +if (( ${#consumers[@]} > 0 && ${#enabled_in[@]} == 0 )); then + echo "REGRESSION: a canonical skill instructs Claude-Code-only task tools that the sandbox does not enable:" >&2 + printf ' - %s\n' "${consumers[@]}" >&2 + echo " Those tools are absent on Codex and Pi, and on Claude Code 2.1.233+ by default." >&2 + echo " Fix the skill to use the .oh/tasks/ ledger, or enable CLAUDE_CODE_ENABLE_TODO_TOOLS=1 in the sandbox." >&2 + exit 1 +fi + +if (( ${#consumers[@]} == 0 && ${#enabled_in[@]} > 0 )); then + echo "REGRESSION: the sandbox enables CLAUDE_CODE_ENABLE_TODO_TOOLS but no canonical skill needs it:" >&2 + printf ' - %s\n' "${enabled_in[@]}" >&2 + echo " Dead configuration — drop it, or point at the skill that depends on the tools." >&2 + exit 1 +fi + +if (( ${#consumers[@]} == 0 )); then + echo "PASS: no canonical skill depends on the Claude-Code-only task tools, and the sandbox does not enable them" >&2 +else + echo "PASS: ${#consumers[@]} skill(s) instruct the task tools and the sandbox enables them (${enabled_in[*]})" >&2 +fi +exit 0 diff --git a/.oh/skills/delegate/SKILL.md b/.oh/skills/delegate/SKILL.md index be8845fa..858dd862 100644 --- a/.oh/skills/delegate/SKILL.md +++ b/.oh/skills/delegate/SKILL.md @@ -45,7 +45,7 @@ flowchart TD B -->|Yes| C["Step 2: Deep-think task decomposition"] C --> D["Step 3: Build dependency graph"] - D --> E["Step 4: Create tasks + compute waves"] + D --> E["Step 4: Write run ledger to .oh/tasks/"] E --> F{--dry-run?} F -->|Yes| DRY["Report: task graph + wave plan"] DRY --> MEM_DRY[Memory Protocol] @@ -126,11 +126,39 @@ Output the wave plan: - No circular dependencies (if found, report error and stop) - Max 5 concurrent agents per wave (split larger waves into sub-waves) -### 4. Create tasks and track dependencies +### 4. Write the run ledger -Use `TaskCreate` for each task. Then use `TaskUpdate` with `addBlockedBy` to wire dependencies. +The task graph is durable state, not conversation state. A delegation outlives a +context window: `/spec execute` compacts mid-build, sessions die, and another agent +can pick up the worktree. Write the graph to disk before spawning any worker. -If `--dry-run`, output the full task graph and wave plan, then skip to **Step 9**. +**Resolve the run directory** as `.oh/tasks//`: + +- Invoked inside a `/spec execute` task (a `--plan` path under `.oh/tasks//`, + or that folder is the current task): reuse that ``. +- Otherwise: `delegate--`, created if absent. + +**Write two files, both owned by this skill:** + +| File | Contents | +|------|----------| +| `delegate-graph.json` | Every task's ID, title, description, `dependsOn`, files, complexity, model override plus its reason, thinking level, acceptance criteria, assigned wave, and `status` (`pending`/`running`/`completed`/`FAIL`/`BLOCKED`) | +| `delegate-log.txt` | Append-only run log; one line per wave boundary and per status change | + +Never write `prd.json` or `progress.txt`. Those belong to the Advisor +(`.oh/tasks/README.md`), and `progress.txt` in particular must not be edited by hand. +This skill's two files sit beside them without collision. + +Both live under `.oh/tasks/`, which is gitignored — that is correct for run state. +Stage them with `git add -f` only when a PR must carry the delegation as evidence. + +**Resume rather than restart.** If `delegate-graph.json` already exists in the resolved +directory, read it first. Re-run only tasks whose status is `pending`, `FAIL`, or +`BLOCKED`; treat `completed` tasks as done and pass their summaries forward as +prior-wave context. A resumed run appends to `delegate-log.txt`; it never truncates it. + +If `--dry-run`, write neither file — output the full task graph and wave plan, then +skip to **Step 7**. ### 5. Execute waves @@ -169,7 +197,9 @@ If any field is missing, either add it or downgrade the task to flat execution ( **b) Collect results** -After all agents in the wave complete, update each task via `TaskUpdate`: +After all agents in the wave complete, set each task's `status` and `summary` in +`delegate-graph.json`, then append the wave's outcome to `delegate-log.txt`. Write both +before spawning the next wave — a crash between waves must leave the graph readable. | Task | Status | Summary | Files Changed | |------|--------|---------|---------------| diff --git a/CHANGELOG.md b/CHANGELOG.md index 9a3cee6b..f79b6f0f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,8 +8,12 @@ Update policy and release automation live in [`/git`](.claude/skills/git/SKILL.m ## [Unreleased] +### Added +- Add `skills-task-tool-coupling.sh`, a tier-A probe holding the canonical skill pack and the sandbox in agreement about the Claude-Code-only task tools ([#886](https://github.com/mifunedev/openharness/issues/886)). + ### Fixed - Give the five probes that shipped without one a `# source:` header, so every probe records the lesson it closes and the `source` column in `RESULTS.md` is fully populated ([#889](https://github.com/mifunedev/openharness/issues/889)). +- `/delegate` no longer instructs the Claude-Code-only `TaskCreate`/`TaskUpdate` from the provider-shared skill pack; its wave graph persists to a `.oh/tasks//` run ledger ([#886](https://github.com/mifunedev/openharness/issues/886)). ## [0.5.1] - 2026-08-29 From 9bc367ecb991313e1ac4986688aac003e399f319 Mon Sep 17 00:00:00 2001 From: Ryan Eggleston Date: Sun, 30 Aug 2026 18:48:29 -0600 Subject: [PATCH 03/23] FROM task/898-single-home-mount TO development (#899) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * task: collapse sandbox storage to one $HOME mount Replace the eleven per-tool named volumes with a single mount at /home/sandbox, selectable as a Docker-managed volume (_workspace) or a host bind via storage.homePath in oh.json. The image's home is moved to /opt/home-seed at build time so an empty named volume never auto-copies; entrypoint's seed_home restores it into either mount kind with cp -a -n, which also backfills new dotfiles on an image upgrade without clobbering operator state. Retire the projectRoot/OH_PROJECT_ROOT config knob: the checkout is fixed at /home/sandbox/harness, nested inside the home mount. Replace the UID-sync 'find -xdev' with an explicit $OH_PROJECT_ROOT prune. -xdev only pruned the checkout because /home/sandbox was the container overlay; once it is a mount they share a device. * task: retarget the herdr image-docs persistence assertion at the single home mount * task: address audit findings on the single-$HOME-mount change seed_home no longer uses 'cp -a -n'. That form skips existing regular files but still rewrites the mode, owner and mtime of directories the destination already has, so every boot relaxed ~/.ssh from 0700 to the image's 0755 and reset the other pre-existing home directories likewise. It now walks the seed and copies only entries the mount lacks, creating a missing directory with the seed's own metadata and never touching an existing path. It also returns non-zero on a real copy failure instead of swallowing it, and the caller warns. oh destroy branches on storage.homePath rather than appending a correction: with a host bind, nothing is deleted and sign-in does not start over, so the volume list and that warning are wrong to print. The volume list is now project-prefixed to match 'docker volume ls'. Reject reserved host paths for storage.homePath. The ownership repair is now an unbounded recursive chown over the mount, so '/', '/home' or '/usr' would hand the sandbox uid a host tree. Fix the migration recipe: 'docker cp :/home/sandbox ' lands the copy at /sandbox when exists. Use the trailing '/.' form and drop the checkout copy that docker cp reads through the bind mount. Document that storage.homePath is invisible to VS Code 'Reopen in Container', which would otherwise give a second, silently diverging home. Harden oh-home-mount.sh with the directory-metadata and copy-failure cases that let the original bug through, plus the missing rm -rf assertion; scope the -xdev ban to lines mentioning /home/sandbox. Refresh the stale project-root seam section in .oh/README.md, which still described the removed knob and cited the deleted probe. * task: drop local agent memory and cron runtime state committed by mistake * task: seed the home mount per top-level entry, not per file A live boot on a real image caught the previous form: walking the whole seed tree and copying file by file spends its time in the uv cache under ~/.cache/uv, and it aborted partway through with EEXIST, leaving a home with four of sixteen entries. seed_home now copies each top-level entry the mount lacks with a single 'cp -a', and never touches one it already has. Verified on a live boot: a clean volume gets all sixteen entries in 7s, ~/.ssh ends at 0700, and an operator edit to ~/.zshrc plus a 0700 ~/.ssh both survive a restart. The trade is explicit and documented: a new file inside a directory the mount already has is not merged in. That matches what the per-tool volumes did, while a new top-level entry from an image upgrade still arrives. oh-home-mount.sh asserts the contract in both directions and now rejects the two wrong implementations - 'cp -a -n' and the whole-tree walk - rather than only the first. Bake ~/.ssh at 0700 in the image so the seeded mode is right before the entrypoint's chmod rather than because of it. * task: ignore per-session agent memory and cron runtime state .oh/memory/ and .oh/crons/ hold local agent scratch — session logs, prompt-miner dumps, a cron pid and log. Nothing in either is tracked on any branch, but neither was ignored, so a `git add -A` sweeps 37 files of operator session state into the commit. It happened twice while building this branch. Ignore both, following the wiki-corpus convention already in this file: local by default, `git add -f` for anything worth keeping. --- .devcontainer/Dockerfile | 12 +- .devcontainer/docker-compose.image-only.yml | 45 +--- .devcontainer/docker-compose.yml | 58 ++---- .devcontainer/entrypoint.sh | 60 +++--- .gitignore | 6 + .oh/README.md | 14 +- .oh/cli/src/__tests__/destroy.test.ts | 2 +- .oh/cli/src/commands/init.ts | 17 +- .oh/cli/src/commands/lifecycle.ts | 33 ++- .../src/lib/__tests__/config-render.test.ts | 5 +- .oh/cli/src/lib/__tests__/oh-config.test.ts | 11 +- .oh/cli/src/lib/config-render.ts | 4 +- .oh/cli/src/lib/oh-config.ts | 49 ++++- .oh/evals/RESULTS.md | 193 +++++++++--------- .oh/evals/probes/oh-home-mount.sh | 142 +++++++++++++ .oh/evals/probes/oh-image-only-deploy.sh | 14 +- .oh/evals/probes/project-root-seam.sh | 48 ----- .oh/evals/probes/worktrees-layout.sh | 4 +- .oh/scripts/__tests__/entrypoint.test.ts | 41 ++-- .oh/scripts/__tests__/herdr-default.test.ts | 18 +- .../__tests__/provision-python.test.ts | 5 +- .../__tests__/sandbox-healthcheck.test.ts | 2 +- CHANGELOG.md | 7 + docs/configuration.md | 2 +- docs/deployment-prebuilt-image.md | 21 +- docs/harnesses/claude-code.md | 2 +- docs/harnesses/deepagents.md | 4 +- docs/harnesses/grok-build.md | 6 +- docs/harnesses/hermes.md | 2 +- docs/installation.md | 83 ++++++-- docs/integrations/github.md | 2 +- docs/integrations/herdr.md | 6 +- docs/integrations/langfuse.md | 2 +- docs/lifecycle-commands.md | 16 +- docs/security-considerations.md | 2 +- oh.json | 2 +- 36 files changed, 581 insertions(+), 359 deletions(-) create mode 100755 .oh/evals/probes/oh-home-mount.sh delete mode 100755 .oh/evals/probes/project-root-seam.sh diff --git a/.devcontainer/Dockerfile b/.devcontainer/Dockerfile index 694576aa..af7babe0 100644 --- a/.devcontainer/Dockerfile +++ b/.devcontainer/Dockerfile @@ -1,7 +1,6 @@ FROM node:22-trixie-slim -ARG OH_PROJECT_ROOT=/home/sandbox/harness -ENV OH_PROJECT_ROOT=${OH_PROJECT_ROOT} +ENV OH_PROJECT_ROOT=/home/sandbox/harness ENV TZ=America/Denver @@ -127,11 +126,14 @@ ENV UV_TOOL_DIR=/home/sandbox/.local/share/uv/tools ENV UV_TOOL_BIN_DIR=/home/sandbox/.local/bin ENV UV_PYTHON_INSTALL_DIR=/home/sandbox/.local/share/uv/python ENV UV_CACHE_DIR=/home/sandbox/.cache/uv -RUN install -d -o sandbox -g sandbox \ +RUN install -d -o sandbox -g sandbox -m 0700 /home/sandbox/.ssh \ + && install -d -o sandbox -g sandbox \ /home/sandbox/.local \ /home/sandbox/.local/share \ /home/sandbox/.local/share/uv \ /home/sandbox/.cache \ + /home/sandbox/.config \ + /home/sandbox/.herdr \ "$UV_TOOL_DIR" "$UV_TOOL_BIN_DIR" "$UV_PYTHON_INSTALL_DIR" "$UV_CACHE_DIR" \ && chown -R sandbox:sandbox /opt/uv 2>/dev/null || true \ && if [ -d /usr/local/lib/hermes-agent ]; then chown -R sandbox:sandbox /usr/local/lib/hermes-agent; fi @@ -189,6 +191,10 @@ RUN if [ "${INSTALL_PYTHON_KERNEL}" = "true" ]; then \ COPY .devcontainer/entrypoint.sh /usr/local/bin/entrypoint.sh RUN chmod +x /usr/local/bin/entrypoint.sh +RUN mv /home/sandbox /opt/home-seed \ + && rm -rf /opt/home-seed/harness \ + && install -d -o sandbox -g sandbox -m 0755 /home/sandbox + WORKDIR ${OH_PROJECT_ROOT} ENTRYPOINT ["entrypoint.sh"] diff --git a/.devcontainer/docker-compose.image-only.yml b/.devcontainer/docker-compose.image-only.yml index 04d92da1..95062956 100644 --- a/.devcontainer/docker-compose.image-only.yml +++ b/.devcontainer/docker-compose.image-only.yml @@ -2,12 +2,14 @@ # =================================================================== # Standalone recipe for running the PUBLISHED image with no repo checkout # and no local image build — there is no parent-directory bind mount and -# no build stanza in this file. The workspace and control plane (.oh/, -# agent state) live entirely in the named volume `oh_workspace`, seeded -# from the image's baked /opt/oh-seed on first boot by entrypoint.sh (see -# OH_IMAGE_ONLY handling). Edits made inside the running sandbox persist -# in the volume across image pulls and container recreation, not in the -# image itself. +# no build stanza in this file. Everything the sandbox persists, workspace +# and control plane included, lives in the single /home/sandbox mount: the +# named volume ${SANDBOX_NAME}_workspace by default, or an absolute HOST +# path when OH_HOME_MOUNT is set. The workspace at /home/sandbox/harness is +# a directory inside that mount, seeded from the image's baked /opt/oh-seed +# on first boot by entrypoint.sh (see OH_IMAGE_ONLY handling); the rest of +# the home comes from /opt/home-seed via seed_home. Edits made inside the +# running sandbox persist across image pulls and container recreation. # # Usage: # docker compose -f .devcontainer/docker-compose.image-only.yml up -d @@ -23,20 +25,7 @@ services: pull_policy: ${OH_PULL_POLICY:-always} container_name: ${SANDBOX_NAME:-openharness} volumes: - - oh_workspace:${OH_PROJECT_ROOT:-/home/sandbox/harness} - - claude-auth:/home/sandbox/.claude - - codex-auth:/home/sandbox/.codex - - pi-auth:/home/sandbox/.pi - - opencode-auth:/home/sandbox/.local/share/opencode - - grok-auth:/home/sandbox/.grok - - deepagents-auth:/home/sandbox/.deepagents - - herdr-data:/home/sandbox/.herdr - - cloudflared-auth:/home/sandbox/.cloudflared - - ssh-config:/home/sandbox/.ssh - # config-dir covers all of ~/.config (gh CLI auth under ~/.config/gh - # included); entrypoint.sh chowns both to the sandbox UID (chmod 700 .ssh). - - config-dir:/home/sandbox/.config - - cc-safety-net:/home/sandbox/.cc-safety-net + - ${OH_HOME_MOUNT:-workspace}:/home/sandbox extra_hosts: - "host.docker.internal:host-gateway" environment: @@ -54,7 +43,6 @@ services: - GIT_USER_EMAIL=${GIT_USER_EMAIL:-} - GH_TOKEN=${GH_TOKEN:-} - XAI_API_KEY=${XAI_API_KEY:-} - - OH_PROJECT_ROOT=${OH_PROJECT_ROOT:-/home/sandbox/harness} - OH_IMAGE_ONLY=1 - CRON_AGENT_BIN=${CRON_AGENT_BIN:-claude} - LANGFUSE_BASE_URL=${LANGFUSE_BASE_URL:-} @@ -65,7 +53,7 @@ services: entrypoint: /usr/local/bin/entrypoint.sh command: sleep infinity healthcheck: - test: ["CMD", "bash", "${OH_PROJECT_ROOT:-/home/sandbox/harness}/.oh/scripts/sandbox-healthcheck.sh"] + test: ["CMD", "bash", "/home/sandbox/harness/.oh/scripts/sandbox-healthcheck.sh"] interval: 30s timeout: 10s retries: 3 @@ -73,15 +61,4 @@ services: restart: unless-stopped volumes: - oh_workspace: - claude-auth: - codex-auth: - pi-auth: - opencode-auth: - grok-auth: - deepagents-auth: - herdr-data: - cloudflared-auth: - ssh-config: - config-dir: - cc-safety-net: + workspace: diff --git a/.devcontainer/docker-compose.yml b/.devcontainer/docker-compose.yml index 6c41a6e0..aaa05520 100644 --- a/.devcontainer/docker-compose.yml +++ b/.devcontainer/docker-compose.yml @@ -1,17 +1,16 @@ # Main Sandbox Docker Compose Configuration # =========================================== -# Base sandbox compose. The base ships with zero in-tree overlays; -# agent auth and tool state persist in named volumes managed by Docker -# (claude-auth, codex-auth, pi-auth, opencode-auth, grok-auth, -# deepagents-auth, herdr-data, cloudflared-auth, ssh-config, config-dir), -# and the bind-mounted repo's UID is synced into the sandbox user by -# entrypoint.sh on first boot. ~/.ssh (ssh-config) and ~/.config -# (config-dir) get their own volumes so user SSH keys and all XDG tool -# config — including the gh CLI auth that lives under ~/.config/gh and -# Herdr settings — survive container restart/rebuild. -# Grok Build uses ~/.grok for user state/auth, persisted by grok-auth. -# Hermes uses the project-local bind-mounted `.hermes/` directory as -# HERMES_HOME for all runtime state, including auth.json (gitignored). +# The sandbox persists everything under the agent's home directory through a +# single mount at /home/sandbox. Leave OH_HOME_MOUNT unset and Docker manages +# it as the named volume ${SANDBOX_NAME}_workspace; set it (via +# storage.homePath in oh.json) to an absolute HOST path and the same mount +# becomes a bind. The image ships its home baked at /opt/home-seed, which +# entrypoint.sh's seed_home restores into whichever mount lands there, so both +# kinds behave identically. +# +# The repo checkout is bind-mounted at /home/sandbox/harness, nested inside +# that mount; the daemon orders mounts by destination depth, so the checkout +# always lands on top. Its location is fixed, not configurable. # # Downstream harness packs and Pi extensions can register additional # overlays by appending paths to composeOverrides[] in oh.json; @@ -41,27 +40,13 @@ services: context: .. dockerfile: .devcontainer/Dockerfile args: - OH_PROJECT_ROOT: ${OH_PROJECT_ROOT:-/home/sandbox/harness} INSTALL_OPENCODE: ${INSTALL_OPENCODE:-false} INSTALL_GROK_BUILD: ${INSTALL_GROK_BUILD:-false} INSTALL_DEEPAGENTS: ${INSTALL_DEEPAGENTS:-false} INSTALL_HERMES: ${INSTALL_HERMES:-false} volumes: - - ..:${OH_PROJECT_ROOT:-/home/sandbox/harness} - - claude-auth:/home/sandbox/.claude - - codex-auth:/home/sandbox/.codex - - pi-auth:/home/sandbox/.pi - - opencode-auth:/home/sandbox/.local/share/opencode - - grok-auth:/home/sandbox/.grok - - deepagents-auth:/home/sandbox/.deepagents - - herdr-data:/home/sandbox/.herdr - - cloudflared-auth:/home/sandbox/.cloudflared - - ssh-config:/home/sandbox/.ssh - # config-dir covers all of ~/.config (gh CLI auth included); - # entrypoint.sh chowns both dirs to the synced sandbox UID on boot - # (and chmod 700 on .ssh). - - config-dir:/home/sandbox/.config - - cc-safety-net:/home/sandbox/.cc-safety-net + - ${OH_HOME_MOUNT:-workspace}:/home/sandbox + - ..:/home/sandbox/harness extra_hosts: - "host.docker.internal:host-gateway" environment: @@ -82,8 +67,7 @@ services: - INSTALL_AGENT_BROWSER=${INSTALL_AGENT_BROWSER:-false} - SKIP_PNPM_INSTALL=${SKIP_PNPM_INSTALL:-0} - INSTALL_HERMES=${INSTALL_HERMES:-false} - - HERMES_HOME=${OH_PROJECT_ROOT:-/home/sandbox/harness}/.hermes - - OH_PROJECT_ROOT=${OH_PROJECT_ROOT:-/home/sandbox/harness} + - HERMES_HOME=/home/sandbox/harness/.hermes - HERMES_DASHBOARD=${HERMES_DASHBOARD:-false} - HERMES_DASHBOARD_PORT=${HERMES_DASHBOARD_PORT:-9119} - CRON_AGENT_BIN=${CRON_AGENT_BIN:-claude} @@ -95,7 +79,7 @@ services: entrypoint: /usr/local/bin/entrypoint.sh command: sleep infinity healthcheck: - test: ["CMD", "bash", "${OH_PROJECT_ROOT:-/home/sandbox/harness}/.oh/scripts/sandbox-healthcheck.sh"] + test: ["CMD", "bash", "/home/sandbox/harness/.oh/scripts/sandbox-healthcheck.sh"] interval: 30s timeout: 10s retries: 3 @@ -103,14 +87,4 @@ services: restart: unless-stopped volumes: - claude-auth: - codex-auth: - pi-auth: - opencode-auth: - grok-auth: - deepagents-auth: - herdr-data: - cloudflared-auth: - ssh-config: - config-dir: - cc-safety-net: + workspace: diff --git a/.devcontainer/entrypoint.sh b/.devcontainer/entrypoint.sh index a2c1b8cc..7365c944 100644 --- a/.devcontainer/entrypoint.sh +++ b/.devcontainer/entrypoint.sh @@ -17,31 +17,7 @@ sandbox_ownership() { repair_home_mount_ownership() { local owner owner="$(sandbox_ownership)" - echo "[entrypoint] repairing sandbox auth mount ownership as $owner" - - for dir in .claude .codex .pi .prime .grok .deepagents .herdr .cloudflared .config .cc-safety-net .ssh; do - if [ -d "/home/sandbox/$dir" ]; then - chown -hR "$owner" "/home/sandbox/$dir" 2>/dev/null || true - [ "$dir" = ".ssh" ] && chmod 700 "/home/sandbox/$dir" 2>/dev/null || true - fi - done - - # Legacy Hermes home state may exist from earlier layouts. Do not recurse - # into $HERMES_HOME when it points at the bind-mounted checkout; that - if [ -d "/home/sandbox/.hermes" ]; then - chown -hR "$owner" "/home/sandbox/.hermes" 2>/dev/null || true - fi - - for parent in /home/sandbox/.local /home/sandbox/.local/share /home/sandbox/.config; do - if [ -d "$parent" ]; then - chown -h "$owner" "$parent" 2>/dev/null || true - fi - done - - OPENCODE_STATE="/home/sandbox/.local/share/opencode" - if [ -d "$OPENCODE_STATE" ]; then - chown -hR "$owner" "$OPENCODE_STATE" 2>/dev/null || true - fi + echo "[entrypoint] repairing sandbox home mount ownership as $owner" install -d -o sandbox -g sandbox \ /home/sandbox/.local/share/uv \ @@ -49,12 +25,30 @@ repair_home_mount_ownership() { /home/sandbox/.local/share/uv/python \ /home/sandbox/.cache \ /home/sandbox/.cache/uv 2>/dev/null || true - for uv_dir in /home/sandbox/.local/share/uv /home/sandbox/.cache/uv; do - if [ -d "$uv_dir" ]; then - chown -hR "$owner" "$uv_dir" 2>/dev/null || true + + find /home/sandbox -path "$OH_PROJECT_ROOT" -prune -o \ + -exec chown -h "$owner" {} + 2>/dev/null || true + + if [ -d /home/sandbox/.ssh ]; then + chmod 700 /home/sandbox/.ssh 2>/dev/null || true + fi +} + +# >>> seed_home >>> +seed_home() { + local dest="${1:-/home/sandbox}" + local src="${OH_HOME_SEED_SRC:-/opt/home-seed}" + [ -d "$src" ] || return 0 + mkdir -p "$dest" || return 1 + local name + while IFS= read -r -d '' name; do + if [ -e "$dest/$name" ] || [ -L "$dest/$name" ]; then + continue fi - done + cp -a "$src/$name" "$dest/$name" || return 1 + done < <(cd "$src" && find . -mindepth 1 -maxdepth 1 -printf '%P\0') } +# <<< seed_home <<< # >>> seed_workspace_volume >>> seed_workspace_volume() { @@ -85,12 +79,13 @@ seed_workspace_volume() { } # <<< seed_workspace_volume <<< -repair_home_mount_ownership - -# ─── Host UID reconciliation ──────────────────────────────────────── OH_PROJECT_ROOT="${OH_PROJECT_ROOT:-/home/sandbox/harness}" HARNESS="${HARNESS:-$OH_PROJECT_ROOT}" +seed_home /home/sandbox || echo "[entrypoint] WARNING: home seed incomplete; some baked dotfiles may be missing" >&2 + +# ─── Host UID reconciliation ──────────────────────────────────────── + uid_reconcile_step() { local description="$1" shift @@ -131,7 +126,6 @@ elif [ -d "$HARNESS_DIR" ]; then fi if [ "$HOST_UID" != "$SANDBOX_UID" ]; then uid_reconcile_step "set sandbox UID to host UID $HOST_UID" usermod -u "$HOST_UID" sandbox || UID_GID_SYNC_OK=false - uid_reconcile_step "repair sandbox-owned files after UID/GID sync" find /home/sandbox -xdev -uid "$SANDBOX_UID" -exec chown -h "$HOST_UID:$HOST_GID" {} + || UID_GID_SYNC_OK=false if [ "$UID_GID_SYNC_OK" = "true" ]; then echo "[entrypoint] sandbox UID synced to host ($SANDBOX_UID → $HOST_UID, $SANDBOX_GID → $HOST_GID)" else diff --git a/.gitignore b/.gitignore index 7875947c..e4a9924b 100644 --- a/.gitignore +++ b/.gitignore @@ -78,3 +78,9 @@ packages/*/node_modules/ !.oh/skills/wiki/corpus/raw/ .oh/skills/wiki/corpus/raw/* !.oh/skills/wiki/corpus/raw/README.md + +# Per-session agent memory and cron runtime state — local scratch, like the +# wiki corpus above. Nothing here is tracked on any branch; the operator +# whitelists an entry with `git add -f` if one is ever worth keeping. +.oh/memory/ +.oh/crons/ diff --git a/.oh/README.md b/.oh/README.md index 8e0fe3a9..510707fd 100644 --- a/.oh/README.md +++ b/.oh/README.md @@ -170,12 +170,14 @@ source instead of the bundled `.oh/templates/`. ## Project-root seam -`OH_PROJECT_ROOT` (default `/home/sandbox/harness`) is the single source of truth for -the container workspace path. All devcontainer and `.oh/scripts` consumers derive their -paths from `${OH_PROJECT_ROOT:-/home/sandbox/harness}` rather than the bare literal. -`HARNESS` is kept as a back-compat alias (`HARNESS="${HARNESS:-$OH_PROJECT_ROOT}"`); -prefer `$OH_PROJECT_ROOT` in new code. This is Phase 1 of [#531](https://github.com/mifunedev/openharness/issues/531) toward `oh init`. -The seam contract is guarded by `.oh/evals/probes/project-root-seam.sh`. +`OH_PROJECT_ROOT` is `/home/sandbox/harness`, fixed. It is no longer configurable: +the sandbox home is one mount at `/home/sandbox` and the checkout is nested inside +it, so a relocatable project root buys nothing ([#898](https://github.com/mifunedev/openharness/issues/898)). +The image pins it (`ENV OH_PROJECT_ROOT=/home/sandbox/harness`), and devcontainer +and `.oh/scripts` consumers keep reading `${OH_PROJECT_ROOT:-/home/sandbox/harness}` +rather than the bare literal. `HARNESS` remains a back-compat alias +(`HARNESS="${HARNESS:-$OH_PROJECT_ROOT}"`); prefer `$OH_PROJECT_ROOT` in new code. +The fixed definition is guarded by `.oh/evals/probes/worktrees-layout.sh`. ## devcontainer layout diff --git a/.oh/cli/src/__tests__/destroy.test.ts b/.oh/cli/src/__tests__/destroy.test.ts index 03cfcfa4..9a13d5ff 100644 --- a/.oh/cli/src/__tests__/destroy.test.ts +++ b/.oh/cli/src/__tests__/destroy.test.ts @@ -115,7 +115,7 @@ describe("named volumes — what destroy is about to delete", () => { it("reads the real repository's volumes so the prompt cannot drift from compose", () => { const root = join(HERE, "..", "..", "..", ".."); - expect(namedVolumes(root)).toContain("claude-auth"); + expect(namedVolumes(root)).toContain("workspace"); }); }); diff --git a/.oh/cli/src/commands/init.ts b/.oh/cli/src/commands/init.ts index 20852ab4..c1db4fd7 100644 --- a/.oh/cli/src/commands/init.ts +++ b/.oh/cli/src/commands/init.ts @@ -641,9 +641,6 @@ const ENV_TO_CONFIG: Record = { TZ: (c, v) => { c.timezone = v; }, - OH_PROJECT_ROOT: (c, v) => { - c.projectRoot = v; - }, GIT_USER_NAME: (c, v) => { section(c, "git").userName = v; }, @@ -763,6 +760,20 @@ async function runWizard( const gitEmail = await askFn("Git user email:"); if (gitEmail) section(config, "git").userEmail = gitEmail; + prompt.info(""); + prompt.info("The sandbox home holds every agent login, the gh token, and the SSH keys."); + prompt.info("Give an absolute HOST path to keep it somewhere you can back up."); + prompt.info("Use a dedicated empty directory — the sandbox takes ownership of it."); + prompt.info(`Leave blank and Docker manages it as \`${config.name ?? ""}_workspace\`.`); + const homePath = (await askFn("Persistent home host path [blank = Docker-managed volume]:")).trim(); + if (homePath !== "") { + if (homePath.startsWith("/")) { + section(config, "storage").homePath = homePath; + } else { + prompt.warn(`Ignoring "${homePath}" — it must be an absolute host path starting with /.`); + } + } + prompt.step(2, 5, "Optional installs"); const installs: { key: string; field: string; desc: string }[] = [ { key: "opencode", field: "opencode", desc: "OpenCode TUI coding agent" }, diff --git a/.oh/cli/src/commands/lifecycle.ts b/.oh/cli/src/commands/lifecycle.ts index eff217ba..477c87a8 100644 --- a/.oh/cli/src/commands/lifecycle.ts +++ b/.oh/cli/src/commands/lifecycle.ts @@ -321,18 +321,29 @@ export async function runDestroy(opts: DestroyOptions, io: LifecycleIO): Promise return 1; } - const volumes = namedVolumes(root); + const homePath = readOhConfig(ohConfigPath(root)).storage?.homePath; io.stdout(`\n${prompt.bold(`oh destroy — ${name}`)}\n\n`); - io.stdout("`docker compose down -v` removes the containers and deletes\n"); - io.stdout( - volumes.length > 0 - ? `these named volumes with everything in them:\n\n ${volumes.join("\n ")}\n\n` - : "every named volume this project owns, with everything in them.\n\n", - ); - io.stdout( - "That is the provider authentication those volumes hold — every agent CLI\n" + - "login, the gh CLI token, and the SSH keys. Sign-in starts over.\n\n", - ); + + if (homePath !== undefined && homePath !== "") { + io.stdout("`docker compose down -v` removes the containers.\n"); + io.stdout( + `This sandbox keeps its home on a host bind at ${homePath}, which \`down -v\`\n` + + "does not touch — every agent CLI login, the gh CLI token, and the SSH keys\n" + + "stay there. Delete that directory yourself if you also want them gone.\n\n", + ); + } else { + const volumes = namedVolumes(root).map((volume) => `${name}_${volume}`); + io.stdout("`docker compose down -v` removes the containers and deletes\n"); + io.stdout( + volumes.length > 0 + ? `these named volumes with everything in them:\n\n ${volumes.join("\n ")}\n\n` + : "every named volume this project owns, with everything in them.\n\n", + ); + io.stdout( + "That is the provider authentication those volumes hold — every agent CLI\n" + + "login, the gh CLI token, and the SSH keys. Sign-in starts over.\n\n", + ); + } const askFn = io.ask ?? prompt.ask; const answer = (await askFn(`Type the sandbox name \`${name}\` to destroy it:`)).trim(); diff --git a/.oh/cli/src/lib/__tests__/config-render.test.ts b/.oh/cli/src/lib/__tests__/config-render.test.ts index 27e5af98..ce0d3a0e 100644 --- a/.oh/cli/src/lib/__tests__/config-render.test.ts +++ b/.oh/cli/src/lib/__tests__/config-render.test.ts @@ -9,7 +9,7 @@ import { defaultOhConfig, type OhConfig } from "../oh-config.js"; const REPO_ROOT = join(dirname(fileURLToPath(import.meta.url)), "..", "..", "..", "..", ".."); const DEVCONTAINER = join(REPO_ROOT, ".devcontainer"); -const RETIRED = ["WORKTREES_DIR", "PROJECTS_DIR", "CRONS_DIR"]; +const RETIRED = ["WORKTREES_DIR", "PROJECTS_DIR", "CRONS_DIR", "OH_PROJECT_ROOT"]; function composeInterpolatedVars(): string[] { const found = new Set(); @@ -24,6 +24,7 @@ function composeInterpolatedVars(): string[] { function fullConfig(): OhConfig { const config = defaultOhConfig("demo"); config.git = { userName: "Ada", userEmail: "ada@example.com" }; + config.storage = { homePath: "/srv/oh-home" }; config.access = { ssh: true, sshPort: 2022, @@ -51,7 +52,7 @@ describe("renderComposeEnv", () => { const text = renderComposeEnv(fullConfig()); expect(text).toContain("SANDBOX_NAME=demo"); expect(text).toContain("TZ=America/Los_Angeles"); - expect(text).toContain("OH_PROJECT_ROOT=/home/sandbox/harness"); + expect(text).toContain("OH_HOME_MOUNT=/srv/oh-home"); expect(text).toContain("GIT_USER_NAME=Ada"); expect(text).toContain("GIT_USER_EMAIL=ada@example.com"); expect(text).toContain("INSTALL_OPENCODE=false"); diff --git a/.oh/cli/src/lib/__tests__/oh-config.test.ts b/.oh/cli/src/lib/__tests__/oh-config.test.ts index 7c1e46be..d0e571f4 100644 --- a/.oh/cli/src/lib/__tests__/oh-config.test.ts +++ b/.oh/cli/src/lib/__tests__/oh-config.test.ts @@ -105,7 +105,16 @@ describe("validateOhConfig", () => { const cases: Array<[string, unknown, RegExp]> = [ ["name", { name: 1 }, /^oh\.json: name must be a string$/], ["timezone", { timezone: true }, /^oh\.json: timezone must be a string$/], - ["projectRoot", { projectRoot: [] }, /^oh\.json: projectRoot must be a string$/], + [ + "storage.homePath", + { storage: { homePath: [] } }, + /^oh\.json: storage\.homePath must be a string$/, + ], + [ + "storage.homePath relative", + { storage: { homePath: "oh-home" } }, + /^oh\.json: storage\.homePath must be an absolute host path$/, + ], ["git", { git: "me" }, /^oh\.json: git must be an object$/], ["git.userName", { git: { userName: 7 } }, /^oh\.json: git\.userName must be a string$/], ["git.userEmail", { git: { userEmail: 7 } }, /^oh\.json: git\.userEmail must be a string$/], diff --git a/.oh/cli/src/lib/config-render.ts b/.oh/cli/src/lib/config-render.ts index 4be304ec..276b28b3 100644 --- a/.oh/cli/src/lib/config-render.ts +++ b/.oh/cli/src/lib/config-render.ts @@ -1,7 +1,7 @@ import type { OhConfig } from "./oh-config.js"; import { isSecretKey } from "./secrets.js"; -const RETIRED_KEYS = ["WORKTREES_DIR", "PROJECTS_DIR", "CRONS_DIR"] as const; +const RETIRED_KEYS = ["WORKTREES_DIR", "PROJECTS_DIR", "CRONS_DIR", "OH_PROJECT_ROOT"] as const; export interface RenderedVar { key: string; @@ -17,7 +17,7 @@ export function renderComposeVars(config: OhConfig): RenderedVar[] { put("SANDBOX_NAME", config.name); put("TZ", config.timezone); - put("OH_PROJECT_ROOT", config.projectRoot); + put("OH_HOME_MOUNT", config.storage?.homePath); put("GIT_USER_NAME", config.git?.userName); put("GIT_USER_EMAIL", config.git?.userEmail); diff --git a/.oh/cli/src/lib/oh-config.ts b/.oh/cli/src/lib/oh-config.ts index b6aeccc0..f09caac4 100644 --- a/.oh/cli/src/lib/oh-config.ts +++ b/.oh/cli/src/lib/oh-config.ts @@ -5,6 +5,26 @@ import { assertInRoot } from "./env-file.js"; const OH_CONFIG_FILE = "oh.json"; const OH_CONFIG_MODE = 0o644; +const RESERVED_HOME_PATHS: readonly string[] = [ + "/", + "/bin", + "/boot", + "/dev", + "/etc", + "/home", + "/lib", + "/opt", + "/proc", + "/root", + "/run", + "/sbin", + "/srv", + "/sys", + "/tmp", + "/usr", + "/var", +]; + export type ImageMode = "build" | "image"; export type PullPolicy = "missing" | "always" | "never"; @@ -48,6 +68,10 @@ export interface ImageSettings { pullPolicy?: PullPolicy; } +export interface StorageSettings { + homePath?: string; +} + export interface CloudSettings { apiUrl?: string; } @@ -74,8 +98,8 @@ export interface OhConfig { version: 1; name?: string; timezone?: string; - projectRoot?: string; git?: GitIdentity; + storage?: StorageSettings; install?: InstallFlags; access?: AccessSettings; hermesDashboard?: HermesDashboardSettings; @@ -97,8 +121,8 @@ export function defaultOhConfig(name: string): OhConfig { version: 1, name, timezone: "America/Los_Angeles", - projectRoot: "/home/sandbox/harness", git: {}, + storage: {}, install: { opencode: false, grokBuild: false, @@ -171,7 +195,24 @@ export function validateOhConfig(value: unknown): OhConfig { expectString(record, "name"); expectString(record, "timezone"); - expectString(record, "projectRoot"); + + const storage = expectSection(record, "storage"); + if (storage) { + expectString(storage, "homePath", "storage."); + const homePath = storage.homePath; + if (typeof homePath === "string" && homePath !== "") { + if (!homePath.startsWith("/")) { + throw fieldError("storage.homePath", "must be an absolute host path"); + } + const normalized = homePath.replace(/\/+$/, "") || "/"; + if (RESERVED_HOME_PATHS.includes(normalized)) { + throw fieldError( + "storage.homePath", + "must be a dedicated directory — the sandbox takes ownership of everything under it", + ); + } + } + } const git = expectSection(record, "git"); if (git) { @@ -304,7 +345,6 @@ export interface OhConfigField { export const OH_CONFIG_FIELDS: readonly OhConfigField[] = [ { path: "name", type: "string" }, { path: "timezone", type: "string" }, - { path: "projectRoot", type: "string" }, { path: "git.userName", type: "string" }, { path: "git.userEmail", type: "string" }, { path: "install.opencode", type: "boolean" }, @@ -324,6 +364,7 @@ export const OH_CONFIG_FIELDS: readonly OhConfigField[] = [ { path: "image.ref", type: "string" }, { path: "image.mode", type: "enum", values: ["build", "image"] }, { path: "image.pullPolicy", type: "enum", values: ["missing", "always", "never"] }, + { path: "storage.homePath", type: "string" }, { path: "cloud.apiUrl", type: "string" }, { path: "langfuse.baseUrl", type: "string" }, { path: "langfuse.privacyPreset", type: "enum", values: LANGFUSE_PRIVACY_PRESETS }, diff --git a/.oh/evals/RESULTS.md b/.oh/evals/RESULTS.md index 7beb104f..2ab5e22c 100644 --- a/.oh/evals/RESULTS.md +++ b/.oh/evals/RESULTS.md @@ -6,97 +6,106 @@ probe id; git history is the time series.** Schema and exit-code semantics are i | probe | tier | last-run (UTC) | status | source | |-------|------|----------------|--------|--------| -| advisor-monitored-loop | A | 2026-08-29 06:52 | PASS | conversation 2026-06-19 (single-owner implementation workflow, issue #257) | -| agent-browser-cli | A | 2026-08-29 06:52 | PASS | retro lesson 2026-06-07 (agent-browser 0.8.5 CLI) | -| agents-identity-contract | A | 2026-08-29 06:52 | PASS | issue #854 — T3-style root identity, glossary, and skill-owned procedures | -| artifact-contract-audit | A | 2026-08-29 06:52 | PASS | issue #583/#645 — production /audit implementation Gate 1 behavior | -| audit-dispatcher-contract | A | 2026-08-29 06:52 | PASS | issue #645 — audit consolidation public taxonomy | -| audit-implementation-behavior | A | 2026-08-29 06:52 | PASS | issue #645 — implementation root/repo/browser behavior | -| audit-pr-acquire | A | 2026-08-29 06:52 | PASS | issue #645 — production PR acquisition behavior | -| audit-pr-classifier | A | 2026-08-29 06:52 | PASS | issue #645 — deterministic focused and queue PR classifier | -| audit-run-root-contract | A | 2026-08-29 06:52 | PASS | issue #645 — executable immutable audit root/run correlation | -| audit-shellcheck-coverage | A | 2026-08-29 06:52 | PASS | issue #645 — private audit scripts require release and CI lint coverage | -| audit-stale-references | A | 2026-08-29 06:52 | PASS | issue #645 — clean-breaking audit migration | -| boot-lint-glob | A | 2026-08-29 06:52 | PASS | issue #90, issue #120 | -| builder-skill-consolidation | A | 2026-08-29 06:52 | PASS | issue #643 — consolidate artifact builders behind one /builder dispatcher | -| capability-benchmark-schema | A | 2026-08-29 06:52 | PASS | issue #167 — capability benchmark instrument | -| cc-safety-net-wiring | A | 2026-08-29 06:52 | PASS | .oh/tasks/cc-safety-net/prd.json US-007 2026-07-19 | -| changelog-entry-length | A | 2026-08-29 06:52 | PASS | conversation 2026-08-24 — CHANGELOG.md grew to 259KB of bullet prose because "one line" was unquantified | -| cleanup-tasks-scoped-guard | A | 2026-08-29 06:52 | PASS | issue #85 | -| cleanup-tasks-worktree-grooming | A | 2026-08-29 06:52 | PASS | issue #168; issue #327 | -| close-issues-on-development | A | 2026-08-29 06:52 | PASS | issue #841 (closing keywords never fire because the default branch is main) 2026-08-26 | -| codex-stale-response-retry | A | 2026-08-29 06:52 | PASS | issue #506 — Codex previous_response_not_found RCA | -| compose-config-path-parity | A | 2026-08-29 06:52 | PASS | ? | -| context-tier-size-budget | A | 2026-08-29 06:52 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-007) — the always-on tier was 85,256 B | -| cron-claude-codex-fallback | A | 2026-08-29 06:52 | PASS | conversation 2026-06-12 (default Codex fallback for crons) | -| cron-watchdog | A | 2026-08-29 06:52 | PASS | issues #130/#453 (cron runtime watchdog + legacy system-cron reaping) 2026-06-19 | -| curl-bash-safe-alternatives | A | 2026-08-29 06:52 | PASS | vet-run/vet integration — public curl|bash examples need review-first alternatives | -| datasets-schema | A | 2026-08-29 06:52 | PASS | issue #196 — .oh/evals/datasets verifiable trajectory corpus (Repo2RLEnv-inspired) | -| debugmcp-availability | A | 2026-08-29 06:52 | SKIPPED | issue #297 — DebugMCP MCP debug-server availability | -| delegate-model-effort-policy | A | 2026-08-29 06:52 | PASS | conversation 2026-07-11 (delegate model inheritance and thinking policy) | -| devtcp-hook | A | 2026-08-29 06:52 | PASS | retro lesson 2026-06-10 (zsh /dev/tcp) | -| docker-inspect-env-guard | A | 2026-08-29 06:52 | PASS | operator directive 2026-08-08 (agents keep the docker socket, but must | -| docs-build-fast-path | A | 2026-08-29 06:52 | PASS | #455 — docs builds must stay out of fast harness/eval/release gates; #536 — docs site externalized to openharness-web; docs markdown relocated to docs/ | -| drift-check-cron-staleness-glob | A | 2026-08-29 06:52 | PASS | issue #98; issue #225 (restart-required cron frontmatter/config drift) | -| entrypoint-pnpm-manifest-fingerprint | A | 2026-08-29 06:52 | PASS | issue #521 (manifest-aware sandbox installs) 2026-07-01 | -| env-schema-parity | A | 2026-08-29 06:52 | PASS | ? | -| eval-ci-gate | A | 2026-08-29 06:52 | PASS | #103 — eval probe suite gated in CI | -| eval-gate | A | 2026-08-29 06:52 | PASS | retro lesson 2026-06-11 (eval-gate) | -| eval-results-atomic | A | 2026-08-29 06:52 | PASS | issue #83 (eval-results-atomic-write) | -| eval-runner-exit | A | 2026-08-29 06:52 | PASS | retro lesson 2026-06-11 (eval-runner-exit) #29 | -| eval-runs-once-per-cycle | A | 2026-08-29 06:52 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-006) — /eval ran 3x per cycle on the | -| execution-target-contract | A | 2026-08-29 06:52 | PASS | issue #733 (ExecutionTarget contract + Docker Compose adapter) 2026-08-10 | -| get-oh-bootstrap | A | 2026-08-29 06:52 | PASS | get-oh.sh bootstrap — the Node-bootstrapping host-side path to the standalone `oh` CLI (also on npm as @mifune/openharness; see oh-npm-package.sh) | -| git-skill | A | 2026-08-29 06:52 | PASS | conversation 2026-06-15 — rules are not always supported; git workflow must be the /git skill | -| harness-audit-empty-output-gate | A | 2026-08-29 06:52 | PASS | issue #246 — /audit harness must fail closed on empty auditor outputs | -| harness-ci-core-paths | A | 2026-08-29 06:52 | PASS | #165 — core sandbox config files must trigger harness CI | -| harness-ci-hooks-paths | A | 2026-08-29 06:52 | PASS | issue #202 — credential/security hook changes must trigger harness CI | -| harness-yaml-migration | A | 2026-08-29 06:52 | PASS | ? | -| health-check-docker-stats | A | 2026-08-29 06:52 | PASS | retro lesson 2026-06-10 (docker stats vs ps Size) | -| health-check-socket-degrade | A | 2026-08-29 06:52 | PASS | issue #762 (refs #756) — /health-check degrades to one statement, not nine failures | -| heartbeat-logging-contract | A | 2026-08-29 06:52 | PASS | issue #447 (heartbeat log append hardening) 2026-06-18 | -| make-oh-lifecycle-parity | A | 2026-08-29 06:52 | PASS | the Makefile/oh surface-gap consolidation — two front doors onto | -| markitdown-wiki-ingest | A | 2026-08-29 06:52 | PASS | issue #649 — pinned local-document normalization contract for /wiki ingest | -| next-dev-prod | A | 2026-08-29 06:52 | SKIPPED | retro lesson 2026-06-04 | -| oh-devcontainer-restructure | A | 2026-08-29 06:52 | PASS | consolidate devcontainer — .oh/devcontainer/ folded back into .devcontainer/ | -| oh-image-only-deploy | A | 2026-08-29 06:52 | PASS | .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only deploy) | -| oh-init-headless-config | A | 2026-08-29 06:52 | PASS | ? | -| oh-init-scaffold | A | 2026-08-29 06:52 | PASS | issue #531 Phase 2 | -| oh-npm-package | A | 2026-08-29 06:52 | PASS | npm publish path for the standalone `oh` CLI (@mifune/openharness) — alternative to get-oh.sh | -| oh-payload-manifest | A | 2026-08-29 06:52 | PASS | issue #531 follow-on (.oh payload manifest — oh update ships a declared allowlist) | -| oh-sandbox-image-mode | A | 2026-08-29 06:52 | PASS | conversation 2026-07-05 (basic Docker deployment — prebuilt-image mode) | -| oh-shipped-repo-overridable | A | 2026-08-29 06:52 | PASS | issue #531 follow-on (de-hardcode residual — shipped .oh shell scripts keep the upstream repo overridable) | -| oh-standalone-lifecycle | A | 2026-08-29 06:52 | PASS | issue #564 | -| oh-update | A | 2026-08-29 06:52 | PASS | issue #531 Phase 3 (oh update — upgrade only the .oh control plane) | -| operator-config-guard | A | 2026-08-29 06:52 | PASS | operator directives 2026-08-06 (.config/ and settings.local.json are operator-only) | -| pnpm-audit-ci-gate | A | 2026-08-29 06:52 | PASS | issue #171 — pnpm security audits must run in CI | -| post-bridge-publish-confirmation | A | 2026-08-29 06:52 | PASS | #523 — post-bridge live publishing requires an explicit final confirmation gate | -| prd-output-path-contract | A | 2026-08-29 06:52 | PASS | retro lesson 2026-06-19 | -| project-root-seam | A | 2026-08-29 06:52 | PASS | issue #531 Phase 1 (OH_PROJECT_ROOT project-root seam) 2026-06-26 | -| prompt-miner-schema-compat | A | 2026-08-29 06:52 | PASS | issue #253 — prompt-miner JSONL schema-drift guard | -| prompt-miner-symlink-entrypoint | A | 2026-08-29 06:52 | PASS | issue #663 — prompt-miner engine no-ops via the documented .claude/skills symlink | -| prompt-miner-weakness-record | A | 2026-08-29 06:52 | PASS | issue #580 — prompt-miner weakness-record (WH-xxx) cluster output | -| protected-path-deletion | A | 2026-08-29 06:52 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-001) — the critique gate was deleted, | -| protected-paths-resolve | A | 2026-08-29 06:52 | PASS | issue #753 — .claude/protected-paths.txt named 7 paths that did not exist. | -| registry-portability-gate | A | 2026-08-29 06:52 | PASS | issue #758 | -| registry-portability | A | 2026-08-29 06:52 | SKIPPED | issue #758 | -| retro-deterministic-contract | A | 2026-08-29 06:52 | PASS | issue #443 — /retro deterministic output and self-contained helper contract | -| rl-delegation-write-worker | A | 2026-08-29 06:52 | PASS | retro lesson 2026-06-10 (rl-delegation) #57 | -| rlm-context-budget | A | 2026-08-29 06:52 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-006 | -| runtime-preflight-gate | A | 2026-08-29 06:52 | PASS | issue #806 § B1 (open sandbox.substrate vs sandbox.runtime selector); | -| sandbox-boot-guard-ci | A | 2026-08-29 06:52 | PASS | issue #449 (sandbox image build CI guard) 2026-06-19; | -| skill-paths | A | 2026-08-29 06:52 | PASS | issue #43 — stale path references; extended by issue #69 — apps/->packages/ rename guard; extended by issue #870 — deleted .oh/agents/advisor.md | -| skills-dir-clean | A | 2026-08-29 06:52 | PASS | conversation 2026-06-29 — Pi parses every top-level `.md` in the skills | -| skills-vendored | A | 2026-08-29 06:52 | PASS | absorb .mifune submodule into .oh — the skills/agents/hooks pack is vendored | -| slack-admin-command-surface | A | 2026-08-29 06:52 | PASS | issue #354 — Slack bridge docs must distinguish Pi /msg-bridge commands from Slack DM admin text handlers | -| spec-family-contract | A | 2026-08-29 06:52 | PASS | issue #265; spec-simplification issue #816; workflow authority issue #854 | -| spec-ready-finalization | A | 2026-08-29 06:52 | PASS | issue #134; spec-simplification issue #816; workflow authority issue #854 | -| ste-checker-contract | A | 2026-08-29 06:52 | PASS | issue #750 PR audit — the /ste checker had four fail-open paths (unclosed | -| submitted-by-trailers | A | 2026-08-29 06:52 | PASS | conversation 2026-06-12 (commit attribution trailers); the single-owner | -| sync-skill-contract | A | 2026-08-29 06:52 | PASS | issue #331 — /sync dispatcher skill (bidirectional origin↔upstream sync) | -| tool-catalog-boundary | A | 2026-08-29 06:52 | PASS | agent-browser's exclusion from the harness catalog (#821) and the | -| weigh-scorer-contract | A | 2026-08-29 06:52 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-003 (2026-06-27) | -| wiki-readme-index | A | 2026-08-29 06:52 | PASS | issue #132 — wiki README index drift guard | -| workflow-boundaries | A | 2026-08-29 06:52 | PASS | conversation 2026-06-19 (workflow consolidation, issue #259); authority moved to /spec in issue #854 | +| advisor-monitored-loop | A | 2026-08-30 17:01 | PASS | conversation 2026-06-19 (single-owner implementation workflow, issue #257) | +| agent-browser-cli | A | 2026-08-30 17:01 | PASS | retro lesson 2026-06-07 (agent-browser 0.8.5 CLI) | +| agents-identity-contract | A | 2026-08-30 17:01 | PASS | issue #854 — T3-style root identity, glossary, and skill-owned procedures | +| artifact-contract-audit | A | 2026-08-30 17:01 | PASS | issue #583/#645 — production /audit implementation Gate 1 behavior | +| audit-dispatcher-contract | A | 2026-08-30 17:01 | PASS | issue #645 — audit consolidation public taxonomy | +| audit-implementation-behavior | A | 2026-08-30 17:01 | PASS | issue #645 — implementation root/repo/browser behavior | +| audit-pr-acquire | A | 2026-08-30 17:01 | PASS | issue #645 — production PR acquisition behavior | +| audit-pr-classifier | A | 2026-08-30 17:01 | PASS | issue #645 — deterministic focused and queue PR classifier | +| audit-run-root-contract | A | 2026-08-30 17:01 | PASS | issue #645 — executable immutable audit root/run correlation | +| audit-shellcheck-coverage | A | 2026-08-30 17:01 | PASS | issue #645 — private audit scripts require release and CI lint coverage | +| audit-stale-references | A | 2026-08-30 17:01 | PASS | issue #645 — clean-breaking audit migration | +| boot-lint-glob | A | 2026-08-30 17:01 | PASS | issue #90, issue #120 | +| builder-skill-consolidation | A | 2026-08-30 17:01 | PASS | issue #643 — consolidate artifact builders behind one /builder dispatcher | +| capability-benchmark-schema | A | 2026-08-30 17:01 | PASS | issue #167 — capability benchmark instrument | +| cc-safety-net-wiring | A | 2026-08-30 17:01 | SKIPPED | .oh/tasks/cc-safety-net/prd.json US-007 2026-07-19 | +| changelog-entry-length | A | 2026-08-30 17:01 | PASS | conversation 2026-08-24 — CHANGELOG.md grew to 259KB of bullet prose because "one line" was unquantified | +| cleanup-tasks-scoped-guard | A | 2026-08-30 17:01 | PASS | issue #85 | +| cleanup-tasks-worktree-grooming | A | 2026-08-30 17:01 | PASS | issue #168; issue #327 | +| cli-publish-typecheck-scope | A | 2026-08-30 17:01 | PASS | release run 33271077312 — v0.5.0 pushed its GHCR image, then failed to publish | +| close-issues-on-development | A | 2026-08-30 17:01 | PASS | issue #841 (closing keywords never fire because the default branch is main) 2026-08-26 | +| codex-stale-response-retry | A | 2026-08-30 17:01 | PASS | issue #506 — Codex previous_response_not_found RCA | +| compose-config-path-parity | A | 2026-08-30 17:01 | PASS | PR #833 (remove harness.yaml — the wrapper and VS Code "Reopen in Container" paths must resolve the same service) 2026-08-26 | +| config-schema-parity | A | 2026-08-30 17:01 | PASS | PR #833 (one schema file — DOCKER_SOCKET, SANDBOX_SSH, OH_SANDBOX_IMAGE, OH_PULL_POLICY, SKIP_PNPM_INSTALL were consumed but undocumented); rewritten for the oh.json/secrets split by PR #887 | +| context-tier-size-budget | A | 2026-08-30 17:01 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-007) — the always-on tier was 85,256 B | +| cron-claude-codex-fallback | A | 2026-08-30 17:01 | PASS | conversation 2026-06-12 (default Codex fallback for crons) | +| cron-watchdog | A | 2026-08-30 17:01 | PASS | issues #130/#453 (cron runtime watchdog + legacy system-cron reaping) 2026-06-19 | +| crons-directory-guide | A | 2026-08-30 17:01 | PASS | issue #874 | +| curl-bash-safe-alternatives | A | 2026-08-30 17:01 | PASS | vet-run/vet integration — public curl|bash examples need review-first alternatives | +| datasets-schema | A | 2026-08-30 17:01 | PASS | issue #196 — .oh/evals/datasets verifiable trajectory corpus (Repo2RLEnv-inspired) | +| debugmcp-availability | A | 2026-08-30 17:01 | SKIPPED | issue #297 — DebugMCP MCP debug-server availability | +| delegate-model-effort-policy | A | 2026-08-30 17:01 | PASS | conversation 2026-07-11 (delegate model inheritance and thinking policy) | +| devtcp-hook | A | 2026-08-30 17:01 | PASS | retro lesson 2026-06-10 (zsh /dev/tcp) | +| docker-inspect-env-guard | A | 2026-08-30 17:01 | PASS | operator directive 2026-08-08 (agents keep the docker socket, but must | +| docs-build-fast-path | A | 2026-08-30 17:01 | PASS | #455 — docs builds must stay out of fast harness/eval/release gates; #536 — docs site externalized to openharness-web; docs markdown relocated to docs/ | +| drift-check-cron-staleness-glob | A | 2026-08-30 17:01 | PASS | issue #98; issue #225 (restart-required cron frontmatter/config drift) | +| entrypoint-pnpm-manifest-fingerprint | A | 2026-08-30 17:01 | PASS | issue #521 (manifest-aware sandbox installs) 2026-07-01 | +| eval-ci-gate | A | 2026-08-30 17:01 | PASS | #103 — eval probe suite gated in CI | +| eval-gate | A | 2026-08-30 17:01 | PASS | retro lesson 2026-06-11 (eval-gate) | +| eval-results-atomic | A | 2026-08-30 17:01 | PASS | issue #83 (eval-results-atomic-write) | +| eval-runner-exit | A | 2026-08-30 17:01 | PASS | retro lesson 2026-06-11 (eval-runner-exit) #29 | +| eval-runs-once-per-cycle | A | 2026-08-30 17:01 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-006) — /eval ran 3x per cycle on the | +| execution-target-contract | A | 2026-08-30 17:01 | PASS | issue #733 (ExecutionTarget contract + Docker Compose adapter) 2026-08-10 | +| get-oh-bootstrap | A | 2026-08-30 17:01 | PASS | get-oh.sh bootstrap — the Node-bootstrapping host-side path to the standalone `oh` CLI (also on npm as @mifune/openharness; see oh-npm-package.sh) | +| git-skill | A | 2026-08-30 17:01 | PASS | conversation 2026-06-15 — rules are not always supported; git workflow must be the /git skill | +| harness-audit-empty-output-gate | A | 2026-08-30 17:01 | PASS | issue #246 — /audit harness must fail closed on empty auditor outputs | +| harness-ci-core-paths | A | 2026-08-30 17:01 | PASS | #165 — core sandbox config files must trigger harness CI | +| harness-ci-hooks-paths | A | 2026-08-30 17:01 | PASS | issue #202 — credential/security hook changes must trigger harness CI | +| harness-yaml-migration | A | 2026-08-30 17:01 | PASS | PR #833 (migrate-harness-yaml.sh — append / uncomment-in-place / preserve / overwrite, plus a silent no-op second run) 2026-08-26 | +| health-check-docker-stats | A | 2026-08-30 17:01 | PASS | retro lesson 2026-06-10 (docker stats vs ps Size) | +| health-check-socket-degrade | A | 2026-08-30 17:01 | PASS | issue #762 (refs #756) — /health-check degrades to one statement, not nine failures | +| heartbeat-logging-contract | A | 2026-08-30 17:01 | PASS | issue #447 (heartbeat log append hardening) 2026-06-18 | +| markitdown-wiki-ingest | A | 2026-08-30 17:01 | PASS | issue #649 — pinned local-document normalization contract for /wiki ingest | +| next-dev-prod | A | 2026-08-30 17:01 | SKIPPED | retro lesson 2026-06-04 | +| oh-compose-env-wiring | A | 2026-08-30 17:01 | PASS | issue #880 (oh as the only front door — oh.json is the non-secret config surface) | +| oh-config-surfaces | A | 2026-08-30 17:01 | REGRESSION | PR #887 (config split across two authored surfaces — a tracked oh.json and a secrets-only root dotenv — with nothing left under $HOME) | +| oh-destroy-guard | A | 2026-08-30 17:01 | PASS | issue #879 — `oh` becomes the only front door, so `make destroy` must | +| oh-devcontainer-restructure | A | 2026-08-30 17:01 | PASS | consolidate devcontainer — .oh/devcontainer/ folded back into .devcontainer/ | +| oh-home-mount | A | 2026-08-30 17:01 | PASS | issue #898 (single $HOME mount) 2026-08-30 | +| oh-image-only-deploy | A | 2026-08-30 17:01 | PASS | .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only deploy) | +| oh-init-headless-config | A | 2026-08-30 17:01 | PASS | PR #827 (installer answers landed in the losing config file); retargeted to the .example.env template by PR #833, then to oh.json by PR #887 | +| oh-init-scaffold | A | 2026-08-30 17:01 | PASS | issue #531 Phase 2 | +| oh-lifecycle-surface | A | 2026-08-30 17:01 | PASS | issue #881 — the Makefile is retired and `oh` is the only front door | +| oh-npm-package | A | 2026-08-30 17:01 | PASS | npm publish path for the standalone `oh` CLI (@mifune/openharness) — alternative to get-oh.sh | +| oh-payload-manifest | A | 2026-08-30 17:01 | PASS | issue #531 follow-on (.oh payload manifest — oh update ships a declared allowlist) | +| oh-sandbox-image-mode | A | 2026-08-30 17:01 | PASS | conversation 2026-07-05 (basic Docker deployment — prebuilt-image mode) | +| oh-shipped-repo-overridable | A | 2026-08-30 17:01 | PASS | issue #531 follow-on (de-hardcode residual — shipped .oh shell scripts keep the upstream repo overridable) | +| oh-standalone-lifecycle | A | 2026-08-30 17:01 | PASS | issue #564 | +| oh-update | A | 2026-08-30 17:01 | PASS | issue #531 Phase 3 (oh update — upgrade only the .oh control plane) | +| operator-config-guard | A | 2026-08-30 17:01 | PASS | operator directives 2026-08-06 (.config/ and settings.local.json are operator-only) | +| pnpm-audit-ci-gate | A | 2026-08-30 17:01 | PASS | issue #171 — pnpm security audits must run in CI | +| post-bridge-publish-confirmation | A | 2026-08-30 17:01 | PASS | #523 — post-bridge live publishing requires an explicit final confirmation gate | +| prd-output-path-contract | A | 2026-08-30 17:01 | PASS | retro lesson 2026-06-19 | +| prompt-miner-schema-compat | A | 2026-08-30 17:01 | PASS | issue #253 — prompt-miner JSONL schema-drift guard | +| prompt-miner-symlink-entrypoint | A | 2026-08-30 17:01 | PASS | issue #663 — prompt-miner engine no-ops via the documented .claude/skills symlink | +| prompt-miner-weakness-record | A | 2026-08-30 17:01 | PASS | issue #580 — prompt-miner weakness-record (WH-xxx) cluster output | +| protected-path-deletion | A | 2026-08-30 17:01 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-001) — the critique gate was deleted, | +| protected-paths-resolve | A | 2026-08-30 17:01 | PASS | issue #753 — .claude/protected-paths.txt named 7 paths that did not exist. | +| registry-portability-gate | A | 2026-08-30 17:01 | PASS | issue #758 | +| registry-portability | A | 2026-08-30 17:01 | SKIPPED | issue #758 | +| retro-deterministic-contract | A | 2026-08-30 17:01 | PASS | issue #443 — /retro deterministic output and self-contained helper contract | +| rl-delegation-write-worker | A | 2026-08-30 17:01 | PASS | retro lesson 2026-06-10 (rl-delegation) #57 | +| rlm-context-budget | A | 2026-08-30 17:01 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-006 | +| runtime-preflight-gate | A | 2026-08-30 17:01 | PASS | issue #806 § B1 (open sandbox.substrate vs sandbox.runtime selector); | +| sandbox-boot-guard-ci | A | 2026-08-30 17:01 | PASS | issue #449 (sandbox image build CI guard) 2026-06-19; | +| sandbox-node-base | A | 2026-08-30 17:01 | PASS | openharness#878 — oh as the only front door, T0 sandbox base image | +| skill-paths | A | 2026-08-30 17:01 | PASS | issue #43 — stale path references; extended by issue #69 — apps/->packages/ rename guard; extended by issue #870 — deleted .oh/agents/advisor.md | +| skills-dir-clean | A | 2026-08-30 17:01 | PASS | conversation 2026-06-29 — Pi parses every top-level `.md` in the skills | +| skills-task-tool-coupling | A | 2026-08-30 17:01 | PASS | council review 2026-08-29 (issue #886) — /delegate instructed Claude-Code-only | +| skills-vendored | A | 2026-08-30 17:01 | PASS | absorb .mifune submodule into .oh — the skills/agents/hooks pack is vendored | +| slack-admin-command-surface | A | 2026-08-30 17:01 | PASS | issue #354 — Slack bridge docs must distinguish Pi /msg-bridge commands from Slack DM admin text handlers | +| spec-family-contract | A | 2026-08-30 17:01 | PASS | issue #265; spec-simplification issue #816; workflow authority issue #854 | +| spec-ready-finalization | A | 2026-08-30 17:01 | PASS | issue #134; spec-simplification issue #816; workflow authority issue #854 | +| ste-checker-contract | A | 2026-08-30 17:01 | PASS | issue #750 PR audit — the /ste checker had four fail-open paths (unclosed | +| submitted-by-trailers | A | 2026-08-30 17:01 | PASS | conversation 2026-06-12 (commit attribution trailers); the single-owner | +| sync-skill-contract | A | 2026-08-30 17:01 | PASS | issue #331 — /sync dispatcher skill (bidirectional origin↔upstream sync) | +| tool-catalog-boundary | A | 2026-08-30 17:01 | PASS | agent-browser's exclusion from the harness catalog (#821) and the | +| version-parity | A | 2026-08-30 17:01 | PASS | conversation 2026-08-29 — the oh CLI became the only lifecycle door, so its | +| weigh-scorer-contract | A | 2026-08-30 17:01 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-003 (2026-06-27) | +| wiki-readme-index | A | 2026-08-30 17:01 | PASS | issue #132 — wiki README index drift guard | +| workflow-boundaries | A | 2026-08-30 17:01 | PASS | conversation 2026-06-19 (workflow consolidation, issue #259); authority moved to /spec in issue #854 | +| worktrees-layout | A | 2026-08-30 17:01 | PASS | issue #872 | diff --git a/.oh/evals/probes/oh-home-mount.sh b/.oh/evals/probes/oh-home-mount.sh new file mode 100755 index 00000000..f8548901 --- /dev/null +++ b/.oh/evals/probes/oh-home-mount.sh @@ -0,0 +1,142 @@ +#!/usr/bin/env bash +# tier: A +# source: issue #898 (single $HOME mount) 2026-08-30 +# desc: One mount at /home/sandbox replaces the per-tool auth volumes; seed_home populates it for both mount kinds; the UID repair prunes the checkout instead of relying on -xdev +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" +COMPOSE_PRIMARY="$ROOT/.devcontainer/docker-compose.yml" +COMPOSE_IO="$ROOT/.devcontainer/docker-compose.image-only.yml" +ENTRYPOINT="$ROOT/.devcontainer/entrypoint.sh" +DOCKERFILE="$ROOT/.devcontainer/Dockerfile" + +for f in "$COMPOSE_PRIMARY" "$COMPOSE_IO" "$ENTRYPOINT" "$DOCKERFILE"; do + [[ -f "$f" ]] || { echo "SKIPPED: missing $f" >&2; exit 2; } +done + +fails=() + +RETIRED_VOLUMES=(claude-auth codex-auth pi-auth opencode-auth grok-auth + deepagents-auth herdr-data cloudflared-auth ssh-config config-dir + cc-safety-net oh_workspace) + +for compose in "$COMPOSE_PRIMARY" "$COMPOSE_IO"; do + label="$(basename "$compose")" + + home_mounts="$(grep -cE '^[[:space:]]*-[[:space:]]*[^[:space:]]+:/home/sandbox$' "$compose" || true)" + if [[ "${home_mounts:-0}" -ne 1 ]]; then + fails+=("$label must have exactly one mount targeting /home/sandbox (found ${home_mounts:-0})") + fi + + grep -Eq '^[[:space:]]*-[[:space:]]*\$\{OH_HOME_MOUNT:-workspace\}:/home/sandbox$' "$compose" \ + || fails+=("$label must mount \${OH_HOME_MOUNT:-workspace} at /home/sandbox so a blank oh.json storage.homePath falls back to the named volume") + + grep -qE '^ workspace:$' "$compose" \ + || fails+=("$label must declare the top-level named volume 'workspace' (compose prefixes it with the project name)") + + if grep -qE '^[[:space:]]*name:[[:space:]]*\S+' <(sed -n '/^volumes:/,$p' "$compose"); then + fails+=("$label must not pin an explicit volume name: the project name: key supplies the _ prefix, and a pinned name removes it") + fi + + for vol in "${RETIRED_VOLUMES[@]}"; do + if grep -qE "^[[:space:]]*-?[[:space:]]*${vol}:" "$compose"; then + fails+=("$label still references the retired per-tool volume '${vol}'") + fi + done +done + +grep -Eq '^[[:space:]]*-[[:space:]]*\.\.:/home/sandbox/harness$' "$COMPOSE_PRIMARY" \ + || fails+=("docker-compose.yml must bind the checkout at the fixed path /home/sandbox/harness, nested inside the home mount") + +if grep -qE '^[[:space:]]*-[[:space:]]*\.\.:' "$COMPOSE_IO"; then + fails+=("docker-compose.image-only.yml must NOT bind a checkout — the workspace is a directory inside the home mount") +fi + +grep -Eq '^RUN mv /home/sandbox /opt/home-seed' "$DOCKERFILE" \ + || fails+=("Dockerfile must move the baked home to /opt/home-seed so a host bind and a named volume are seeded identically") +grep -Fq 'install -d -o sandbox -g sandbox -m 0755 /home/sandbox' "$DOCKERFILE" \ + || fails+=("Dockerfile must leave /home/sandbox empty after staging the seed (an empty named volume must not auto-copy)") +grep -Fq 'rm -rf /opt/home-seed/harness' "$DOCKERFILE" \ + || fails+=("Dockerfile must drop harness/ from the seed, or seed_home would create an empty directory under the checkout bind") + +if grep -E -- '-xdev' "$ENTRYPOINT" | grep -Fq '/home/sandbox'; then + fails+=("entrypoint.sh must not use -xdev under /home/sandbox: once home is a mount it shares a device with the checkout and stops pruning it") +fi +grep -Fq 'find /home/sandbox -path "$OH_PROJECT_ROOT" -prune -o' "$ENTRYPOINT" \ + || fails+=("entrypoint.sh must prune \$OH_PROJECT_ROOT explicitly when repairing home ownership") + +# ── behavioral sim of the fenced seed_home ──────────────────────────── +seed_fn_file="$(mktemp)" +trap 'rm -rf "$seed_fn_file" "${simdir:-}"' EXIT +awk '/^# >>> seed_home >>>$/{f=1;next} /^# <<< seed_home <<<$/{f=0} f' "$ENTRYPOINT" > "$seed_fn_file" + +if [[ ! -s "$seed_fn_file" ]]; then + fails+=("entrypoint.sh must define seed_home inside the '# >>> seed_home >>>' fence so this probe can exercise it") +else + simdir="$(mktemp -d)" + src="$simdir/seed"; mkdir -p "$src/.oh-my-zsh" + printf 'baked\n' > "$src/.zshrc" + printf 'baked\n' > "$src/.oh-my-zsh/marker" + + # (a) fresh dest gets the whole baked home + dest_a="$simdir/a" + ( set -e; source "$seed_fn_file"; OH_HOME_SEED_SRC="$src" seed_home "$dest_a" ) + [[ -f "$dest_a/.zshrc" && -f "$dest_a/.oh-my-zsh/marker" ]] \ + || fails+=("seed sim (a): an empty home mount must receive the baked dotfiles") + + # (b) operator state is never clobbered, and new files still backfill + dest_b="$simdir/b"; mkdir -p "$dest_b" + printf 'mine\n' > "$dest_b/.zshrc" + ( set -e; source "$seed_fn_file"; OH_HOME_SEED_SRC="$src" seed_home "$dest_b" ) + [[ "$(cat "$dest_b/.zshrc")" == "mine" ]] \ + || fails+=("seed sim (b): seed_home must never clobber an existing file") + [[ -f "$dest_b/.oh-my-zsh/marker" ]] \ + || fails+=("seed sim (b): seed_home must still backfill files the dest lacks, so image upgrades land") + + # (b2) an existing top-level DIRECTORY is left entirely alone — mode + # included. A plain `cp -a -n` skips existing files but still rewrites the + # mode of directories, silently relaxing ~/.ssh from 0700 on every boot. + mkdir -p "$src/.ssh"; chmod 755 "$src/.ssh" + printf 'baked\n' > "$src/.ssh/known_hosts" + dest_b2="$simdir/b2"; mkdir -p "$dest_b2/.ssh"; chmod 700 "$dest_b2/.ssh" + ( set -e; source "$seed_fn_file"; OH_HOME_SEED_SRC="$src" seed_home "$dest_b2" ) + [[ "$(stat -c %a "$dest_b2/.ssh")" == "700" ]] \ + || fails+=("seed sim (b2): seed_home must not rewrite the mode of a directory the home mount already has") + [[ ! -e "$dest_b2/.ssh/known_hosts" ]] \ + || fails+=("seed sim (b2): seed_home must not descend into an existing top-level entry — seeding is per top-level entry, so a whole-tree walk would re-copy the uv cache on every boot") + + # (b3) a top-level directory the mount lacks arrives whole, with its mode + dest_b3="$simdir/b3" + ( set -e; source "$seed_fn_file"; OH_HOME_SEED_SRC="$src" seed_home "$dest_b3" ) + [[ "$(stat -c %a "$dest_b3/.ssh")" == "755" ]] \ + || fails+=("seed sim (b3): a directory absent from the home mount must arrive with the seed's mode") + [[ -f "$dest_b3/.ssh/known_hosts" ]] \ + || fails+=("seed sim (b3): a directory absent from the home mount must arrive with its contents") + + # (c) idempotent across boots + ( set -e; source "$seed_fn_file"; OH_HOME_SEED_SRC="$src" seed_home "$dest_a" ) + [[ "$(cat "$dest_a/.zshrc")" == "baked" ]] \ + || fails+=("seed sim (c): a second boot must leave an already-seeded home unchanged") + + # (d) a missing seed source is a no-op, never a boot failure + ( set -e; source "$seed_fn_file"; OH_HOME_SEED_SRC="$simdir/absent" seed_home "$simdir/d" ) \ + || fails+=("seed sim (d): seed_home must return 0 when the baked seed is absent") + + # (e) a genuine copy failure is reported, not swallowed + dest_e="$simdir/e"; mkdir -p "$dest_e"; chmod 500 "$dest_e" + if [[ "$(id -u)" != "0" ]]; then + if ( source "$seed_fn_file"; OH_HOME_SEED_SRC="$src" seed_home "$dest_e" ) 2>/dev/null; then + fails+=("seed sim (e): seed_home must return non-zero when it cannot write the home mount, so a partial seed is visible in the boot log") + fi + fi + chmod 700 "$dest_e" +fi + +if (( ${#fails[@]} > 0 )); then + echo "REGRESSION: single-\$HOME-mount contract broken:" >&2 + printf ' - %s\n' "${fails[@]}" >&2 + exit 1 +fi + +echo "PASS: one \${OH_HOME_MOUNT:-workspace} mount at /home/sandbox in both compose files with the per-tool volumes retired and no pinned volume name; the checkout binds at the fixed /home/sandbox/harness; the Dockerfile stages /opt/home-seed and leaves the image home empty; entrypoint prunes \$OH_PROJECT_ROOT instead of -xdev; seed_home copies whole top-level entries the mount lacks, never touches one it already has (mode included), reports write failures, and no-ops without a seed" >&2 +exit 0 diff --git a/.oh/evals/probes/oh-image-only-deploy.sh b/.oh/evals/probes/oh-image-only-deploy.sh index 68857d18..f09455be 100755 --- a/.oh/evals/probes/oh-image-only-deploy.sh +++ b/.oh/evals/probes/oh-image-only-deploy.sh @@ -6,11 +6,11 @@ # elif, and defines seed_workspace_volume/.image-seeded; a behavioral sim # (fenced function extracted in isolation, no full entrypoint source) proves # fresh-seed, idempotent-reseed, and no-clobber-of-existing-.oh/ behavior; -# docker-compose.image-only.yml mounts a named oh_workspace volume, sets +# docker-compose.image-only.yml mounts the single home volume, sets # OH_IMAGE_ONLY=1, parameterizes image:, sets pull_policy:, and has neither # build: nor a `..:` bind mount; the primary docker-compose.yml still keeps # its `..:` bind mount (regression floor); the deploy doc has dropped the -# "Not yet" placeholder and documents oh_workspace/OH_IMAGE_ONLY; the +# "Not yet" placeholder and documents OH_HOME_MOUNT/OH_IMAGE_ONLY; the # Dockerfile (if present) stages /opt/oh-seed for the entrypoint to seed from. set -euo pipefail @@ -86,8 +86,8 @@ else fi fi -grep -Eq '^[[:space:]]*-[[:space:]]*oh_workspace:\$\{OH_PROJECT_ROOT' "$COMPOSE_IO" \ - || fails+=("docker-compose.image-only.yml must mount a named oh_workspace volume at \${OH_PROJECT_ROOT}") +grep -Eq '^[[:space:]]*-[[:space:]]*\$\{OH_HOME_MOUNT:-workspace\}:/home/sandbox$' "$COMPOSE_IO" \ + || fails+=("docker-compose.image-only.yml must mount \${OH_HOME_MOUNT:-workspace} at /home/sandbox") grep -Fq 'OH_IMAGE_ONLY=1' "$COMPOSE_IO" \ || fails+=("docker-compose.image-only.yml must set OH_IMAGE_ONLY=1 in the container environment") grep -Eq 'image:[[:space:]]*\$\{OH_SANDBOX_IMAGE' "$COMPOSE_IO" \ @@ -115,8 +115,8 @@ else if [[ "${not_yet_count:-0}" -ne 0 ]]; then fails+=("deployment-prebuilt-image.md still contains the 'Not yet' placeholder (${not_yet_count} occurrence(s))") fi - grep -Fq 'oh_workspace' "$DOC" \ - || fails+=("deployment-prebuilt-image.md must mention oh_workspace") + grep -Fq 'OH_HOME_MOUNT' "$DOC" \ + || fails+=("deployment-prebuilt-image.md must mention OH_HOME_MOUNT") grep -Fq 'OH_IMAGE_ONLY' "$DOC" \ || fails+=("deployment-prebuilt-image.md must mention OH_IMAGE_ONLY") fi @@ -142,5 +142,5 @@ if (( ${#fails[@]} > 0 )); then exit 1 fi -echo "PASS: Flavor B (image-only) contract — entrypoint gates OH_IMAGE_ONLY before the host-UID-sync elif and defines seed_workspace_volume/.image-seeded; behavioral sim confirms fresh-seed, idempotent-reseed, and no-clobber-of-existing-.oh/; docker-compose.image-only.yml mounts oh_workspace, sets OH_IMAGE_ONLY=1, parameterizes image:/pull_policy:, and has no build:/'..:' bind mount; primary docker-compose.yml still binds '..:' (regression floor); deploy doc drops the 'Not yet' placeholder and documents oh_workspace/OH_IMAGE_ONLY; Dockerfile stages /opt/oh-seed" >&2 +echo "PASS: Flavor B (image-only) contract — entrypoint gates OH_IMAGE_ONLY before the host-UID-sync elif and defines seed_workspace_volume/.image-seeded; behavioral sim confirms fresh-seed, idempotent-reseed, and no-clobber-of-existing-.oh/; docker-compose.image-only.yml mounts \${OH_HOME_MOUNT:-workspace} at /home/sandbox, sets OH_IMAGE_ONLY=1, parameterizes image:/pull_policy:, and has no build:/'..:' bind mount; primary docker-compose.yml still binds '..:' (regression floor); deploy doc drops the 'Not yet' placeholder and documents OH_HOME_MOUNT/OH_IMAGE_ONLY; Dockerfile stages /opt/oh-seed" >&2 exit 0 diff --git a/.oh/evals/probes/project-root-seam.sh b/.oh/evals/probes/project-root-seam.sh deleted file mode 100755 index 2a043cc2..00000000 --- a/.oh/evals/probes/project-root-seam.sh +++ /dev/null @@ -1,48 +0,0 @@ -#!/usr/bin/env bash -# tier: A -# source: issue #531 Phase 1 (OH_PROJECT_ROOT project-root seam) 2026-06-26 -# desc: Guards the OH_PROJECT_ROOT project-root seam — #531 Phase 1 -set -euo pipefail - -ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" -COMPOSE="$ROOT/.devcontainer/docker-compose.yml" -DOCKERFILE="$ROOT/.devcontainer/Dockerfile" -ENTRYPOINT="$ROOT/.devcontainer/entrypoint.sh" - -[[ -f "$COMPOSE" ]] || { echo "SKIPPED: missing $COMPOSE" >&2; exit 2; } -[[ -f "$DOCKERFILE" ]] || { echo "SKIPPED: missing $DOCKERFILE" >&2; exit 2; } -[[ -f "$ENTRYPOINT" ]] || { echo "SKIPPED: missing $ENTRYPOINT" >&2; exit 2; } - -missing=() - -grep -qE 'OH_PROJECT_ROOT:' "$COMPOSE" \ - || missing+=("compose: OH_PROJECT_ROOT build arg missing") -grep -qE 'OH_PROJECT_ROOT=' "$COMPOSE" \ - || missing+=("compose: OH_PROJECT_ROOT env var missing") - -grep -qF 'ARG OH_PROJECT_ROOT=' "$DOCKERFILE" \ - || missing+=("Dockerfile: ARG OH_PROJECT_ROOT= missing") -grep -qF 'ENV OH_PROJECT_ROOT' "$DOCKERFILE" \ - || missing+=("Dockerfile: ENV OH_PROJECT_ROOT missing") - -grep -qF "OH_PROJECT_ROOT=\"\${OH_PROJECT_ROOT:-/home/sandbox/harness}\"" "$ENTRYPOINT" \ - || missing+=("entrypoint: OH_PROJECT_ROOT seam definition missing") - -grep -qF "HARNESS=\"\${HARNESS:-\$OH_PROJECT_ROOT}\"" "$ENTRYPOINT" \ - || missing+=("entrypoint: HARNESS alias must be \${HARNESS:-\$OH_PROJECT_ROOT}") - -if (( ${#missing[@]} )); then - printf 'REGRESSION: project-root-seam contract missing: %s\n' "${missing[*]}" >&2 - exit 1 -fi - -bare_harness=$(grep -vE '^[[:space:]]*#' "$ENTRYPOINT" \ - | grep -E "HARNESS=['\"]?/home/sandbox/harness" \ - | grep -vE ':-' || true) -if [[ -n "$bare_harness" ]]; then - echo "REGRESSION: entrypoint.sh has a bare unconditional HARNESS=/home/sandbox/harness (defeats the seam)" >&2 - exit 1 -fi - -echo "PASS: OH_PROJECT_ROOT project-root seam defined in compose, Dockerfile, and entrypoint; HARNESS alias chains through the seam" >&2 -exit 0 diff --git a/.oh/evals/probes/worktrees-layout.sh b/.oh/evals/probes/worktrees-layout.sh index 56b07d1a..05fb65e8 100755 --- a/.oh/evals/probes/worktrees-layout.sh +++ b/.oh/evals/probes/worktrees-layout.sh @@ -103,8 +103,8 @@ if bash .oh/scripts/oh-path definitely-not-a-harness-dir --no-create >/dev/null exit 1 fi -if ! grep -Fq 'OH_PROJECT_ROOT' .devcontainer/docker-compose.yml; then - echo "REGRESSION: OH_PROJECT_ROOT was swept up with the retired layout knobs" >&2 +if ! grep -Fq 'ENV OH_PROJECT_ROOT=/home/sandbox/harness' .devcontainer/Dockerfile; then + echo "REGRESSION: OH_PROJECT_ROOT lost its fixed image-level definition" >&2 exit 1 fi diff --git a/.oh/scripts/__tests__/entrypoint.test.ts b/.oh/scripts/__tests__/entrypoint.test.ts index 2d8c1f1a..94103503 100644 --- a/.oh/scripts/__tests__/entrypoint.test.ts +++ b/.oh/scripts/__tests__/entrypoint.test.ts @@ -11,28 +11,45 @@ function entrypoint(): string { return readFileSync(ENTRYPOINT, "utf8"); } -describe("devcontainer entrypoint auth volume ownership", () => { - it("repairs auth mounts with the sandbox user's current numeric uid/gid", () => { +describe("devcontainer entrypoint home mount ownership", () => { + it("repairs the whole home mount with the sandbox user's current numeric uid/gid", () => { const text = entrypoint(); expect(text).toContain("sandbox_ownership()"); expect(text).toContain('$(id -u sandbox)'); expect(text).toContain('$(id -g sandbox)'); expect(text).toContain('owner="$(sandbox_ownership)"'); - expect(text).toContain('chown -hR "$owner" "/home/sandbox/$dir"'); - expect(text).toContain(".local/share/opencode"); - expect(text).toContain("/home/sandbox/.hermes"); - expect(text).toContain("Do not recurse\n # into $HERMES_HOME when it points at the bind-mounted checkout"); + expect(text).toContain('find /home/sandbox -path "$OH_PROJECT_ROOT" -prune -o'); + expect(text).toContain('-exec chown -h "$owner" {} +'); + expect(text).toContain("chmod 700 /home/sandbox/.ssh"); }); - it("runs auth mount repair before and after host UID reconciliation", () => { + it("prunes the checkout rather than relying on -xdev, which the home mount defeats", () => { + const text = entrypoint(); + + expect(text).not.toContain("-xdev"); + expect(text).not.toContain('chown -hR "$owner" "/home/sandbox/$dir"'); + }); + + it("seeds the home mount from the image before host UID reconciliation", () => { + const text = entrypoint(); + const seedFn = text.indexOf("# >>> seed_home >>>"); + const seedCall = text.indexOf("seed_home /home/sandbox"); + const uidSync = text.indexOf("usermod -u \"$HOST_UID\" sandbox"); + + expect(seedFn).toBeGreaterThan(-1); + expect(text).toContain('if [ -e "$dest/$name" ] || [ -L "$dest/$name" ]; then'); + expect(text).toContain('find . -mindepth 1 -maxdepth 1'); + expect(text).toContain('${OH_HOME_SEED_SRC:-/opt/home-seed}'); + expect(seedCall).toBeGreaterThan(seedFn); + expect(uidSync).toBeGreaterThan(seedCall); + }); + + it("runs home mount repair after host UID reconciliation", () => { const text = entrypoint(); - const firstRepair = text.indexOf("repair_home_mount_ownership\n\n# ─── Host UID reconciliation"); const uidSync = text.indexOf("usermod -u \"$HOST_UID\" sandbox"); const secondRepair = text.indexOf("# UID/GID reconciliation can change"); - expect(firstRepair).toBeGreaterThan(-1); - expect(uidSync).toBeGreaterThan(firstRepair); expect(secondRepair).toBeGreaterThan(uidSync); const postUidSync = text.slice(secondRepair); const secondRepairCall = postUidSync.indexOf("repair_home_mount_ownership"); @@ -65,13 +82,11 @@ describe("devcontainer entrypoint auth volume ownership", () => { text.indexOf("# UID/GID reconciliation can change"), ); const usermod = block.indexOf("uid_reconcile_step \"set sandbox UID to host UID $HOST_UID\" usermod -u \"$HOST_UID\" sandbox"); - const chown = block.indexOf("uid_reconcile_step \"repair sandbox-owned files after UID/GID sync\" find /home/sandbox"); const success = block.indexOf("sandbox UID synced to host"); const incomplete = block.indexOf("sandbox UID/GID reconciliation incomplete"); expect(usermod).toBeGreaterThan(-1); - expect(chown).toBeGreaterThan(usermod); - expect(success).toBeGreaterThan(chown); + expect(success).toBeGreaterThan(usermod); expect(incomplete).toBeGreaterThan(success); expect(block).toContain("if [ \"$UID_GID_SYNC_OK\" = \"true\" ]; then"); }); diff --git a/.oh/scripts/__tests__/herdr-default.test.ts b/.oh/scripts/__tests__/herdr-default.test.ts index bb7319ca..1b10150f 100644 --- a/.oh/scripts/__tests__/herdr-default.test.ts +++ b/.oh/scripts/__tests__/herdr-default.test.ts @@ -22,20 +22,18 @@ describe("default Herdr integration", () => { (composeFile) => { const compose = readRepoFile(`.devcontainer/${composeFile}`); - expect(compose).toContain("herdr-data:/home/sandbox/.herdr"); - expect(compose).toContain("config-dir:/home/sandbox/.config"); - expect(compose).not.toContain("/home/sandbox/.config/herdr"); - expect(compose).not.toContain("/home/sandbox/.config/gh"); - expect(compose).toMatch(/^ herdr-data:$/m); - expect(compose).toMatch(/^ config-dir:$/m); + expect(compose).toContain("${OH_HOME_MOUNT:-workspace}:/home/sandbox"); + expect(compose).not.toContain("/home/sandbox/.herdr"); + expect(compose).not.toContain("/home/sandbox/.config"); + expect(compose).toMatch(/^ workspace:$/m); }, ); - it("repairs ownership for Herdr volumes after UID sync", () => { + it("repairs ownership for Herdr state after UID sync", () => { const entrypoint = readRepoFile(".devcontainer/entrypoint.sh"); - expect(entrypoint).toContain(".herdr"); - expect(entrypoint).toMatch(/for dir in .* \.config /); + expect(entrypoint).toContain('find /home/sandbox -path "$OH_PROJECT_ROOT" -prune -o'); + expect(entrypoint).toContain('-exec chown -h "$owner" {} +'); }); it("makes Herdr the first interactive action in canonical onboarding", () => { @@ -64,6 +62,6 @@ describe("default Herdr integration", () => { expect(herdrDocs).toContain("~/.config/herdr"); expect(herdrDocs).toContain("~/.herdr/worktrees"); expect(herdrDocs).not.toContain("herdr update"); - expect(imageDocs).toContain("herdr-data:/home/sandbox/.herdr"); + expect(imageDocs).toContain(":/home/sandbox \\"); }); }); diff --git a/.oh/scripts/__tests__/provision-python.test.ts b/.oh/scripts/__tests__/provision-python.test.ts index 5e434cce..47a5d239 100644 --- a/.oh/scripts/__tests__/provision-python.test.ts +++ b/.oh/scripts/__tests__/provision-python.test.ts @@ -112,8 +112,9 @@ describe("entrypoint uv ownership repair", () => { it("repairs the uv tree on every boot", () => { const text = entrypoint(); expect(text).toContain("/home/sandbox/.local/share/uv/python"); - expect(text).toContain('for uv_dir in /home/sandbox/.local/share/uv /home/sandbox/.cache/uv'); - expect(text).toContain('chown -hR "$owner" "$uv_dir"'); + expect(text).toContain("/home/sandbox/.cache/uv"); + expect(text).toContain('find /home/sandbox -path "$OH_PROJECT_ROOT" -prune -o'); + expect(text).toContain('-exec chown -h "$owner" {} +'); }); it("runs provisioning after provider links and does not abort boot on failure", () => { diff --git a/.oh/scripts/__tests__/sandbox-healthcheck.test.ts b/.oh/scripts/__tests__/sandbox-healthcheck.test.ts index 27a79baf..e5ae10e3 100644 --- a/.oh/scripts/__tests__/sandbox-healthcheck.test.ts +++ b/.oh/scripts/__tests__/sandbox-healthcheck.test.ts @@ -135,7 +135,7 @@ describe("sandbox healthcheck", () => { const compose = readFileSync(COMPOSE, "utf8"); expect(compose).toContain("healthcheck:"); - expect(compose).toContain("${OH_PROJECT_ROOT:-/home/sandbox/harness}/.oh/scripts/sandbox-healthcheck.sh"); + expect(compose).toContain("/home/sandbox/harness/.oh/scripts/sandbox-healthcheck.sh"); expect(compose).toContain("start_period: 300s"); }); diff --git a/CHANGELOG.md b/CHANGELOG.md index f79b6f0f..afd57deb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,7 +8,14 @@ Update policy and release automation live in [`/git`](.claude/skills/git/SKILL.m ## [Unreleased] +### Changed +- **BREAKING:** Persist the sandbox home through one `/home/sandbox` mount, not eleven per-tool volumes; set `storage.homePath` for a host path, else `_workspace` ([#898](https://github.com/mifunedev/openharness/issues/898)). + +### Removed +- **BREAKING:** Retire the `projectRoot` / `OH_PROJECT_ROOT` config knob — the checkout is fixed at `/home/sandbox/harness`, nested inside the home mount ([#898](https://github.com/mifunedev/openharness/issues/898)). + ### Added +- Add `oh-home-mount.sh`, a tier-A probe holding the single-`$HOME`-mount contract: one mount per compose file, the baked `/opt/home-seed`, and the checkout prune that replaces `-xdev` ([#898](https://github.com/mifunedev/openharness/issues/898)). - Add `skills-task-tool-coupling.sh`, a tier-A probe holding the canonical skill pack and the sandbox in agreement about the Claude-Code-only task tools ([#886](https://github.com/mifunedev/openharness/issues/886)). ### Fixed diff --git a/docs/configuration.md b/docs/configuration.md index d850a2fe..199c967e 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -43,7 +43,7 @@ to; `—` means the field is consumed by the `oh` CLI itself and never rendered. | `version` | number | `1` | — | Schema version. Must be `1`. | | `name` | string | directory name | `SANDBOX_NAME` | Container and Compose project name. | | `timezone` | string | `America/Los_Angeles` | `TZ` | Timezone for cron schedules and log timestamps. | -| `projectRoot` | string | `/home/sandbox/harness` | `OH_PROJECT_ROOT` | Container path the repository is mounted at. Leave at the default unless relocating the project root. | +| `storage.homePath` | string | unset | `OH_HOME_MOUNT` | Absolute **host** path for the single `/home/sandbox` mount. Leave unset and Docker manages it as the named volume `_workspace`. Must start with `/`; use a dedicated empty directory, since the sandbox takes ownership of it. A stale `OH_HOME_MOUNT` in `.devcontainer/.env` outranks this value, because the wrapper passes the dotenv last. | ### Git identity inside the sandbox diff --git a/docs/deployment-prebuilt-image.md b/docs/deployment-prebuilt-image.md index 612ba418..48de044b 100644 --- a/docs/deployment-prebuilt-image.md +++ b/docs/deployment-prebuilt-image.md @@ -147,9 +147,11 @@ is a standalone compose file — no `..:` bind mount, no `build:` stanza: docker compose -f .devcontainer/docker-compose.image-only.yml up -d ``` -This pulls and runs the published image with **no clone and no build**. The -workspace and control plane live entirely in the named `oh_workspace` volume -declared in that file, mounted at `$OH_PROJECT_ROOT`. +This pulls and runs the published image with **no clone and no build**. +Everything the sandbox persists — the workspace and control plane at +`/home/sandbox/harness` included — lives in the single `/home/sandbox` mount +declared in that file: the named volume `_workspace` by default, +or an absolute host path when `OH_HOME_MOUNT` is set. ### `OH_IMAGE_ONLY=1` @@ -167,7 +169,7 @@ mount, so its host-UID-sync path is unchanged. ### Seed-to-volume persistence -On the **first boot** against an empty `oh_workspace` volume, the entrypoint +On the **first boot** against an empty home mount, the entrypoint seeds the baked control plane — from the image's `/opt/oh-seed` — into the volume, then writes the marker `.oh/.image-seeded`. From that point on, the **volume is authoritative**: it is the operator-editable copy of `.oh/` (and @@ -207,20 +209,15 @@ NAME=openharness # ── 1. Clear previous state ── DESTRUCTIVE: wipes the seeded workspace ── docker rm -f "$NAME" 2>/dev/null || true -docker volume rm oh_workspace 2>/dev/null || true # the seeded .oh/ control plane +docker volume rm "${NAME}_workspace" 2>/dev/null || true # the whole sandbox home # ── 2. Fresh run (no bind mount, no build) ───────────────────────── docker run -d --name "$NAME" --restart unless-stopped --init \ -e OH_IMAGE_ONLY=1 \ - -e OH_PROJECT_ROOT=/home/sandbox/harness \ -e GIT_USER_NAME="ryaneggz" \ -e GIT_USER_EMAIL="kre8mymedia@gmail.com" \ -e GH_TOKEN="${GH_TOKEN:-}" \ - -v oh_workspace:/home/sandbox/harness \ - -v claude-auth:/home/sandbox/.claude \ - -v config-dir:/home/sandbox/.config \ - -v herdr-data:/home/sandbox/.herdr \ - -v ssh-config:/home/sandbox/.ssh \ + -v "${NAME}_workspace":/home/sandbox \ "$IMAGE" sleep infinity # ── 3. Verify the seed + provider wiring ─────────────────────────── @@ -233,7 +230,7 @@ docker exec "$NAME" bash -lc ' ``` A healthy boot ends with `Providers OK: …` and `SEED_OK`, and the logs show -**no** `protected-paths.txt is missing`. The `oh_workspace` volume is now +**no** `protected-paths.txt is missing`. The home mount is now authoritative — later boots see the `.oh/.image-seeded` marker and skip re-seeding, so your in-container edits persist. diff --git a/docs/harnesses/claude-code.md b/docs/harnesses/claude-code.md index 9bc824d0..b9b6145e 100644 --- a/docs/harnesses/claude-code.md +++ b/docs/harnesses/claude-code.md @@ -38,7 +38,7 @@ claude auth status # confirm you're authenticated the same OAuth flow, but `claude auth login` is the explicit, scriptable path. Credentials are stored in `~/.claude/.credentials.json` inside the sandbox (persisted via the -`claude-auth` volume). The sandbox banner at login indicates whether credentials are present. +`/home/sandbox` mount). The sandbox banner at login indicates whether credentials are present. ## Optional Langfuse observability diff --git a/docs/harnesses/deepagents.md b/docs/harnesses/deepagents.md index beed8d4b..9f74fee7 100644 --- a/docs/harnesses/deepagents.md +++ b/docs/harnesses/deepagents.md @@ -73,7 +73,7 @@ If the command is not found, confirm `INSTALL_DEEPAGENTS=true` is set in DeepAgents reads provider API keys from `~/.deepagents/.env` and CLI defaults from `~/.deepagents/config.toml`. The directory is persisted by -the `deepagents-auth` named volume by default, so credentials survive +the single `/home/sandbox` mount by default, so credentials survive container rebuilds. Create the env file on first use: @@ -99,7 +99,7 @@ durability and review semantics: | Path | Scope | Persistence | Notes | |---|---|---|---| -| `~/.deepagents/` | Per-sandbox user state | Survives rebuilds via the `deepagents-auth` volume | Provider keys, model defaults, memory, skills, sessions live here. **Only place secrets here.** | +| `~/.deepagents/` | Per-sandbox user state | Survives rebuilds via the `/home/sandbox` mount | Provider keys, model defaults, memory, skills, sessions live here. **Only place secrets here.** | | `/.deepagents/` | Per-project | Whatever git decides | Project memory and skills the agent may load from the workspace root. **Treat as project data — follows normal `.gitignore` and code-review rules.** | A repo-local `.deepagents/` directory may be read by the agent and may be diff --git a/docs/harnesses/grok-build.md b/docs/harnesses/grok-build.md index f63e2a62..c31552c1 100644 --- a/docs/harnesses/grok-build.md +++ b/docs/harnesses/grok-build.md @@ -84,7 +84,7 @@ environment variables as convenience secret storage only; users and processes with Docker/container access may be able to inspect them. :::warning Auth precedence -Cached OAuth/session state in `~/.grok/auth.json` takes precedence over `XAI_API_KEY`. If Grok Build appears to ignore a new API key, run `grok logout` or reset the `grok-auth` volume, then try again. +Cached OAuth/session state in `~/.grok/auth.json` takes precedence over `XAI_API_KEY`. If Grok Build appears to ignore a new API key, run `grok logout` or delete `~/.grok`, then try again. ::: ## Common usage @@ -109,7 +109,7 @@ tmux attach -t agent-grok ## State persistence -Open Harness mounts the `grok-auth` named volume at `/home/sandbox/.grok` (`~/.grok`) alongside the other agent auth volumes. This volume persists **Grok user state written under `~/.grok`** across container rebuilds, such as: +Open Harness persists `~/.grok` in the single `/home/sandbox` mount, alongside every other agent's state. That keeps **Grok user state written under `~/.grok`** across container rebuilds, such as: - auth and cached sessions (`auth.json`) - config @@ -119,7 +119,7 @@ Open Harness mounts the `grok-auth` named volume at `/home/sandbox/.grok` (`~/.g - logs :::warning Volume removal deletes Grok state -`oh destroy` and `docker compose down -v` remove named volumes, including `grok-auth`. Use `oh stop` when you want Grok Build state under `~/.grok` to survive. +`oh destroy` and `docker compose down -v` delete the sandbox home volume, `~/.grok` included. Use `oh stop` when you want Grok Build state under `~/.grok` to survive. ::: ## Dangerous flags diff --git a/docs/harnesses/hermes.md b/docs/harnesses/hermes.md index 9d0b4147..e386ddbb 100644 --- a/docs/harnesses/hermes.md +++ b/docs/harnesses/hermes.md @@ -100,7 +100,7 @@ skills used by Claude, Codex, and Pi visible to Hermes by default. Auth lives directly inside `HERMES_HOME` (`~/harness/.hermes/auth.json`). No symlink or named volume is involved: an earlier design symlinked -`auth.json` into a home-scoped `hermes-auth` Docker volume, but that +`auth.json` into a home-scoped Docker volume of its own, but that volume sits on a different filesystem from the bind-mounted checkout and caused Hermes' atomic-replace writes to fail with `EXDEV`. Keeping auth on the same bind-mount device fixes this; the entrypoint heals any diff --git a/docs/installation.md b/docs/installation.md index 7b5eed9b..83372748 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -341,24 +341,77 @@ claude → claude --dangerously-skip-permissions codex → codex --dangerously-bypass-approvals-and-sandbox ``` -### Persistent volumes +### Persistent storage -Auth credentials and Herdr workspace state survive container rebuilds via named Docker volumes: +Everything under the sandbox user's home directory — every agent login, the +GitHub CLI token, the SSH keys, shell history, and any state a tool writes +anywhere in `~` — persists through a **single mount at `/home/sandbox`**. -- `claude-auth` → `~/.claude` (Claude Code OAuth) -- `codex-auth` → `~/.codex` (Codex OAuth) -- `opencode-auth` → `~/.local/share/opencode` (OpenCode OAuth; `auth.json`) -- `pi-auth` → `~/.pi` (Pi Agent OAuth) -- `deepagents-auth` → `~/.deepagents` (DeepAgents provider keys, memory, skills, sessions; used when DeepAgents is enabled (`install.deepagents: true` in `oh.json`)). Repo-local `.deepagents/` is **project data** and follows normal `.gitignore` and code-review rules — never put secrets there. -- `hermes-auth` → `~/.hermes` (Hermes auth only; non-auth runtime state defaults to project-local `~/harness/.hermes` when Hermes is enabled (`install.hermes: true` in `oh.json`)) -- `grok-auth` → `~/.grok` (all Grok Build user state: auth, config, sessions, memory, skills/plugins, logs; mounted alongside the other agent auth volumes and used by Grok Build when `install.grokBuild: true` in `oh.json`). Cached OAuth/session state in `~/.grok/auth.json` takes precedence over `XAI_API_KEY`; if an API key seems ignored, run `grok logout` or reset the volume. -- `cloudflared-auth` → `~/.cloudflared` (Cloudflare tunnel credentials, when used) -- `ssh-config` → `~/.ssh` (user SSH keys / known_hosts; entrypoint enforces `chmod 700`) -- `config-dir` → `~/.config` (all XDG tool config, including the GitHub CLI tokens under `~/.config/gh` and Herdr settings) -- `herdr-data` → `~/.herdr` (Herdr-created worktrees and related data; session metadata is under `~/.config/herdr`) +By default Docker manages it as the named volume `_workspace`. +Set `storage.homePath` in `oh.json` to an absolute **host** path and the same +mount becomes a bind, so you can back the sandbox home up, inspect it, or move +it between machines: -Hermes is split: when Hermes is enabled (`install.hermes: true` in `oh.json`), `HERMES_HOME` defaults to the project-local bind-mounted `~/harness/.hermes/` directory, while auth remains in the `~/.hermes` named volume and is linked into the project-local home as `auth.json`. The entrypoint links `.hermes/skills/openharness` to the tracked shared skill directory (`.oh/skills/`) so Hermes sees the same harness skills as Claude, Codex, and Pi without copying them into runtime state. Project-local runtime contents are gitignored except `.hermes/README.md`; `oh destroy` removes the auth volume but not the bind-mounted project runtime directory. +```bash +oh config set storage.homePath /srv/openharness-home +``` + +`oh init` asks for this path during the Project step; leave it blank to keep the +Docker-managed volume. Use a **dedicated, empty** directory — the sandbox takes +ownership of everything in it, so never point it at your own host `$HOME`. + +The repository checkout is bind-mounted at `/home/sandbox/harness`, nested +inside that mount. Its location is fixed, not configurable. + +The image ships its baked home at `/opt/home-seed`. On every boot the entrypoint +copies in each **top-level** entry the mount does not already have, and never +touches one it does — not even its permissions. A fresh mount comes up complete; +an image upgrade adds whatever new top-level entries it introduced (a new agent +CLI's `~/.newtool`, say) and leaves everything you already have alone. It does +not merge new files into a directory the mount already has, which is what the +per-tool volumes did before. + +Hermes is split: when Hermes is enabled (`install.hermes: true` in `oh.json`), +`HERMES_HOME` defaults to the project-local bind-mounted `~/harness/.hermes/` +directory. The entrypoint links `.hermes/skills/openharness` to the tracked +shared skill directory (`.oh/skills/`) so Hermes sees the same harness skills as +Claude, Codex, and Pi without copying them into runtime state. Project-local +runtime contents are gitignored except `.hermes/README.md`. + +`oh destroy` and `docker compose down -v` delete the named volume and everything +in it — provider credentials included; use `oh stop` when you want them to +survive. When `storage.homePath` points at a host bind, `down -v` cannot remove +it, and `oh destroy` says so. + +#### Migrating from the per-tool volumes + +Releases before this change kept eleven separate volumes (`claude-auth`, +`config-dir`, `ssh-config`, and so on). They are not migrated automatically. +**Before** upgrading, copy the old home out of the still-running container: + +```bash +mkdir -p /srv/openharness-home +docker cp :/home/sandbox/. /srv/openharness-home +rm -rf /srv/openharness-home/harness +oh config set storage.homePath /srv/openharness-home +``` + +The trailing `/.` matters: without it `docker cp` places the copy at +`/srv/openharness-home/sandbox/` instead of unpacking its contents, and the +sandbox comes up freshly seeded as though nothing was migrated. The `rm -rf` +drops the copy of the repository checkout — `docker cp` reads through the bind +mount, so the archive includes `harness/` with its `.git` and `node_modules`, +which can be several GB and is shadowed by the checkout bind at runtime anyway. + +Then rebuild. To stay on a Docker-managed volume instead, copy that directory +into the new volume once: + +```bash +docker run --rm -v _workspace:/to -v /srv/openharness-home:/from \ + alpine cp -a /from/. /to/ +``` -`oh destroy` and `docker compose down -v` remove named volumes, including Herdr state and provider credentials; use `oh stop` when you want them to survive. +Skipping this loses every agent login and the SSH keys; nothing else breaks, and +you simply sign in again. Downstream harness packs and Pi extensions can introduce additional volumes or bind-mount overlays by adding paths to `composeOverrides[]` in the tracked `oh.json`. That list is the one place overlay paths live, and only `oh` applies it: VS Code "Reopen in Container" reads `.devcontainer/docker-compose.yml` alone and applies [no overlays at all](lifecycle-commands.md#vs-code-reopen-in-container-applies-no-overlays). diff --git a/docs/integrations/github.md b/docs/integrations/github.md index fadf3ccd..fed901a5 100644 --- a/docs/integrations/github.md +++ b/docs/integrations/github.md @@ -81,4 +81,4 @@ gh issue view 42 ## Persisting credentials across restarts -The `gh` token is stored inside the container at `~/.config/gh/`, backed by the named `config-dir` volume that persists all of `~/.config`. The token survives `docker compose down` and `docker compose up` cycles. `docker compose down -v` removes the volume — re-run `gh auth login` after a `down -v`, or register your own compose overlay in `config.json` `composeOverrides[]` to bind-mount a host directory to `~/.config/`. +The `gh` token is stored inside the container at `~/.config/gh/`, which persists in the single `/home/sandbox` mount. The token survives `docker compose down` and `docker compose up` cycles. `docker compose down -v` deletes the volume — re-run `gh auth login` after a `down -v`, or set `storage.homePath` in `oh.json` to keep the sandbox home on a host path that `down -v` cannot touch. diff --git a/docs/integrations/herdr.md b/docs/integrations/herdr.md index 9f14173b..f5d90481 100644 --- a/docs/integrations/herdr.md +++ b/docs/integrations/herdr.md @@ -40,8 +40,10 @@ herdr integration status ## Persistence -- `~/.config/herdr` (in the shared `config-dir` volume): configuration, logs, and session metadata. -- `~/.herdr` (in `herdr-data`): Herdr-created worktrees and related data. +- `~/.config/herdr`: configuration, logs, and session metadata. +- `~/.herdr`: Herdr-created worktrees and related data. + +Both persist in the single `/home/sandbox` mount. `oh stop` and normal rebuilds preserve metadata and layout in these volumes, but stopped containers do not preserve running agent, test, or server processes. `oh destroy` runs Compose with `-v` and removes the volumes too. diff --git a/docs/integrations/langfuse.md b/docs/integrations/langfuse.md index df79ed72..13388f3e 100644 --- a/docs/integrations/langfuse.md +++ b/docs/integrations/langfuse.md @@ -382,7 +382,7 @@ your self-hosted URL. Other optional fields are `LANGFUSE_USER_ID`, The plugin installs and is enabled at **user scope**. Configuration is stored by Claude's plugin configuration and OS-keychain mechanisms according to upstream. -Open Harness persists `~/.claude` on the `claude-auth` volume, but OS-keychain +Open Harness persists `~/.claude` in the `/home/sandbox` mount, but OS-keychain availability and persistence are platform-dependent; verify the plugin remains configured after a sandbox rebuild. You can enter the same Langfuse key pair in Pi and Claude Code, but their saved configurations are independent. Do not put diff --git a/docs/lifecycle-commands.md b/docs/lifecycle-commands.md index b618ad91..4ac82749 100644 --- a/docs/lifecycle-commands.md +++ b/docs/lifecycle-commands.md @@ -67,13 +67,17 @@ configuration, so they stay host-only rather than failing halfway. ## `oh destroy` and its confirmation policy -`down -v` wipes the named volumes, and those volumes hold provider +`down -v` wipes the sandbox home volume, and that volume holds provider authentication. `oh destroy` is therefore the only lifecycle verb that asks before it runs. It names the volumes it is about to delete — read from `.devcontainer/docker-compose.yml`, not hardcoded — then requires you to type the sandbox name. A blank line, a wrong name, or anything else aborts with a non-zero exit and removes nothing. +When `storage.homePath` points the home mount at a host path, `down -v` cannot +delete it. `oh destroy` says so and leaves the directory in place; remove it +yourself if you want it gone. + Non-interactive use is gated on an explicit flag. When stdin is not a terminal and `--yes` is absent, `oh destroy` refuses outright rather than assume consent. @@ -107,5 +111,15 @@ Non-secret `oh.json` settings only reach compose when `oh` renders them, so on this path each variable falls back to its default in `.devcontainer/docker-compose.yml`. +:::danger `storage.homePath` is ignored on this path +`OH_HOME_MOUNT` is one of those rendered-only variables, so *Reopen in +Container* falls back to the Docker-managed `_workspace` volume even when +`storage.homePath` points the sandbox home at a host directory. That is a +**second, separate home**: agent logins made through `oh sandbox` are not there, +and the two diverge silently from then on. + +If you set `storage.homePath`, always provision with `oh sandbox` and attach. +::: + If you need any overlay, provision with `oh sandbox` and then use *Dev Containers: Attach to Running Container* instead of *Reopen in Container*. diff --git a/docs/security-considerations.md b/docs/security-considerations.md index 7cf88bc3..605772aa 100644 --- a/docs/security-considerations.md +++ b/docs/security-considerations.md @@ -41,7 +41,7 @@ templates are committed. - `**/auth.json` and `**/.credentials.json` (`.gitignore:63-64`) — provider auth blobs. - **Template allowlist:** the *tracked* files are templates that hold no real secrets — e.g. [`.env.example`](../.env.example) and `.claude/.example.env.claude`. The operator copies `.env.example` to the real (gitignored, mode-`0600`) root `.env`; `install.sh` seeds it and `oh secret set ` edits one key. `.devcontainer/.env` is a symlink to that file, so VS Code "Reopen in Container" reads the same one. The `.env.example` header spells this out, including the warning that the compose default for `SANDBOX_PASSWORD` (`test1234`) is weak and public and must be overridden on any network-reachable deployment. - **Split by kind:** non-secret settings live in the *tracked* [`oh.json`](../oh.json), never in `.env`. The split is enforced in code — `.oh/cli/src/lib/secrets.ts` owns the secret allow-list and `.oh/cli/src/lib/config-render.ts` refuses to render an allow-listed secret into the compose environment. See [Configuration](configuration.md). -- **In the sandbox:** auth/state persists in Docker **named volumes** (`claude-auth`, `codex-auth`, `pi-auth`, `ssh-config`, `config-dir`, …), not in the repo — see [`.devcontainer/docker-compose.yml:31-41`](../.devcontainer/docker-compose.yml). +- **In the sandbox:** auth/state persists in the single `/home/sandbox` mount — the named volume `_workspace`, or a host path when `storage.homePath` is set — not in the repo. See [`.devcontainer/docker-compose.yml`](../.devcontainer/docker-compose.yml). **What this does not do:** it does not scan commit *contents* for secrets pasted into a tracked file by mistake. That is the job of the diff --git a/oh.json b/oh.json index 06504465..c17aad65 100644 --- a/oh.json +++ b/oh.json @@ -2,8 +2,8 @@ "version": 1, "name": "openharness", "timezone": "America/Los_Angeles", - "projectRoot": "/home/sandbox/harness", "git": {}, + "storage": {}, "install": { "opencode": false, "grokBuild": false, From bee80601dfebbd21a56a0fc895d3b04849b9d49d Mon Sep 17 00:00:00 2001 From: Ryan Eggleston Date: Sun, 30 Aug 2026 18:56:27 -0600 Subject: [PATCH 04/23] FROM task/900-slim-sandbox-image TO task/898-single-home-mount (#901) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * task: collapse sandbox storage to one $HOME mount Replace the eleven per-tool named volumes with a single mount at /home/sandbox, selectable as a Docker-managed volume (_workspace) or a host bind via storage.homePath in oh.json. The image's home is moved to /opt/home-seed at build time so an empty named volume never auto-copies; entrypoint's seed_home restores it into either mount kind with cp -a -n, which also backfills new dotfiles on an image upgrade without clobbering operator state. Retire the projectRoot/OH_PROJECT_ROOT config knob: the checkout is fixed at /home/sandbox/harness, nested inside the home mount. Replace the UID-sync 'find -xdev' with an explicit $OH_PROJECT_ROOT prune. -xdev only pruned the checkout because /home/sandbox was the container overlay; once it is a mount they share a device. * task: retarget the herdr image-docs persistence assertion at the single home mount * task: address audit findings on the single-$HOME-mount change seed_home no longer uses 'cp -a -n'. That form skips existing regular files but still rewrites the mode, owner and mtime of directories the destination already has, so every boot relaxed ~/.ssh from 0700 to the image's 0755 and reset the other pre-existing home directories likewise. It now walks the seed and copies only entries the mount lacks, creating a missing directory with the seed's own metadata and never touching an existing path. It also returns non-zero on a real copy failure instead of swallowing it, and the caller warns. oh destroy branches on storage.homePath rather than appending a correction: with a host bind, nothing is deleted and sign-in does not start over, so the volume list and that warning are wrong to print. The volume list is now project-prefixed to match 'docker volume ls'. Reject reserved host paths for storage.homePath. The ownership repair is now an unbounded recursive chown over the mount, so '/', '/home' or '/usr' would hand the sandbox uid a host tree. Fix the migration recipe: 'docker cp :/home/sandbox ' lands the copy at /sandbox when exists. Use the trailing '/.' form and drop the checkout copy that docker cp reads through the bind mount. Document that storage.homePath is invisible to VS Code 'Reopen in Container', which would otherwise give a second, silently diverging home. Harden oh-home-mount.sh with the directory-metadata and copy-failure cases that let the original bug through, plus the missing rm -rf assertion; scope the -xdev ban to lines mentioning /home/sandbox. Refresh the stale project-root seam section in .oh/README.md, which still described the removed knob and cited the deleted probe. * task: drop local agent memory and cron runtime state committed by mistake * task: seed the home mount per top-level entry, not per file A live boot on a real image caught the previous form: walking the whole seed tree and copying file by file spends its time in the uv cache under ~/.cache/uv, and it aborted partway through with EEXIST, leaving a home with four of sixteen entries. seed_home now copies each top-level entry the mount lacks with a single 'cp -a', and never touches one it already has. Verified on a live boot: a clean volume gets all sixteen entries in 7s, ~/.ssh ends at 0700, and an operator edit to ~/.zshrc plus a 0700 ~/.ssh both survive a restart. The trade is explicit and documented: a new file inside a directory the mount already has is not merged in. That matches what the per-tool volumes did, while a new top-level entry from an image upgrade still arrives. oh-home-mount.sh asserts the contract in both directions and now rejects the two wrong implementations - 'cp -a -n' and the whole-tree walk - rather than only the first. Bake ~/.ssh at 0700 in the image so the seeded mode is right before the entrypoint's chmod rather than because of it. * task: ignore per-session agent memory and cron runtime state .oh/memory/ and .oh/crons/ hold local agent scratch — session logs, prompt-miner dumps, a cron pid and log. Nothing in either is tracked on any branch, but neither was ignored, so a `git add -A` sweeps 37 files of operator session state into the commit. It happened twice while building this branch. Ignore both, following the wiki-corpus convention already in this file: local by default, `git add -f` for anything worth keeping. * task: slim the sandbox image by ~540 MB Stop shipping build caches inside the baked home seed, and stop shipping the seed twice. The home seed measured 358 MB, of which 103 MB was pure build cache: ~/.npm (31 MB, left by the pi npm install) and ~/.cache/uv (72 MB, left by provision-python.sh). Neither is read at runtime -- the entrypoint's repair_home_mount_ownership install -d's the uv cache on every boot, and npm recreates its own. Both purges fold into the RUN that produces them. RUN mv /home/sandbox /opt/home-seed moved data across a layer boundary, so the image stored the whole seed twice. The Dockerfile is now three stages: base (through the sandbox user and the UV_* env), home (everything that writes into /home/sandbox), and final, which brings the seed in with COPY --from=home --chown=sandbox:sandbox. The final stage also removes the useradd skel files that the old wholesale mv used to take with it, preserving the empty-home invariant that keeps an empty named volume from auto-copying. Runtime NPM_USER_PREFIX and UV_* still point at /home/sandbox, so pi self-updates and uv tool install keep persisting into the mount. .dockerignore now excludes .pnpm-store and the untracked .pi build outputs, which were pulling ~2 GB into every local build context. Two oracles move with the mechanism: oh-home-mount.sh pinned the literal RUN mv spelling and now asserts the invariant (seed staged at /opt/home-seed, empty home, no harness/), and sandbox-base-image.test.ts end-anchored the FROM line, which a stage alias necessarily breaks. Measured with docker system df -v: 4.0 GB -> 3.46 GB. * fix: close four oracle gaps an adversarial audit found in the seed slimming The Dockerfile restructure held up under an entry-by-entry comparison of the two built images -- 29,992 seed entries, zero differences in type, mode, or owner, and a path delta that is exactly the intended cache purge. The defects were all in the guards. image-seed-hygiene.sh compared textual line numbers in a file where instruction order is not execution order, so it passed a Dockerfile that still shipped both caches: delete the purges from the home stage and add them as the first instruction of the final stage, and the line-number check is satisfied while the seed is copied from an unpurged stage. It now parses stages, anchors on the staging instruction itself rather than any mention of /opt/home-seed, resolves which stage the seed is copied from, and requires a purge inside that stage. The single-stage RUN mv form still works, and the previously fragile anchor no longer false-positives on an earlier ENV referencing the same path. oh-home-mount.sh accepted COPY --from=base (whose home is bare useradd skel, so every baked dotfile would be lost) and COPY --from=alpine:latest. The source must now be a stage declared in the same Dockerfile, and not base. sandbox-base-image.test.ts stopped pinning anything once there were three FROM lines -- any one matching satisfied it, so FROM ubuntu:24.04 AS final passed. A new case walks the FROM lines in order and asserts every image is either the pinned base or a stage defined earlier. COPY --from creates its destination at the default 0755 and --chown does not restore a mode, so /opt/home-seed silently went 0700 -> 0755 versus the RUN mv era. Restored, and now asserted by the probe, since nothing caught it. Each fix is mutation-verified: the audit's mutant and all four original negative controls exit 1, the false-positive control stays green, and both bad COPY --from sources are rejected by both probes. * task: refresh eval scoreboard after merge --- .devcontainer/Dockerfile | 38 ++-- .dockerignore | 3 + .oh/evals/RESULTS.md | 203 +++++++++--------- .oh/evals/probes/image-seed-hygiene.sh | 132 ++++++++++++ .oh/evals/probes/oh-home-mount.sh | 15 +- .../__tests__/sandbox-base-image.test.ts | 18 +- .oh/tasks/slim-sandbox-image/prd.md | 202 +++++++++++++++++ CHANGELOG.md | 1 + 8 files changed, 494 insertions(+), 118 deletions(-) create mode 100755 .oh/evals/probes/image-seed-hygiene.sh create mode 100644 .oh/tasks/slim-sandbox-image/prd.md diff --git a/.devcontainer/Dockerfile b/.devcontainer/Dockerfile index af7babe0..60fe7347 100644 --- a/.devcontainer/Dockerfile +++ b/.devcontainer/Dockerfile @@ -1,4 +1,4 @@ -FROM node:22-trixie-slim +FROM node:22-trixie-slim AS base ENV OH_PROJECT_ROOT=/home/sandbox/harness @@ -126,6 +126,9 @@ ENV UV_TOOL_DIR=/home/sandbox/.local/share/uv/tools ENV UV_TOOL_BIN_DIR=/home/sandbox/.local/bin ENV UV_PYTHON_INSTALL_DIR=/home/sandbox/.local/share/uv/python ENV UV_CACHE_DIR=/home/sandbox/.cache/uv + +FROM base AS home +ARG AGENTS="claude-code,codex,pi-coding-agent" RUN install -d -o sandbox -g sandbox -m 0700 /home/sandbox/.ssh \ && install -d -o sandbox -g sandbox \ /home/sandbox/.local \ @@ -134,15 +137,14 @@ RUN install -d -o sandbox -g sandbox -m 0700 /home/sandbox/.ssh \ /home/sandbox/.cache \ /home/sandbox/.config \ /home/sandbox/.herdr \ - "$UV_TOOL_DIR" "$UV_TOOL_BIN_DIR" "$UV_PYTHON_INSTALL_DIR" "$UV_CACHE_DIR" \ - && chown -R sandbox:sandbox /opt/uv 2>/dev/null || true \ - && if [ -d /usr/local/lib/hermes-agent ]; then chown -R sandbox:sandbox /usr/local/lib/hermes-agent; fi + "$UV_TOOL_DIR" "$UV_TOOL_BIN_DIR" "$UV_PYTHON_INSTALL_DIR" "$UV_CACHE_DIR" RUN set -e; \ install -d -o sandbox -g sandbox "$NPM_USER_PREFIX"; \ if [[ ",${AGENTS}," == *",pi-coding-agent,"* ]]; then \ su - sandbox -c 'npm --prefix "$HOME/.local" install -g --ignore-scripts @earendil-works/pi-coding-agent'; \ - fi + fi; \ + rm -rf /home/sandbox/.npm RUN su - sandbox -c "RUNZSH=no CHSH=no KEEP_ZSHRC=yes sh -c \"\$(curl -fsSL https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh)\" --unattended" \ && su - sandbox -c "git clone --depth 1 https://github.com/zsh-users/zsh-autosuggestions /home/sandbox/.oh-my-zsh/custom/plugins/zsh-autosuggestions" \ @@ -169,15 +171,9 @@ RUN su - sandbox -c "git config --global --add safe.directory ${OH_PROJECT_ROOT} RUN echo "cd ${OH_PROJECT_ROOT} 2>/dev/null" >> /home/sandbox/.bashrc -RUN mkdir -p /.devcontainer && echo '{"workspaceFolder":"/home/sandbox/harness","remoteUser":"sandbox"}' > /.devcontainer/devcontainer.json - -LABEL devcontainer.metadata='[{"remoteUser":"sandbox","workspaceFolder":"/home/sandbox/harness"}]' -LABEL org.opencontainers.image.licenses="Apache-2.0" - COPY --chown=sandbox:sandbox .oh/install/ /home/sandbox/install/ RUN chmod +x /home/sandbox/install/*.sh -# boot (entrypoint OH_IMAGE_ONLY / seed_workspace_volume, OH_IMAGE_SEED_SRC). COPY --chown=sandbox:sandbox . /opt/oh-seed/ ARG INSTALL_PYTHON_KERNEL=true @@ -186,13 +182,27 @@ RUN if [ "${INSTALL_PYTHON_KERNEL}" = "true" ]; then \ su - sandbox -c "OH_PYTHON_VERSION='${OH_PYTHON_VERSION}' bash /opt/oh-seed/.oh/scripts/provision-python.sh"; \ else \ echo "Skipping Python kernel provisioning (INSTALL_PYTHON_KERNEL=false)"; \ - fi + fi \ + && rm -rf /home/sandbox/.cache/uv + +FROM base AS final +RUN chown -R sandbox:sandbox /opt/uv 2>/dev/null || true \ + && if [ -d /usr/local/lib/hermes-agent ]; then chown -R sandbox:sandbox /usr/local/lib/hermes-agent; fi + +RUN mkdir -p /.devcontainer && echo '{"workspaceFolder":"/home/sandbox/harness","remoteUser":"sandbox"}' > /.devcontainer/devcontainer.json + +LABEL devcontainer.metadata='[{"remoteUser":"sandbox","workspaceFolder":"/home/sandbox/harness"}]' +LABEL org.opencontainers.image.licenses="Apache-2.0" + +# boot (entrypoint OH_IMAGE_ONLY / seed_workspace_volume, OH_IMAGE_SEED_SRC). +COPY --chown=sandbox:sandbox . /opt/oh-seed/ COPY .devcontainer/entrypoint.sh /usr/local/bin/entrypoint.sh RUN chmod +x /usr/local/bin/entrypoint.sh -RUN mv /home/sandbox /opt/home-seed \ - && rm -rf /opt/home-seed/harness \ +COPY --from=home --chown=sandbox:sandbox /home/sandbox /opt/home-seed +RUN rm -rf /opt/home-seed/harness /home/sandbox \ + && chmod 0700 /opt/home-seed \ && install -d -o sandbox -g sandbox -m 0755 /home/sandbox WORKDIR ${OH_PROJECT_ROOT} diff --git a/.dockerignore b/.dockerignore index 73aa368f..ff1bdfca 100644 --- a/.dockerignore +++ b/.dockerignore @@ -15,3 +15,6 @@ projects/* !projects/CLAUDE.md .git/ node_modules/ +.pnpm-store/ +.pi/bridge/ +.pi/npm/ diff --git a/.oh/evals/RESULTS.md b/.oh/evals/RESULTS.md index 2ab5e22c..ee53d0d4 100644 --- a/.oh/evals/RESULTS.md +++ b/.oh/evals/RESULTS.md @@ -6,106 +6,107 @@ probe id; git history is the time series.** Schema and exit-code semantics are i | probe | tier | last-run (UTC) | status | source | |-------|------|----------------|--------|--------| -| advisor-monitored-loop | A | 2026-08-30 17:01 | PASS | conversation 2026-06-19 (single-owner implementation workflow, issue #257) | -| agent-browser-cli | A | 2026-08-30 17:01 | PASS | retro lesson 2026-06-07 (agent-browser 0.8.5 CLI) | -| agents-identity-contract | A | 2026-08-30 17:01 | PASS | issue #854 — T3-style root identity, glossary, and skill-owned procedures | -| artifact-contract-audit | A | 2026-08-30 17:01 | PASS | issue #583/#645 — production /audit implementation Gate 1 behavior | -| audit-dispatcher-contract | A | 2026-08-30 17:01 | PASS | issue #645 — audit consolidation public taxonomy | -| audit-implementation-behavior | A | 2026-08-30 17:01 | PASS | issue #645 — implementation root/repo/browser behavior | -| audit-pr-acquire | A | 2026-08-30 17:01 | PASS | issue #645 — production PR acquisition behavior | -| audit-pr-classifier | A | 2026-08-30 17:01 | PASS | issue #645 — deterministic focused and queue PR classifier | -| audit-run-root-contract | A | 2026-08-30 17:01 | PASS | issue #645 — executable immutable audit root/run correlation | -| audit-shellcheck-coverage | A | 2026-08-30 17:01 | PASS | issue #645 — private audit scripts require release and CI lint coverage | -| audit-stale-references | A | 2026-08-30 17:01 | PASS | issue #645 — clean-breaking audit migration | -| boot-lint-glob | A | 2026-08-30 17:01 | PASS | issue #90, issue #120 | -| builder-skill-consolidation | A | 2026-08-30 17:01 | PASS | issue #643 — consolidate artifact builders behind one /builder dispatcher | -| capability-benchmark-schema | A | 2026-08-30 17:01 | PASS | issue #167 — capability benchmark instrument | -| cc-safety-net-wiring | A | 2026-08-30 17:01 | SKIPPED | .oh/tasks/cc-safety-net/prd.json US-007 2026-07-19 | -| changelog-entry-length | A | 2026-08-30 17:01 | PASS | conversation 2026-08-24 — CHANGELOG.md grew to 259KB of bullet prose because "one line" was unquantified | -| cleanup-tasks-scoped-guard | A | 2026-08-30 17:01 | PASS | issue #85 | -| cleanup-tasks-worktree-grooming | A | 2026-08-30 17:01 | PASS | issue #168; issue #327 | -| cli-publish-typecheck-scope | A | 2026-08-30 17:01 | PASS | release run 33271077312 — v0.5.0 pushed its GHCR image, then failed to publish | -| close-issues-on-development | A | 2026-08-30 17:01 | PASS | issue #841 (closing keywords never fire because the default branch is main) 2026-08-26 | -| codex-stale-response-retry | A | 2026-08-30 17:01 | PASS | issue #506 — Codex previous_response_not_found RCA | -| compose-config-path-parity | A | 2026-08-30 17:01 | PASS | PR #833 (remove harness.yaml — the wrapper and VS Code "Reopen in Container" paths must resolve the same service) 2026-08-26 | -| config-schema-parity | A | 2026-08-30 17:01 | PASS | PR #833 (one schema file — DOCKER_SOCKET, SANDBOX_SSH, OH_SANDBOX_IMAGE, OH_PULL_POLICY, SKIP_PNPM_INSTALL were consumed but undocumented); rewritten for the oh.json/secrets split by PR #887 | -| context-tier-size-budget | A | 2026-08-30 17:01 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-007) — the always-on tier was 85,256 B | -| cron-claude-codex-fallback | A | 2026-08-30 17:01 | PASS | conversation 2026-06-12 (default Codex fallback for crons) | -| cron-watchdog | A | 2026-08-30 17:01 | PASS | issues #130/#453 (cron runtime watchdog + legacy system-cron reaping) 2026-06-19 | -| crons-directory-guide | A | 2026-08-30 17:01 | PASS | issue #874 | -| curl-bash-safe-alternatives | A | 2026-08-30 17:01 | PASS | vet-run/vet integration — public curl|bash examples need review-first alternatives | -| datasets-schema | A | 2026-08-30 17:01 | PASS | issue #196 — .oh/evals/datasets verifiable trajectory corpus (Repo2RLEnv-inspired) | -| debugmcp-availability | A | 2026-08-30 17:01 | SKIPPED | issue #297 — DebugMCP MCP debug-server availability | -| delegate-model-effort-policy | A | 2026-08-30 17:01 | PASS | conversation 2026-07-11 (delegate model inheritance and thinking policy) | -| devtcp-hook | A | 2026-08-30 17:01 | PASS | retro lesson 2026-06-10 (zsh /dev/tcp) | -| docker-inspect-env-guard | A | 2026-08-30 17:01 | PASS | operator directive 2026-08-08 (agents keep the docker socket, but must | -| docs-build-fast-path | A | 2026-08-30 17:01 | PASS | #455 — docs builds must stay out of fast harness/eval/release gates; #536 — docs site externalized to openharness-web; docs markdown relocated to docs/ | -| drift-check-cron-staleness-glob | A | 2026-08-30 17:01 | PASS | issue #98; issue #225 (restart-required cron frontmatter/config drift) | -| entrypoint-pnpm-manifest-fingerprint | A | 2026-08-30 17:01 | PASS | issue #521 (manifest-aware sandbox installs) 2026-07-01 | -| eval-ci-gate | A | 2026-08-30 17:01 | PASS | #103 — eval probe suite gated in CI | -| eval-gate | A | 2026-08-30 17:01 | PASS | retro lesson 2026-06-11 (eval-gate) | -| eval-results-atomic | A | 2026-08-30 17:01 | PASS | issue #83 (eval-results-atomic-write) | -| eval-runner-exit | A | 2026-08-30 17:01 | PASS | retro lesson 2026-06-11 (eval-runner-exit) #29 | -| eval-runs-once-per-cycle | A | 2026-08-30 17:01 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-006) — /eval ran 3x per cycle on the | -| execution-target-contract | A | 2026-08-30 17:01 | PASS | issue #733 (ExecutionTarget contract + Docker Compose adapter) 2026-08-10 | -| get-oh-bootstrap | A | 2026-08-30 17:01 | PASS | get-oh.sh bootstrap — the Node-bootstrapping host-side path to the standalone `oh` CLI (also on npm as @mifune/openharness; see oh-npm-package.sh) | -| git-skill | A | 2026-08-30 17:01 | PASS | conversation 2026-06-15 — rules are not always supported; git workflow must be the /git skill | -| harness-audit-empty-output-gate | A | 2026-08-30 17:01 | PASS | issue #246 — /audit harness must fail closed on empty auditor outputs | -| harness-ci-core-paths | A | 2026-08-30 17:01 | PASS | #165 — core sandbox config files must trigger harness CI | -| harness-ci-hooks-paths | A | 2026-08-30 17:01 | PASS | issue #202 — credential/security hook changes must trigger harness CI | -| harness-yaml-migration | A | 2026-08-30 17:01 | PASS | PR #833 (migrate-harness-yaml.sh — append / uncomment-in-place / preserve / overwrite, plus a silent no-op second run) 2026-08-26 | -| health-check-docker-stats | A | 2026-08-30 17:01 | PASS | retro lesson 2026-06-10 (docker stats vs ps Size) | -| health-check-socket-degrade | A | 2026-08-30 17:01 | PASS | issue #762 (refs #756) — /health-check degrades to one statement, not nine failures | -| heartbeat-logging-contract | A | 2026-08-30 17:01 | PASS | issue #447 (heartbeat log append hardening) 2026-06-18 | -| markitdown-wiki-ingest | A | 2026-08-30 17:01 | PASS | issue #649 — pinned local-document normalization contract for /wiki ingest | -| next-dev-prod | A | 2026-08-30 17:01 | SKIPPED | retro lesson 2026-06-04 | -| oh-compose-env-wiring | A | 2026-08-30 17:01 | PASS | issue #880 (oh as the only front door — oh.json is the non-secret config surface) | -| oh-config-surfaces | A | 2026-08-30 17:01 | REGRESSION | PR #887 (config split across two authored surfaces — a tracked oh.json and a secrets-only root dotenv — with nothing left under $HOME) | -| oh-destroy-guard | A | 2026-08-30 17:01 | PASS | issue #879 — `oh` becomes the only front door, so `make destroy` must | -| oh-devcontainer-restructure | A | 2026-08-30 17:01 | PASS | consolidate devcontainer — .oh/devcontainer/ folded back into .devcontainer/ | -| oh-home-mount | A | 2026-08-30 17:01 | PASS | issue #898 (single $HOME mount) 2026-08-30 | -| oh-image-only-deploy | A | 2026-08-30 17:01 | PASS | .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only deploy) | -| oh-init-headless-config | A | 2026-08-30 17:01 | PASS | PR #827 (installer answers landed in the losing config file); retargeted to the .example.env template by PR #833, then to oh.json by PR #887 | -| oh-init-scaffold | A | 2026-08-30 17:01 | PASS | issue #531 Phase 2 | -| oh-lifecycle-surface | A | 2026-08-30 17:01 | PASS | issue #881 — the Makefile is retired and `oh` is the only front door | -| oh-npm-package | A | 2026-08-30 17:01 | PASS | npm publish path for the standalone `oh` CLI (@mifune/openharness) — alternative to get-oh.sh | -| oh-payload-manifest | A | 2026-08-30 17:01 | PASS | issue #531 follow-on (.oh payload manifest — oh update ships a declared allowlist) | -| oh-sandbox-image-mode | A | 2026-08-30 17:01 | PASS | conversation 2026-07-05 (basic Docker deployment — prebuilt-image mode) | -| oh-shipped-repo-overridable | A | 2026-08-30 17:01 | PASS | issue #531 follow-on (de-hardcode residual — shipped .oh shell scripts keep the upstream repo overridable) | -| oh-standalone-lifecycle | A | 2026-08-30 17:01 | PASS | issue #564 | -| oh-update | A | 2026-08-30 17:01 | PASS | issue #531 Phase 3 (oh update — upgrade only the .oh control plane) | -| operator-config-guard | A | 2026-08-30 17:01 | PASS | operator directives 2026-08-06 (.config/ and settings.local.json are operator-only) | -| pnpm-audit-ci-gate | A | 2026-08-30 17:01 | PASS | issue #171 — pnpm security audits must run in CI | -| post-bridge-publish-confirmation | A | 2026-08-30 17:01 | PASS | #523 — post-bridge live publishing requires an explicit final confirmation gate | -| prd-output-path-contract | A | 2026-08-30 17:01 | PASS | retro lesson 2026-06-19 | -| prompt-miner-schema-compat | A | 2026-08-30 17:01 | PASS | issue #253 — prompt-miner JSONL schema-drift guard | -| prompt-miner-symlink-entrypoint | A | 2026-08-30 17:01 | PASS | issue #663 — prompt-miner engine no-ops via the documented .claude/skills symlink | -| prompt-miner-weakness-record | A | 2026-08-30 17:01 | PASS | issue #580 — prompt-miner weakness-record (WH-xxx) cluster output | -| protected-path-deletion | A | 2026-08-30 17:01 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-001) — the critique gate was deleted, | -| protected-paths-resolve | A | 2026-08-30 17:01 | PASS | issue #753 — .claude/protected-paths.txt named 7 paths that did not exist. | -| registry-portability-gate | A | 2026-08-30 17:01 | PASS | issue #758 | -| registry-portability | A | 2026-08-30 17:01 | SKIPPED | issue #758 | -| retro-deterministic-contract | A | 2026-08-30 17:01 | PASS | issue #443 — /retro deterministic output and self-contained helper contract | -| rl-delegation-write-worker | A | 2026-08-30 17:01 | PASS | retro lesson 2026-06-10 (rl-delegation) #57 | -| rlm-context-budget | A | 2026-08-30 17:01 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-006 | -| runtime-preflight-gate | A | 2026-08-30 17:01 | PASS | issue #806 § B1 (open sandbox.substrate vs sandbox.runtime selector); | -| sandbox-boot-guard-ci | A | 2026-08-30 17:01 | PASS | issue #449 (sandbox image build CI guard) 2026-06-19; | -| sandbox-node-base | A | 2026-08-30 17:01 | PASS | openharness#878 — oh as the only front door, T0 sandbox base image | -| skill-paths | A | 2026-08-30 17:01 | PASS | issue #43 — stale path references; extended by issue #69 — apps/->packages/ rename guard; extended by issue #870 — deleted .oh/agents/advisor.md | -| skills-dir-clean | A | 2026-08-30 17:01 | PASS | conversation 2026-06-29 — Pi parses every top-level `.md` in the skills | -| skills-task-tool-coupling | A | 2026-08-30 17:01 | PASS | council review 2026-08-29 (issue #886) — /delegate instructed Claude-Code-only | -| skills-vendored | A | 2026-08-30 17:01 | PASS | absorb .mifune submodule into .oh — the skills/agents/hooks pack is vendored | -| slack-admin-command-surface | A | 2026-08-30 17:01 | PASS | issue #354 — Slack bridge docs must distinguish Pi /msg-bridge commands from Slack DM admin text handlers | -| spec-family-contract | A | 2026-08-30 17:01 | PASS | issue #265; spec-simplification issue #816; workflow authority issue #854 | -| spec-ready-finalization | A | 2026-08-30 17:01 | PASS | issue #134; spec-simplification issue #816; workflow authority issue #854 | -| ste-checker-contract | A | 2026-08-30 17:01 | PASS | issue #750 PR audit — the /ste checker had four fail-open paths (unclosed | -| submitted-by-trailers | A | 2026-08-30 17:01 | PASS | conversation 2026-06-12 (commit attribution trailers); the single-owner | -| sync-skill-contract | A | 2026-08-30 17:01 | PASS | issue #331 — /sync dispatcher skill (bidirectional origin↔upstream sync) | -| tool-catalog-boundary | A | 2026-08-30 17:01 | PASS | agent-browser's exclusion from the harness catalog (#821) and the | -| version-parity | A | 2026-08-30 17:01 | PASS | conversation 2026-08-29 — the oh CLI became the only lifecycle door, so its | -| weigh-scorer-contract | A | 2026-08-30 17:01 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-003 (2026-06-27) | -| wiki-readme-index | A | 2026-08-30 17:01 | PASS | issue #132 — wiki README index drift guard | -| workflow-boundaries | A | 2026-08-30 17:01 | PASS | conversation 2026-06-19 (workflow consolidation, issue #259); authority moved to /spec in issue #854 | -| worktrees-layout | A | 2026-08-30 17:01 | PASS | issue #872 | +| advisor-monitored-loop | A | 2026-08-31 00:49 | PASS | conversation 2026-06-19 (single-owner implementation workflow, issue #257) | +| agent-browser-cli | A | 2026-08-31 00:49 | PASS | retro lesson 2026-06-07 (agent-browser 0.8.5 CLI) | +| agents-identity-contract | A | 2026-08-31 00:49 | PASS | issue #854 — T3-style root identity, glossary, and skill-owned procedures | +| artifact-contract-audit | A | 2026-08-31 00:49 | PASS | issue #583/#645 — production /audit implementation Gate 1 behavior | +| audit-dispatcher-contract | A | 2026-08-31 00:49 | PASS | issue #645 — audit consolidation public taxonomy | +| audit-implementation-behavior | A | 2026-08-31 00:49 | PASS | issue #645 — implementation root/repo/browser behavior | +| audit-pr-acquire | A | 2026-08-31 00:49 | PASS | issue #645 — production PR acquisition behavior | +| audit-pr-classifier | A | 2026-08-31 00:49 | PASS | issue #645 — deterministic focused and queue PR classifier | +| audit-run-root-contract | A | 2026-08-31 00:49 | PASS | issue #645 — executable immutable audit root/run correlation | +| audit-shellcheck-coverage | A | 2026-08-31 00:49 | PASS | issue #645 — private audit scripts require release and CI lint coverage | +| audit-stale-references | A | 2026-08-31 00:49 | PASS | issue #645 — clean-breaking audit migration | +| boot-lint-glob | A | 2026-08-31 00:49 | PASS | issue #90, issue #120 | +| builder-skill-consolidation | A | 2026-08-31 00:49 | PASS | issue #643 — consolidate artifact builders behind one /builder dispatcher | +| capability-benchmark-schema | A | 2026-08-31 00:49 | PASS | issue #167 — capability benchmark instrument | +| cc-safety-net-wiring | A | 2026-08-31 00:49 | SKIPPED | .oh/tasks/cc-safety-net/prd.json US-007 2026-07-19 | +| changelog-entry-length | A | 2026-08-31 00:49 | PASS | conversation 2026-08-24 — CHANGELOG.md grew to 259KB of bullet prose because "one line" was unquantified | +| cleanup-tasks-scoped-guard | A | 2026-08-31 00:49 | PASS | issue #85 | +| cleanup-tasks-worktree-grooming | A | 2026-08-31 00:49 | PASS | issue #168; issue #327 | +| cli-publish-typecheck-scope | A | 2026-08-31 00:49 | PASS | release run 33271077312 — v0.5.0 pushed its GHCR image, then failed to publish | +| close-issues-on-development | A | 2026-08-31 00:49 | PASS | issue #841 (closing keywords never fire because the default branch is main) 2026-08-26 | +| codex-stale-response-retry | A | 2026-08-31 00:49 | PASS | issue #506 — Codex previous_response_not_found RCA | +| compose-config-path-parity | A | 2026-08-31 00:49 | PASS | PR #833 (remove harness.yaml — the wrapper and VS Code "Reopen in Container" paths must resolve the same service) 2026-08-26 | +| config-schema-parity | A | 2026-08-31 00:49 | PASS | PR #833 (one schema file — DOCKER_SOCKET, SANDBOX_SSH, OH_SANDBOX_IMAGE, OH_PULL_POLICY, SKIP_PNPM_INSTALL were consumed but undocumented); rewritten for the oh.json/secrets split by PR #887 | +| context-tier-size-budget | A | 2026-08-31 00:49 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-007) — the always-on tier was 85,256 B | +| cron-claude-codex-fallback | A | 2026-08-31 00:49 | PASS | conversation 2026-06-12 (default Codex fallback for crons) | +| cron-watchdog | A | 2026-08-31 00:49 | PASS | issues #130/#453 (cron runtime watchdog + legacy system-cron reaping) 2026-06-19 | +| crons-directory-guide | A | 2026-08-31 00:49 | PASS | issue #874 | +| curl-bash-safe-alternatives | A | 2026-08-31 00:49 | PASS | vet-run/vet integration — public curl|bash examples need review-first alternatives | +| datasets-schema | A | 2026-08-31 00:49 | PASS | issue #196 — .oh/evals/datasets verifiable trajectory corpus (Repo2RLEnv-inspired) | +| debugmcp-availability | A | 2026-08-31 00:49 | SKIPPED | issue #297 — DebugMCP MCP debug-server availability | +| delegate-model-effort-policy | A | 2026-08-31 00:49 | PASS | conversation 2026-07-11 (delegate model inheritance and thinking policy) | +| devtcp-hook | A | 2026-08-31 00:49 | PASS | retro lesson 2026-06-10 (zsh /dev/tcp) | +| docker-inspect-env-guard | A | 2026-08-31 00:49 | PASS | operator directive 2026-08-08 (agents keep the docker socket, but must | +| docs-build-fast-path | A | 2026-08-31 00:49 | PASS | #455 — docs builds must stay out of fast harness/eval/release gates; #536 — docs site externalized to openharness-web; docs markdown relocated to docs/ | +| drift-check-cron-staleness-glob | A | 2026-08-31 00:49 | PASS | issue #98; issue #225 (restart-required cron frontmatter/config drift) | +| entrypoint-pnpm-manifest-fingerprint | A | 2026-08-31 00:49 | PASS | issue #521 (manifest-aware sandbox installs) 2026-07-01 | +| eval-ci-gate | A | 2026-08-31 00:49 | PASS | #103 — eval probe suite gated in CI | +| eval-gate | A | 2026-08-31 00:49 | PASS | retro lesson 2026-06-11 (eval-gate) | +| eval-results-atomic | A | 2026-08-31 00:49 | PASS | issue #83 (eval-results-atomic-write) | +| eval-runner-exit | A | 2026-08-31 00:49 | PASS | retro lesson 2026-06-11 (eval-runner-exit) #29 | +| eval-runs-once-per-cycle | A | 2026-08-31 00:49 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-006) — /eval ran 3x per cycle on the | +| execution-target-contract | A | 2026-08-31 00:49 | PASS | issue #733 (ExecutionTarget contract + Docker Compose adapter) 2026-08-10 | +| get-oh-bootstrap | A | 2026-08-31 00:49 | PASS | get-oh.sh bootstrap — the Node-bootstrapping host-side path to the standalone `oh` CLI (also on npm as @mifune/openharness; see oh-npm-package.sh) | +| git-skill | A | 2026-08-31 00:49 | PASS | conversation 2026-06-15 — rules are not always supported; git workflow must be the /git skill | +| harness-audit-empty-output-gate | A | 2026-08-31 00:49 | PASS | issue #246 — /audit harness must fail closed on empty auditor outputs | +| harness-ci-core-paths | A | 2026-08-31 00:49 | PASS | #165 — core sandbox config files must trigger harness CI | +| harness-ci-hooks-paths | A | 2026-08-31 00:49 | PASS | issue #202 — credential/security hook changes must trigger harness CI | +| harness-yaml-migration | A | 2026-08-31 00:49 | PASS | PR #833 (migrate-harness-yaml.sh — append / uncomment-in-place / preserve / overwrite, plus a silent no-op second run) 2026-08-26 | +| health-check-docker-stats | A | 2026-08-31 00:49 | PASS | retro lesson 2026-06-10 (docker stats vs ps Size) | +| health-check-socket-degrade | A | 2026-08-31 00:49 | PASS | issue #762 (refs #756) — /health-check degrades to one statement, not nine failures | +| heartbeat-logging-contract | A | 2026-08-31 00:49 | PASS | issue #447 (heartbeat log append hardening) 2026-06-18 | +| image-seed-hygiene | A | 2026-08-31 00:49 | PASS | issue #900 (slim the sandbox image) 2026-08-30 | +| markitdown-wiki-ingest | A | 2026-08-31 00:49 | PASS | issue #649 — pinned local-document normalization contract for /wiki ingest | +| next-dev-prod | A | 2026-08-31 00:49 | SKIPPED | retro lesson 2026-06-04 | +| oh-compose-env-wiring | A | 2026-08-31 00:49 | PASS | issue #880 (oh as the only front door — oh.json is the non-secret config surface) | +| oh-config-surfaces | A | 2026-08-31 00:49 | REGRESSION | PR #887 (config split across two authored surfaces — a tracked oh.json and a secrets-only root dotenv — with nothing left under $HOME) | +| oh-destroy-guard | A | 2026-08-31 00:49 | PASS | issue #879 — `oh` becomes the only front door, so `make destroy` must | +| oh-devcontainer-restructure | A | 2026-08-31 00:49 | PASS | consolidate devcontainer — .oh/devcontainer/ folded back into .devcontainer/ | +| oh-home-mount | A | 2026-08-31 00:49 | PASS | issue #898 (single $HOME mount) 2026-08-30 | +| oh-image-only-deploy | A | 2026-08-31 00:49 | PASS | .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only deploy) | +| oh-init-headless-config | A | 2026-08-31 00:49 | PASS | PR #827 (installer answers landed in the losing config file); retargeted to the .example.env template by PR #833, then to oh.json by PR #887 | +| oh-init-scaffold | A | 2026-08-31 00:49 | PASS | issue #531 Phase 2 | +| oh-lifecycle-surface | A | 2026-08-31 00:49 | PASS | issue #881 — the Makefile is retired and `oh` is the only front door | +| oh-npm-package | A | 2026-08-31 00:49 | PASS | npm publish path for the standalone `oh` CLI (@mifune/openharness) — alternative to get-oh.sh | +| oh-payload-manifest | A | 2026-08-31 00:49 | PASS | issue #531 follow-on (.oh payload manifest — oh update ships a declared allowlist) | +| oh-sandbox-image-mode | A | 2026-08-31 00:49 | PASS | conversation 2026-07-05 (basic Docker deployment — prebuilt-image mode) | +| oh-shipped-repo-overridable | A | 2026-08-31 00:49 | PASS | issue #531 follow-on (de-hardcode residual — shipped .oh shell scripts keep the upstream repo overridable) | +| oh-standalone-lifecycle | A | 2026-08-31 00:49 | PASS | issue #564 | +| oh-update | A | 2026-08-31 00:49 | PASS | issue #531 Phase 3 (oh update — upgrade only the .oh control plane) | +| operator-config-guard | A | 2026-08-31 00:49 | PASS | operator directives 2026-08-06 (.config/ and settings.local.json are operator-only) | +| pnpm-audit-ci-gate | A | 2026-08-31 00:49 | PASS | issue #171 — pnpm security audits must run in CI | +| post-bridge-publish-confirmation | A | 2026-08-31 00:49 | PASS | #523 — post-bridge live publishing requires an explicit final confirmation gate | +| prd-output-path-contract | A | 2026-08-31 00:49 | PASS | retro lesson 2026-06-19 | +| prompt-miner-schema-compat | A | 2026-08-31 00:49 | PASS | issue #253 — prompt-miner JSONL schema-drift guard | +| prompt-miner-symlink-entrypoint | A | 2026-08-31 00:49 | PASS | issue #663 — prompt-miner engine no-ops via the documented .claude/skills symlink | +| prompt-miner-weakness-record | A | 2026-08-31 00:49 | PASS | issue #580 — prompt-miner weakness-record (WH-xxx) cluster output | +| protected-path-deletion | A | 2026-08-31 00:49 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-001) — the critique gate was deleted, | +| protected-paths-resolve | A | 2026-08-31 00:49 | PASS | issue #753 — .claude/protected-paths.txt named 7 paths that did not exist. | +| registry-portability-gate | A | 2026-08-31 00:49 | PASS | issue #758 | +| registry-portability | A | 2026-08-31 00:49 | SKIPPED | issue #758 | +| retro-deterministic-contract | A | 2026-08-31 00:49 | PASS | issue #443 — /retro deterministic output and self-contained helper contract | +| rl-delegation-write-worker | A | 2026-08-31 00:49 | PASS | retro lesson 2026-06-10 (rl-delegation) #57 | +| rlm-context-budget | A | 2026-08-31 00:49 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-006 | +| runtime-preflight-gate | A | 2026-08-31 00:49 | PASS | issue #806 § B1 (open sandbox.substrate vs sandbox.runtime selector); | +| sandbox-boot-guard-ci | A | 2026-08-31 00:49 | PASS | issue #449 (sandbox image build CI guard) 2026-06-19; | +| sandbox-node-base | A | 2026-08-31 00:49 | PASS | openharness#878 — oh as the only front door, T0 sandbox base image | +| skill-paths | A | 2026-08-31 00:49 | PASS | issue #43 — stale path references; extended by issue #69 — apps/->packages/ rename guard; extended by issue #870 — deleted .oh/agents/advisor.md | +| skills-dir-clean | A | 2026-08-31 00:49 | PASS | conversation 2026-06-29 — Pi parses every top-level `.md` in the skills | +| skills-task-tool-coupling | A | 2026-08-31 00:49 | PASS | council review 2026-08-29 (issue #886) — /delegate instructed Claude-Code-only | +| skills-vendored | A | 2026-08-31 00:49 | PASS | absorb .mifune submodule into .oh — the skills/agents/hooks pack is vendored | +| slack-admin-command-surface | A | 2026-08-31 00:49 | PASS | issue #354 — Slack bridge docs must distinguish Pi /msg-bridge commands from Slack DM admin text handlers | +| spec-family-contract | A | 2026-08-31 00:49 | PASS | issue #265; spec-simplification issue #816; workflow authority issue #854 | +| spec-ready-finalization | A | 2026-08-31 00:49 | PASS | issue #134; spec-simplification issue #816; workflow authority issue #854 | +| ste-checker-contract | A | 2026-08-31 00:49 | PASS | issue #750 PR audit — the /ste checker had four fail-open paths (unclosed | +| submitted-by-trailers | A | 2026-08-31 00:49 | PASS | conversation 2026-06-12 (commit attribution trailers); the single-owner | +| sync-skill-contract | A | 2026-08-31 00:49 | PASS | issue #331 — /sync dispatcher skill (bidirectional origin↔upstream sync) | +| tool-catalog-boundary | A | 2026-08-31 00:49 | PASS | agent-browser's exclusion from the harness catalog (#821) and the | +| version-parity | A | 2026-08-31 00:49 | PASS | conversation 2026-08-29 — the oh CLI became the only lifecycle door, so its | +| weigh-scorer-contract | A | 2026-08-31 00:49 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-003 (2026-06-27) | +| wiki-readme-index | A | 2026-08-31 00:49 | PASS | issue #132 — wiki README index drift guard | +| workflow-boundaries | A | 2026-08-31 00:49 | PASS | conversation 2026-06-19 (workflow consolidation, issue #259); authority moved to /spec in issue #854 | +| worktrees-layout | A | 2026-08-31 00:49 | PASS | issue #872 | diff --git a/.oh/evals/probes/image-seed-hygiene.sh b/.oh/evals/probes/image-seed-hygiene.sh new file mode 100755 index 00000000..58addb26 --- /dev/null +++ b/.oh/evals/probes/image-seed-hygiene.sh @@ -0,0 +1,132 @@ +#!/usr/bin/env bash +# tier: A +# source: issue #900 (slim the sandbox image) 2026-08-30 +# desc: The baked home seed ships no build caches (~/.npm, ~/.cache/uv are purged inside the stage the seed is copied from), /opt/home-seed keeps the 0700 mode of the home it replaces, and the build context excludes .pnpm-store and the .pi build outputs +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" +DOCKERFILE="$ROOT/.devcontainer/Dockerfile" +DOCKERIGNORE="$ROOT/.dockerignore" + +for f in "$DOCKERFILE" "$DOCKERIGNORE"; do + [[ -f "$f" ]] || { echo "SKIPPED: missing $f" >&2; exit 2; } +done + +fails=() + +instructions="$(grep -nvE '^[[:space:]]*(#|$)' "$DOCKERFILE" || true)" + +declare -A line_stage=() +declare -A stage_defined=() +cur_stage='' +while IFS= read -r entry; do + [[ -n "$entry" ]] || continue + ln="${entry%%:*}" + text="${entry#*:}" + if [[ "$text" =~ ^[[:space:]]*[Ff][Rr][Oo][Mm][[:space:]] ]]; then + if [[ "$text" =~ [[:space:]][Aa][Ss][[:space:]]+([A-Za-z0-9_.-]+)[[:space:]]*$ ]]; then + cur_stage="${BASH_REMATCH[1]}" + else + cur_stage='' + fi + stage_defined["$cur_stage"]=1 + fi + line_stage["$ln"]="$cur_stage" +done <<< "$instructions" + +mv_re='^[[:space:]]*RUN[[:space:]]+mv[[:space:]]+/home/sandbox[[:space:]]+/opt/home-seed[[:space:]]*$' +copy_re='^[[:space:]]*COPY[[:space:]].*--from=([^[:space:]]+).*[[:space:]]/home/sandbox[[:space:]]+/opt/home-seed[[:space:]]*$' + +stage_line="" +seed_stage="" +seed_form="" +while IFS= read -r entry; do + [[ -n "$entry" ]] || continue + ln="${entry%%:*}" + text="${entry#*:}" + if [[ "$text" =~ $mv_re ]]; then + stage_line="$ln"; seed_form="mv"; seed_stage="${line_stage[$ln]}"; break + fi + if [[ "$text" =~ $copy_re ]]; then + stage_line="$ln"; seed_form="copy"; seed_stage="${BASH_REMATCH[1]}"; break + fi +done <<< "$instructions" + +if [[ -z "$stage_line" ]]; then + fails+=("Dockerfile must stage the baked home at /opt/home-seed with 'RUN mv /home/sandbox /opt/home-seed' or 'COPY --from= /home/sandbox /opt/home-seed' (no instruction does)") +elif [[ "$seed_form" == "copy" && -z "${stage_defined[$seed_stage]:-}" ]]; then + fails+=("the /opt/home-seed staging COPY at line $stage_line sources --from=$seed_stage, which is not a stage defined in this Dockerfile; the seed must come from a stage this build produces") + stage_line="" +fi + +npm_cache_re='rm -rf[^;&|]*(/home/sandbox/\.npm|\$\{?HOME\}?/\.npm)([[:space:]]|/|$)' +uv_cache_re='rm -rf[^;&|]*(/home/sandbox/\.cache/uv|\$\{?HOME\}?/\.cache/uv|\$\{?UV_CACHE_DIR\}?)([[:space:]]|/|$)' + +check_purge() { + local label="$1" re="$2" entry ln found_any="" ok="" + [[ -n "$stage_line" ]] || return 0 + while IFS= read -r entry; do + [[ -n "$entry" ]] || continue + ln="${entry%%:*}" + [[ "${entry#*:}" =~ $re ]] || continue + found_any=1 + [[ "${line_stage[$ln]}" == "$seed_stage" ]] || continue + if [[ "$seed_form" == "mv" ]] && (( ln >= stage_line )); then continue; fi + ok=1 + done <<< "$instructions" + + if [[ -z "$found_any" ]]; then + fails+=("Dockerfile never removes the $label build cache from /home/sandbox; it ships inside the home seed staged at /opt/home-seed") + elif [[ -z "$ok" && "$seed_form" == "mv" ]]; then + fails+=("Dockerfile removes the $label build cache at or after the home is staged at /opt/home-seed (line $stage_line); the purge must run before staging") + elif [[ -z "$ok" ]]; then + fails+=("Dockerfile removes the $label build cache, but not inside the stage '$seed_stage' whose /home/sandbox becomes the seed staged at /opt/home-seed (line $stage_line); a purge in any other stage leaves the shipped seed unchanged") + fi +} + +check_purge "npm (~/.npm)" "$npm_cache_re" +check_purge "uv (~/.cache/uv)" "$uv_cache_re" + +if [[ -n "$stage_line" ]] \ + && ! grep -qE 'chmod([[:space:]]+-[^[:space:]]+)*[[:space:]]+0?700[[:space:]]+/opt/home-seed([[:space:]]|$)' "$DOCKERFILE" \ + && ! grep -qE '^[[:space:]]*COPY[[:space:]].*--chmod=0?700.*[[:space:]]/opt/home-seed([[:space:]]|$)' "$DOCKERFILE"; then + fails+=("/opt/home-seed must be mode 0700, matching the 0700 Debian useradd -m home it replaces; 'COPY --from' creates the destination 0755 and --chown does not restore the mode, so the final stage must chmod it") +fi + +ignored() { + grep -qE "^[[:space:]]*(\*\*/)?$1/?[[:space:]]*$" "$DOCKERIGNORE" +} + +ignored '\.pnpm-store' \ + || fails+=(".dockerignore must exclude .pnpm-store — a multi-GB gitignored pnpm content-addressable store that otherwise ships to the daemon on every local build") + +for out in '\.pi/bridge' '\.pi/npm'; do + ignored "$out" \ + || fails+=(".dockerignore must exclude ${out//\\/} — an untracked .pi build output (see .pi/.gitignore)") +done + +# The .pi exclusions must be surgical: /opt/oh-seed still needs every tracked +# .pi file, so a blanket .pi exclusion without re-includes is a regression. +if grep -qE '^[[:space:]]*(\*\*/)?\.pi/?[[:space:]]*$' "$DOCKERIGNORE" \ + && ! grep -qE '^[[:space:]]*!\.pi/' "$DOCKERIGNORE"; then + fails+=(".dockerignore excludes all of .pi/ without re-including the tracked files /opt/oh-seed needs; follow the exclude-then-re-include pattern already used for .claude/*") +fi + +if command -v git >/dev/null 2>&1 && git -C "$ROOT" rev-parse --git-dir >/dev/null 2>&1; then + while IFS= read -r tracked; do + case "$tracked" in + .pi/bridge/*|.pi/npm/*|.pnpm-store/*) + fails+=(".dockerignore excludes '$tracked', which is tracked in git and must reach the build context") + ;; + esac + done < <(git -C "$ROOT" ls-files .pi .pnpm-store) +fi + +if (( ${#fails[@]} > 0 )); then + echo "REGRESSION: sandbox image seed hygiene broken:" >&2 + printf ' - %s\n' "${fails[@]}" >&2 + exit 1 +fi + +echo "PASS: the Dockerfile purges ~/.npm and ~/.cache/uv inside the stage whose /home/sandbox becomes the seed staged at /opt/home-seed, stages that seed 0700, and .dockerignore keeps .pnpm-store and the .pi build outputs out of the build context without dropping any tracked .pi file" >&2 +exit 0 diff --git a/.oh/evals/probes/oh-home-mount.sh b/.oh/evals/probes/oh-home-mount.sh index f8548901..a529c2e0 100755 --- a/.oh/evals/probes/oh-home-mount.sh +++ b/.oh/evals/probes/oh-home-mount.sh @@ -52,8 +52,19 @@ if grep -qE '^[[:space:]]*-[[:space:]]*\.\.:' "$COMPOSE_IO"; then fails+=("docker-compose.image-only.yml must NOT bind a checkout — the workspace is a directory inside the home mount") fi -grep -Eq '^RUN mv /home/sandbox /opt/home-seed' "$DOCKERFILE" \ - || fails+=("Dockerfile must move the baked home to /opt/home-seed so a host bind and a named volume are seeded identically") +dockerfile_stages="$(grep -oiE '^[[:space:]]*FROM[[:space:]]+[^[:space:]]+[[:space:]]+AS[[:space:]]+[A-Za-z0-9_.-]+[[:space:]]*$' "$DOCKERFILE" | awk '{print $NF}')" +seed_copy="$(grep -E '^[[:space:]]*COPY[[:space:]].*--from=[^[:space:]]+.*[[:space:]]/home/sandbox[[:space:]]+/opt/home-seed[[:space:]]*$' "$DOCKERFILE" | head -n1)" +seed_from="$(printf '%s' "$seed_copy" | grep -oE -- '--from=[^[:space:]]+' | head -n1 | cut -d= -f2)" + +if grep -Eq '^[[:space:]]*RUN[[:space:]]+mv[[:space:]]+/home/sandbox[[:space:]]+/opt/home-seed[[:space:]]*$' "$DOCKERFILE"; then + : +elif [[ -z "$seed_from" ]]; then + fails+=("Dockerfile must stage the baked home at /opt/home-seed (RUN mv, or COPY --from= /home/sandbox /opt/home-seed) so a host bind and a named volume are seeded identically") +elif ! printf '%s\n' "$dockerfile_stages" | grep -qxF "$seed_from"; then + fails+=("the /opt/home-seed staging COPY sources --from=$seed_from, which is not a stage defined in this Dockerfile; an external image's /home/sandbox is not the home this build bakes") +elif [[ "$seed_from" == "base" ]]; then + fails+=("the /opt/home-seed staging COPY sources --from=base, whose /home/sandbox is bare useradd -m skel; the seed must come from the stage that bakes the dotfiles, agent state, and toolchain homes") +fi grep -Fq 'install -d -o sandbox -g sandbox -m 0755 /home/sandbox' "$DOCKERFILE" \ || fails+=("Dockerfile must leave /home/sandbox empty after staging the seed (an empty named volume must not auto-copy)") grep -Fq 'rm -rf /opt/home-seed/harness' "$DOCKERFILE" \ diff --git a/.oh/scripts/__tests__/sandbox-base-image.test.ts b/.oh/scripts/__tests__/sandbox-base-image.test.ts index 5f787767..c73ca3cd 100644 --- a/.oh/scripts/__tests__/sandbox-base-image.test.ts +++ b/.oh/scripts/__tests__/sandbox-base-image.test.ts @@ -8,11 +8,27 @@ const dockerfile = readFileSync(path.join(repoRoot, ".devcontainer/Dockerfile"), describe("sandbox base image", () => { it("builds from the official Node image on Debian Trixie", () => { - expect(dockerfile).toMatch(/^FROM node:22-trixie-slim$/m); + expect(dockerfile).toMatch(/^FROM node:22-trixie-slim( AS \S+)?$/m); expect(dockerfile).not.toContain("debian:bookworm-slim"); expect(dockerfile).not.toContain("deb.nodesource.com"); }); + it("derives every stage transitively from the pinned base", () => { + const froms = dockerfile + .split("\n") + .filter((line) => /^FROM\s/.test(line)) + .map((line) => line.trim().split(/\s+/)); + expect(froms.length).toBeGreaterThan(0); + + const defined = new Set(); + const foreign: string[] = []; + for (const [, image, asKeyword, name] of froms) { + if (image !== "node:22-trixie-slim" && !defined.has(image)) foreign.push(image); + if (asKeyword?.toLowerCase() === "as" && name) defined.add(name); + } + expect(foreign).toEqual([]); + }); + it("tracks Trixie for Docker's apt repository", () => { expect(dockerfile).toContain("https://download.docker.com/linux/debian trixie stable"); expect(dockerfile).not.toContain("https://download.docker.com/linux/debian bookworm stable"); diff --git a/.oh/tasks/slim-sandbox-image/prd.md b/.oh/tasks/slim-sandbox-image/prd.md new file mode 100644 index 00000000..5e3c1f82 --- /dev/null +++ b/.oh/tasks/slim-sandbox-image/prd.md @@ -0,0 +1,202 @@ +# PRD — Slim the sandbox image (issue #900) + +## 1. Overview + +Issue #900 proposes shrinking `.devcontainer/Dockerfile` to "base runtime + `oh` +CLI" and moving the agent CLIs, oh-my-zsh, the uv/python tree, and herdr to +runtime provisioning, on the theory that the single `/home/sandbox` mount from +#898/#899 now makes runtime installs persist. + +**Measurement rejects that premise.** The three largest artifacts in the image — +`@openai/codex` (336 MB), `@anthropic-ai/claude-code` (239 MB), `cc-safety-net` +(25 MB) — install to `/usr/lib/node_modules`, **outside** the mount. Moving them +to `oh harness install` would make them re-download on every container recreate: +a regression in the exact property #898 established. See §7. + +What #900 *can* legitimately reach is the 358 MB baked home at +`/opt/home-seed`. There the fix is not runtime provisioning — it is deleting +bytes that never needed to ship. + +This PRD delivers the image reduction #900 wants, without moving any install to +boot time, without a new version-pin surface, and without breaking offline first +boot or the prebuilt-image "docker run and go" promise. + +## 2. Goals + +| Goal | Measure | +|---|---| +| Stop shipping build caches in the home seed | `/opt/home-seed` contains no `.npm` and no `.cache/uv` | +| Stop shipping the home seed twice | The image stores the seed once, not once per side of `mv` | +| Stop pulling 2 GB of untracked junk into local build contexts | `.pnpm-store` and `.pi` build outputs excluded by `.dockerignore` | +| Change nothing an operator can observe | `verify-sandbox-image.sh` and the boot guard pass unchanged | + +Target: **~460 MB off the image**, **~2 GB off local build context**. + +## 3. Measured baseline + +`sandbox-openharness:latest`, measured in-container on 2026-08-30: + +| Path | Size | Produced at | Movable? | +|---|---|---|---| +| `/home/sandbox` (the seed) | **358 MB** | — | — | +| `.local` (pi 136 MB + uv python 95 MB + oh kernel 71 MB) | 238 MB | `Dockerfile:141-144,185` | keep | +| `.cache/uv` | **72 MB** | `Dockerfile:185` | **delete — build cache** | +| `.npm` | **31 MB** | `Dockerfile:141-144` | **delete — build cache** | +| `.oh-my-zsh` + 3 plugins | 18 MB | `Dockerfile:147-151` | keep | + +Build context (repo root, not excluded by `.dockerignore`): +`.pnpm-store` **1.5 GB**, `.pi` **509 MB**. + +Outside the mount, therefore *not* addressable by #900's idea without a +persistence regression: codex 336 MB, claude-code 239 MB, cc-safety-net 25 MB +(`/usr/lib/node_modules`); uv 49 MB, herdr 19 MB (`/usr/local/bin`); pnpm global +28 MB; `/opt/oh` 38 MB; hermes 2.0 GB when `INSTALL_HERMES=true`. + +**The `mv` doubles the seed.** `Dockerfile:194` (`RUN mv /home/sandbox +/opt/home-seed`) moves data across a layer boundary: the new layer holds a full +copy while every earlier layer still holds the original. Verified with a +synthetic control (busybox + 100 MB blob, with and without the `mv`): +**107,117,540 → 212,007,913 bytes.** So every byte removed from the seed is +worth two bytes of image, and storing the seed once is worth another 255 MB. + +## 4. Scope + +### In scope — Phase 1: purge build caches (independently shippable) + +1. In `.devcontainer/Dockerfile`, after the python-kernel layer + (`Dockerfile:185-189`) and before the seed staging layer + (`Dockerfile:194`), remove the build caches from the home: + `rm -rf /home/sandbox/.npm /home/sandbox/.cache/uv`. + Prefer folding the removal into the *producing* layers where that does not + complicate them; a dedicated layer is acceptable because the `mv` layer that + follows re-materializes the tree anyway. +2. Recreate `/home/sandbox/.cache/uv` as an empty `sandbox`-owned directory if + its absence would change runtime behavior — `entrypoint.sh`'s + `repair_home_mount_ownership` already `install -d`s it, so verify before + adding anything. +3. Extend `.dockerignore` with `.pnpm-store` and the untracked `.pi` build + outputs (`.pi/bridge`, `.pi/npm`), preserving the existing tracked-file + re-include pattern used for `.claude/*`. +4. New Tier-A probe `.oh/evals/probes/image-seed-hygiene.sh` asserting the + Dockerfile purges both caches before staging the seed, and that + `.dockerignore` excludes the store directories. + +### In scope — Phase 2: store the seed once (attempt; drop if CI regresses) + +5. Restructure `.devcontainer/Dockerfile` into stages so the home is produced in + a builder stage and lands in the final image via + `COPY --from= /home/sandbox /opt/home-seed`, with the final stage + creating an empty `/home/sandbox`. The natural split is at `Dockerfile:128`: + everything through the user creation is shared base; `Dockerfile:129-189` is + the home stage; the final stage keeps the non-home work (the `/opt/uv` and + hermes `chown -R` at `Dockerfile:138-139`, `/opt/oh-seed` at + `Dockerfile:181`, the entrypoint, labels, `WORKDIR`). +6. Runtime `ENV NPM_USER_PREFIX` / `UV_*` keep pointing at `/home/sandbox` — + semantics unchanged. Pi self-updates and `uv tool install` must keep + persisting into the mount. +7. Rewrite `.oh/evals/probes/oh-home-mount.sh:55-62`, which currently pins the + *spelling* of the mechanism (`RUN mv /home/sandbox /opt/home-seed`), to + assert the *invariant* instead: the seed is staged at `/opt/home-seed`, + `/home/sandbox` is empty in the image, and `harness/` is absent from the + seed. This rewrite ships in the same commit as the restructure. + +**Phase 2 is conditional.** If the builder stage lengthens the +`sandbox-boot-guard` build materially, or the restructure cannot preserve every +`verify-sandbox-image.sh` assertion, drop it from this PR and file it as a +follow-up. Phase 1 ships either way. + +### Out of scope + +- Moving any install to runtime provisioning (§7). +- Relocating `NPM_USER_PREFIX` / `UV_*` out of `$HOME` (§7). +- Flipping `INSTALL_PYTHON_KERNEL` to `false` (−166 MB). This is the first + change that would make first boot depend on the network; it is a policy + decision about default capability, not image hygiene. File separately. +- Anything about hermes (2.0 GB, already opt-in and already runtime-installable). +- `projects/mifunedev/openharness-web`. + +## 5. Success criteria + +1. `docker build` produces an image whose `/opt/home-seed` contains no `.npm` + and no `.cache/uv`; total image size drops by ≥ 200 MB (Phase 1 alone) and + ≥ 400 MB (both phases), measured against the same commit's pre-change build. +2. `.oh/scripts/verify-sandbox-image.sh` passes unchanged — every pin, version, + and UID/GID assertion still holds. +3. A booted container is behaviorally identical: `pi`, `claude`, `codex`, + `herdr`, `uv`, `python` all resolve; `~/.oh-my-zsh` and `~/.zshrc` present; + `oh` works; the python kernel is provisioned. +4. `bash .claude/skills/eval/run.sh` exits 0 (no regressions), and + `cd .oh/cli && npm test` passes. +5. Both CI workflows green: `sandbox-boot-guard` and `sandbox-compatibility`. +6. First boot still works with no network beyond the image pull. + +## 6. Constraints (AGENTS.md) + +- No explanatory comments in tracked code. The Dockerfile's existing header + comments are compose/Docker commentary, not tracked application code; do not + add new ones justifying the change — put that in the PR body. +- `.oh/` is canonical; never patch a generated mirror. +- Smallest realistic change. Resist scope creep into §4's out-of-scope list. +- Remote and disconnected operation must survive. + +## 7. Rejected alternatives + +**#900 as literally written** — agent CLIs / oh-my-zsh / uv / herdr moved to +runtime. Rejected on three counts. (a) The 600 MB prize (codex, claude-code, +cc-safety-net) lives outside the mount, so moving it to runtime *removes* +persistence rather than adding it; making it persist first requires relocating +the root npm global prefix into `$HOME`, which grows the seed by 600 MB. +(b) `.oh/cli/src/lib/harnesses/catalog.ts:23,33,44-52,63,88` installs +**unpinned** (`npm install -g @anthropic-ai/claude-code`, no version), so +runtime provisioning means *inventing* a pin-and-verify surface that build time +already has for free. (c) It converts `verify-sandbox-image.sh` from an image +oracle into a booted-container oracle, makes first boot network-dependent, and +contradicts `docs/deployment-prebuilt-image.md:15-19` ("a sandbox comes up in +the time it takes to pull"). + +**The middle path — relocate `NPM_USER_PREFIX`/`UV_*` to `/opt`.** Cuts the seed +to ~18 MB but saves zero image bytes (the same data moves to a different layer) +while destroying the persistence #898 bought: pi self-updates and +`uv tool install` results would stop surviving recreate. Strictly worse than +Phase 2, which achieves a larger reduction and keeps persistence. + +**BuildKit `--mount=type=cache` instead of `rm -rf`.** Would also solve the +cache problem, but adds a BuildKit requirement to a Dockerfile that must build +under plain `docker build` in both CI jobs. `rm -rf` is the smaller truthful +model. + +## 8. Build-time oracles that must keep passing + +These assert build-time facts and are the regression surface for this work: + +- `.oh/scripts/verify-sandbox-image.sh:10,19,48-52,102-119,133` — base codename, + apt suites, UID/GID, node/pnpm pins, herdr version + checksum, tool versions. + Invoked by both CI workflows. +- `.oh/evals/probes/cc-safety-net-wiring.sh:107-111` — the `cc-safety-net@` pin. +- `.oh/scripts/__tests__/herdr-default.test.ts:11-17` — `HERDR_VERSION` and both + arch sha256s. +- `.oh/scripts/__tests__/sandbox-base-image.test.ts:11`, + `.oh/evals/probes/sandbox-node-base.sh:21-26`, + `.oh/evals/probes/runtime-preflight-gate.sh:56-64` — `FROM` pin parity. +- `.oh/evals/probes/oh-home-mount.sh:55-62` — **expected to change in Phase 2 + only**, per §4.7. +- `.oh/evals/probes/tool-catalog-boundary.sh:35` — fails if + `INSTALL_AGENT_BROWSER` appears in the Dockerfile. Pre-existing rule that some + installs belong at boot; do not disturb it. +- `.oh/scripts/__tests__/provision-python.test.ts:80-88` — asserts the exact + `install -d` block, slicing to `indexOf("# Pi self-updates")`, **a comment + that no longer exists in the Dockerfile**, so the anchor silently degrades to + "rest of file". Fix the anchor opportunistically if Phase 1 touches that + region; do not expand scope to chase it otherwise. +- `.oh/evals/probes/{oh-image-only-deploy,sandbox-boot-guard-ci,oh-devcontainer-restructure,worktrees-layout}.sh` + — Dockerfile-coupled; re-run and confirm. + +## 9. Delivery + +One PR, branch `task/900-slim-sandbox-image`, cut from and targeting +`task/898-single-home-mount` (PR #899 is green but unmerged, and Phase 2 depends +on its `/opt/home-seed` staging). Retarget to `development` once #899 merges — +GitHub does that automatically. + +`Closes #900` in the body. CHANGELOG entry under `### Changed`, one sentence, +≤ 250 chars, linking the PR. diff --git a/CHANGELOG.md b/CHANGELOG.md index afd57deb..5ad8fc9b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,7 @@ Update policy and release automation live in [`/git`](.claude/skills/git/SKILL.m ### Changed - **BREAKING:** Persist the sandbox home through one `/home/sandbox` mount, not eleven per-tool volumes; set `storage.homePath` for a host path, else `_workspace` ([#898](https://github.com/mifunedev/openharness/issues/898)). +- Shrink the sandbox image ~540 MB: drop build caches from the baked home seed, stage the seed once via a builder stage, and keep untracked build output out of the build context ([#900](https://github.com/mifunedev/openharness/issues/900)). ### Removed - **BREAKING:** Retire the `projectRoot` / `OH_PROJECT_ROOT` config knob — the checkout is fixed at `/home/sandbox/harness`, nested inside the home mount ([#898](https://github.com/mifunedev/openharness/issues/898)). From 09dfa8c481aaac5a409ce0407e7328bd9f445fdf Mon Sep 17 00:00:00 2001 From: Ryan Eggleston Date: Mon, 31 Aug 2026 14:29:36 -0600 Subject: [PATCH 05/23] FROM task/902-provision-harnesses TO development (#903) * task: provision the default harnesses into the home mount oh is meant to split by execution target -- on the host it provisions the host or the sandbox, and inside the sandbox it provisions the sandbox with harnesses and tools. The second half did not work for the two harnesses most people use. claude-code and codex carried installUser: "root", which against the local execution target becomes sudo -n -- npm install -g, and /etc/sudoers.d/sandbox grants sandbox ALL=(ALL) ALL with no NOPASSWD. sudo -n true returns "a password is required". Both now match the pi entry directly above them: installUser "sandbox", npm --prefix /home/sandbox/.local install -g. That lands them inside the home mount, so they also survive container recreate and can be upgraded in place in a running remote sandbox rather than requiring an image rebuild. claude-code deliberately does not get --ignore-scripts. Its postinstall copies the native binary over a placeholder; with the flag the install succeeds and claude --version then fails with "claude native binary not installed". Verified both ways against a scratch prefix. provision-harnesses.sh follows provision-python.sh: the same mode flag, the same root to gosu sandbox re-exec, the same ownership diagnostics, the same die-with- the-command-to-re-run style. --print-env is absent because this provisioner exports nothing downstream. It reads the catalog through oh harness list --json and installs through oh harness install, so the shell knows no ids, packages, prefixes, or argv, and the TypeScript catalog stays the only description. No default harness carries a version pin today, so an existing install is never replaced and the script says so in its own output rather than implying it refreshes. The entrypoint hook runs after link-providers.sh, not before. link-providers' only binary dependency is cc-safety-net, which stays baked, and it is the boot-critical hard gate; a network-dependent best-effort step does not belong in front of the step that decides whether the boot is viable. Provisioning warns and continues, so an offline sandbox still comes up as a usable shell. BAKE_HARNESSES defaults to true and gates only the $AGENTS loop, not the whole RUN. INSTALL_OPENCODE and INSTALL_GROK_BUILD are separate opt-ins and turning them off as a side effect would be a silent regression. Nothing leaves the image in this change. * fix: bound the boot path and close four more provisioning defects An adversarial audit of #903 found five defects that six green checks missed. The serious one is a boot hang. oh harness list --json probes every entry in the catalog, not just the three defaults, and one of them is t3code, whose verifyArgv is npx --no-install t3 --version. npx contacts the registry, and probeInstalled passed no timeoutMs, so spawnSync waited without bound. Against an unreachable registry the auditor's run was still going at 2m30 when their own timeout killed it. On any boot where DNS resolves but the registry does not answer, the entrypoint blocks before sleep infinity, exceeds the 300s start_period, and never goes healthy -- and restart: unless-stopped does not rescue an unhealthy-but-alive container. Warn-and-continue cannot help, because a hang never reaches the if !. It hangs while listing, before any install, so BAKE_HARNESSES=true did not avoid it either. Bounded at three layers, because each fails differently: a 15s timeoutMs on the probe spawn, reported as unknown rather than a crash; a --defaults filter on oh harness list so the boot path probes three entries instead of nine and never runs npx; and a timeout wrapper on the entrypoint call so the boot is bounded whatever the CLI does. Measured against the auditor's exact command: 2m30 and killed, to 17.0s full-catalog and 1.55s with --defaults. The install loop read from a herestring while installs run with stdio inherit, so an installer that reads stdin consumed the rest of the loop. Reproduced with a stub: three missing harnesses, one installed, exit 0, success printed. Latent with npm, live the moment a default uses the curl | bash shape two catalog entries already use. Installs now read from /dev/null. The script force-exported OH_EXECUTION_TARGET=local, which short-circuits the in-container check, while the prefix is hardcoded to /home/sandbox/.local, and every error told the operator to re-run with no mention of where. On the host that provisioned the host. It now refuses unless inside the sandbox, reusing the CLI's own runningInsideSandbox predicate rather than inventing a check, and the entrypoint asserts the local target explicitly -- the documented raw docker run recipe never passes SANDBOX_NAME, so the guard would otherwise have silently skipped provisioning for the prebuilt-image flavor. The probe asserted that ARG BAKE_HARNESSES was declared, not that anything used it: deleting the gate left it green. It now checks the ARG is referenced by the RUN that installs $AGENTS and by the one that bakes pi, and that both stages declare it. Deleting either declaration also used to pass. ARG is stage-scoped, so BAKE_HARNESSES=false unbaked claude-code and codex but left pi baked in the home stage while the else-branch claimed otherwise. The home stage now declares and honors the flag. Also: OH_SANDBOX_USER was advertised but illusory, since the catalog hardcodes the user and prefix; it is gone. The final log line no longer claims to have provisioned anything in --verify mode. --- .devcontainer/Dockerfile | 26 ++- .devcontainer/entrypoint.sh | 7 + .oh/cli/src/__tests__/harness-catalog.test.ts | 77 +++++++ .oh/cli/src/__tests__/harness.test.ts | 54 ++++- .oh/cli/src/cli.ts | 22 +- .oh/cli/src/commands/harness.ts | 18 +- .oh/cli/src/lib/harnesses/catalog.ts | 26 ++- .oh/evals/RESULTS.md | 205 +++++++++--------- .oh/evals/probes/harness-home-provisioning.sh | 110 ++++++++++ .oh/scripts/provision-harnesses.sh | 158 ++++++++++++++ CHANGELOG.md | 1 + docs/harnesses/overview.md | 4 +- 12 files changed, 582 insertions(+), 126 deletions(-) create mode 100755 .oh/evals/probes/harness-home-provisioning.sh create mode 100755 .oh/scripts/provision-harnesses.sh diff --git a/.devcontainer/Dockerfile b/.devcontainer/Dockerfile index 60fe7347..beff0791 100644 --- a/.devcontainer/Dockerfile +++ b/.devcontainer/Dockerfile @@ -86,6 +86,7 @@ SHELL ["/bin/bash", "-c"] ARG AGENTS="claude-code,codex,pi-coding-agent" ARG INSTALL_OPENCODE=false ARG INSTALL_GROK_BUILD=false +ARG BAKE_HARNESSES=true RUN set -e; \ declare -A PKG=( \ @@ -94,13 +95,15 @@ RUN set -e; \ [pi-coding-agent]=@earendil-works/pi-coding-agent \ [opencode]=opencode-ai \ ); \ - IFS=',' read -ra agents <<< "$AGENTS"; \ - for a in "${agents[@]}"; do \ - if [ "$a" = "pi-coding-agent" ]; then continue; fi; \ - pkg="${PKG[$a]:-}"; \ - if [ -n "$pkg" ]; then npm install -g "$pkg"; \ - else echo "Unknown agent: $a"; exit 1; fi; \ - done; \ + if [ "${BAKE_HARNESSES}" = "true" ]; then \ + IFS=',' read -ra agents <<< "$AGENTS"; \ + for a in "${agents[@]}"; do \ + if [ "$a" = "pi-coding-agent" ]; then continue; fi; \ + pkg="${PKG[$a]:-}"; \ + if [ -n "$pkg" ]; then npm install -g "$pkg"; \ + else echo "Unknown agent: $a"; exit 1; fi; \ + done; \ + else echo "Skipping baked Claude Code and Codex installs (BAKE_HARNESSES=${BAKE_HARNESSES}); .oh/scripts/provision-harnesses.sh installs every kind:\"default\" harness into /home/sandbox/.local at boot"; fi; \ if [ "${INSTALL_OPENCODE}" = "true" ]; then npm install -g opencode-ai; \ else echo "Skipping OpenCode CLI install (INSTALL_OPENCODE=false)"; fi; \ if [ "${INSTALL_GROK_BUILD}" = "true" ]; then \ @@ -129,6 +132,7 @@ ENV UV_CACHE_DIR=/home/sandbox/.cache/uv FROM base AS home ARG AGENTS="claude-code,codex,pi-coding-agent" +ARG BAKE_HARNESSES=true RUN install -d -o sandbox -g sandbox -m 0700 /home/sandbox/.ssh \ && install -d -o sandbox -g sandbox \ /home/sandbox/.local \ @@ -141,9 +145,11 @@ RUN install -d -o sandbox -g sandbox -m 0700 /home/sandbox/.ssh \ RUN set -e; \ install -d -o sandbox -g sandbox "$NPM_USER_PREFIX"; \ - if [[ ",${AGENTS}," == *",pi-coding-agent,"* ]]; then \ - su - sandbox -c 'npm --prefix "$HOME/.local" install -g --ignore-scripts @earendil-works/pi-coding-agent'; \ - fi; \ + if [ "${BAKE_HARNESSES}" = "true" ]; then \ + if [[ ",${AGENTS}," == *",pi-coding-agent,"* ]]; then \ + su - sandbox -c 'npm --prefix "$HOME/.local" install -g --ignore-scripts @earendil-works/pi-coding-agent'; \ + fi; \ + else echo "Skipping baked Pi install (BAKE_HARNESSES=${BAKE_HARNESSES}); .oh/scripts/provision-harnesses.sh installs it into /home/sandbox/.local at boot"; fi; \ rm -rf /home/sandbox/.npm RUN su - sandbox -c "RUNZSH=no CHSH=no KEEP_ZSHRC=yes sh -c \"\$(curl -fsSL https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh)\" --unattended" \ diff --git a/.devcontainer/entrypoint.sh b/.devcontainer/entrypoint.sh index 7365c944..cbe342a9 100644 --- a/.devcontainer/entrypoint.sh +++ b/.devcontainer/entrypoint.sh @@ -150,6 +150,13 @@ if [ -x "$HARNESS/.oh/scripts/link-providers.sh" ]; then fi fi +if [ "${OH_PROVISION_HARNESSES:-true}" = "true" ] \ + && [ -x "$HARNESS/.oh/scripts/provision-harnesses.sh" ]; then + if ! OH_EXECUTION_TARGET=local timeout "${OH_PROVISION_HARNESSES_TIMEOUT:-180}" bash "$HARNESS/.oh/scripts/provision-harnesses.sh"; then + echo "[entrypoint] WARNING: harness provisioning did not complete; run: bash .oh/scripts/provision-harnesses.sh" >&2 + fi +fi + if [ "${OH_PROVISION_PYTHON:-true}" = "true" ] \ && [ -x "$HARNESS/.oh/scripts/provision-python.sh" ]; then if ! bash "$HARNESS/.oh/scripts/provision-python.sh"; then diff --git a/.oh/cli/src/__tests__/harness-catalog.test.ts b/.oh/cli/src/__tests__/harness-catalog.test.ts index f2f342c1..cc9b8bd7 100644 --- a/.oh/cli/src/__tests__/harness-catalog.test.ts +++ b/.oh/cli/src/__tests__/harness-catalog.test.ts @@ -16,6 +16,32 @@ const read = (rel: string): string => readFileSync(join(REPO_ROOT, rel), "utf8") const DOCKERFILE = read(".devcontainer/Dockerfile"); const COMPOSE_YML = read(".devcontainer/docker-compose.yml"); const CONFIG_DOC = read("docs/configuration.md"); +const ENTRYPOINT = read(".devcontainer/entrypoint.sh"); +const NPM_USER_PREFIX = "/home/sandbox/.local"; + +const BAKE_GATE = 'if [ "${BAKE_HARNESSES}" = "true" ]'; + +function dockerfileStage(stage: string): string { + const lines = DOCKERFILE.split("\n"); + const start = lines.findIndex((l) => new RegExp(`^FROM .* AS ${stage}$`).test(l)); + if (start === -1) return ""; + const rest = lines.slice(start + 1); + const end = rest.findIndex((l) => l.startsWith("FROM ")); + return (end === -1 ? rest : rest.slice(0, end)).join("\n"); +} + +function dockerfileRunWith(needle: string): string { + const blocks: string[] = []; + let buf: string | null = null; + for (const line of DOCKERFILE.split("\n")) { + if (buf === null && !line.startsWith("RUN ")) continue; + buf = buf === null ? line : `${buf}\n${line}`; + if (line.endsWith("\\")) continue; + blocks.push(buf); + buf = null; + } + return blocks.find((b) => b.includes(needle)) ?? ""; +} function versionPins(argv: readonly string[]): string[] { const pins = new Set(); @@ -127,6 +153,57 @@ describe("harness catalog", () => { } }); + describe("default harnesses install into the home mount, not the image", () => { + const defaults = HARNESS_CATALOG.filter((h) => h.kind === "default"); + + it("covers claude-code, codex and pi", () => { + expect(defaults.map((h) => h.id).sort()).toEqual(["claude-code", "codex", "pi"]); + }); + + it("declares NPM_USER_PREFIX as the prefix the catalog installs into", () => { + expect(DOCKERFILE).toContain(`ENV NPM_USER_PREFIX="${NPM_USER_PREFIX}"`); + }); + + it.each(defaults.map((h) => [h.id, h] as const))( + "%s: installs as the sandbox user into NPM_USER_PREFIX", + (_id, h) => { + expect(h.installUser).toBe("sandbox"); + expect(h.installArgv).toContain(NPM_USER_PREFIX); + }, + ); + + it("keeps claude-code's postinstall, which copies the native binary over the placeholder", () => { + expect(findHarness("claude-code")!.installArgv).not.toContain("--ignore-scripts"); + }); + + it("lets the image bake be turned off, and provisions the same harnesses at boot", () => { + expect(DOCKERFILE).toMatch(/^ARG BAKE_HARNESSES=true$/m); + expect(ENTRYPOINT).toContain("OH_PROVISION_HARNESSES"); + expect(ENTRYPOINT).toContain(".oh/scripts/provision-harnesses.sh"); + }); + + it.each(["base", "home"])( + "%s declares BAKE_HARNESSES, which ARG scopes to that stage alone", + (stage) => { + expect(dockerfileStage(stage)).toMatch(/^ARG BAKE_HARNESSES/m); + }, + ); + + it("gates every baked default install on BAKE_HARNESSES, pi included", () => { + expect(dockerfileRunWith("read -ra agents")).toContain(BAKE_GATE); + expect(dockerfileRunWith("--ignore-scripts @earendil-works/pi-coding-agent")).toContain( + BAKE_GATE, + ); + }); + + it("bounds the boot-path provisioner so an unreachable registry cannot stall the entrypoint", () => { + expect(ENTRYPOINT).toMatch( + /timeout "\$\{OH_PROVISION_HARNESSES_TIMEOUT:-\d+\}" bash "\$HARNESS\/\.oh\/scripts\/provision-harnesses\.sh"/, + ); + expect(ENTRYPOINT).toContain("WARNING: harness provisioning did not complete"); + }); + }); + it("findHarness resolves known ids and rejects unknown ones", () => { expect(findHarness("opencode")?.harnessKey).toBe("opencode"); expect(findHarness("grok-build")?.harnessKey).toBe("grok_build"); diff --git a/.oh/cli/src/__tests__/harness.test.ts b/.oh/cli/src/__tests__/harness.test.ts index 277951dc..06b9840f 100644 --- a/.oh/cli/src/__tests__/harness.test.ts +++ b/.oh/cli/src/__tests__/harness.test.ts @@ -9,6 +9,7 @@ vi.mock("node:os", async (importOriginal) => { import { dirname, join } from "node:path"; import { fileURLToPath } from "node:url"; import { + PROBE_TIMEOUT_MS, runHarnessInstall, runHarnessList, runHarnessStatus, @@ -49,14 +50,19 @@ function makeRepo(): string { interface RecordedCall { cmd: string; args: string[]; + timeoutMs?: number; } function makeRunner( reply: (cmd: string, args: string[]) => RunResult | undefined = () => undefined, ): { calls: RecordedCall[]; run: LifecycleRunner } { const calls: RecordedCall[] = []; - const run: LifecycleRunner = (cmd, args) => { - calls.push({ cmd, args: [...args] }); + const run: LifecycleRunner = (cmd, args, opts) => { + calls.push({ + cmd, + args: [...args], + ...(opts.timeoutMs !== undefined ? { timeoutMs: opts.timeoutMs } : {}), + }); return reply(cmd, args) ?? { status: 0, stdout: "", stderr: "" }; }; return { calls, run }; @@ -414,6 +420,50 @@ describe("runHarnessList", () => { }); }); +describe("runHarnessList — a hung verify probe cannot stall the boot path", () => { + const INSIDE_SANDBOX: NodeJS.ProcessEnv = { OH_EXECUTION_TARGET: "local" }; + + it("bounds every probe spawn with a timeout", async () => { + const root = makeRepo(); + const { calls, run } = makeRunner(); + await runHarnessList({ cwd: root, run, env: INSIDE_SANDBOX, json: true }, makeIo().io); + expect(calls.length).toBeGreaterThan(0); + for (const call of calls) expect(call.timeoutMs).toBe(PROBE_TIMEOUT_MS); + }); + + it("reports a timed-out probe as unknown rather than throwing", async () => { + const root = makeRepo(); + const { run } = makeRunner((cmd) => + cmd === "npx" + ? { status: null, error: { code: "ETIMEDOUT", message: "spawnSync npx ETIMEDOUT" } } + : undefined, + ); + const { out, io } = makeIo(); + expect(await runHarnessList({ cwd: root, run, env: INSIDE_SANDBOX, json: true }, io)).toBe(0); + const parsed = JSON.parse(text(out)); + expect(parsed.find((h: { id: string }) => h.id === "t3code").installed).toBeNull(); + expect(parsed.find((h: { id: string }) => h.id === "claude-code").installed).toBe(true); + }); + + it("--defaults probes only the default harnesses, never the registry-touching ones", async () => { + const root = makeRepo(); + const { calls, run } = makeRunner(); + const { out, io } = makeIo(); + expect( + await runHarnessList( + { cwd: root, run, env: INSIDE_SANDBOX, json: true, defaultsOnly: true }, + io, + ), + ).toBe(0); + expect(JSON.parse(text(out)).map((h: { id: string }) => h.id)).toEqual([ + "claude-code", + "codex", + "pi", + ]); + expect(calls.map((c) => c.cmd).sort()).toEqual(["claude", "codex", "pi"]); + }); +}); + describe("runHarnessStatus", () => { it("with no name behaves like list", async () => { const root = makeRepo(); diff --git a/.oh/cli/src/cli.ts b/.oh/cli/src/cli.ts index 0c4efe9a..733ab131 100644 --- a/.oh/cli/src/cli.ts +++ b/.oh/cli/src/cli.ts @@ -268,7 +268,7 @@ export function printHarnessHelp(): void { process.stdout.write(`oh harness — Install and inspect agent CLI harnesses Usage: - oh harness list List known harnesses and their state + oh harness list [--defaults] List known harnesses and their state oh harness install Install a harness into the sandbox oh harness status [name] Show installed/enabled state @@ -281,6 +281,7 @@ and exits 0. Flags: --persist-only Only set the oh.json install.* field (no container work) --no-persist Live-install only; leave oh.json unchanged + --defaults List only kind:"default" harnesses (list) --json Machine-readable output (list/status) Harnesses: @@ -820,11 +821,18 @@ export interface HarnessArgs { name?: string; persistOnly: boolean; noPersist: boolean; + defaultsOnly: boolean; json: boolean; } export function parseHarnessArgs(rest: string[]): ParseResult { - const args: HarnessArgs = { help: false, persistOnly: false, noPersist: false, json: false }; + const args: HarnessArgs = { + help: false, + persistOnly: false, + noPersist: false, + defaultsOnly: false, + json: false, + }; if (rest.length === 0 || isHelpFlag(rest[0])) { return { ok: true, args: { ...args, help: true } }; } @@ -835,6 +843,8 @@ export function parseHarnessArgs(rest: string[]): ParseResult { args.persistOnly = true; } else if (token === "--no-persist") { args.noPersist = true; + } else if (token === "--defaults") { + args.defaultsOnly = true; } else if (token === "--json") { args.json = true; } else if (token.startsWith("-")) { @@ -861,6 +871,12 @@ export function parseHarnessArgs(rest: string[]): ParseResult { if (sub === "list" && name !== undefined) { return { ok: false, error: `oh harness list: unexpected argument "${name}"` }; } + if (args.defaultsOnly && sub !== "list") { + return { + ok: false, + error: `oh harness ${sub}: --defaults applies to \`oh harness list\` only`, + }; + } if (args.persistOnly && args.noPersist) { return { ok: false, @@ -1335,7 +1351,7 @@ async function main(argv: string[]): Promise { stderr: (s) => process.stderr.write(s), }; if (a.subcommand === "list") { - return await runHarnessList({ json: a.json }, io); + return await runHarnessList({ json: a.json, defaultsOnly: a.defaultsOnly }, io); } if (a.subcommand === "status") { return await runHarnessStatus(a.name, { json: a.json }, io); diff --git a/.oh/cli/src/commands/harness.ts b/.oh/cli/src/commands/harness.ts index d7c89d51..5050e405 100644 --- a/.oh/cli/src/commands/harness.ts +++ b/.oh/cli/src/commands/harness.ts @@ -12,6 +12,7 @@ import { setInstallFlag, } from "../lib/env-file.js"; import { + defaultHarnesses, findHarness, harnessIds, HARNESS_CATALOG, @@ -30,6 +31,7 @@ export interface HarnessOptions { run?: LifecycleRunner; json?: boolean; env?: NodeJS.ProcessEnv; + defaultsOnly?: boolean; } export interface HarnessInstallOptions extends HarnessOptions { @@ -46,6 +48,8 @@ interface HarnessState { docs: string; } +export const PROBE_TIMEOUT_MS = 15_000; + function isReachable(status: string): boolean { return status === "ready" || status === "starting"; } @@ -73,6 +77,7 @@ async function probeInstalled( argv: [...entry.verifyArgv], user: "sandbox", stdio: "capture", + timeoutMs: PROBE_TIMEOUT_MS, }); return r.exitCode === 0; } catch (err) { @@ -85,9 +90,9 @@ async function collectStates( root: string, run: LifecycleRunner, env?: NodeJS.ProcessEnv, - only?: HarnessEntry, + only?: readonly HarnessEntry[], ): Promise { - const entries = only ? [only] : [...HARNESS_CATALOG]; + const entries = only ? [...only] : [...HARNESS_CATALOG]; const target = targetFor(root, run, env); let reachable = false; @@ -140,7 +145,12 @@ function renderTable(states: HarnessState[], io: HarnessIO): void { export async function runHarnessList(opts: HarnessOptions, io: HarnessIO): Promise { const run = opts.run ?? spawnRunner; const root = resolveProjectRoot(opts.cwd); - const states = await collectStates(root, run, opts.env); + const states = await collectStates( + root, + run, + opts.env, + opts.defaultsOnly === true ? defaultHarnesses() : undefined, + ); if (opts.json) { io.stdout(`${JSON.stringify(states, null, 2)}\n`); } else { @@ -169,7 +179,7 @@ export async function runHarnessStatus( if (!only) return unknownHarness(name, io); } - const states = await collectStates(root, run, opts.env, only); + const states = await collectStates(root, run, opts.env, only ? [only] : undefined); if (opts.json) { io.stdout(`${JSON.stringify(only ? states[0] : states, null, 2)}\n`); } else { diff --git a/.oh/cli/src/lib/harnesses/catalog.ts b/.oh/cli/src/lib/harnesses/catalog.ts index 11be7a7c..b6c0f408 100644 --- a/.oh/cli/src/lib/harnesses/catalog.ts +++ b/.oh/cli/src/lib/harnesses/catalog.ts @@ -22,8 +22,15 @@ export const HARNESS_CATALOG: readonly HarnessEntry[] = [ id: "claude-code", title: "Claude Code", binary: "claude", - installArgv: ["npm", "install", "-g", "@anthropic-ai/claude-code"], - installUser: "root", + installArgv: [ + "npm", + "--prefix", + "/home/sandbox/.local", + "install", + "-g", + "@anthropic-ai/claude-code", + ], + installUser: "sandbox", verifyArgv: ["claude", "--version"], docsPath: "docs/harnesses/claude-code.md", kind: "default", @@ -32,8 +39,15 @@ export const HARNESS_CATALOG: readonly HarnessEntry[] = [ id: "codex", title: "Codex", binary: "codex", - installArgv: ["npm", "install", "-g", "@openai/codex"], - installUser: "root", + installArgv: [ + "npm", + "--prefix", + "/home/sandbox/.local", + "install", + "-g", + "@openai/codex", + ], + installUser: "sandbox", verifyArgv: ["codex", "--version"], docsPath: "docs/harnesses/codex.md", kind: "default", @@ -138,6 +152,10 @@ export const HARNESS_CATALOG: readonly HarnessEntry[] = [ }, ]; +export function defaultHarnesses(): readonly HarnessEntry[] { + return HARNESS_CATALOG.filter((h) => h.kind === "default"); +} + export function findHarness(id: string): HarnessEntry | undefined { return HARNESS_CATALOG.find((h) => h.id === id); } diff --git a/.oh/evals/RESULTS.md b/.oh/evals/RESULTS.md index ee53d0d4..a6c7ac50 100644 --- a/.oh/evals/RESULTS.md +++ b/.oh/evals/RESULTS.md @@ -6,107 +6,108 @@ probe id; git history is the time series.** Schema and exit-code semantics are i | probe | tier | last-run (UTC) | status | source | |-------|------|----------------|--------|--------| -| advisor-monitored-loop | A | 2026-08-31 00:49 | PASS | conversation 2026-06-19 (single-owner implementation workflow, issue #257) | -| agent-browser-cli | A | 2026-08-31 00:49 | PASS | retro lesson 2026-06-07 (agent-browser 0.8.5 CLI) | -| agents-identity-contract | A | 2026-08-31 00:49 | PASS | issue #854 — T3-style root identity, glossary, and skill-owned procedures | -| artifact-contract-audit | A | 2026-08-31 00:49 | PASS | issue #583/#645 — production /audit implementation Gate 1 behavior | -| audit-dispatcher-contract | A | 2026-08-31 00:49 | PASS | issue #645 — audit consolidation public taxonomy | -| audit-implementation-behavior | A | 2026-08-31 00:49 | PASS | issue #645 — implementation root/repo/browser behavior | -| audit-pr-acquire | A | 2026-08-31 00:49 | PASS | issue #645 — production PR acquisition behavior | -| audit-pr-classifier | A | 2026-08-31 00:49 | PASS | issue #645 — deterministic focused and queue PR classifier | -| audit-run-root-contract | A | 2026-08-31 00:49 | PASS | issue #645 — executable immutable audit root/run correlation | -| audit-shellcheck-coverage | A | 2026-08-31 00:49 | PASS | issue #645 — private audit scripts require release and CI lint coverage | -| audit-stale-references | A | 2026-08-31 00:49 | PASS | issue #645 — clean-breaking audit migration | -| boot-lint-glob | A | 2026-08-31 00:49 | PASS | issue #90, issue #120 | -| builder-skill-consolidation | A | 2026-08-31 00:49 | PASS | issue #643 — consolidate artifact builders behind one /builder dispatcher | -| capability-benchmark-schema | A | 2026-08-31 00:49 | PASS | issue #167 — capability benchmark instrument | -| cc-safety-net-wiring | A | 2026-08-31 00:49 | SKIPPED | .oh/tasks/cc-safety-net/prd.json US-007 2026-07-19 | -| changelog-entry-length | A | 2026-08-31 00:49 | PASS | conversation 2026-08-24 — CHANGELOG.md grew to 259KB of bullet prose because "one line" was unquantified | -| cleanup-tasks-scoped-guard | A | 2026-08-31 00:49 | PASS | issue #85 | -| cleanup-tasks-worktree-grooming | A | 2026-08-31 00:49 | PASS | issue #168; issue #327 | -| cli-publish-typecheck-scope | A | 2026-08-31 00:49 | PASS | release run 33271077312 — v0.5.0 pushed its GHCR image, then failed to publish | -| close-issues-on-development | A | 2026-08-31 00:49 | PASS | issue #841 (closing keywords never fire because the default branch is main) 2026-08-26 | -| codex-stale-response-retry | A | 2026-08-31 00:49 | PASS | issue #506 — Codex previous_response_not_found RCA | -| compose-config-path-parity | A | 2026-08-31 00:49 | PASS | PR #833 (remove harness.yaml — the wrapper and VS Code "Reopen in Container" paths must resolve the same service) 2026-08-26 | -| config-schema-parity | A | 2026-08-31 00:49 | PASS | PR #833 (one schema file — DOCKER_SOCKET, SANDBOX_SSH, OH_SANDBOX_IMAGE, OH_PULL_POLICY, SKIP_PNPM_INSTALL were consumed but undocumented); rewritten for the oh.json/secrets split by PR #887 | -| context-tier-size-budget | A | 2026-08-31 00:49 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-007) — the always-on tier was 85,256 B | -| cron-claude-codex-fallback | A | 2026-08-31 00:49 | PASS | conversation 2026-06-12 (default Codex fallback for crons) | -| cron-watchdog | A | 2026-08-31 00:49 | PASS | issues #130/#453 (cron runtime watchdog + legacy system-cron reaping) 2026-06-19 | -| crons-directory-guide | A | 2026-08-31 00:49 | PASS | issue #874 | -| curl-bash-safe-alternatives | A | 2026-08-31 00:49 | PASS | vet-run/vet integration — public curl|bash examples need review-first alternatives | -| datasets-schema | A | 2026-08-31 00:49 | PASS | issue #196 — .oh/evals/datasets verifiable trajectory corpus (Repo2RLEnv-inspired) | -| debugmcp-availability | A | 2026-08-31 00:49 | SKIPPED | issue #297 — DebugMCP MCP debug-server availability | -| delegate-model-effort-policy | A | 2026-08-31 00:49 | PASS | conversation 2026-07-11 (delegate model inheritance and thinking policy) | -| devtcp-hook | A | 2026-08-31 00:49 | PASS | retro lesson 2026-06-10 (zsh /dev/tcp) | -| docker-inspect-env-guard | A | 2026-08-31 00:49 | PASS | operator directive 2026-08-08 (agents keep the docker socket, but must | -| docs-build-fast-path | A | 2026-08-31 00:49 | PASS | #455 — docs builds must stay out of fast harness/eval/release gates; #536 — docs site externalized to openharness-web; docs markdown relocated to docs/ | -| drift-check-cron-staleness-glob | A | 2026-08-31 00:49 | PASS | issue #98; issue #225 (restart-required cron frontmatter/config drift) | -| entrypoint-pnpm-manifest-fingerprint | A | 2026-08-31 00:49 | PASS | issue #521 (manifest-aware sandbox installs) 2026-07-01 | -| eval-ci-gate | A | 2026-08-31 00:49 | PASS | #103 — eval probe suite gated in CI | -| eval-gate | A | 2026-08-31 00:49 | PASS | retro lesson 2026-06-11 (eval-gate) | -| eval-results-atomic | A | 2026-08-31 00:49 | PASS | issue #83 (eval-results-atomic-write) | -| eval-runner-exit | A | 2026-08-31 00:49 | PASS | retro lesson 2026-06-11 (eval-runner-exit) #29 | -| eval-runs-once-per-cycle | A | 2026-08-31 00:49 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-006) — /eval ran 3x per cycle on the | -| execution-target-contract | A | 2026-08-31 00:49 | PASS | issue #733 (ExecutionTarget contract + Docker Compose adapter) 2026-08-10 | -| get-oh-bootstrap | A | 2026-08-31 00:49 | PASS | get-oh.sh bootstrap — the Node-bootstrapping host-side path to the standalone `oh` CLI (also on npm as @mifune/openharness; see oh-npm-package.sh) | -| git-skill | A | 2026-08-31 00:49 | PASS | conversation 2026-06-15 — rules are not always supported; git workflow must be the /git skill | -| harness-audit-empty-output-gate | A | 2026-08-31 00:49 | PASS | issue #246 — /audit harness must fail closed on empty auditor outputs | -| harness-ci-core-paths | A | 2026-08-31 00:49 | PASS | #165 — core sandbox config files must trigger harness CI | -| harness-ci-hooks-paths | A | 2026-08-31 00:49 | PASS | issue #202 — credential/security hook changes must trigger harness CI | -| harness-yaml-migration | A | 2026-08-31 00:49 | PASS | PR #833 (migrate-harness-yaml.sh — append / uncomment-in-place / preserve / overwrite, plus a silent no-op second run) 2026-08-26 | -| health-check-docker-stats | A | 2026-08-31 00:49 | PASS | retro lesson 2026-06-10 (docker stats vs ps Size) | -| health-check-socket-degrade | A | 2026-08-31 00:49 | PASS | issue #762 (refs #756) — /health-check degrades to one statement, not nine failures | -| heartbeat-logging-contract | A | 2026-08-31 00:49 | PASS | issue #447 (heartbeat log append hardening) 2026-06-18 | -| image-seed-hygiene | A | 2026-08-31 00:49 | PASS | issue #900 (slim the sandbox image) 2026-08-30 | -| markitdown-wiki-ingest | A | 2026-08-31 00:49 | PASS | issue #649 — pinned local-document normalization contract for /wiki ingest | -| next-dev-prod | A | 2026-08-31 00:49 | SKIPPED | retro lesson 2026-06-04 | -| oh-compose-env-wiring | A | 2026-08-31 00:49 | PASS | issue #880 (oh as the only front door — oh.json is the non-secret config surface) | -| oh-config-surfaces | A | 2026-08-31 00:49 | REGRESSION | PR #887 (config split across two authored surfaces — a tracked oh.json and a secrets-only root dotenv — with nothing left under $HOME) | -| oh-destroy-guard | A | 2026-08-31 00:49 | PASS | issue #879 — `oh` becomes the only front door, so `make destroy` must | -| oh-devcontainer-restructure | A | 2026-08-31 00:49 | PASS | consolidate devcontainer — .oh/devcontainer/ folded back into .devcontainer/ | -| oh-home-mount | A | 2026-08-31 00:49 | PASS | issue #898 (single $HOME mount) 2026-08-30 | -| oh-image-only-deploy | A | 2026-08-31 00:49 | PASS | .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only deploy) | -| oh-init-headless-config | A | 2026-08-31 00:49 | PASS | PR #827 (installer answers landed in the losing config file); retargeted to the .example.env template by PR #833, then to oh.json by PR #887 | -| oh-init-scaffold | A | 2026-08-31 00:49 | PASS | issue #531 Phase 2 | -| oh-lifecycle-surface | A | 2026-08-31 00:49 | PASS | issue #881 — the Makefile is retired and `oh` is the only front door | -| oh-npm-package | A | 2026-08-31 00:49 | PASS | npm publish path for the standalone `oh` CLI (@mifune/openharness) — alternative to get-oh.sh | -| oh-payload-manifest | A | 2026-08-31 00:49 | PASS | issue #531 follow-on (.oh payload manifest — oh update ships a declared allowlist) | -| oh-sandbox-image-mode | A | 2026-08-31 00:49 | PASS | conversation 2026-07-05 (basic Docker deployment — prebuilt-image mode) | -| oh-shipped-repo-overridable | A | 2026-08-31 00:49 | PASS | issue #531 follow-on (de-hardcode residual — shipped .oh shell scripts keep the upstream repo overridable) | -| oh-standalone-lifecycle | A | 2026-08-31 00:49 | PASS | issue #564 | -| oh-update | A | 2026-08-31 00:49 | PASS | issue #531 Phase 3 (oh update — upgrade only the .oh control plane) | -| operator-config-guard | A | 2026-08-31 00:49 | PASS | operator directives 2026-08-06 (.config/ and settings.local.json are operator-only) | -| pnpm-audit-ci-gate | A | 2026-08-31 00:49 | PASS | issue #171 — pnpm security audits must run in CI | -| post-bridge-publish-confirmation | A | 2026-08-31 00:49 | PASS | #523 — post-bridge live publishing requires an explicit final confirmation gate | -| prd-output-path-contract | A | 2026-08-31 00:49 | PASS | retro lesson 2026-06-19 | -| prompt-miner-schema-compat | A | 2026-08-31 00:49 | PASS | issue #253 — prompt-miner JSONL schema-drift guard | -| prompt-miner-symlink-entrypoint | A | 2026-08-31 00:49 | PASS | issue #663 — prompt-miner engine no-ops via the documented .claude/skills symlink | -| prompt-miner-weakness-record | A | 2026-08-31 00:49 | PASS | issue #580 — prompt-miner weakness-record (WH-xxx) cluster output | -| protected-path-deletion | A | 2026-08-31 00:49 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-001) — the critique gate was deleted, | -| protected-paths-resolve | A | 2026-08-31 00:49 | PASS | issue #753 — .claude/protected-paths.txt named 7 paths that did not exist. | -| registry-portability-gate | A | 2026-08-31 00:49 | PASS | issue #758 | -| registry-portability | A | 2026-08-31 00:49 | SKIPPED | issue #758 | -| retro-deterministic-contract | A | 2026-08-31 00:49 | PASS | issue #443 — /retro deterministic output and self-contained helper contract | -| rl-delegation-write-worker | A | 2026-08-31 00:49 | PASS | retro lesson 2026-06-10 (rl-delegation) #57 | -| rlm-context-budget | A | 2026-08-31 00:49 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-006 | -| runtime-preflight-gate | A | 2026-08-31 00:49 | PASS | issue #806 § B1 (open sandbox.substrate vs sandbox.runtime selector); | -| sandbox-boot-guard-ci | A | 2026-08-31 00:49 | PASS | issue #449 (sandbox image build CI guard) 2026-06-19; | -| sandbox-node-base | A | 2026-08-31 00:49 | PASS | openharness#878 — oh as the only front door, T0 sandbox base image | -| skill-paths | A | 2026-08-31 00:49 | PASS | issue #43 — stale path references; extended by issue #69 — apps/->packages/ rename guard; extended by issue #870 — deleted .oh/agents/advisor.md | -| skills-dir-clean | A | 2026-08-31 00:49 | PASS | conversation 2026-06-29 — Pi parses every top-level `.md` in the skills | -| skills-task-tool-coupling | A | 2026-08-31 00:49 | PASS | council review 2026-08-29 (issue #886) — /delegate instructed Claude-Code-only | -| skills-vendored | A | 2026-08-31 00:49 | PASS | absorb .mifune submodule into .oh — the skills/agents/hooks pack is vendored | -| slack-admin-command-surface | A | 2026-08-31 00:49 | PASS | issue #354 — Slack bridge docs must distinguish Pi /msg-bridge commands from Slack DM admin text handlers | -| spec-family-contract | A | 2026-08-31 00:49 | PASS | issue #265; spec-simplification issue #816; workflow authority issue #854 | -| spec-ready-finalization | A | 2026-08-31 00:49 | PASS | issue #134; spec-simplification issue #816; workflow authority issue #854 | -| ste-checker-contract | A | 2026-08-31 00:49 | PASS | issue #750 PR audit — the /ste checker had four fail-open paths (unclosed | -| submitted-by-trailers | A | 2026-08-31 00:49 | PASS | conversation 2026-06-12 (commit attribution trailers); the single-owner | -| sync-skill-contract | A | 2026-08-31 00:49 | PASS | issue #331 — /sync dispatcher skill (bidirectional origin↔upstream sync) | -| tool-catalog-boundary | A | 2026-08-31 00:49 | PASS | agent-browser's exclusion from the harness catalog (#821) and the | -| version-parity | A | 2026-08-31 00:49 | PASS | conversation 2026-08-29 — the oh CLI became the only lifecycle door, so its | -| weigh-scorer-contract | A | 2026-08-31 00:49 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-003 (2026-06-27) | -| wiki-readme-index | A | 2026-08-31 00:49 | PASS | issue #132 — wiki README index drift guard | -| workflow-boundaries | A | 2026-08-31 00:49 | PASS | conversation 2026-06-19 (workflow consolidation, issue #259); authority moved to /spec in issue #854 | -| worktrees-layout | A | 2026-08-31 00:49 | PASS | issue #872 | +| advisor-monitored-loop | A | 2026-08-31 01:42 | PASS | conversation 2026-06-19 (single-owner implementation workflow, issue #257) | +| agent-browser-cli | A | 2026-08-31 01:42 | PASS | retro lesson 2026-06-07 (agent-browser 0.8.5 CLI) | +| agents-identity-contract | A | 2026-08-31 01:42 | PASS | issue #854 — T3-style root identity, glossary, and skill-owned procedures | +| artifact-contract-audit | A | 2026-08-31 01:42 | PASS | issue #583/#645 — production /audit implementation Gate 1 behavior | +| audit-dispatcher-contract | A | 2026-08-31 01:42 | PASS | issue #645 — audit consolidation public taxonomy | +| audit-implementation-behavior | A | 2026-08-31 01:42 | PASS | issue #645 — implementation root/repo/browser behavior | +| audit-pr-acquire | A | 2026-08-31 01:42 | PASS | issue #645 — production PR acquisition behavior | +| audit-pr-classifier | A | 2026-08-31 01:42 | PASS | issue #645 — deterministic focused and queue PR classifier | +| audit-run-root-contract | A | 2026-08-31 01:42 | PASS | issue #645 — executable immutable audit root/run correlation | +| audit-shellcheck-coverage | A | 2026-08-31 01:42 | PASS | issue #645 — private audit scripts require release and CI lint coverage | +| audit-stale-references | A | 2026-08-31 01:42 | PASS | issue #645 — clean-breaking audit migration | +| boot-lint-glob | A | 2026-08-31 01:42 | PASS | issue #90, issue #120 | +| builder-skill-consolidation | A | 2026-08-31 01:42 | PASS | issue #643 — consolidate artifact builders behind one /builder dispatcher | +| capability-benchmark-schema | A | 2026-08-31 01:42 | PASS | issue #167 — capability benchmark instrument | +| cc-safety-net-wiring | A | 2026-08-31 01:42 | SKIPPED | .oh/tasks/cc-safety-net/prd.json US-007 2026-07-19 | +| changelog-entry-length | A | 2026-08-31 01:42 | PASS | conversation 2026-08-24 — CHANGELOG.md grew to 259KB of bullet prose because "one line" was unquantified | +| cleanup-tasks-scoped-guard | A | 2026-08-31 01:42 | PASS | issue #85 | +| cleanup-tasks-worktree-grooming | A | 2026-08-31 01:42 | PASS | issue #168; issue #327 | +| cli-publish-typecheck-scope | A | 2026-08-31 01:42 | PASS | release run 33271077312 — v0.5.0 pushed its GHCR image, then failed to publish | +| close-issues-on-development | A | 2026-08-31 01:42 | PASS | issue #841 (closing keywords never fire because the default branch is main) 2026-08-26 | +| codex-stale-response-retry | A | 2026-08-31 01:42 | PASS | issue #506 — Codex previous_response_not_found RCA | +| compose-config-path-parity | A | 2026-08-31 01:42 | PASS | PR #833 (remove harness.yaml — the wrapper and VS Code "Reopen in Container" paths must resolve the same service) 2026-08-26 | +| config-schema-parity | A | 2026-08-31 01:42 | PASS | PR #833 (one schema file — DOCKER_SOCKET, SANDBOX_SSH, OH_SANDBOX_IMAGE, OH_PULL_POLICY, SKIP_PNPM_INSTALL were consumed but undocumented); rewritten for the oh.json/secrets split by PR #887 | +| context-tier-size-budget | A | 2026-08-31 01:42 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-007) — the always-on tier was 85,256 B | +| cron-claude-codex-fallback | A | 2026-08-31 01:42 | PASS | conversation 2026-06-12 (default Codex fallback for crons) | +| cron-watchdog | A | 2026-08-31 01:42 | PASS | issues #130/#453 (cron runtime watchdog + legacy system-cron reaping) 2026-06-19 | +| crons-directory-guide | A | 2026-08-31 01:42 | PASS | issue #874 | +| curl-bash-safe-alternatives | A | 2026-08-31 01:42 | PASS | vet-run/vet integration — public curl|bash examples need review-first alternatives | +| datasets-schema | A | 2026-08-31 01:42 | PASS | issue #196 — .oh/evals/datasets verifiable trajectory corpus (Repo2RLEnv-inspired) | +| debugmcp-availability | A | 2026-08-31 01:42 | SKIPPED | issue #297 — DebugMCP MCP debug-server availability | +| delegate-model-effort-policy | A | 2026-08-31 01:42 | PASS | conversation 2026-07-11 (delegate model inheritance and thinking policy) | +| devtcp-hook | A | 2026-08-31 01:42 | PASS | retro lesson 2026-06-10 (zsh /dev/tcp) | +| docker-inspect-env-guard | A | 2026-08-31 01:42 | PASS | operator directive 2026-08-08 (agents keep the docker socket, but must | +| docs-build-fast-path | A | 2026-08-31 01:42 | PASS | #455 — docs builds must stay out of fast harness/eval/release gates; #536 — docs site externalized to openharness-web; docs markdown relocated to docs/ | +| drift-check-cron-staleness-glob | A | 2026-08-31 01:42 | PASS | issue #98; issue #225 (restart-required cron frontmatter/config drift) | +| entrypoint-pnpm-manifest-fingerprint | A | 2026-08-31 01:42 | PASS | issue #521 (manifest-aware sandbox installs) 2026-07-01 | +| eval-ci-gate | A | 2026-08-31 01:42 | PASS | #103 — eval probe suite gated in CI | +| eval-gate | A | 2026-08-31 01:42 | PASS | retro lesson 2026-06-11 (eval-gate) | +| eval-results-atomic | A | 2026-08-31 01:42 | PASS | issue #83 (eval-results-atomic-write) | +| eval-runner-exit | A | 2026-08-31 01:42 | PASS | retro lesson 2026-06-11 (eval-runner-exit) #29 | +| eval-runs-once-per-cycle | A | 2026-08-31 01:42 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-006) — /eval ran 3x per cycle on the | +| execution-target-contract | A | 2026-08-31 01:42 | PASS | issue #733 (ExecutionTarget contract + Docker Compose adapter) 2026-08-10 | +| get-oh-bootstrap | A | 2026-08-31 01:42 | PASS | get-oh.sh bootstrap — the Node-bootstrapping host-side path to the standalone `oh` CLI (also on npm as @mifune/openharness; see oh-npm-package.sh) | +| git-skill | A | 2026-08-31 01:42 | PASS | conversation 2026-06-15 — rules are not always supported; git workflow must be the /git skill | +| harness-audit-empty-output-gate | A | 2026-08-31 01:42 | PASS | issue #246 — /audit harness must fail closed on empty auditor outputs | +| harness-ci-core-paths | A | 2026-08-31 01:42 | PASS | #165 — core sandbox config files must trigger harness CI | +| harness-ci-hooks-paths | A | 2026-08-31 01:42 | PASS | issue #202 — credential/security hook changes must trigger harness CI | +| harness-home-provisioning | A | 2026-08-31 01:42 | PASS | #902 — `oh harness install` must work from inside the sandbox, where | +| harness-yaml-migration | A | 2026-08-31 01:42 | PASS | PR #833 (migrate-harness-yaml.sh — append / uncomment-in-place / preserve / overwrite, plus a silent no-op second run) 2026-08-26 | +| health-check-docker-stats | A | 2026-08-31 01:42 | PASS | retro lesson 2026-06-10 (docker stats vs ps Size) | +| health-check-socket-degrade | A | 2026-08-31 01:42 | PASS | issue #762 (refs #756) — /health-check degrades to one statement, not nine failures | +| heartbeat-logging-contract | A | 2026-08-31 01:42 | PASS | issue #447 (heartbeat log append hardening) 2026-06-18 | +| image-seed-hygiene | A | 2026-08-31 01:42 | PASS | issue #900 (slim the sandbox image) 2026-08-30 | +| markitdown-wiki-ingest | A | 2026-08-31 01:42 | PASS | issue #649 — pinned local-document normalization contract for /wiki ingest | +| next-dev-prod | A | 2026-08-31 01:42 | SKIPPED | retro lesson 2026-06-04 | +| oh-compose-env-wiring | A | 2026-08-31 01:42 | PASS | issue #880 (oh as the only front door — oh.json is the non-secret config surface) | +| oh-config-surfaces | A | 2026-08-31 01:42 | REGRESSION | PR #887 (config split across two authored surfaces — a tracked oh.json and a secrets-only root dotenv — with nothing left under $HOME) | +| oh-destroy-guard | A | 2026-08-31 01:42 | PASS | issue #879 — `oh` becomes the only front door, so `make destroy` must | +| oh-devcontainer-restructure | A | 2026-08-31 01:42 | PASS | consolidate devcontainer — .oh/devcontainer/ folded back into .devcontainer/ | +| oh-home-mount | A | 2026-08-31 01:42 | PASS | issue #898 (single $HOME mount) 2026-08-30 | +| oh-image-only-deploy | A | 2026-08-31 01:42 | PASS | .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only deploy) | +| oh-init-headless-config | A | 2026-08-31 01:42 | PASS | PR #827 (installer answers landed in the losing config file); retargeted to the .example.env template by PR #833, then to oh.json by PR #887 | +| oh-init-scaffold | A | 2026-08-31 01:42 | PASS | issue #531 Phase 2 | +| oh-lifecycle-surface | A | 2026-08-31 01:42 | PASS | issue #881 — the Makefile is retired and `oh` is the only front door | +| oh-npm-package | A | 2026-08-31 01:42 | PASS | npm publish path for the standalone `oh` CLI (@mifune/openharness) — alternative to get-oh.sh | +| oh-payload-manifest | A | 2026-08-31 01:42 | PASS | issue #531 follow-on (.oh payload manifest — oh update ships a declared allowlist) | +| oh-sandbox-image-mode | A | 2026-08-31 01:42 | PASS | conversation 2026-07-05 (basic Docker deployment — prebuilt-image mode) | +| oh-shipped-repo-overridable | A | 2026-08-31 01:42 | PASS | issue #531 follow-on (de-hardcode residual — shipped .oh shell scripts keep the upstream repo overridable) | +| oh-standalone-lifecycle | A | 2026-08-31 01:42 | PASS | issue #564 | +| oh-update | A | 2026-08-31 01:42 | PASS | issue #531 Phase 3 (oh update — upgrade only the .oh control plane) | +| operator-config-guard | A | 2026-08-31 01:42 | PASS | operator directives 2026-08-06 (.config/ and settings.local.json are operator-only) | +| pnpm-audit-ci-gate | A | 2026-08-31 01:42 | PASS | issue #171 — pnpm security audits must run in CI | +| post-bridge-publish-confirmation | A | 2026-08-31 01:42 | PASS | #523 — post-bridge live publishing requires an explicit final confirmation gate | +| prd-output-path-contract | A | 2026-08-31 01:42 | PASS | retro lesson 2026-06-19 | +| prompt-miner-schema-compat | A | 2026-08-31 01:42 | PASS | issue #253 — prompt-miner JSONL schema-drift guard | +| prompt-miner-symlink-entrypoint | A | 2026-08-31 01:42 | PASS | issue #663 — prompt-miner engine no-ops via the documented .claude/skills symlink | +| prompt-miner-weakness-record | A | 2026-08-31 01:42 | PASS | issue #580 — prompt-miner weakness-record (WH-xxx) cluster output | +| protected-path-deletion | A | 2026-08-31 01:42 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-001) — the critique gate was deleted, | +| protected-paths-resolve | A | 2026-08-31 01:42 | PASS | issue #753 — .claude/protected-paths.txt named 7 paths that did not exist. | +| registry-portability-gate | A | 2026-08-31 01:42 | PASS | issue #758 | +| registry-portability | A | 2026-08-31 01:42 | SKIPPED | issue #758 | +| retro-deterministic-contract | A | 2026-08-31 01:42 | PASS | issue #443 — /retro deterministic output and self-contained helper contract | +| rl-delegation-write-worker | A | 2026-08-31 01:42 | PASS | retro lesson 2026-06-10 (rl-delegation) #57 | +| rlm-context-budget | A | 2026-08-31 01:42 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-006 | +| runtime-preflight-gate | A | 2026-08-31 01:42 | PASS | issue #806 § B1 (open sandbox.substrate vs sandbox.runtime selector); | +| sandbox-boot-guard-ci | A | 2026-08-31 01:42 | PASS | issue #449 (sandbox image build CI guard) 2026-06-19; | +| sandbox-node-base | A | 2026-08-31 01:42 | PASS | openharness#878 — oh as the only front door, T0 sandbox base image | +| skill-paths | A | 2026-08-31 01:42 | PASS | issue #43 — stale path references; extended by issue #69 — apps/->packages/ rename guard; extended by issue #870 — deleted .oh/agents/advisor.md | +| skills-dir-clean | A | 2026-08-31 01:42 | PASS | conversation 2026-06-29 — Pi parses every top-level `.md` in the skills | +| skills-task-tool-coupling | A | 2026-08-31 01:42 | PASS | council review 2026-08-29 (issue #886) — /delegate instructed Claude-Code-only | +| skills-vendored | A | 2026-08-31 01:42 | PASS | absorb .mifune submodule into .oh — the skills/agents/hooks pack is vendored | +| slack-admin-command-surface | A | 2026-08-31 01:42 | PASS | issue #354 — Slack bridge docs must distinguish Pi /msg-bridge commands from Slack DM admin text handlers | +| spec-family-contract | A | 2026-08-31 01:42 | PASS | issue #265; spec-simplification issue #816; workflow authority issue #854 | +| spec-ready-finalization | A | 2026-08-31 01:42 | PASS | issue #134; spec-simplification issue #816; workflow authority issue #854 | +| ste-checker-contract | A | 2026-08-31 01:42 | PASS | issue #750 PR audit — the /ste checker had four fail-open paths (unclosed | +| submitted-by-trailers | A | 2026-08-31 01:42 | PASS | conversation 2026-06-12 (commit attribution trailers); the single-owner | +| sync-skill-contract | A | 2026-08-31 01:42 | PASS | issue #331 — /sync dispatcher skill (bidirectional origin↔upstream sync) | +| tool-catalog-boundary | A | 2026-08-31 01:42 | PASS | agent-browser's exclusion from the harness catalog (#821) and the | +| version-parity | A | 2026-08-31 01:42 | PASS | conversation 2026-08-29 — the oh CLI became the only lifecycle door, so its | +| weigh-scorer-contract | A | 2026-08-31 01:42 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-003 (2026-06-27) | +| wiki-readme-index | A | 2026-08-31 01:42 | PASS | issue #132 — wiki README index drift guard | +| workflow-boundaries | A | 2026-08-31 01:42 | PASS | conversation 2026-06-19 (workflow consolidation, issue #259); authority moved to /spec in issue #854 | +| worktrees-layout | A | 2026-08-31 01:42 | PASS | issue #872 | diff --git a/.oh/evals/probes/harness-home-provisioning.sh b/.oh/evals/probes/harness-home-provisioning.sh new file mode 100755 index 00000000..17f70163 --- /dev/null +++ b/.oh/evals/probes/harness-home-provisioning.sh @@ -0,0 +1,110 @@ +#!/usr/bin/env bash +# tier: A +# source: #902 — `oh harness install` must work from inside the sandbox, where +# sudo has no NOPASSWD, so default harnesses install into the home mount +# desc: every kind:"default" harness installs as the sandbox user into +# NPM_USER_PREFIX, claude-code keeps its postinstall, and the boot path +# carries the OH_PROVISION_HARNESSES guard and its provisioner. +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" +CATALOG="$ROOT/.oh/cli/src/lib/harnesses/catalog.ts" +ENTRY="$ROOT/.devcontainer/entrypoint.sh" +DOCKERFILE="$ROOT/.devcontainer/Dockerfile" +PROVISIONER="$ROOT/.oh/scripts/provision-harnesses.sh" + +for f in "$CATALOG" "$ENTRY" "$DOCKERFILE"; do + if [[ ! -f $f ]]; then + echo "SKIPPED: absent: $f" >&2 + exit 2 + fi +done + +PREFIX=$(sed -n 's/^ENV NPM_USER_PREFIX="\([^"]*\)".*/\1/p' "$DOCKERFILE" | head -1) +if [[ -z $PREFIX ]]; then + echo "SKIPPED: Dockerfile declares no ENV NPM_USER_PREFIX to anchor the install prefix" >&2 + exit 2 +fi + +missing=() + +entries=$(awk ' + /^ \{$/ { buf=""; inb=1; next } + /^ \},$/ { if (inb) print buf; inb=0; next } + inb { buf = buf $0 " " } +' "$CATALOG") + +defaults=0 +while IFS= read -r entry; do + [[ $entry == *'kind: "default"'* ]] || continue + defaults=$((defaults + 1)) + id=$(sed -n 's/.*id: "\([^"]*\)".*/\1/p' <<<"$entry") + if [[ $entry == *'installUser: "root"'* ]]; then + missing+=("harnesses/catalog.ts: default harness \"$id\" installs as root — inside the sandbox that becomes \`sudo -n\`, and /etc/sudoers.d/sandbox has no NOPASSWD") + fi + if [[ $entry != *"$PREFIX"* ]]; then + missing+=("harnesses/catalog.ts: default harness \"$id\" does not install into $PREFIX — a baked install under /usr/lib/node_modules cannot be upgraded by a running sandbox") + fi + if [[ $id == "claude-code" && $entry == *"--ignore-scripts"* ]]; then + missing+=("harnesses/catalog.ts: claude-code uses --ignore-scripts — its postinstall copies the native binary over the placeholder, so \`claude --version\` fails with 'claude native binary not installed'") + fi +done <<<"$entries" + +if ((defaults == 0)); then + echo "SKIPPED: no kind:\"default\" harness parsed out of $CATALOG" >&2 + exit 2 +fi + +grep -qF 'OH_PROVISION_HARNESSES' "$ENTRY" \ + || missing+=("entrypoint.sh: no OH_PROVISION_HARNESSES guard — nothing provisions harnesses into the home mount at boot") +grep -qF 'provision-harnesses.sh' "$ENTRY" \ + || missing+=("entrypoint.sh: does not call .oh/scripts/provision-harnesses.sh") +grep -qF 'WARNING: harness provisioning did not complete' "$ENTRY" \ + || missing+=("entrypoint.sh: harness provisioning does not warn-and-continue — an offline sandbox must still come up as a usable shell") +[[ -x $PROVISIONER ]] \ + || missing+=(".oh/scripts/provision-harnesses.sh: missing or not executable") +run_block_with() { + awk -v needle="$1" ' + function flush() { + if (index(buf, needle)) print buf + buf = "" + } + /^RUN / { buf = $0; cont = ($0 ~ /\\$/); if (!cont) flush(); next } + cont { buf = buf "\n" $0; cont = ($0 ~ /\\$/); if (!cont) flush() } + ' "$DOCKERFILE" +} + +BAKE_GATE='if [ "${BAKE_HARNESSES}" = "true" ]' + +stage_body() { + awk -v stage="$1" ' + /^FROM / { inb = ($0 ~ ("AS " stage "$")); next } + inb + ' "$DOCKERFILE" +} + +for stage in base home; do + stage_body "$stage" | grep -qE '^ARG BAKE_HARNESSES' \ + || missing+=("Dockerfile: the $stage stage does not declare ARG BAKE_HARNESSES — ARG is stage-scoped, so the build arg is empty there and every \${BAKE_HARNESSES} test in that stage reads as unset") +done + +AGENTS_BLOCK=$(run_block_with 'read -ra agents') +if [[ -z $AGENTS_BLOCK ]]; then + missing+=("Dockerfile: found no RUN that loops over \$AGENTS — the probe cannot tell whether BAKE_HARNESSES gates the bake") +elif [[ $AGENTS_BLOCK != *"$BAKE_GATE"* ]]; then + missing+=("Dockerfile: the RUN that installs \$AGENTS does not test \${BAKE_HARNESSES} — ARG BAKE_HARNESSES is declared but dead, so BAKE_HARNESSES=false still bakes the agent CLIs") +fi + +PI_BLOCK=$(run_block_with '--ignore-scripts @earendil-works/pi-coding-agent') +if [[ -z $PI_BLOCK ]]; then + missing+=("Dockerfile: found no RUN that bakes pi into \$NPM_USER_PREFIX — the probe cannot tell whether BAKE_HARNESSES gates it") +elif [[ $PI_BLOCK != *"$BAKE_GATE"* ]]; then + missing+=("Dockerfile: the RUN that bakes pi does not test \${BAKE_HARNESSES} — ARG is stage-scoped, so BAKE_HARNESSES=false unbakes claude and codex but leaves pi in the image") +fi + +if ((${#missing[@]})); then + printf 'REGRESSION: %s\n' "${missing[@]}" >&2 + exit 1 +fi + +echo "PASS: all $defaults default harnesses install as the sandbox user into $PREFIX, and the boot path provisions them" >&2 diff --git a/.oh/scripts/provision-harnesses.sh b/.oh/scripts/provision-harnesses.sh new file mode 100755 index 00000000..b1f9f2cf --- /dev/null +++ b/.oh/scripts/provision-harnesses.sh @@ -0,0 +1,158 @@ +#!/usr/bin/env bash + +set -euo pipefail + +SANDBOX_USER="sandbox" +OH_BIN="${OH_BIN:-oh}" + +MODE="provision" +case "${1:-}" in + --verify) MODE="verify" ;; + "") ;; + *) echo "usage: $(basename "$0") [--verify]" >&2; exit 2 ;; +esac + +log() { echo "[provision-harnesses] $*"; } + +die() { + echo "[provision-harnesses] ERROR: $1" >&2 + shift + for line in "$@"; do echo "[provision-harnesses] $line" >&2; done + exit 1 +} + +inside_sandbox() { + case "${OH_EXECUTION_TARGET:-}" in + local) return 0 ;; + docker-compose) return 1 ;; + esac + [ -f /.dockerenv ] && [ -n "${SANDBOX_NAME:-}" ] +} + +inside_sandbox || die \ + "this provisions /home/$SANDBOX_USER/.local inside the sandbox and must not run on the host" \ + "open a sandbox shell first:" \ + " oh shell" \ + " bash .oh/scripts/provision-harnesses.sh" + +export OH_EXECUTION_TARGET=local + +if [ "$(id -u)" = "0" ]; then + if ! id "$SANDBOX_USER" >/dev/null 2>&1; then + die "user '$SANDBOX_USER' does not exist" \ + "this script provisions the sandbox image's agent user; rebuild the image:" \ + " oh sandbox" + fi + USER_HOME=$(getent passwd "$SANDBOX_USER" | cut -d: -f6) + [ -n "$USER_HOME" ] || die "cannot resolve home directory for '$SANDBOX_USER'" + + install -d -o "$SANDBOX_USER" -g "$SANDBOX_USER" \ + "$USER_HOME/.local" \ + "$USER_HOME/.local/bin" \ + "$USER_HOME/.local/lib" \ + "$USER_HOME/.npm" 2>/dev/null || true + + if command -v gosu >/dev/null 2>&1; then + exec gosu "$SANDBOX_USER" env HOME="$USER_HOME" "$0" "$@" + fi + exec su "$SANDBOX_USER" -s /bin/bash -c "HOME='$USER_HOME' OH_EXECUTION_TARGET=local '$0' $*" +fi + +HOME="${HOME:-$(getent passwd "$(id -u)" | cut -d: -f6)}" +export HOME + +NPM_USER_PREFIX="${NPM_USER_PREFIX:-$HOME/.local}" +export NPM_USER_PREFIX +export PATH="$NPM_USER_PREFIX/bin:$PATH" + +check_writable() { + local dir="$1" + if [ ! -d "$dir" ]; then + mkdir -p "$dir" 2>/dev/null && return 0 + local parent; parent=$(dirname "$dir") + die "cannot create $dir (parent $parent is owned by $(stat -c '%U:%G' "$parent" 2>/dev/null || echo unknown))" \ + "this is an ownership bug in provisioning, not something to fix with 'sudo npm' —" \ + "a root-owned harness under $NPM_USER_PREFIX is unusable by the '$SANDBOX_USER' user." \ + "repair from the host or as root:" \ + " docker exec -u root chown -R $SANDBOX_USER:$SANDBOX_USER $parent" + fi + if [ ! -w "$dir" ]; then + die "$dir is not writable by $(id -un) (owned by $(stat -c '%U:%G' "$dir" 2>/dev/null || echo unknown))" \ + "do not work around this with 'sudo npm' — it installs under /usr/lib/node_modules," \ + "which no running sandbox can upgrade in place." \ + "repair from the host or as root:" \ + " docker exec -u root chown -R $SANDBOX_USER:$SANDBOX_USER $dir" + fi +} + +for d in "$NPM_USER_PREFIX" "$NPM_USER_PREFIX/bin" "$NPM_USER_PREFIX/lib" "$HOME/.npm"; do + [ "$MODE" = "verify" ] && [ ! -d "$d" ] && continue + check_writable "$d" +done + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" +cd "$ROOT" + +command -v "$OH_BIN" >/dev/null 2>&1 || die \ + "the oh CLI is not on PATH as '$OH_BIN'" \ + "the image installs it to /usr/local/bin/oh; rebuild the sandbox image:" \ + " oh sandbox" + +command -v jq >/dev/null 2>&1 || die \ + "jq is not on PATH" \ + "the image installs it with apt; rebuild the sandbox image:" \ + " oh sandbox" + +STATES="" +if ! STATES="$("$OH_BIN" harness list --defaults --json 2>/dev/null)" || [ -z "$STATES" ]; then + die "'$OH_BIN harness list --defaults --json' produced no catalog" \ + "the CLI at $(command -v "$OH_BIN") predates \`oh harness\`; the harness catalog" \ + "is the only source of truth for what to install, so there is nothing to provision." \ + "rebuild the sandbox image from this control plane:" \ + " oh sandbox" +fi + +DEFAULTS="$(jq -r '.[] | select(.kind == "default") | "\(.id)\t\(.installed)"' <<<"$STATES")" +[ -n "$DEFAULTS" ] || die \ + "the harness catalog declares no default harnesses" \ + "check .oh/cli/src/lib/harnesses/catalog.ts" + +missing=() +failed=() + +while IFS=$'\t' read -r id installed; do + [ -n "$id" ] || continue + if [ "$installed" = "true" ]; then + log "OK $id present (unpinned — an existing install is never replaced)" + continue + fi + if [ "$MODE" = "verify" ]; then + missing+=("$id") + continue + fi + log "installing $id into $NPM_USER_PREFIX" + if "$OH_BIN" harness install "$id" --no-persist Date: Mon, 31 Aug 2026 14:34:20 -0600 Subject: [PATCH 06/23] FROM task/904-unbake-harnesses TO development (#905) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * task: provision the default harnesses into the home mount oh is meant to split by execution target -- on the host it provisions the host or the sandbox, and inside the sandbox it provisions the sandbox with harnesses and tools. The second half did not work for the two harnesses most people use. claude-code and codex carried installUser: "root", which against the local execution target becomes sudo -n -- npm install -g, and /etc/sudoers.d/sandbox grants sandbox ALL=(ALL) ALL with no NOPASSWD. sudo -n true returns "a password is required". Both now match the pi entry directly above them: installUser "sandbox", npm --prefix /home/sandbox/.local install -g. That lands them inside the home mount, so they also survive container recreate and can be upgraded in place in a running remote sandbox rather than requiring an image rebuild. claude-code deliberately does not get --ignore-scripts. Its postinstall copies the native binary over a placeholder; with the flag the install succeeds and claude --version then fails with "claude native binary not installed". Verified both ways against a scratch prefix. provision-harnesses.sh follows provision-python.sh: the same mode flag, the same root to gosu sandbox re-exec, the same ownership diagnostics, the same die-with- the-command-to-re-run style. --print-env is absent because this provisioner exports nothing downstream. It reads the catalog through oh harness list --json and installs through oh harness install, so the shell knows no ids, packages, prefixes, or argv, and the TypeScript catalog stays the only description. No default harness carries a version pin today, so an existing install is never replaced and the script says so in its own output rather than implying it refreshes. The entrypoint hook runs after link-providers.sh, not before. link-providers' only binary dependency is cc-safety-net, which stays baked, and it is the boot-critical hard gate; a network-dependent best-effort step does not belong in front of the step that decides whether the boot is viable. Provisioning warns and continues, so an offline sandbox still comes up as a usable shell. BAKE_HARNESSES defaults to true and gates only the $AGENTS loop, not the whole RUN. INSTALL_OPENCODE and INSTALL_GROK_BUILD are separate opt-ins and turning them off as a side effect would be a silent regression. Nothing leaves the image in this change. * fix: bound the boot path and close four more provisioning defects An adversarial audit of #903 found five defects that six green checks missed. The serious one is a boot hang. oh harness list --json probes every entry in the catalog, not just the three defaults, and one of them is t3code, whose verifyArgv is npx --no-install t3 --version. npx contacts the registry, and probeInstalled passed no timeoutMs, so spawnSync waited without bound. Against an unreachable registry the auditor's run was still going at 2m30 when their own timeout killed it. On any boot where DNS resolves but the registry does not answer, the entrypoint blocks before sleep infinity, exceeds the 300s start_period, and never goes healthy -- and restart: unless-stopped does not rescue an unhealthy-but-alive container. Warn-and-continue cannot help, because a hang never reaches the if !. It hangs while listing, before any install, so BAKE_HARNESSES=true did not avoid it either. Bounded at three layers, because each fails differently: a 15s timeoutMs on the probe spawn, reported as unknown rather than a crash; a --defaults filter on oh harness list so the boot path probes three entries instead of nine and never runs npx; and a timeout wrapper on the entrypoint call so the boot is bounded whatever the CLI does. Measured against the auditor's exact command: 2m30 and killed, to 17.0s full-catalog and 1.55s with --defaults. The install loop read from a herestring while installs run with stdio inherit, so an installer that reads stdin consumed the rest of the loop. Reproduced with a stub: three missing harnesses, one installed, exit 0, success printed. Latent with npm, live the moment a default uses the curl | bash shape two catalog entries already use. Installs now read from /dev/null. The script force-exported OH_EXECUTION_TARGET=local, which short-circuits the in-container check, while the prefix is hardcoded to /home/sandbox/.local, and every error told the operator to re-run with no mention of where. On the host that provisioned the host. It now refuses unless inside the sandbox, reusing the CLI's own runningInsideSandbox predicate rather than inventing a check, and the entrypoint asserts the local target explicitly -- the documented raw docker run recipe never passes SANDBOX_NAME, so the guard would otherwise have silently skipped provisioning for the prebuilt-image flavor. The probe asserted that ARG BAKE_HARNESSES was declared, not that anything used it: deleting the gate left it green. It now checks the ARG is referenced by the RUN that installs $AGENTS and by the one that bakes pi, and that both stages declare it. Deleting either declaration also used to pass. ARG is stage-scoped, so BAKE_HARNESSES=false unbaked claude-code and codex but left pi baked in the home stage while the else-branch claimed otherwise. The home stage now declares and honors the flag. Also: OH_SANDBOX_USER was advertised but illusory, since the catalog hardcodes the user and prefix; it is gone. The final log line no longer claims to have provisioned anything in --verify mode. * task: stop baking the default harnesses into the sandbox image PR #903 wired provision-harnesses.sh into the boot path but shipped it behind ARG BAKE_HARNESSES=true, so every default harness was already present when the provisioner ran and the install path never executed. All four defects that PR's audit found lived in code a green CI run and a normal boot both skip. Delete the bake rather than flip its default: remove ARG BAKE_HARNESSES, ARG AGENTS, the PKG map and the $AGENTS loop in `base`, and the gated pi install in `home`. A build arg that can re-bake is a dormant path that would restore both the shadowed /usr/lib/node_modules copy and the untested boot install. Make the install path CI-visible, since it is now load-bearing on every boot: - The boot smoke asserts the outcome — each default harness resolves under NPM_USER_PREFIX via `type -P`, is owned by the reconciled sandbox uid, and prints its own version — and refuses to pass when the catalog reports no defaults. It boots on a fresh home volume, so this runs real npm work. - verify-sandbox-image.sh gains the negative: reading the catalog out of the image itself, no kind:"default" harness may be installed. - start_period goes 300s -> 600s in both compose files to cover the install, and the boot-guard probe now derives the smoke deadline from the healthcheck window instead of pinning a literal that a start_period bump could invert. - The probe and unit assertions invert from "the bake is gated" to "no default harness package appears in the Dockerfile", reading the package names out of installArgv so they cannot drift from the catalog. cc-safety-net stays baked. Opt-in INSTALL_* harnesses are untouched. Costs this accepts, documented in installation.md: a first boot on a fresh home mount needs network and runs 60-180s longer; an offline first boot yields a usable shell with no agent CLIs; ~/.npm now lives in the home mount. Closes #904 * task: cite the measured first-boot provisioning time --- .devcontainer/Dockerfile | 29 +-- .devcontainer/docker-compose.image-only.yml | 7 +- .devcontainer/docker-compose.yml | 7 +- .github/workflows/sandbox-boot-guard.yml | 20 +- .oh/cli/src/__tests__/harness-catalog.test.ts | 50 ++--- .oh/cli/src/commands/harness.ts | 2 + .oh/evals/RESULTS.md | 206 +++++++++--------- .oh/evals/probes/harness-home-provisioning.sh | 72 +++--- .oh/evals/probes/sandbox-boot-guard-ci.sh | 29 ++- .../__tests__/sandbox-boot-smoke.test.ts | 55 +++++ .../__tests__/sandbox-healthcheck.test.ts | 8 +- .../__tests__/verify-sandbox-image.test.ts | 53 +++++ .oh/scripts/sandbox-boot-smoke.sh | 78 ++++++- .oh/scripts/verify-sandbox-image.sh | 29 ++- CHANGELOG.md | 3 + docs/deployment-prebuilt-image.md | 9 +- docs/harnesses/opencode.md | 2 +- docs/harnesses/overview.md | 2 +- docs/installation.md | 25 ++- docs/quickstart.md | 11 +- 20 files changed, 479 insertions(+), 218 deletions(-) diff --git a/.devcontainer/Dockerfile b/.devcontainer/Dockerfile index beff0791..ffc38e4b 100644 --- a/.devcontainer/Dockerfile +++ b/.devcontainer/Dockerfile @@ -83,27 +83,10 @@ RUN corepack enable && corepack prepare pnpm@10.33.0 --activate \ SHELL ["/bin/bash", "-c"] -ARG AGENTS="claude-code,codex,pi-coding-agent" ARG INSTALL_OPENCODE=false ARG INSTALL_GROK_BUILD=false -ARG BAKE_HARNESSES=true RUN set -e; \ - declare -A PKG=( \ - [claude-code]=@anthropic-ai/claude-code \ - [codex]=@openai/codex \ - [pi-coding-agent]=@earendil-works/pi-coding-agent \ - [opencode]=opencode-ai \ - ); \ - if [ "${BAKE_HARNESSES}" = "true" ]; then \ - IFS=',' read -ra agents <<< "$AGENTS"; \ - for a in "${agents[@]}"; do \ - if [ "$a" = "pi-coding-agent" ]; then continue; fi; \ - pkg="${PKG[$a]:-}"; \ - if [ -n "$pkg" ]; then npm install -g "$pkg"; \ - else echo "Unknown agent: $a"; exit 1; fi; \ - done; \ - else echo "Skipping baked Claude Code and Codex installs (BAKE_HARNESSES=${BAKE_HARNESSES}); .oh/scripts/provision-harnesses.sh installs every kind:\"default\" harness into /home/sandbox/.local at boot"; fi; \ if [ "${INSTALL_OPENCODE}" = "true" ]; then npm install -g opencode-ai; \ else echo "Skipping OpenCode CLI install (INSTALL_OPENCODE=false)"; fi; \ if [ "${INSTALL_GROK_BUILD}" = "true" ]; then \ @@ -131,8 +114,6 @@ ENV UV_PYTHON_INSTALL_DIR=/home/sandbox/.local/share/uv/python ENV UV_CACHE_DIR=/home/sandbox/.cache/uv FROM base AS home -ARG AGENTS="claude-code,codex,pi-coding-agent" -ARG BAKE_HARNESSES=true RUN install -d -o sandbox -g sandbox -m 0700 /home/sandbox/.ssh \ && install -d -o sandbox -g sandbox \ /home/sandbox/.local \ @@ -143,14 +124,8 @@ RUN install -d -o sandbox -g sandbox -m 0700 /home/sandbox/.ssh \ /home/sandbox/.herdr \ "$UV_TOOL_DIR" "$UV_TOOL_BIN_DIR" "$UV_PYTHON_INSTALL_DIR" "$UV_CACHE_DIR" -RUN set -e; \ - install -d -o sandbox -g sandbox "$NPM_USER_PREFIX"; \ - if [ "${BAKE_HARNESSES}" = "true" ]; then \ - if [[ ",${AGENTS}," == *",pi-coding-agent,"* ]]; then \ - su - sandbox -c 'npm --prefix "$HOME/.local" install -g --ignore-scripts @earendil-works/pi-coding-agent'; \ - fi; \ - else echo "Skipping baked Pi install (BAKE_HARNESSES=${BAKE_HARNESSES}); .oh/scripts/provision-harnesses.sh installs it into /home/sandbox/.local at boot"; fi; \ - rm -rf /home/sandbox/.npm +RUN install -d -o sandbox -g sandbox "$NPM_USER_PREFIX" \ + && rm -rf /home/sandbox/.npm RUN su - sandbox -c "RUNZSH=no CHSH=no KEEP_ZSHRC=yes sh -c \"\$(curl -fsSL https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh)\" --unattended" \ && su - sandbox -c "git clone --depth 1 https://github.com/zsh-users/zsh-autosuggestions /home/sandbox/.oh-my-zsh/custom/plugins/zsh-autosuggestions" \ diff --git a/.devcontainer/docker-compose.image-only.yml b/.devcontainer/docker-compose.image-only.yml index 95062956..1ef9f6b1 100644 --- a/.devcontainer/docker-compose.image-only.yml +++ b/.devcontainer/docker-compose.image-only.yml @@ -57,7 +57,12 @@ services: interval: 30s timeout: 10s retries: 3 - start_period: 300s + # Boot now installs the default harnesses into the home mount instead of + # unpacking them from the image (entrypoint.sh, bounded by + # OH_PROVISION_HARNESSES_TIMEOUT). A cold first boot on a fresh home mount + # therefore spends up to 180s on npm before the control plane is reachable. + # 300s left ~120s for the rest of boot; 600s keeps the same headroom. + start_period: 600s restart: unless-stopped volumes: diff --git a/.devcontainer/docker-compose.yml b/.devcontainer/docker-compose.yml index aaa05520..3cd0fba0 100644 --- a/.devcontainer/docker-compose.yml +++ b/.devcontainer/docker-compose.yml @@ -83,7 +83,12 @@ services: interval: 30s timeout: 10s retries: 3 - start_period: 300s + # Boot now installs the default harnesses into the home mount instead of + # unpacking them from the image (entrypoint.sh, bounded by + # OH_PROVISION_HARNESSES_TIMEOUT). A cold first boot on a fresh home mount + # therefore spends up to 180s on npm before the control plane is reachable. + # 300s left ~120s for the rest of boot; 600s keeps the same headroom. + start_period: 600s restart: unless-stopped volumes: diff --git a/.github/workflows/sandbox-boot-guard.yml b/.github/workflows/sandbox-boot-guard.yml index 448a863a..f4b4c9f8 100644 --- a/.github/workflows/sandbox-boot-guard.yml +++ b/.github/workflows/sandbox-boot-guard.yml @@ -92,7 +92,7 @@ jobs: # The devcontainer bind-mounts the checkout (..:/home/sandbox/harness) and # can run a synchronous `pnpm install` during boot when the root dependency # tree is missing or its manifest marker is stale — a cold install can - # overrun the healthcheck's ~390s unhealthy deadline and fail the boot with + # overrun the healthcheck's ~690s unhealthy deadline and fail the boot with # "missing required tmux session: cron-system". Seeding node_modules here # moves that slow/variable install OUT of the healthcheck-timed window; the # sandbox smoke step below also sets SKIP_PNPM_INSTALL=1 so the pre-seeded @@ -132,14 +132,24 @@ jobs: sh -c 'ls -la /mnt/node_modules/croner/package.json' echo "OK: pre-seeded node_modules is visible to the daemon that boots the sandbox" - - name: Boot sandbox and verify healthcheck + # This step is the ONLY place the boot-time harness install runs end to end. + # It boots on a fresh home volume (the smoke tears down with `down -v`), so + # provision-harnesses.sh does real npm work every run and the smoke asserts + # the result. Do not set OH_PROVISION_HARNESSES=false here to speed it up — + # that would return the install path to being untested dead code. + - name: Boot sandbox and verify healthcheck (exercises boot-time harness provisioning) env: SANDBOX_NAME: openharness-boot-guard-${{ github.run_id }} - BOOT_SMOKE_TIMEOUT_SECONDS: "900" + # Boot now installs the default harnesses into a fresh home volume + # (#904) before the control plane comes up, so the smoke deadline has + # to clear the compose healthcheck's 600s start_period plus the + # 3x30s retry window (~690s), not just the old 300s one. + BOOT_SMOKE_TIMEOUT_SECONDS: "1200" BOOT_SMOKE_INTERVAL_SECONDS: "10" # Deps are pre-seeded above and proven visible; SKIP_PNPM_INSTALL=1 # keeps the in-container install out of the healthcheck-timed window. - # This is the Option C contract: install slowness can no longer trip - # the ~390s unhealthy deadline. Boot is now just the cron-system spawn. + # This is the Option C contract: pnpm slowness can no longer trip the + # ~690s unhealthy deadline. What remains inside that window is the + # cron-system spawn and the boot-time harness install. SKIP_PNPM_INSTALL: "1" run: bash .oh/scripts/sandbox-boot-smoke.sh diff --git a/.oh/cli/src/__tests__/harness-catalog.test.ts b/.oh/cli/src/__tests__/harness-catalog.test.ts index cc9b8bd7..b6424492 100644 --- a/.oh/cli/src/__tests__/harness-catalog.test.ts +++ b/.oh/cli/src/__tests__/harness-catalog.test.ts @@ -19,29 +19,10 @@ const CONFIG_DOC = read("docs/configuration.md"); const ENTRYPOINT = read(".devcontainer/entrypoint.sh"); const NPM_USER_PREFIX = "/home/sandbox/.local"; -const BAKE_GATE = 'if [ "${BAKE_HARNESSES}" = "true" ]'; - -function dockerfileStage(stage: string): string { - const lines = DOCKERFILE.split("\n"); - const start = lines.findIndex((l) => new RegExp(`^FROM .* AS ${stage}$`).test(l)); - if (start === -1) return ""; - const rest = lines.slice(start + 1); - const end = rest.findIndex((l) => l.startsWith("FROM ")); - return (end === -1 ? rest : rest.slice(0, end)).join("\n"); -} - -function dockerfileRunWith(needle: string): string { - const blocks: string[] = []; - let buf: string | null = null; - for (const line of DOCKERFILE.split("\n")) { - if (buf === null && !line.startsWith("RUN ")) continue; - buf = buf === null ? line : `${buf}\n${line}`; - if (line.endsWith("\\")) continue; - blocks.push(buf); - buf = null; - } - return blocks.find((b) => b.includes(needle)) ?? ""; -} +// Comments may legitimately name a harness package; only instructions may not. +const DOCKERFILE_CODE = DOCKERFILE.split("\n") + .filter((l) => !/^\s*#/.test(l)) + .join("\n"); function versionPins(argv: readonly string[]): string[] { const pins = new Set(); @@ -176,24 +157,25 @@ describe("harness catalog", () => { expect(findHarness("claude-code")!.installArgv).not.toContain("--ignore-scripts"); }); - it("lets the image bake be turned off, and provisions the same harnesses at boot", () => { - expect(DOCKERFILE).toMatch(/^ARG BAKE_HARNESSES=true$/m); + it("provisions the default harnesses at boot rather than baking them", () => { expect(ENTRYPOINT).toContain("OH_PROVISION_HARNESSES"); expect(ENTRYPOINT).toContain(".oh/scripts/provision-harnesses.sh"); }); - it.each(["base", "home"])( - "%s declares BAKE_HARNESSES, which ARG scopes to that stage alone", - (stage) => { - expect(dockerfileStage(stage)).toMatch(/^ARG BAKE_HARNESSES/m); + it.each(defaults.map((h) => [h.id, h] as const))( + "%s: its npm package is absent from the Dockerfile", + (id, h) => { + const pkg = h.installArgv[h.installArgv.length - 1]; + expect(pkg, `${id} declares no install package`).toMatch(/^(@[^/]+\/)?[^-].*/); + expect( + DOCKERFILE_CODE, + `${id} is baked into the image; it belongs to provision-harnesses.sh`, + ).not.toContain(pkg); }, ); - it("gates every baked default install on BAKE_HARNESSES, pi included", () => { - expect(dockerfileRunWith("read -ra agents")).toContain(BAKE_GATE); - expect(dockerfileRunWith("--ignore-scripts @earendil-works/pi-coding-agent")).toContain( - BAKE_GATE, - ); + it("keeps no build arg that could re-bake the default harnesses", () => { + expect(DOCKERFILE_CODE).not.toMatch(/^ARG (BAKE_HARNESSES|AGENTS)=/m); }); it("bounds the boot-path provisioner so an unreachable registry cannot stall the entrypoint", () => { diff --git a/.oh/cli/src/commands/harness.ts b/.oh/cli/src/commands/harness.ts index 5050e405..cdb61c83 100644 --- a/.oh/cli/src/commands/harness.ts +++ b/.oh/cli/src/commands/harness.ts @@ -42,6 +42,7 @@ export interface HarnessInstallOptions extends HarnessOptions { interface HarnessState { id: string; title: string; + binary: string; kind: string; enabled: boolean | null; installed: boolean | null; @@ -107,6 +108,7 @@ async function collectStates( states.push({ id: entry.id, title: entry.title, + binary: entry.binary, kind: entry.kind, enabled: entry.harnessKey === undefined ? null : isInstallFlagEnabled(root, entry.harnessKey), diff --git a/.oh/evals/RESULTS.md b/.oh/evals/RESULTS.md index a6c7ac50..0d1ab988 100644 --- a/.oh/evals/RESULTS.md +++ b/.oh/evals/RESULTS.md @@ -6,108 +6,108 @@ probe id; git history is the time series.** Schema and exit-code semantics are i | probe | tier | last-run (UTC) | status | source | |-------|------|----------------|--------|--------| -| advisor-monitored-loop | A | 2026-08-31 01:42 | PASS | conversation 2026-06-19 (single-owner implementation workflow, issue #257) | -| agent-browser-cli | A | 2026-08-31 01:42 | PASS | retro lesson 2026-06-07 (agent-browser 0.8.5 CLI) | -| agents-identity-contract | A | 2026-08-31 01:42 | PASS | issue #854 — T3-style root identity, glossary, and skill-owned procedures | -| artifact-contract-audit | A | 2026-08-31 01:42 | PASS | issue #583/#645 — production /audit implementation Gate 1 behavior | -| audit-dispatcher-contract | A | 2026-08-31 01:42 | PASS | issue #645 — audit consolidation public taxonomy | -| audit-implementation-behavior | A | 2026-08-31 01:42 | PASS | issue #645 — implementation root/repo/browser behavior | -| audit-pr-acquire | A | 2026-08-31 01:42 | PASS | issue #645 — production PR acquisition behavior | -| audit-pr-classifier | A | 2026-08-31 01:42 | PASS | issue #645 — deterministic focused and queue PR classifier | -| audit-run-root-contract | A | 2026-08-31 01:42 | PASS | issue #645 — executable immutable audit root/run correlation | -| audit-shellcheck-coverage | A | 2026-08-31 01:42 | PASS | issue #645 — private audit scripts require release and CI lint coverage | -| audit-stale-references | A | 2026-08-31 01:42 | PASS | issue #645 — clean-breaking audit migration | -| boot-lint-glob | A | 2026-08-31 01:42 | PASS | issue #90, issue #120 | -| builder-skill-consolidation | A | 2026-08-31 01:42 | PASS | issue #643 — consolidate artifact builders behind one /builder dispatcher | -| capability-benchmark-schema | A | 2026-08-31 01:42 | PASS | issue #167 — capability benchmark instrument | -| cc-safety-net-wiring | A | 2026-08-31 01:42 | SKIPPED | .oh/tasks/cc-safety-net/prd.json US-007 2026-07-19 | -| changelog-entry-length | A | 2026-08-31 01:42 | PASS | conversation 2026-08-24 — CHANGELOG.md grew to 259KB of bullet prose because "one line" was unquantified | -| cleanup-tasks-scoped-guard | A | 2026-08-31 01:42 | PASS | issue #85 | -| cleanup-tasks-worktree-grooming | A | 2026-08-31 01:42 | PASS | issue #168; issue #327 | -| cli-publish-typecheck-scope | A | 2026-08-31 01:42 | PASS | release run 33271077312 — v0.5.0 pushed its GHCR image, then failed to publish | -| close-issues-on-development | A | 2026-08-31 01:42 | PASS | issue #841 (closing keywords never fire because the default branch is main) 2026-08-26 | -| codex-stale-response-retry | A | 2026-08-31 01:42 | PASS | issue #506 — Codex previous_response_not_found RCA | -| compose-config-path-parity | A | 2026-08-31 01:42 | PASS | PR #833 (remove harness.yaml — the wrapper and VS Code "Reopen in Container" paths must resolve the same service) 2026-08-26 | -| config-schema-parity | A | 2026-08-31 01:42 | PASS | PR #833 (one schema file — DOCKER_SOCKET, SANDBOX_SSH, OH_SANDBOX_IMAGE, OH_PULL_POLICY, SKIP_PNPM_INSTALL were consumed but undocumented); rewritten for the oh.json/secrets split by PR #887 | -| context-tier-size-budget | A | 2026-08-31 01:42 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-007) — the always-on tier was 85,256 B | -| cron-claude-codex-fallback | A | 2026-08-31 01:42 | PASS | conversation 2026-06-12 (default Codex fallback for crons) | -| cron-watchdog | A | 2026-08-31 01:42 | PASS | issues #130/#453 (cron runtime watchdog + legacy system-cron reaping) 2026-06-19 | -| crons-directory-guide | A | 2026-08-31 01:42 | PASS | issue #874 | -| curl-bash-safe-alternatives | A | 2026-08-31 01:42 | PASS | vet-run/vet integration — public curl|bash examples need review-first alternatives | -| datasets-schema | A | 2026-08-31 01:42 | PASS | issue #196 — .oh/evals/datasets verifiable trajectory corpus (Repo2RLEnv-inspired) | -| debugmcp-availability | A | 2026-08-31 01:42 | SKIPPED | issue #297 — DebugMCP MCP debug-server availability | -| delegate-model-effort-policy | A | 2026-08-31 01:42 | PASS | conversation 2026-07-11 (delegate model inheritance and thinking policy) | -| devtcp-hook | A | 2026-08-31 01:42 | PASS | retro lesson 2026-06-10 (zsh /dev/tcp) | -| docker-inspect-env-guard | A | 2026-08-31 01:42 | PASS | operator directive 2026-08-08 (agents keep the docker socket, but must | -| docs-build-fast-path | A | 2026-08-31 01:42 | PASS | #455 — docs builds must stay out of fast harness/eval/release gates; #536 — docs site externalized to openharness-web; docs markdown relocated to docs/ | -| drift-check-cron-staleness-glob | A | 2026-08-31 01:42 | PASS | issue #98; issue #225 (restart-required cron frontmatter/config drift) | -| entrypoint-pnpm-manifest-fingerprint | A | 2026-08-31 01:42 | PASS | issue #521 (manifest-aware sandbox installs) 2026-07-01 | -| eval-ci-gate | A | 2026-08-31 01:42 | PASS | #103 — eval probe suite gated in CI | -| eval-gate | A | 2026-08-31 01:42 | PASS | retro lesson 2026-06-11 (eval-gate) | -| eval-results-atomic | A | 2026-08-31 01:42 | PASS | issue #83 (eval-results-atomic-write) | -| eval-runner-exit | A | 2026-08-31 01:42 | PASS | retro lesson 2026-06-11 (eval-runner-exit) #29 | -| eval-runs-once-per-cycle | A | 2026-08-31 01:42 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-006) — /eval ran 3x per cycle on the | -| execution-target-contract | A | 2026-08-31 01:42 | PASS | issue #733 (ExecutionTarget contract + Docker Compose adapter) 2026-08-10 | -| get-oh-bootstrap | A | 2026-08-31 01:42 | PASS | get-oh.sh bootstrap — the Node-bootstrapping host-side path to the standalone `oh` CLI (also on npm as @mifune/openharness; see oh-npm-package.sh) | -| git-skill | A | 2026-08-31 01:42 | PASS | conversation 2026-06-15 — rules are not always supported; git workflow must be the /git skill | -| harness-audit-empty-output-gate | A | 2026-08-31 01:42 | PASS | issue #246 — /audit harness must fail closed on empty auditor outputs | -| harness-ci-core-paths | A | 2026-08-31 01:42 | PASS | #165 — core sandbox config files must trigger harness CI | -| harness-ci-hooks-paths | A | 2026-08-31 01:42 | PASS | issue #202 — credential/security hook changes must trigger harness CI | -| harness-home-provisioning | A | 2026-08-31 01:42 | PASS | #902 — `oh harness install` must work from inside the sandbox, where | -| harness-yaml-migration | A | 2026-08-31 01:42 | PASS | PR #833 (migrate-harness-yaml.sh — append / uncomment-in-place / preserve / overwrite, plus a silent no-op second run) 2026-08-26 | -| health-check-docker-stats | A | 2026-08-31 01:42 | PASS | retro lesson 2026-06-10 (docker stats vs ps Size) | -| health-check-socket-degrade | A | 2026-08-31 01:42 | PASS | issue #762 (refs #756) — /health-check degrades to one statement, not nine failures | -| heartbeat-logging-contract | A | 2026-08-31 01:42 | PASS | issue #447 (heartbeat log append hardening) 2026-06-18 | -| image-seed-hygiene | A | 2026-08-31 01:42 | PASS | issue #900 (slim the sandbox image) 2026-08-30 | -| markitdown-wiki-ingest | A | 2026-08-31 01:42 | PASS | issue #649 — pinned local-document normalization contract for /wiki ingest | -| next-dev-prod | A | 2026-08-31 01:42 | SKIPPED | retro lesson 2026-06-04 | -| oh-compose-env-wiring | A | 2026-08-31 01:42 | PASS | issue #880 (oh as the only front door — oh.json is the non-secret config surface) | -| oh-config-surfaces | A | 2026-08-31 01:42 | REGRESSION | PR #887 (config split across two authored surfaces — a tracked oh.json and a secrets-only root dotenv — with nothing left under $HOME) | -| oh-destroy-guard | A | 2026-08-31 01:42 | PASS | issue #879 — `oh` becomes the only front door, so `make destroy` must | -| oh-devcontainer-restructure | A | 2026-08-31 01:42 | PASS | consolidate devcontainer — .oh/devcontainer/ folded back into .devcontainer/ | -| oh-home-mount | A | 2026-08-31 01:42 | PASS | issue #898 (single $HOME mount) 2026-08-30 | -| oh-image-only-deploy | A | 2026-08-31 01:42 | PASS | .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only deploy) | -| oh-init-headless-config | A | 2026-08-31 01:42 | PASS | PR #827 (installer answers landed in the losing config file); retargeted to the .example.env template by PR #833, then to oh.json by PR #887 | -| oh-init-scaffold | A | 2026-08-31 01:42 | PASS | issue #531 Phase 2 | -| oh-lifecycle-surface | A | 2026-08-31 01:42 | PASS | issue #881 — the Makefile is retired and `oh` is the only front door | -| oh-npm-package | A | 2026-08-31 01:42 | PASS | npm publish path for the standalone `oh` CLI (@mifune/openharness) — alternative to get-oh.sh | -| oh-payload-manifest | A | 2026-08-31 01:42 | PASS | issue #531 follow-on (.oh payload manifest — oh update ships a declared allowlist) | -| oh-sandbox-image-mode | A | 2026-08-31 01:42 | PASS | conversation 2026-07-05 (basic Docker deployment — prebuilt-image mode) | -| oh-shipped-repo-overridable | A | 2026-08-31 01:42 | PASS | issue #531 follow-on (de-hardcode residual — shipped .oh shell scripts keep the upstream repo overridable) | -| oh-standalone-lifecycle | A | 2026-08-31 01:42 | PASS | issue #564 | -| oh-update | A | 2026-08-31 01:42 | PASS | issue #531 Phase 3 (oh update — upgrade only the .oh control plane) | -| operator-config-guard | A | 2026-08-31 01:42 | PASS | operator directives 2026-08-06 (.config/ and settings.local.json are operator-only) | -| pnpm-audit-ci-gate | A | 2026-08-31 01:42 | PASS | issue #171 — pnpm security audits must run in CI | -| post-bridge-publish-confirmation | A | 2026-08-31 01:42 | PASS | #523 — post-bridge live publishing requires an explicit final confirmation gate | -| prd-output-path-contract | A | 2026-08-31 01:42 | PASS | retro lesson 2026-06-19 | -| prompt-miner-schema-compat | A | 2026-08-31 01:42 | PASS | issue #253 — prompt-miner JSONL schema-drift guard | -| prompt-miner-symlink-entrypoint | A | 2026-08-31 01:42 | PASS | issue #663 — prompt-miner engine no-ops via the documented .claude/skills symlink | -| prompt-miner-weakness-record | A | 2026-08-31 01:42 | PASS | issue #580 — prompt-miner weakness-record (WH-xxx) cluster output | -| protected-path-deletion | A | 2026-08-31 01:42 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-001) — the critique gate was deleted, | -| protected-paths-resolve | A | 2026-08-31 01:42 | PASS | issue #753 — .claude/protected-paths.txt named 7 paths that did not exist. | -| registry-portability-gate | A | 2026-08-31 01:42 | PASS | issue #758 | -| registry-portability | A | 2026-08-31 01:42 | SKIPPED | issue #758 | -| retro-deterministic-contract | A | 2026-08-31 01:42 | PASS | issue #443 — /retro deterministic output and self-contained helper contract | -| rl-delegation-write-worker | A | 2026-08-31 01:42 | PASS | retro lesson 2026-06-10 (rl-delegation) #57 | -| rlm-context-budget | A | 2026-08-31 01:42 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-006 | -| runtime-preflight-gate | A | 2026-08-31 01:42 | PASS | issue #806 § B1 (open sandbox.substrate vs sandbox.runtime selector); | -| sandbox-boot-guard-ci | A | 2026-08-31 01:42 | PASS | issue #449 (sandbox image build CI guard) 2026-06-19; | -| sandbox-node-base | A | 2026-08-31 01:42 | PASS | openharness#878 — oh as the only front door, T0 sandbox base image | -| skill-paths | A | 2026-08-31 01:42 | PASS | issue #43 — stale path references; extended by issue #69 — apps/->packages/ rename guard; extended by issue #870 — deleted .oh/agents/advisor.md | -| skills-dir-clean | A | 2026-08-31 01:42 | PASS | conversation 2026-06-29 — Pi parses every top-level `.md` in the skills | -| skills-task-tool-coupling | A | 2026-08-31 01:42 | PASS | council review 2026-08-29 (issue #886) — /delegate instructed Claude-Code-only | -| skills-vendored | A | 2026-08-31 01:42 | PASS | absorb .mifune submodule into .oh — the skills/agents/hooks pack is vendored | -| slack-admin-command-surface | A | 2026-08-31 01:42 | PASS | issue #354 — Slack bridge docs must distinguish Pi /msg-bridge commands from Slack DM admin text handlers | -| spec-family-contract | A | 2026-08-31 01:42 | PASS | issue #265; spec-simplification issue #816; workflow authority issue #854 | -| spec-ready-finalization | A | 2026-08-31 01:42 | PASS | issue #134; spec-simplification issue #816; workflow authority issue #854 | -| ste-checker-contract | A | 2026-08-31 01:42 | PASS | issue #750 PR audit — the /ste checker had four fail-open paths (unclosed | -| submitted-by-trailers | A | 2026-08-31 01:42 | PASS | conversation 2026-06-12 (commit attribution trailers); the single-owner | -| sync-skill-contract | A | 2026-08-31 01:42 | PASS | issue #331 — /sync dispatcher skill (bidirectional origin↔upstream sync) | -| tool-catalog-boundary | A | 2026-08-31 01:42 | PASS | agent-browser's exclusion from the harness catalog (#821) and the | -| version-parity | A | 2026-08-31 01:42 | PASS | conversation 2026-08-29 — the oh CLI became the only lifecycle door, so its | -| weigh-scorer-contract | A | 2026-08-31 01:42 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-003 (2026-06-27) | -| wiki-readme-index | A | 2026-08-31 01:42 | PASS | issue #132 — wiki README index drift guard | -| workflow-boundaries | A | 2026-08-31 01:42 | PASS | conversation 2026-06-19 (workflow consolidation, issue #259); authority moved to /spec in issue #854 | -| worktrees-layout | A | 2026-08-31 01:42 | PASS | issue #872 | +| advisor-monitored-loop | A | 2026-08-31 02:05 | PASS | conversation 2026-06-19 (single-owner implementation workflow, issue #257) | +| agent-browser-cli | A | 2026-08-31 02:05 | PASS | retro lesson 2026-06-07 (agent-browser 0.8.5 CLI) | +| agents-identity-contract | A | 2026-08-31 02:05 | PASS | issue #854 — T3-style root identity, glossary, and skill-owned procedures | +| artifact-contract-audit | A | 2026-08-31 02:05 | PASS | issue #583/#645 — production /audit implementation Gate 1 behavior | +| audit-dispatcher-contract | A | 2026-08-31 02:05 | PASS | issue #645 — audit consolidation public taxonomy | +| audit-implementation-behavior | A | 2026-08-31 02:05 | PASS | issue #645 — implementation root/repo/browser behavior | +| audit-pr-acquire | A | 2026-08-31 02:05 | PASS | issue #645 — production PR acquisition behavior | +| audit-pr-classifier | A | 2026-08-31 02:05 | PASS | issue #645 — deterministic focused and queue PR classifier | +| audit-run-root-contract | A | 2026-08-31 02:05 | PASS | issue #645 — executable immutable audit root/run correlation | +| audit-shellcheck-coverage | A | 2026-08-31 02:05 | PASS | issue #645 — private audit scripts require release and CI lint coverage | +| audit-stale-references | A | 2026-08-31 02:05 | PASS | issue #645 — clean-breaking audit migration | +| boot-lint-glob | A | 2026-08-31 02:05 | PASS | issue #90, issue #120 | +| builder-skill-consolidation | A | 2026-08-31 02:05 | PASS | issue #643 — consolidate artifact builders behind one /builder dispatcher | +| capability-benchmark-schema | A | 2026-08-31 02:05 | PASS | issue #167 — capability benchmark instrument | +| cc-safety-net-wiring | A | 2026-08-31 02:05 | SKIPPED | .oh/tasks/cc-safety-net/prd.json US-007 2026-07-19 | +| changelog-entry-length | A | 2026-08-31 02:05 | PASS | conversation 2026-08-24 — CHANGELOG.md grew to 259KB of bullet prose because "one line" was unquantified | +| cleanup-tasks-scoped-guard | A | 2026-08-31 02:05 | PASS | issue #85 | +| cleanup-tasks-worktree-grooming | A | 2026-08-31 02:05 | PASS | issue #168; issue #327 | +| cli-publish-typecheck-scope | A | 2026-08-31 02:05 | PASS | release run 33271077312 — v0.5.0 pushed its GHCR image, then failed to publish | +| close-issues-on-development | A | 2026-08-31 02:05 | PASS | issue #841 (closing keywords never fire because the default branch is main) 2026-08-26 | +| codex-stale-response-retry | A | 2026-08-31 02:05 | PASS | issue #506 — Codex previous_response_not_found RCA | +| compose-config-path-parity | A | 2026-08-31 02:05 | PASS | PR #833 (remove harness.yaml — the wrapper and VS Code "Reopen in Container" paths must resolve the same service) 2026-08-26 | +| config-schema-parity | A | 2026-08-31 02:05 | PASS | PR #833 (one schema file — DOCKER_SOCKET, SANDBOX_SSH, OH_SANDBOX_IMAGE, OH_PULL_POLICY, SKIP_PNPM_INSTALL were consumed but undocumented); rewritten for the oh.json/secrets split by PR #887 | +| context-tier-size-budget | A | 2026-08-31 02:05 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-007) — the always-on tier was 85,256 B | +| cron-claude-codex-fallback | A | 2026-08-31 02:05 | PASS | conversation 2026-06-12 (default Codex fallback for crons) | +| cron-watchdog | A | 2026-08-31 02:05 | PASS | issues #130/#453 (cron runtime watchdog + legacy system-cron reaping) 2026-06-19 | +| crons-directory-guide | A | 2026-08-31 02:05 | PASS | issue #874 | +| curl-bash-safe-alternatives | A | 2026-08-31 02:05 | PASS | vet-run/vet integration — public curl|bash examples need review-first alternatives | +| datasets-schema | A | 2026-08-31 02:05 | PASS | issue #196 — .oh/evals/datasets verifiable trajectory corpus (Repo2RLEnv-inspired) | +| debugmcp-availability | A | 2026-08-31 02:05 | SKIPPED | issue #297 — DebugMCP MCP debug-server availability | +| delegate-model-effort-policy | A | 2026-08-31 02:05 | PASS | conversation 2026-07-11 (delegate model inheritance and thinking policy) | +| devtcp-hook | A | 2026-08-31 02:05 | PASS | retro lesson 2026-06-10 (zsh /dev/tcp) | +| docker-inspect-env-guard | A | 2026-08-31 02:05 | PASS | operator directive 2026-08-08 (agents keep the docker socket, but must | +| docs-build-fast-path | A | 2026-08-31 02:05 | PASS | #455 — docs builds must stay out of fast harness/eval/release gates; #536 — docs site externalized to openharness-web; docs markdown relocated to docs/ | +| drift-check-cron-staleness-glob | A | 2026-08-31 02:05 | PASS | issue #98; issue #225 (restart-required cron frontmatter/config drift) | +| entrypoint-pnpm-manifest-fingerprint | A | 2026-08-31 02:05 | PASS | issue #521 (manifest-aware sandbox installs) 2026-07-01 | +| eval-ci-gate | A | 2026-08-31 02:05 | PASS | #103 — eval probe suite gated in CI | +| eval-gate | A | 2026-08-31 02:05 | PASS | retro lesson 2026-06-11 (eval-gate) | +| eval-results-atomic | A | 2026-08-31 02:05 | PASS | issue #83 (eval-results-atomic-write) | +| eval-runner-exit | A | 2026-08-31 02:05 | PASS | retro lesson 2026-06-11 (eval-runner-exit) #29 | +| eval-runs-once-per-cycle | A | 2026-08-31 02:05 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-006) — /eval ran 3x per cycle on the | +| execution-target-contract | A | 2026-08-31 02:05 | PASS | issue #733 (ExecutionTarget contract + Docker Compose adapter) 2026-08-10 | +| get-oh-bootstrap | A | 2026-08-31 02:05 | PASS | get-oh.sh bootstrap — the Node-bootstrapping host-side path to the standalone `oh` CLI (also on npm as @mifune/openharness; see oh-npm-package.sh) | +| git-skill | A | 2026-08-31 02:05 | PASS | conversation 2026-06-15 — rules are not always supported; git workflow must be the /git skill | +| harness-audit-empty-output-gate | A | 2026-08-31 02:05 | PASS | issue #246 — /audit harness must fail closed on empty auditor outputs | +| harness-ci-core-paths | A | 2026-08-31 02:05 | PASS | #165 — core sandbox config files must trigger harness CI | +| harness-ci-hooks-paths | A | 2026-08-31 02:05 | PASS | issue #202 — credential/security hook changes must trigger harness CI | +| harness-home-provisioning | A | 2026-08-31 02:05 | PASS | #902 — `oh harness install` must work from inside the sandbox, where | +| harness-yaml-migration | A | 2026-08-31 02:05 | PASS | PR #833 (migrate-harness-yaml.sh — append / uncomment-in-place / preserve / overwrite, plus a silent no-op second run) 2026-08-26 | +| health-check-docker-stats | A | 2026-08-31 02:05 | PASS | retro lesson 2026-06-10 (docker stats vs ps Size) | +| health-check-socket-degrade | A | 2026-08-31 02:05 | PASS | issue #762 (refs #756) — /health-check degrades to one statement, not nine failures | +| heartbeat-logging-contract | A | 2026-08-31 02:05 | PASS | issue #447 (heartbeat log append hardening) 2026-06-18 | +| image-seed-hygiene | A | 2026-08-31 02:05 | PASS | issue #900 (slim the sandbox image) 2026-08-30 | +| markitdown-wiki-ingest | A | 2026-08-31 02:05 | PASS | issue #649 — pinned local-document normalization contract for /wiki ingest | +| next-dev-prod | A | 2026-08-31 02:05 | SKIPPED | retro lesson 2026-06-04 | +| oh-compose-env-wiring | A | 2026-08-31 02:05 | PASS | issue #880 (oh as the only front door — oh.json is the non-secret config surface) | +| oh-config-surfaces | A | 2026-08-31 02:05 | REGRESSION | PR #887 (config split across two authored surfaces — a tracked oh.json and a secrets-only root dotenv — with nothing left under $HOME) | +| oh-destroy-guard | A | 2026-08-31 02:05 | PASS | issue #879 — `oh` becomes the only front door, so `make destroy` must | +| oh-devcontainer-restructure | A | 2026-08-31 02:05 | PASS | consolidate devcontainer — .oh/devcontainer/ folded back into .devcontainer/ | +| oh-home-mount | A | 2026-08-31 02:05 | PASS | issue #898 (single $HOME mount) 2026-08-30 | +| oh-image-only-deploy | A | 2026-08-31 02:05 | PASS | .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only deploy) | +| oh-init-headless-config | A | 2026-08-31 02:05 | PASS | PR #827 (installer answers landed in the losing config file); retargeted to the .example.env template by PR #833, then to oh.json by PR #887 | +| oh-init-scaffold | A | 2026-08-31 02:05 | PASS | issue #531 Phase 2 | +| oh-lifecycle-surface | A | 2026-08-31 02:05 | PASS | issue #881 — the Makefile is retired and `oh` is the only front door | +| oh-npm-package | A | 2026-08-31 02:05 | PASS | npm publish path for the standalone `oh` CLI (@mifune/openharness) — alternative to get-oh.sh | +| oh-payload-manifest | A | 2026-08-31 02:05 | PASS | issue #531 follow-on (.oh payload manifest — oh update ships a declared allowlist) | +| oh-sandbox-image-mode | A | 2026-08-31 02:05 | PASS | conversation 2026-07-05 (basic Docker deployment — prebuilt-image mode) | +| oh-shipped-repo-overridable | A | 2026-08-31 02:05 | PASS | issue #531 follow-on (de-hardcode residual — shipped .oh shell scripts keep the upstream repo overridable) | +| oh-standalone-lifecycle | A | 2026-08-31 02:05 | PASS | issue #564 | +| oh-update | A | 2026-08-31 02:05 | PASS | issue #531 Phase 3 (oh update — upgrade only the .oh control plane) | +| operator-config-guard | A | 2026-08-31 02:05 | PASS | operator directives 2026-08-06 (.config/ and settings.local.json are operator-only) | +| pnpm-audit-ci-gate | A | 2026-08-31 02:05 | PASS | issue #171 — pnpm security audits must run in CI | +| post-bridge-publish-confirmation | A | 2026-08-31 02:05 | PASS | #523 — post-bridge live publishing requires an explicit final confirmation gate | +| prd-output-path-contract | A | 2026-08-31 02:05 | PASS | retro lesson 2026-06-19 | +| prompt-miner-schema-compat | A | 2026-08-31 02:05 | PASS | issue #253 — prompt-miner JSONL schema-drift guard | +| prompt-miner-symlink-entrypoint | A | 2026-08-31 02:05 | PASS | issue #663 — prompt-miner engine no-ops via the documented .claude/skills symlink | +| prompt-miner-weakness-record | A | 2026-08-31 02:05 | PASS | issue #580 — prompt-miner weakness-record (WH-xxx) cluster output | +| protected-path-deletion | A | 2026-08-31 02:05 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-001) — the critique gate was deleted, | +| protected-paths-resolve | A | 2026-08-31 02:05 | PASS | issue #753 — .claude/protected-paths.txt named 7 paths that did not exist. | +| registry-portability-gate | A | 2026-08-31 02:05 | PASS | issue #758 | +| registry-portability | A | 2026-08-31 02:05 | SKIPPED | issue #758 | +| retro-deterministic-contract | A | 2026-08-31 02:05 | PASS | issue #443 — /retro deterministic output and self-contained helper contract | +| rl-delegation-write-worker | A | 2026-08-31 02:05 | PASS | retro lesson 2026-06-10 (rl-delegation) #57 | +| rlm-context-budget | A | 2026-08-31 02:05 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-006 | +| runtime-preflight-gate | A | 2026-08-31 02:05 | PASS | issue #806 § B1 (open sandbox.substrate vs sandbox.runtime selector); | +| sandbox-boot-guard-ci | A | 2026-08-31 02:05 | PASS | issue #449 (sandbox image build CI guard) 2026-06-19; | +| sandbox-node-base | A | 2026-08-31 02:05 | PASS | openharness#878 — oh as the only front door, T0 sandbox base image | +| skill-paths | A | 2026-08-31 02:05 | PASS | issue #43 — stale path references; extended by issue #69 — apps/->packages/ rename guard; extended by issue #870 — deleted .oh/agents/advisor.md | +| skills-dir-clean | A | 2026-08-31 02:05 | PASS | conversation 2026-06-29 — Pi parses every top-level `.md` in the skills | +| skills-task-tool-coupling | A | 2026-08-31 02:05 | PASS | council review 2026-08-29 (issue #886) — /delegate instructed Claude-Code-only | +| skills-vendored | A | 2026-08-31 02:05 | PASS | absorb .mifune submodule into .oh — the skills/agents/hooks pack is vendored | +| slack-admin-command-surface | A | 2026-08-31 02:05 | PASS | issue #354 — Slack bridge docs must distinguish Pi /msg-bridge commands from Slack DM admin text handlers | +| spec-family-contract | A | 2026-08-31 02:05 | PASS | issue #265; spec-simplification issue #816; workflow authority issue #854 | +| spec-ready-finalization | A | 2026-08-31 02:05 | PASS | issue #134; spec-simplification issue #816; workflow authority issue #854 | +| ste-checker-contract | A | 2026-08-31 02:05 | PASS | issue #750 PR audit — the /ste checker had four fail-open paths (unclosed | +| submitted-by-trailers | A | 2026-08-31 02:05 | PASS | conversation 2026-06-12 (commit attribution trailers); the single-owner | +| sync-skill-contract | A | 2026-08-31 02:05 | PASS | issue #331 — /sync dispatcher skill (bidirectional origin↔upstream sync) | +| tool-catalog-boundary | A | 2026-08-31 02:05 | PASS | agent-browser's exclusion from the harness catalog (#821) and the | +| version-parity | A | 2026-08-31 02:05 | PASS | conversation 2026-08-29 — the oh CLI became the only lifecycle door, so its | +| weigh-scorer-contract | A | 2026-08-31 02:05 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-003 (2026-06-27) | +| wiki-readme-index | A | 2026-08-31 02:05 | PASS | issue #132 — wiki README index drift guard | +| workflow-boundaries | A | 2026-08-31 02:05 | PASS | conversation 2026-06-19 (workflow consolidation, issue #259); authority moved to /spec in issue #854 | +| worktrees-layout | A | 2026-08-31 02:05 | PASS | issue #872 | diff --git a/.oh/evals/probes/harness-home-provisioning.sh b/.oh/evals/probes/harness-home-provisioning.sh index 17f70163..0799c9cd 100755 --- a/.oh/evals/probes/harness-home-provisioning.sh +++ b/.oh/evals/probes/harness-home-provisioning.sh @@ -2,9 +2,12 @@ # tier: A # source: #902 — `oh harness install` must work from inside the sandbox, where # sudo has no NOPASSWD, so default harnesses install into the home mount +# source: #904 — the image must not bake a default harness, or the boot-time +# install path is dead code that CI and a normal boot both skip # desc: every kind:"default" harness installs as the sandbox user into -# NPM_USER_PREFIX, claude-code keeps its postinstall, and the boot path -# carries the OH_PROVISION_HARNESSES guard and its provisioner. +# NPM_USER_PREFIX, claude-code keeps its postinstall, no default harness +# package appears in the Dockerfile, and the boot path carries the +# OH_PROVISION_HARNESSES guard and its provisioner. set -euo pipefail ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" @@ -63,43 +66,44 @@ grep -qF 'WARNING: harness provisioning did not complete' "$ENTRY" \ || missing+=("entrypoint.sh: harness provisioning does not warn-and-continue — an offline sandbox must still come up as a usable shell") [[ -x $PROVISIONER ]] \ || missing+=(".oh/scripts/provision-harnesses.sh: missing or not executable") -run_block_with() { - awk -v needle="$1" ' - function flush() { - if (index(buf, needle)) print buf - buf = "" - } - /^RUN / { buf = $0; cont = ($0 ~ /\\$/); if (!cont) flush(); next } - cont { buf = buf "\n" $0; cont = ($0 ~ /\\$/); if (!cont) flush() } - ' "$DOCKERFILE" +# #904: the image must not bake any kind:"default" harness. The install target is +# the home mount, so a copy under /usr/lib/node_modules shadows it with one no +# running sandbox can upgrade — and, worse, makes the boot-time install path +# dead code that never runs and never gets tested. The package names come from +# the catalog itself, so this cannot drift. +strip_dockerfile_comments() { + grep -vE '^[[:space:]]*#' "$DOCKERFILE" } -BAKE_GATE='if [ "${BAKE_HARNESSES}" = "true" ]' +DOCKERFILE_CODE=$(strip_dockerfile_comments) -stage_body() { - awk -v stage="$1" ' - /^FROM / { inb = ($0 ~ ("AS " stage "$")); next } - inb - ' "$DOCKERFILE" -} - -for stage in base home; do - stage_body "$stage" | grep -qE '^ARG BAKE_HARNESSES' \ - || missing+=("Dockerfile: the $stage stage does not declare ARG BAKE_HARNESSES — ARG is stage-scoped, so the build arg is empty there and every \${BAKE_HARNESSES} test in that stage reads as unset") -done +pkgs=0 +while IFS= read -r entry; do + [[ $entry == *'kind: "default"'* ]] || continue + id=$(sed -n 's/.*id: "\([^"]*\)".*/\1/p' <<<"$entry") + # The package specifier is the last element of installArgv. Read it from that + # array alone — `binary` and `verifyArgv` also hold bare names, and matching + # those would test the wrong string ("claude" appears in the Dockerfile's + # shell alias; "@anthropic-ai/claude-code" is what must not). + argv=$(sed -n 's/.*installArgv: \[\(.*\)\], *installUser.*/\1/p' <<<"$entry") + pkg=$(grep -oE '"[^"]+"' <<<"$argv" | tr -d '"' | tail -1) + if [[ -z $pkg || $pkg == -* ]]; then + missing+=("harnesses/catalog.ts: could not read an install package out of default harness \"$id\" — the no-bake check cannot be applied to it") + continue + fi + pkgs=$((pkgs + 1)) + if grep -qF -- "$pkg" <<<"$DOCKERFILE_CODE"; then + missing+=("Dockerfile: names $pkg — default harness \"$id\" is baked into the image again; it belongs to .oh/scripts/provision-harnesses.sh, which installs it into $PREFIX at boot") + fi +done <<<"$entries" -AGENTS_BLOCK=$(run_block_with 'read -ra agents') -if [[ -z $AGENTS_BLOCK ]]; then - missing+=("Dockerfile: found no RUN that loops over \$AGENTS — the probe cannot tell whether BAKE_HARNESSES gates the bake") -elif [[ $AGENTS_BLOCK != *"$BAKE_GATE"* ]]; then - missing+=("Dockerfile: the RUN that installs \$AGENTS does not test \${BAKE_HARNESSES} — ARG BAKE_HARNESSES is declared but dead, so BAKE_HARNESSES=false still bakes the agent CLIs") +if ((pkgs == 0)); then + echo "SKIPPED: parsed no install package out of any kind:\"default\" catalog entry, so the no-bake check would pass vacuously" >&2 + exit 2 fi -PI_BLOCK=$(run_block_with '--ignore-scripts @earendil-works/pi-coding-agent') -if [[ -z $PI_BLOCK ]]; then - missing+=("Dockerfile: found no RUN that bakes pi into \$NPM_USER_PREFIX — the probe cannot tell whether BAKE_HARNESSES gates it") -elif [[ $PI_BLOCK != *"$BAKE_GATE"* ]]; then - missing+=("Dockerfile: the RUN that bakes pi does not test \${BAKE_HARNESSES} — ARG is stage-scoped, so BAKE_HARNESSES=false unbakes claude and codex but leaves pi in the image") +if grep -qE '^ARG (BAKE_HARNESSES|AGENTS)=' <<<"$DOCKERFILE_CODE"; then + missing+=("Dockerfile: ARG BAKE_HARNESSES/AGENTS is back — a build-arg that re-bakes the default harnesses is a dormant path that reintroduces the shadowed install and un-exercises the boot provisioner") fi if ((${#missing[@]})); then @@ -107,4 +111,4 @@ if ((${#missing[@]})); then exit 1 fi -echo "PASS: all $defaults default harnesses install as the sandbox user into $PREFIX, and the boot path provisions them" >&2 +echo "PASS: all $defaults default harnesses install as the sandbox user into $PREFIX, none of the $pkgs packages is baked into the image, and the boot path provisions them" >&2 diff --git a/.oh/evals/probes/sandbox-boot-guard-ci.sh b/.oh/evals/probes/sandbox-boot-guard-ci.sh index fc6dd713..e603e14b 100755 --- a/.oh/evals/probes/sandbox-boot-guard-ci.sh +++ b/.oh/evals/probes/sandbox-boot-guard-ci.sh @@ -51,7 +51,34 @@ has '--tag "sandbox-${SANDBOX_NAME}"' "compose image tag for smoke boot" has 'bash .oh/scripts/sandbox-boot-smoke.sh' "boot smoke healthcheck invocation" has 'name: Validate sandbox compose and image build' "the named boot guard job" has 'bash .oh/scripts/verify-sandbox-image.sh' "reusable image verifier invocation" -has 'BOOT_SMOKE_TIMEOUT_SECONDS: "900"' "bounded boot smoke timeout" +# The smoke deadline must clear the compose healthcheck's own unhealthy deadline +# (start_period + interval x retries), or the smoke times out before the boot it +# is measuring has had its full allowance. Derive both sides — pinning a literal +# lets a start_period bump silently invert the relationship. +smoke_timeout=$(grep -Eo 'BOOT_SMOKE_TIMEOUT_SECONDS: *"?[0-9]+' <<<"$text" | grep -Eo '[0-9]+$' | head -1) +if [[ -z $smoke_timeout ]]; then + missing+=("bounded boot smoke timeout (no BOOT_SMOKE_TIMEOUT_SECONDS)") +else + COMPOSE_FILE="$ROOT/.devcontainer/docker-compose.yml" + hc=$(awk '/^ *healthcheck:/ {inb=1} inb && /^ *(interval|retries|start_period):/ {print} inb && /^ *restart:/ {inb=0}' "$COMPOSE_FILE") + interval=$(grep -Eo 'interval: *[0-9]+' <<<"$hc" | grep -Eo '[0-9]+' | head -1) + retries=$(grep -Eo 'retries: *[0-9]+' <<<"$hc" | grep -Eo '[0-9]+' | head -1) + start_period=$(grep -Eo 'start_period: *[0-9]+' <<<"$hc" | grep -Eo '[0-9]+' | head -1) + if [[ -z $interval || -z $retries || -z $start_period ]]; then + missing+=("could not read the sandbox healthcheck window out of .devcontainer/docker-compose.yml") + else + deadline=$((start_period + interval * retries)) + if ((smoke_timeout <= deadline)); then + missing+=("BOOT_SMOKE_TIMEOUT_SECONDS=$smoke_timeout does not clear the healthcheck unhealthy deadline of ${deadline}s (start_period ${start_period}s + ${interval}s x ${retries}) — the smoke would time out before the boot it measures") + fi + fi +fi + +# #904: boot-time harness provisioning is exercised nowhere else. Turning it off +# here to save CI minutes would restore it to untested dead code. +if grep -Eq 'OH_PROVISION_HARNESSES: *"?false' <<<"$text"; then + missing+=("the boot guard disables OH_PROVISION_HARNESSES — this job is the only place the boot-time harness install runs") +fi has 'Sandbox boot guard only' "comment explaining non-release intent" if grep -Eq 'docker[[:space:]]+push|--push([[:space:]]|$)|docker/login-action|docker/login|ghcr\.io|[[:alnum:]._-]+\.[[:alnum:]._-]+/.+:.+|packages:[[:space:]]*write|secrets\.' <<<"$text"; then diff --git a/.oh/scripts/__tests__/sandbox-boot-smoke.test.ts b/.oh/scripts/__tests__/sandbox-boot-smoke.test.ts index 59dcca6a..9eceef08 100644 --- a/.oh/scripts/__tests__/sandbox-boot-smoke.test.ts +++ b/.oh/scripts/__tests__/sandbox-boot-smoke.test.ts @@ -16,6 +16,8 @@ function fixture( runtimeExecFails?: boolean; runtimeUid?: string; markerOwner?: string; + harnessProbeFails?: boolean; + noDefaultHarnesses?: boolean; } = {}, ) { const runtimeUid = opts.runtimeUid ?? HOST_UID; @@ -74,6 +76,31 @@ case "$1" in printf '%s\n' ${JSON.stringify(markerOwner)} exit 0 ;; + *"oh harness list --defaults --json"*) + cat <<'JSON' +${ + opts.noDefaultHarnesses + ? "[]" + : `[ + { "id": "claude-code", "title": "Claude Code", "binary": "claude", "kind": "default", "enabled": null, "installed": true, "docs": "x" }, + { "id": "pi", "title": "Pi", "binary": "pi", "kind": "default", "enabled": null, "installed": true, "docs": "x" } +]` +} +JSON + exit 0 + ;; + *"type -P"*) + if [ "${opts.harnessProbeFails ? "1" : "0"}" = "1" ]; then + echo 'is not on PATH under /home/sandbox/.local (type -P gave: /usr/bin/claude)' >&2 + exit 1 + fi + printf '1.2.3\n' + exit 0 + ;; + *"id -u sandbox"*) + printf '%s\n' ${JSON.stringify(runtimeUid)} + exit 0 + ;; esac echo 'sandbox healthcheck ok' exit 0 @@ -138,6 +165,34 @@ describe("sandbox boot smoke", () => { expect(result.stdout).toContain( `sandbox user, bind mount, and sandbox-created files all resolve to ${HOST_UID}:${HOST_GID}`, ); + expect(dockerCalls).toContain("oh harness list --defaults --json"); + expect(dockerCalls).toContain("type -P"); + expect(result.stdout).toContain("claude-code provisioned at boot -> 1.2.3"); + expect(result.stdout).toContain("pi provisioned at boot -> 1.2.3"); + }); + + // #904 deleted the image bake, so this assertion is the only thing standing + // between a silently broken boot-time install and a green pipeline. + it("fails when a default harness was not provisioned into the home mount", () => { + const fx = fixture({ harnessProbeFails: true }); + + const result = runSmoke(fx); + + expect(result.status).toBe(1); + expect(result.stderr).toContain( + "default harness 'claude-code' was not provisioned into the home mount at boot", + ); + expect(result.stderr).toContain("type -P gave: /usr/bin/claude"); + expect(readFileSync(fx.composeLog, "utf8")).toContain("down -v --remove-orphans"); + }); + + it("refuses to pass vacuously when the catalog reports no default harnesses", () => { + const fx = fixture({ noDefaultHarnesses: true }); + + const result = runSmoke(fx); + + expect(result.status).toBe(1); + expect(result.stderr).toContain('reported no kind:"default" harnesses'); }); it("fails when the runtime sandbox user does not match the checkout owner", () => { diff --git a/.oh/scripts/__tests__/sandbox-healthcheck.test.ts b/.oh/scripts/__tests__/sandbox-healthcheck.test.ts index e5ae10e3..27b36ff2 100644 --- a/.oh/scripts/__tests__/sandbox-healthcheck.test.ts +++ b/.oh/scripts/__tests__/sandbox-healthcheck.test.ts @@ -136,7 +136,13 @@ describe("sandbox healthcheck", () => { expect(compose).toContain("healthcheck:"); expect(compose).toContain("/home/sandbox/harness/.oh/scripts/sandbox-healthcheck.sh"); - expect(compose).toContain("start_period: 300s"); + // Boot installs the default harnesses into the home mount (#904), bounded + // by OH_PROVISION_HARNESSES_TIMEOUT (180s). The start period has to cover + // that plus the rest of boot, so assert the floor rather than a literal + // that a reduction could slip past. + const startPeriod = /start_period: (\d+)s/.exec(compose); + expect(startPeriod, "compose declares no healthcheck start_period").not.toBeNull(); + expect(Number(startPeriod![1])).toBeGreaterThanOrEqual(600); }); it("delegates tmux checks to the sandbox user when Docker invokes as root", () => { diff --git a/.oh/scripts/__tests__/verify-sandbox-image.test.ts b/.oh/scripts/__tests__/verify-sandbox-image.test.ts index 11ca5ec8..0ce05a8d 100644 --- a/.oh/scripts/__tests__/verify-sandbox-image.test.ts +++ b/.oh/scripts/__tests__/verify-sandbox-image.test.ts @@ -23,6 +23,9 @@ type Overrides = Partial<{ missingTool: string; nonVersionTool: string; platformWarning: string; + bakedHarnesses: boolean; + noDefaultHarnesses: boolean; + harnessCatalogFails: boolean; }>; function fixture(o: Overrides = {}) { @@ -44,6 +47,9 @@ function fixture(o: Overrides = {}) { missingTool: "", nonVersionTool: "", platformWarning: "", + bakedHarnesses: false, + noDefaultHarnesses: false, + harnessCatalogFails: false, ...o, }; @@ -62,6 +68,22 @@ case "$cmd" in "pnpm --version") printf '%s\\n' ${JSON.stringify(v.pnpm)} ;; "herdr --version") printf '%s\\n' ${JSON.stringify(v.herdr)} ;; *sha256sum*) printf '%s /usr/local/bin/herdr\\n' ${JSON.stringify(v.herdrSha)} ;; + *"oh harness list --defaults --json"*) + if [ "${v.harnessCatalogFails ? "1" : "0"}" = "1" ]; then + echo 'not an OpenHarness-equipped repo' >&2 + exit 1 + fi + cat <<'JSON' +${ + v.noDefaultHarnesses + ? "[]" + : `[ + { "id": "claude-code", "binary": "claude", "kind": "default", "installed": ${v.bakedHarnesses} }, + { "id": "pi", "binary": "pi", "kind": "default", "installed": false } +]` +} +JSON + ;; *) if [ -n ${JSON.stringify(v.missingTool)} ] && [ "$cmd" = ${JSON.stringify(v.missingTool)} ]; then echo 'command not found' >&2 @@ -175,4 +197,35 @@ describe("verify-sandbox-image", () => { expect(result.status).toBe(0); expect(result.stdout).toContain("matches the arm64 (aarch64) Dockerfile checksum pin"); }); + + // #904: the default harnesses moved out of the image and into the boot path. + // A baked copy under /usr/lib/node_modules shadows the home-mount install and + // silently un-exercises the provisioner, so the image must not carry one. + it("passes an image that bakes no default harness", () => { + const result = run(fixture()); + + expect(result.status).toBe(0); + expect(result.stdout).toContain("no default harness is baked into the image"); + }); + + it("rejects an image that bakes a default harness", () => { + const result = run(fixture({ bakedHarnesses: true })); + + expect(result.status).toBe(1); + expect(result.stderr).toContain("the image ships baked default harnesses: claude-code (claude)"); + }); + + it("refuses to pass vacuously when the image catalog lists no default harness", () => { + const result = run(fixture({ noDefaultHarnesses: true })); + + expect(result.status).toBe(1); + expect(result.stderr).toContain("would pass vacuously"); + }); + + it("fails loudly when the harness catalog cannot be read out of the image", () => { + const result = run(fixture({ harnessCatalogFails: true })); + + expect(result.status).toBe(1); + expect(result.stderr).toContain("could not read the harness catalog from the image"); + }); }); diff --git a/.oh/scripts/sandbox-boot-smoke.sh b/.oh/scripts/sandbox-boot-smoke.sh index c51bcbb9..08046939 100755 --- a/.oh/scripts/sandbox-boot-smoke.sh +++ b/.oh/scripts/sandbox-boot-smoke.sh @@ -81,6 +81,78 @@ verify_bind_ownership() { echo "sandbox boot smoke: sandbox user, bind mount, and sandbox-created files all resolve to $host_uid:$host_gid" } +# Under emulation `docker exec` can prefix output with a platform warning; take +# the first line that actually carries content. +first_real_line() { + grep -vE "^WARNING: The requested image's platform" | grep -m1 -E '[^[:space:]]' || true +} + +# The default harnesses are no longer baked into the image (#904); the boot path +# installs them into the home mount. That install therefore runs on EVERY fresh +# boot, and nothing else in CI exercises it — this is its only oracle. Assert the +# outcome, not the log line: each default harness must resolve to a real binary +# under NPM_USER_PREFIX, owned by the sandbox user, that prints its own version. +verify_default_harnesses() { + local cid="$1" + local prefix="${NPM_USER_PREFIX:-/home/sandbox/.local}" + local states ids binary sandbox_uid out line + + if ! command -v jq >/dev/null 2>&1; then + echo "sandbox boot smoke failed: jq is required on the runner to read the harness catalog JSON" >&2 + return 1 + fi + + if ! states=$(docker exec -u sandbox "$cid" bash -lc 'oh harness list --defaults --json' 2>/tmp/sandbox-boot-smoke-harness.err); then + echo "sandbox boot smoke failed: 'oh harness list --defaults --json' did not run in the booted sandbox" >&2 + cat /tmp/sandbox-boot-smoke-harness.err >&2 || true + return 1 + fi + + ids=$(jq -r '.[] | select(.kind == "default") | .id' <<<"$states") + if [ -z "$ids" ]; then + echo "sandbox boot smoke failed: the harness catalog reported no kind:\"default\" harnesses, so this check would pass vacuously" >&2 + return 1 + fi + + sandbox_uid=$(docker exec "$cid" id -u sandbox) + + local failed=0 + while IFS= read -r id; do + [ -n "$id" ] || continue + binary=$(jq -r --arg id "$id" '.[] | select(.id == $id) | .binary' <<<"$states") + if [ -z "$binary" ] || [ "$binary" = "null" ]; then + echo "sandbox boot smoke failed: default harness '$id' declares no binary to check" >&2 + failed=1 + continue + fi + if ! out=$(docker exec -u sandbox "$cid" bash -lc " + set -e + path=\$(type -P '$binary') + case \"\$path\" in + $prefix/*) ;; + *) echo \"is not on PATH under $prefix (type -P gave: '\$path')\" >&2; exit 1 ;; + esac + owner=\$(stat -Lc %u \"\$path\") + [ \"\$owner\" = '$sandbox_uid' ] || { echo \"binary is owned by uid \$owner, not sandbox ($sandbox_uid)\" >&2; exit 1; } + \"\$path\" --version + " 2>&1); then + echo "sandbox boot smoke failed: default harness '$id' was not provisioned into the home mount at boot" >&2 + printf ' %s\n' "$out" >&2 + failed=1 + continue + fi + line=$(first_real_line <<<"$out") + if ! grep -Eq '(^|[^[:alnum:]])v?[0-9]+([.][0-9]+)+([^[:alnum:]]|$)' <<<"$line"; then + echo "sandbox boot smoke failed: '$binary --version' printed no numeric version: $line" >&2 + failed=1 + continue + fi + echo "sandbox boot smoke: $id provisioned at boot -> $line" + done <<<"$ids" + + [ "$failed" = "0" ] +} + trap teardown EXIT # shellcheck disable=SC2086 # BOOT_SMOKE_UP_ARGS is an intentional argv fragment for CI tuning. @@ -106,7 +178,11 @@ while [ "$(date +%s)" -le "$end" ]; do status_diagnostics "$cid" exit 1 fi - echo "sandbox boot smoke ok: $SERVICE ($cid) passed $HEALTH_CMD, Herdr runtime, and bind-ownership checks" + if ! verify_default_harnesses "$cid"; then + status_diagnostics "$cid" + exit 1 + fi + echo "sandbox boot smoke ok: $SERVICE ($cid) passed $HEALTH_CMD, Herdr runtime, bind-ownership, and boot-provisioned harness checks" exit 0 fi last_status=$(docker inspect -f '{{if .State.Health}}{{.State.Health.Status}}{{else}}no-healthcheck{{end}}' "$cid" 2>/dev/null || echo "inspect-failed") diff --git a/.oh/scripts/verify-sandbox-image.sh b/.oh/scripts/verify-sandbox-image.sh index c875013e..18342b2a 100755 --- a/.oh/scripts/verify-sandbox-image.sh +++ b/.oh/scripts/verify-sandbox-image.sh @@ -1,7 +1,8 @@ #!/usr/bin/env bash # Verify a built sandbox image: base distribution, apt suites, the sandbox # UID/GID contract, the Node/pnpm pins, the Herdr checksum, and version output -# from every required default tool. Usage: verify-sandbox-image.sh +# from every required default tool, and that no kind:"default" harness is baked +# into it. Usage: verify-sandbox-image.sh set -euo pipefail @@ -137,6 +138,32 @@ for tool in "gh --version" "docker --version" "docker compose version" \ fi done +# The image must NOT ship the default harnesses (#904). They are the in-sandbox +# CLI's responsibility and are installed into the home mount at boot, so a +# default harness found here means the bake came back and the home mount's copy +# is shadowed by an unupgradable one under /usr/lib/node_modules. The catalog in +# the image is the source of truth for which ids are default, so this cannot +# drift from harnesses/catalog.ts. +if defaults_json=$(run 'cd /opt/oh-seed && OH_EXECUTION_TARGET=local oh harness list --defaults --json' 2>/tmp/verify-sandbox-defaults.err); then + if command -v jq >/dev/null 2>&1; then + default_ids=$(jq -r '.[] | select(.kind == "default") | .id' <<<"$defaults_json") + if [ -z "$default_ids" ]; then + fail "the image's harness catalog reports no kind:\"default\" harnesses — the unbaked-image check would pass vacuously" + else + baked=$(jq -r '.[] | select(.kind == "default" and .installed == true) | "\(.id) (\(.binary))"' <<<"$defaults_json") + if [ -n "$baked" ]; then + fail "the image ships baked default harnesses: $(tr '\n' ' ' <<<"$baked")— these must be provisioned into /home/sandbox/.local at boot, not baked" + else + ok "no default harness is baked into the image ($(tr '\n' ' ' <<<"$default_ids"))" + fi + fi + else + fail "jq is required to read the image's harness catalog JSON" + fi +else + fail "could not read the harness catalog from the image: $(cat /tmp/verify-sandbox-defaults.err 2>/dev/null | head -3)" +fi + if ((${#failures[@]})); then printf '\nverify-sandbox-image: %d check(s) failed\n' "${#failures[@]}" >&2 printf ' - %s\n' "${failures[@]}" >&2 diff --git a/CHANGELOG.md b/CHANGELOG.md index 5ecce67e..6d2b6452 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,13 +11,16 @@ Update policy and release automation live in [`/git`](.claude/skills/git/SKILL.m ### Changed - **BREAKING:** Persist the sandbox home through one `/home/sandbox` mount, not eleven per-tool volumes; set `storage.homePath` for a host path, else `_workspace` ([#898](https://github.com/mifunedev/openharness/issues/898)). - Shrink the sandbox image ~540 MB: drop build caches from the baked home seed, stage the seed once via a builder stage, and keep untracked build output out of the build context ([#900](https://github.com/mifunedev/openharness/issues/900)). +- **BREAKING:** Stop baking Claude Code, Codex, and Pi into the image; boot installs them into the home mount, so a first boot needs network and runs 60-180s longer ([#904](https://github.com/mifunedev/openharness/issues/904)). ### Removed +- Remove the `BAKE_HARNESSES` and `AGENTS` build args along with the image bake they gated; the harness catalog is the only source of truth for what gets installed ([#904](https://github.com/mifunedev/openharness/issues/904)). - **BREAKING:** Retire the `projectRoot` / `OH_PROJECT_ROOT` config knob — the checkout is fixed at `/home/sandbox/harness`, nested inside the home mount ([#898](https://github.com/mifunedev/openharness/issues/898)). ### Added - Provision the default harnesses into `/home/sandbox/.local` at boot, gated by `OH_PROVISION_HARNESSES`, so `oh harness install` also works from inside the sandbox ([#902](https://github.com/mifunedev/openharness/issues/902)). - Add `oh-home-mount.sh`, a tier-A probe holding the single-`$HOME`-mount contract: one mount per compose file, the baked `/opt/home-seed`, and the checkout prune that replaces `-xdev` ([#898](https://github.com/mifunedev/openharness/issues/898)). +- Assert boot-provisioned harnesses in the boot smoke and reject a baked default harness in `verify-sandbox-image.sh`, so CI exercises the install path ([#904](https://github.com/mifunedev/openharness/issues/904)). - Add `skills-task-tool-coupling.sh`, a tier-A probe holding the canonical skill pack and the sandbox in agreement about the Claude-Code-only task tools ([#886](https://github.com/mifunedev/openharness/issues/886)). ### Fixed diff --git a/docs/deployment-prebuilt-image.md b/docs/deployment-prebuilt-image.md index 48de044b..b934d978 100644 --- a/docs/deployment-prebuilt-image.md +++ b/docs/deployment-prebuilt-image.md @@ -234,9 +234,16 @@ A healthy boot ends with `Providers OK: …` and `SEED_OK`, and the logs show authoritative — later boots see the `.oh/.image-seeded` marker and skip re-seeding, so your in-container edits persist. +The same first boot also installs the default harnesses (Claude Code, Codex, Pi) +into `/home/sandbox/.local`; they are not baked into the image. Expect the boot +to run 60–180s longer than the `sleep 8` above and to need network — check with +`docker exec "$NAME" bash -lc 'oh harness list --defaults'`. If the registry was +unreachable the container still comes up; re-run +`docker exec "$NAME" bash -lc 'bash /home/sandbox/harness/.oh/scripts/provision-harnesses.sh'`. + ```bash # ── 4. Attach an interactive shell (once the container is stable) ── -# Optional: block until the healthcheck reports healthy (start_period ~300s). +# Optional: block until the healthcheck reports healthy (start_period ~600s). until [ "$(docker inspect -f '{{.State.Health.Status}}' "$NAME" 2>/dev/null)" = healthy ]; do echo "waiting for $NAME to become healthy…"; sleep 5 done diff --git a/docs/harnesses/opencode.md b/docs/harnesses/opencode.md index ea3413bf..c6bc5bfb 100644 --- a/docs/harnesses/opencode.md +++ b/docs/harnesses/opencode.md @@ -4,7 +4,7 @@ title: "OpenCode" # OpenCode -OpenCode is a terminal coding agent that can run interactively or execute one-shot tasks. It is an optional image-level runtime in Open Harness; the default sandbox image ships Claude Code, Codex, and Pi only. +OpenCode is a terminal coding agent that can run interactively or execute one-shot tasks. It is an optional image-level runtime in Open Harness; the default harnesses — Claude Code, Codex, and Pi — are provisioned into `~/.local` at boot instead of being baked into the image. ## Install (optional) diff --git a/docs/harnesses/overview.md b/docs/harnesses/overview.md index d49b2954..23bb0eba 100644 --- a/docs/harnesses/overview.md +++ b/docs/harnesses/overview.md @@ -76,7 +76,7 @@ prime-agent --version # Prime Agent (not preinstalled — oh harness install p ## Authentication -Open Harness ships Claude Code, Codex, and Pi in the default image. Authenticate at least one default harness before use; authenticate optional harnesses after enabling their install flags: +Open Harness provisions Claude Code, Codex, and Pi into `~/.local` on first boot rather than baking them into the image. Authenticate at least one default harness before use; authenticate optional harnesses after enabling their install flags: - **Claude Code**: run `claude` and follow the OAuth prompt (see [Claude Code](./claude-code.md)). - **Codex**: run `codex login` (see [Codex](./codex.md)). diff --git a/docs/installation.md b/docs/installation.md index 83372748..2223e93a 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -280,7 +280,26 @@ Docker's apt repository tracks the `trixie` suite. Cloudflare's stays on `bookwo ### AI agent CLIs -Default CLIs are always present. Optional CLIs are excluded from the default image; `oh harness install ` flips the matching `install.*` field in `oh.json` and installs it. +Default CLIs are not baked into the image. The entrypoint runs +`.oh/scripts/provision-harnesses.sh` on every boot, which installs any missing +default harness into `~/.local` — inside the home mount — as the `sandbox` user. +That is what makes `oh harness install ` able to upgrade one in place: a copy +under `/usr/lib/node_modules` would be root-owned and unwritable from a running +sandbox. Consequences worth knowing: + +- A **first boot on a fresh home mount needs network**. Measured at 21s on a + GitHub Actions runner; budget 60–180s on a slower link. The compose + healthcheck's `start_period` is 600s to cover it. +- If the registry is unreachable the sandbox still comes up as a usable shell, + with a warning and no agent CLIs. Re-run + `bash .oh/scripts/provision-harnesses.sh` once you have network. +- An existing install is never replaced, so the provisioner is a no-op on every + boot after the first. Upgrade deliberately with `oh harness install `. +- npm's cache now lives in the home mount at `~/.npm` and grows across upgrades. + `npm cache clean --force` reclaims it. +- Set `OH_PROVISION_HARNESSES=false` to skip the step entirely. + +Optional CLIs are excluded from the default image; `oh harness install ` flips the matching `install.*` field in `oh.json` and installs it. | Tool | Command | Source | Status | |------|---------|--------|--------| @@ -305,8 +324,8 @@ Default CLIs are always present. Optional CLIs are excluded from the default ima ### DevOps & infrastructure `oh tool list` reports which of these are present, and `oh tool status ` -adds a version where the tool has a verified version flag. They are baked into -the image, so there is nothing to install. +adds a version where the tool has a verified version flag. Unlike the agent +CLIs above, these are baked into the image, so there is nothing to install. | Tool | Purpose | |------|---------| diff --git a/docs/quickstart.md b/docs/quickstart.md index c3740163..024d73cd 100644 --- a/docs/quickstart.md +++ b/docs/quickstart.md @@ -119,9 +119,14 @@ continue to run independently under tmux. ## Set up agents inside Herdr -The default sandbox ships with Claude Code, Codex, and Pi. OpenCode, -DeepAgents, Hermes, and Grok Build are optional image-level installs; T3 Code runs on -demand via the `/t3` skill or direct `npx`. Authenticate at least one harness before use. +The sandbox provisions Claude Code, Codex, and Pi into `~/.local` on first boot — +they live in the home mount, not the image, so `oh harness install ` upgrades +them in place without a rebuild. A first boot on a fresh home mount therefore needs +network access and takes a minute or two longer; the sandbox still comes up as a +usable shell if the registry is unreachable, and you can retry with +`bash .oh/scripts/provision-harnesses.sh`. OpenCode, DeepAgents, Hermes, and Grok +Build are optional image-level installs; T3 Code runs on demand via the `/t3` skill +or direct `npx`. Authenticate at least one harness before use. > **Simplest cross-provider login — device mode via `/login`.** The most straightforward path > that works the same across most harnesses: launch the agent in **interactive mode**, run From 744e5c34b1531b63a7fe9f35c893550f8dac7699 Mon Sep 17 00:00:00 2001 From: Ryan Eggleston Date: Mon, 31 Aug 2026 14:37:54 -0600 Subject: [PATCH 07/23] FROM task/906-tools-out-of-image TO development (#907) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * task: provision the default harnesses into the home mount oh is meant to split by execution target -- on the host it provisions the host or the sandbox, and inside the sandbox it provisions the sandbox with harnesses and tools. The second half did not work for the two harnesses most people use. claude-code and codex carried installUser: "root", which against the local execution target becomes sudo -n -- npm install -g, and /etc/sudoers.d/sandbox grants sandbox ALL=(ALL) ALL with no NOPASSWD. sudo -n true returns "a password is required". Both now match the pi entry directly above them: installUser "sandbox", npm --prefix /home/sandbox/.local install -g. That lands them inside the home mount, so they also survive container recreate and can be upgraded in place in a running remote sandbox rather than requiring an image rebuild. claude-code deliberately does not get --ignore-scripts. Its postinstall copies the native binary over a placeholder; with the flag the install succeeds and claude --version then fails with "claude native binary not installed". Verified both ways against a scratch prefix. provision-harnesses.sh follows provision-python.sh: the same mode flag, the same root to gosu sandbox re-exec, the same ownership diagnostics, the same die-with- the-command-to-re-run style. --print-env is absent because this provisioner exports nothing downstream. It reads the catalog through oh harness list --json and installs through oh harness install, so the shell knows no ids, packages, prefixes, or argv, and the TypeScript catalog stays the only description. No default harness carries a version pin today, so an existing install is never replaced and the script says so in its own output rather than implying it refreshes. The entrypoint hook runs after link-providers.sh, not before. link-providers' only binary dependency is cc-safety-net, which stays baked, and it is the boot-critical hard gate; a network-dependent best-effort step does not belong in front of the step that decides whether the boot is viable. Provisioning warns and continues, so an offline sandbox still comes up as a usable shell. BAKE_HARNESSES defaults to true and gates only the $AGENTS loop, not the whole RUN. INSTALL_OPENCODE and INSTALL_GROK_BUILD are separate opt-ins and turning them off as a side effect would be a silent regression. Nothing leaves the image in this change. * fix: bound the boot path and close four more provisioning defects An adversarial audit of #903 found five defects that six green checks missed. The serious one is a boot hang. oh harness list --json probes every entry in the catalog, not just the three defaults, and one of them is t3code, whose verifyArgv is npx --no-install t3 --version. npx contacts the registry, and probeInstalled passed no timeoutMs, so spawnSync waited without bound. Against an unreachable registry the auditor's run was still going at 2m30 when their own timeout killed it. On any boot where DNS resolves but the registry does not answer, the entrypoint blocks before sleep infinity, exceeds the 300s start_period, and never goes healthy -- and restart: unless-stopped does not rescue an unhealthy-but-alive container. Warn-and-continue cannot help, because a hang never reaches the if !. It hangs while listing, before any install, so BAKE_HARNESSES=true did not avoid it either. Bounded at three layers, because each fails differently: a 15s timeoutMs on the probe spawn, reported as unknown rather than a crash; a --defaults filter on oh harness list so the boot path probes three entries instead of nine and never runs npx; and a timeout wrapper on the entrypoint call so the boot is bounded whatever the CLI does. Measured against the auditor's exact command: 2m30 and killed, to 17.0s full-catalog and 1.55s with --defaults. The install loop read from a herestring while installs run with stdio inherit, so an installer that reads stdin consumed the rest of the loop. Reproduced with a stub: three missing harnesses, one installed, exit 0, success printed. Latent with npm, live the moment a default uses the curl | bash shape two catalog entries already use. Installs now read from /dev/null. The script force-exported OH_EXECUTION_TARGET=local, which short-circuits the in-container check, while the prefix is hardcoded to /home/sandbox/.local, and every error told the operator to re-run with no mention of where. On the host that provisioned the host. It now refuses unless inside the sandbox, reusing the CLI's own runningInsideSandbox predicate rather than inventing a check, and the entrypoint asserts the local target explicitly -- the documented raw docker run recipe never passes SANDBOX_NAME, so the guard would otherwise have silently skipped provisioning for the prebuilt-image flavor. The probe asserted that ARG BAKE_HARNESSES was declared, not that anything used it: deleting the gate left it green. It now checks the ARG is referenced by the RUN that installs $AGENTS and by the one that bakes pi, and that both stages declare it. Deleting either declaration also used to pass. ARG is stage-scoped, so BAKE_HARNESSES=false unbaked claude-code and codex but left pi baked in the home stage while the else-branch claimed otherwise. The home stage now declares and honors the flag. Also: OH_SANDBOX_USER was advertised but illusory, since the catalog hardcodes the user and prefix; it is gone. The final log line no longer claims to have provisioned anything in --verify mode. * task: stop baking the default harnesses into the sandbox image PR #903 wired provision-harnesses.sh into the boot path but shipped it behind ARG BAKE_HARNESSES=true, so every default harness was already present when the provisioner ran and the install path never executed. All four defects that PR's audit found lived in code a green CI run and a normal boot both skip. Delete the bake rather than flip its default: remove ARG BAKE_HARNESSES, ARG AGENTS, the PKG map and the $AGENTS loop in `base`, and the gated pi install in `home`. A build arg that can re-bake is a dormant path that would restore both the shadowed /usr/lib/node_modules copy and the untested boot install. Make the install path CI-visible, since it is now load-bearing on every boot: - The boot smoke asserts the outcome — each default harness resolves under NPM_USER_PREFIX via `type -P`, is owned by the reconciled sandbox uid, and prints its own version — and refuses to pass when the catalog reports no defaults. It boots on a fresh home volume, so this runs real npm work. - verify-sandbox-image.sh gains the negative: reading the catalog out of the image itself, no kind:"default" harness may be installed. - start_period goes 300s -> 600s in both compose files to cover the install, and the boot-guard probe now derives the smoke deadline from the healthcheck window instead of pinning a literal that a start_period bump could invert. - The probe and unit assertions invert from "the bake is gated" to "no default harness package appears in the Dockerfile", reading the package names out of installArgv so they cannot drift from the catalog. cc-safety-net stays baked. Opt-in INSTALL_* harnesses are untouched. Costs this accepts, documented in installation.md: a first boot on a fresh home mount needs network and runs 60-180s longer; an offline first boot yields a usable shell with no agent CLIs; ~/.npm now lives in the home mount. Closes #904 * task: cite the measured first-boot provisioning time * task: move herdr and cloudflared out of the image into the tool catalog Per the ownership boundary — the in-sandbox CLI provisions harnesses and tools — herdr and cloudflared are tools, so the image should not carry them. #905 did this for the default harnesses; this does it for the default tools. The obvious template does not work. #897's tailscale entry root-installs to /usr/local/bin, and commands/tool.ts:309 passes stdio:"inherit", so local-target.ts:113-116 selects the INTERACTIVE branch — plain `sudo --`, no -n. /etc/sudoers.d/sandbox grants `sandbox ALL=(ALL) ALL` with no NOPASSWD, so `oh tool install ` hangs on a password prompt no agent can answer. Verified in a running sandbox: `sudo -n -- true` → "a password is required". (#897's tailscale has the same defect; flagged there, not fixed here.) So install to ~/.local/bin as the sandbox user instead, the same correction #900 made for the harnesses. No sudo, survives container recreation in the home mount, and upgradeable in place by a running sandbox. - ToolKind gains "default". herdr 0.7.4 and cloudflared 2026.8.2 become kind:"default", installUser:"sandbox", with per-arch pinned URLs and sha256 verification into $NPM_USER_PREFIX/bin. Checksums measured by downloading both arches, not copied from anywhere. - provision-harnesses.sh generalizes over both catalogs and becomes provision-defaults.sh (OH_PROVISION_DEFAULTS, timeout 180s → 240s). It dies rather than reporting success when neither catalog yields a default. - The Dockerfile loses the herdr RUN, ARG HERDR_VERSION, and the whole cloudflared apt block — with it the bookworm-suite workaround that existed only because Cloudflare publishes no trixie suite. Docker's is now the only third-party apt source. - Both oracles generalize: verify-sandbox-image.sh rejects a baked default harness OR tool, reading each catalog out of the image; the boot smoke asserts every default in both catalogs resolves under NPM_USER_PREFIX, is owned by the sandbox uid, and prints a version. - The herdr version+checksum pin moves from the Dockerfile to the catalog, and herdr-default.test.ts follows it. Costs, documented in installation.md: an offline first boot on a fresh home mount now has no herdr, so `oh shell` lands in a plain shell with tmux as the fallback multiplexer. The entrypoint says so explicitly on failure. Closes #906 --- .devcontainer/Dockerfile | 21 -- .devcontainer/docker-compose.image-only.yml | 10 +- .devcontainer/docker-compose.yml | 10 +- .devcontainer/entrypoint.sh | 9 +- .github/workflows/sandbox-boot-guard.yml | 4 +- .oh/cli/src/__tests__/harness-catalog.test.ts | 10 +- .oh/cli/src/__tests__/tool-catalog.test.ts | 53 ++++- .oh/cli/src/cli.ts | 14 +- .oh/cli/src/commands/tool.ts | 17 +- .oh/cli/src/lib/tools/catalog.ts | 59 ++++- .oh/evals/RESULTS.md | 206 +++++++++--------- ...rovisioning.sh => default-provisioning.sh} | 76 +++++-- .oh/evals/probes/sandbox-boot-guard-ci.sh | 4 +- .oh/scripts/__tests__/herdr-default.test.ts | 19 +- .../__tests__/sandbox-base-image.test.ts | 24 +- .../__tests__/sandbox-boot-smoke.test.ts | 26 ++- .../__tests__/sandbox-healthcheck.test.ts | 2 +- .../__tests__/verify-sandbox-image.test.ts | 72 +++--- ...ion-harnesses.sh => provision-defaults.sh} | 98 +++++---- .oh/scripts/sandbox-boot-smoke.sh | 37 ++-- .oh/scripts/verify-sandbox-image.sh | 100 +++------ CHANGELOG.md | 3 + docs/deployment-prebuilt-image.md | 2 +- docs/installation.md | 36 +-- docs/integrations/herdr.md | 2 +- docs/quickstart.md | 5 +- 26 files changed, 540 insertions(+), 379 deletions(-) rename .oh/evals/probes/{harness-home-provisioning.sh => default-provisioning.sh} (52%) rename .oh/scripts/{provision-harnesses.sh => provision-defaults.sh} (59%) diff --git a/.devcontainer/Dockerfile b/.devcontainer/Dockerfile index ffc38e4b..1207ca0e 100644 --- a/.devcontainer/Dockerfile +++ b/.devcontainer/Dockerfile @@ -28,29 +28,8 @@ RUN install -m 0755 -d /etc/apt/keyrings \ && apt-get update && apt-get install -y --no-install-recommends docker-ce-cli docker-compose-plugin \ && rm -rf /var/lib/apt/lists/* -# Cloudflare publishes no trixie suite (pkg.cloudflare.com/cloudflared/dists/trixie returns HTTP 404); -# the bookworm package is compatible with trixie, so this suite stays pinned to bookworm. -RUN curl -fsSL https://pkg.cloudflare.com/cloudflare-main.gpg \ - -o /usr/share/keyrings/cloudflare-main.gpg \ - && echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/cloudflare-main.gpg] https://pkg.cloudflare.com/cloudflared bookworm main" \ - > /etc/apt/sources.list.d/cloudflared.list \ - && apt-get update && apt-get install -y --no-install-recommends cloudflared \ - && rm -rf /var/lib/apt/lists/* - RUN BUN_INSTALL=/usr/local curl -fsSL https://bun.sh/install | bash -ARG HERDR_VERSION=0.7.4 -RUN case "$(dpkg --print-architecture)" in \ - amd64) herdr_arch=x86_64; herdr_sha=bc0fc02d4ba500f9cac2353a43e67fe036785ecca6eb55378e050fac3c103059 ;; \ - arm64) herdr_arch=aarch64; herdr_sha=544e0002de42806d1ab64ccdef3a7e7414f24717b0b6b022bc9e57d2eefd26a2 ;; \ - *) echo "Unsupported Herdr architecture: $(dpkg --print-architecture)" >&2; exit 1 ;; \ - esac \ - && curl -fsSL "https://github.com/ogulcancelik/herdr/releases/download/v${HERDR_VERSION}/herdr-linux-${herdr_arch}" \ - -o /usr/local/bin/herdr \ - && echo "${herdr_sha} /usr/local/bin/herdr" | sha256sum -c - \ - && chmod 0755 /usr/local/bin/herdr \ - && test "$(herdr --version)" = "herdr ${HERDR_VERSION}" - RUN curl -LsSf https://astral.sh/uv/install.sh | env INSTALLER_NO_MODIFY_PATH=1 sh \ && cp /root/.local/bin/uv /usr/local/bin/uv \ && cp /root/.local/bin/uvx /usr/local/bin/uvx diff --git a/.devcontainer/docker-compose.image-only.yml b/.devcontainer/docker-compose.image-only.yml index 1ef9f6b1..6328e8e4 100644 --- a/.devcontainer/docker-compose.image-only.yml +++ b/.devcontainer/docker-compose.image-only.yml @@ -57,11 +57,11 @@ services: interval: 30s timeout: 10s retries: 3 - # Boot now installs the default harnesses into the home mount instead of - # unpacking them from the image (entrypoint.sh, bounded by - # OH_PROVISION_HARNESSES_TIMEOUT). A cold first boot on a fresh home mount - # therefore spends up to 180s on npm before the control plane is reachable. - # 300s left ~120s for the rest of boot; 600s keeps the same headroom. + # Boot installs the default harnesses AND the default tools (herdr, + # cloudflared) into the home mount instead of unpacking them from the + # image (entrypoint.sh, bounded by OH_PROVISION_DEFAULTS_TIMEOUT). A cold + # first boot on a fresh home mount therefore spends up to 240s downloading + # before the control plane is reachable; 600s leaves room for the rest. start_period: 600s restart: unless-stopped diff --git a/.devcontainer/docker-compose.yml b/.devcontainer/docker-compose.yml index 3cd0fba0..8a99d972 100644 --- a/.devcontainer/docker-compose.yml +++ b/.devcontainer/docker-compose.yml @@ -83,11 +83,11 @@ services: interval: 30s timeout: 10s retries: 3 - # Boot now installs the default harnesses into the home mount instead of - # unpacking them from the image (entrypoint.sh, bounded by - # OH_PROVISION_HARNESSES_TIMEOUT). A cold first boot on a fresh home mount - # therefore spends up to 180s on npm before the control plane is reachable. - # 300s left ~120s for the rest of boot; 600s keeps the same headroom. + # Boot installs the default harnesses AND the default tools (herdr, + # cloudflared) into the home mount instead of unpacking them from the + # image (entrypoint.sh, bounded by OH_PROVISION_DEFAULTS_TIMEOUT). A cold + # first boot on a fresh home mount therefore spends up to 240s downloading + # before the control plane is reachable; 600s leaves room for the rest. start_period: 600s restart: unless-stopped diff --git a/.devcontainer/entrypoint.sh b/.devcontainer/entrypoint.sh index cbe342a9..f1d13501 100644 --- a/.devcontainer/entrypoint.sh +++ b/.devcontainer/entrypoint.sh @@ -150,10 +150,11 @@ if [ -x "$HARNESS/.oh/scripts/link-providers.sh" ]; then fi fi -if [ "${OH_PROVISION_HARNESSES:-true}" = "true" ] \ - && [ -x "$HARNESS/.oh/scripts/provision-harnesses.sh" ]; then - if ! OH_EXECUTION_TARGET=local timeout "${OH_PROVISION_HARNESSES_TIMEOUT:-180}" bash "$HARNESS/.oh/scripts/provision-harnesses.sh"; then - echo "[entrypoint] WARNING: harness provisioning did not complete; run: bash .oh/scripts/provision-harnesses.sh" >&2 +if [ "${OH_PROVISION_DEFAULTS:-true}" = "true" ] \ + && [ -x "$HARNESS/.oh/scripts/provision-defaults.sh" ]; then + if ! OH_EXECUTION_TARGET=local timeout "${OH_PROVISION_DEFAULTS_TIMEOUT:-240}" bash "$HARNESS/.oh/scripts/provision-defaults.sh"; then + echo "[entrypoint] WARNING: default provisioning did not complete; run: bash .oh/scripts/provision-defaults.sh" >&2 + echo "[entrypoint] WARNING: herdr may be unavailable — 'tmux' still works as a fallback multiplexer" >&2 fi fi diff --git a/.github/workflows/sandbox-boot-guard.yml b/.github/workflows/sandbox-boot-guard.yml index f4b4c9f8..1591267e 100644 --- a/.github/workflows/sandbox-boot-guard.yml +++ b/.github/workflows/sandbox-boot-guard.yml @@ -134,8 +134,8 @@ jobs: # This step is the ONLY place the boot-time harness install runs end to end. # It boots on a fresh home volume (the smoke tears down with `down -v`), so - # provision-harnesses.sh does real npm work every run and the smoke asserts - # the result. Do not set OH_PROVISION_HARNESSES=false here to speed it up — + # provision-defaults.sh does real npm work every run and the smoke asserts + # the result. Do not set OH_PROVISION_DEFAULTS=false here to speed it up — # that would return the install path to being untested dead code. - name: Boot sandbox and verify healthcheck (exercises boot-time harness provisioning) env: diff --git a/.oh/cli/src/__tests__/harness-catalog.test.ts b/.oh/cli/src/__tests__/harness-catalog.test.ts index b6424492..47a62bd8 100644 --- a/.oh/cli/src/__tests__/harness-catalog.test.ts +++ b/.oh/cli/src/__tests__/harness-catalog.test.ts @@ -158,8 +158,8 @@ describe("harness catalog", () => { }); it("provisions the default harnesses at boot rather than baking them", () => { - expect(ENTRYPOINT).toContain("OH_PROVISION_HARNESSES"); - expect(ENTRYPOINT).toContain(".oh/scripts/provision-harnesses.sh"); + expect(ENTRYPOINT).toContain("OH_PROVISION_DEFAULTS"); + expect(ENTRYPOINT).toContain(".oh/scripts/provision-defaults.sh"); }); it.each(defaults.map((h) => [h.id, h] as const))( @@ -169,7 +169,7 @@ describe("harness catalog", () => { expect(pkg, `${id} declares no install package`).toMatch(/^(@[^/]+\/)?[^-].*/); expect( DOCKERFILE_CODE, - `${id} is baked into the image; it belongs to provision-harnesses.sh`, + `${id} is baked into the image; it belongs to provision-defaults.sh`, ).not.toContain(pkg); }, ); @@ -180,9 +180,9 @@ describe("harness catalog", () => { it("bounds the boot-path provisioner so an unreachable registry cannot stall the entrypoint", () => { expect(ENTRYPOINT).toMatch( - /timeout "\$\{OH_PROVISION_HARNESSES_TIMEOUT:-\d+\}" bash "\$HARNESS\/\.oh\/scripts\/provision-harnesses\.sh"/, + /timeout "\$\{OH_PROVISION_DEFAULTS_TIMEOUT:-\d+\}" bash "\$HARNESS\/\.oh\/scripts\/provision-defaults\.sh"/, ); - expect(ENTRYPOINT).toContain("WARNING: harness provisioning did not complete"); + expect(ENTRYPOINT).toContain("WARNING: default provisioning did not complete"); }); }); diff --git a/.oh/cli/src/__tests__/tool-catalog.test.ts b/.oh/cli/src/__tests__/tool-catalog.test.ts index 37e483f2..1e72346a 100644 --- a/.oh/cli/src/__tests__/tool-catalog.test.ts +++ b/.oh/cli/src/__tests__/tool-catalog.test.ts @@ -25,8 +25,27 @@ describe("tool catalog shape", () => { ]); }); - it("has exactly one installable tool", () => { - expect(installableToolIds()).toEqual(["agent-browser"]); + it("makes exactly the default and opt-in tools installable", () => { + expect(installableToolIds()).toEqual(["agent-browser", "herdr", "cloudflared"]); + for (const t of TOOL_CATALOG) { + // A kind:"default" tool is provisioned at boot through `oh tool install`, + // so it MUST be installable; a baked-in one must not be. + if (t.kind === "default") expect(t.installArgv, t.id).toBeDefined(); + if (t.kind === "baked-in") expect(t.installArgv, t.id).toBeUndefined(); + } + }); + + // #906: commands/tool.ts installs with stdio:"inherit", so local-target.ts + // picks plain `sudo --` for a root install — and /etc/sudoers.d/sandbox has + // no NOPASSWD. A root-installed default would hang an agent on a password + // prompt, and could not be upgraded by the running sandbox afterwards. + it("installs every default tool as the sandbox user into the home mount", () => { + for (const t of TOOL_CATALOG) { + if (t.kind !== "default") continue; + expect(t.installUser, t.id).toBe("sandbox"); + expect(t.installArgv!.join("\n"), t.id).toContain("NPM_USER_PREFIX"); + expect(t.installArgv!.join("\n"), t.id).toContain("sha256sum -c -"); + } }); it("makes every non-installable tool say why", () => { @@ -49,17 +68,29 @@ describe("tool catalog shape", () => { it("declares a version probe only where the flag is a safe standard", () => { const withVersion = TOOL_CATALOG.filter((t) => t.versionArgv !== undefined).map((t) => t.id); - expect(withVersion).toEqual(["cloudflared", "docker-cli", "gh"]); + expect(withVersion).toEqual(["herdr", "cloudflared", "docker-cli", "gh"]); for (const t of TOOL_CATALOG) { if (t.versionArgv) expect(t.versionArgv, t.id).toEqual([t.binary, "--version"]); } }); it("passes argv arrays with no interpolation this process performs", () => { + // The hazard is a JS template literal that Node expands before the argv + // ever reaches a shell. A `bash -lc` script body legitimately contains + // ${...} for the shell IN the container to expand, so exempt that one + // token and forbid backticks in the catalog source instead. + expect( + read(".oh/cli/src/lib/tools/catalog.ts"), + "a template literal with ${...} would be expanded by Node before any shell sees it", + ).not.toMatch(/`[^`]*\$\{/s); for (const t of TOOL_CATALOG) { for (const argv of [t.installArgv, t.verifyArgv, t.versionArgv]) { if (!argv) continue; - for (const token of argv) expect(token, `${t.id}: ${token}`).not.toContain("${"); + const shellBody = argv[0] === "bash" && argv[1] === "-lc" ? 2 : -1; + argv.forEach((token, i) => { + if (i === shellBody) return; + expect(token, `${t.id}: ${token}`).not.toContain("${"); + }); } } }); @@ -153,8 +184,20 @@ describe("baked-in tools", () => { it("are each actually in the Dockerfile", () => { const dockerfile = read(".devcontainer/Dockerfile"); + const baked = TOOL_CATALOG.filter((t) => t.kind === "baked-in"); + expect(baked.length, "no baked-in tool left to check").toBeGreaterThan(0); + for (const t of baked) { + expect(dockerfile, t.id).toContain(t.binary); + } + }); + + // #906: herdr and cloudflared moved to kind:"default". The inverse of the + // check above — a default tool must NOT be in the Dockerfile — lives in + // .oh/evals/probes/default-provisioning.sh, which matches on the pinned + // project URL rather than the bare binary name. + it("no longer claims herdr or cloudflared", () => { for (const id of ["herdr", "cloudflared"]) { - expect(dockerfile, id).toContain(id); + expect(findTool(id)!.kind, id).toBe("default"); } }); }); diff --git a/.oh/cli/src/cli.ts b/.oh/cli/src/cli.ts index 733ab131..46886fd3 100644 --- a/.oh/cli/src/cli.ts +++ b/.oh/cli/src/cli.ts @@ -370,7 +370,7 @@ runtime (see \`oh runtime\`) — a headless browser, a tunnel client, the GitHub CLI. Usage: - oh tool list List known tools and their state + oh tool list [--defaults] List known tools and their state oh tool status [name] Show installed state and version oh tool install Install a tool into the sandbox @@ -387,6 +387,7 @@ Flags: --no-persist Live-install only; leave oh.json unchanged --yes Accept a large download without prompting --json Machine-readable output (list/status) + --defaults List only kind:"default" tools (list) Tools: ${toolIds().map((t) => ` ${t}`).join("\n")} @@ -943,6 +944,7 @@ interface ToolArgs { noPersist: boolean; yes: boolean; json: boolean; + defaultsOnly: boolean; subcommand?: "list" | "install" | "status"; name?: string; } @@ -950,6 +952,7 @@ interface ToolArgs { export function parseToolArgs(rest: string[]): ParseResult { const args: ToolArgs = { help: false, persistOnly: false, noPersist: false, yes: false, json: false, + defaultsOnly: false, }; if (rest.length === 0 || isHelpFlag(rest[0])) { return { ok: true, args: { ...args, help: true } }; @@ -961,6 +964,7 @@ export function parseToolArgs(rest: string[]): ParseResult { else if (token === "--no-persist") args.noPersist = true; else if (token === "--yes" || token === "-y") args.yes = true; else if (token === "--json") args.json = true; + else if (token === "--defaults") args.defaultsOnly = true; else if (token.startsWith("-")) { return { ok: false, error: `oh tool: unknown flag "${token}"` }; } else positionals.push(token); @@ -989,6 +993,12 @@ export function parseToolArgs(rest: string[]): ParseResult { error: "oh tool: --persist-only conflicts with --no-persist — pass at most one", }; } + if (args.defaultsOnly && sub !== "list") { + return { + ok: false, + error: `oh tool ${sub}: --defaults applies to \`oh tool list\` only`, + }; + } args.subcommand = sub; if (name !== undefined) args.name = name; @@ -1405,7 +1415,7 @@ async function main(argv: string[]): Promise { stderr: (s) => process.stderr.write(s), }; if (a.subcommand === "list") { - return await runToolList({ json: a.json }, io); + return await runToolList({ json: a.json, defaultsOnly: a.defaultsOnly }, io); } if (a.subcommand === "status") { return await runToolStatus(a.name, { json: a.json }, io); diff --git a/.oh/cli/src/commands/tool.ts b/.oh/cli/src/commands/tool.ts index f181db8e..02af4298 100644 --- a/.oh/cli/src/commands/tool.ts +++ b/.oh/cli/src/commands/tool.ts @@ -13,6 +13,7 @@ import { setInstallFlag, } from "../lib/env-file.js"; import { + defaultTools, findTool, installableToolIds, toolIds, @@ -32,6 +33,7 @@ export interface ToolOptions { cwd?: string; run?: LifecycleRunner; json?: boolean; + defaultsOnly?: boolean; env?: NodeJS.ProcessEnv; } @@ -44,6 +46,7 @@ export interface ToolInstallOptions extends ToolOptions { interface ToolRow { id: string; title: string; + binary: string; kind: string; enabled: boolean | null; installed: boolean | null; @@ -107,9 +110,9 @@ async function collectRows( root: string, run: LifecycleRunner, env?: NodeJS.ProcessEnv, - only?: ToolEntry, + only?: readonly ToolEntry[], ): Promise { - const entries = only ? [only] : [...TOOL_CATALOG]; + const entries = only ? [...only] : [...TOOL_CATALOG]; const target = targetFor(root, run, env); let reachable = false; @@ -125,6 +128,7 @@ async function collectRows( rows.push({ id: entry.id, title: entry.title, + binary: entry.binary, kind: entry.kind, enabled: entry.toolKey === undefined ? null : isInstallFlagEnabled(root, entry.toolKey), @@ -175,7 +179,12 @@ function renderDetail(rows: ToolRow[], io: ToolIO): void { export async function runToolList(opts: ToolOptions, io: ToolIO): Promise { const run = opts.run ?? spawnRunner; const root = resolveProjectRoot(opts.cwd); - const rows = await collectRows(root, run, opts.env); + const rows = await collectRows( + root, + run, + opts.env, + opts.defaultsOnly === true ? defaultTools() : undefined, + ); if (opts.json) { io.stdout(`${JSON.stringify(rows, null, 2)}\n`); } else { @@ -204,7 +213,7 @@ export async function runToolStatus( if (!only) return unknownTool(name, io); } - const rows = await collectRows(root, run, opts.env, only); + const rows = await collectRows(root, run, opts.env, only ? [only] : undefined); if (opts.json) { io.stdout(`${JSON.stringify(only ? rows[0] : rows, null, 2)}\n`); } else { diff --git a/.oh/cli/src/lib/tools/catalog.ts b/.oh/cli/src/lib/tools/catalog.ts index aebb3e84..fac0d638 100644 --- a/.oh/cli/src/lib/tools/catalog.ts +++ b/.oh/cli/src/lib/tools/catalog.ts @@ -1,6 +1,7 @@ export type ToolKind = | "baked-in" + | "default" | "opt-in"; export interface ToolEntry { @@ -42,22 +43,64 @@ export const TOOL_CATALOG: readonly ToolEntry[] = Object.freeze([ Object.freeze({ id: "herdr", title: "Herdr", - kind: "baked-in", + kind: "default", binary: "herdr", verifyArgv: Object.freeze(["bash", "-lc", "command -v herdr >/dev/null"]), - notInstallableReason: - "herdr is installed in the base image with a pinned, sha256-verified binary (.devcontainer/Dockerfile). Rebuild the image to change it.", + versionArgv: Object.freeze(["herdr", "--version"]), + installArgv: Object.freeze([ + "bash", + "-lc", + [ + "set -e", + 'version=0.7.4', + 'case "$(dpkg --print-architecture)" in', + " amd64) arch=x86_64; sha=bc0fc02d4ba500f9cac2353a43e67fe036785ecca6eb55378e050fac3c103059 ;;", + " arm64) arch=aarch64; sha=544e0002de42806d1ab64ccdef3a7e7414f24717b0b6b022bc9e57d2eefd26a2 ;;", + ' *) echo "no pinned Herdr build for $(dpkg --print-architecture)" >&2; exit 1 ;;', + "esac", + 'prefix="${NPM_USER_PREFIX:-$HOME/.local}"', + 'tmp="$(mktemp -d)"', + "trap 'rm -rf \"$tmp\"' EXIT", + 'curl -fsSL "https://github.com/ogulcancelik/herdr/releases/download/v$version/herdr-linux-$arch" -o "$tmp/herdr"', + 'echo "$sha $tmp/herdr" | sha256sum -c -', + 'install -d "$prefix/bin"', + 'install -m 0755 "$tmp/herdr" "$prefix/bin/herdr"', + 'test "$("$prefix/bin/herdr" --version)" = "herdr $version"', + ].join("\n"), + ]), + installUser: "sandbox", docsPath: TOOLS_DOC, }), Object.freeze({ id: "cloudflared", title: "cloudflared", - kind: "baked-in", + kind: "default", binary: "cloudflared", verifyArgv: Object.freeze(["bash", "-lc", "command -v cloudflared >/dev/null"]), versionArgv: Object.freeze(["cloudflared", "--version"]), - notInstallableReason: - "cloudflared is installed in the base image from Cloudflare's apt repository (.devcontainer/Dockerfile). Rebuild the image to change it.", + installArgv: Object.freeze([ + "bash", + "-lc", + [ + "set -e", + "version=2026.8.2", + 'case "$(dpkg --print-architecture)" in', + " amd64) sha=fcfb02b575a52ca1af2e3267af4e1517bcdeb30ac48c834c69abaed3c0576ad2 ;;", + " arm64) sha=7747d94570fb390cf47dcb4f9555c193c6355cda9793f0d878d9049e5d6a7790 ;;", + ' *) echo "no pinned cloudflared build for $(dpkg --print-architecture)" >&2; exit 1 ;;', + "esac", + 'arch="$(dpkg --print-architecture)"', + 'prefix="${NPM_USER_PREFIX:-$HOME/.local}"', + 'tmp="$(mktemp -d)"', + "trap 'rm -rf \"$tmp\"' EXIT", + 'curl -fsSL "https://github.com/cloudflare/cloudflared/releases/download/$version/cloudflared-linux-$arch" -o "$tmp/cloudflared"', + 'echo "$sha $tmp/cloudflared" | sha256sum -c -', + 'install -d "$prefix/bin"', + 'install -m 0755 "$tmp/cloudflared" "$prefix/bin/cloudflared"', + '"$prefix/bin/cloudflared" --version >/dev/null', + ].join("\n"), + ]), + installUser: "sandbox", docsPath: TOOLS_DOC, }), Object.freeze({ @@ -92,6 +135,10 @@ export function toolIds(): string[] { return TOOL_CATALOG.map((t) => t.id); } +export function defaultTools(): readonly ToolEntry[] { + return TOOL_CATALOG.filter((t) => t.kind === "default"); +} + export function installableToolIds(): string[] { return TOOL_CATALOG.filter((t) => t.installArgv !== undefined).map((t) => t.id); } diff --git a/.oh/evals/RESULTS.md b/.oh/evals/RESULTS.md index 0d1ab988..319a405a 100644 --- a/.oh/evals/RESULTS.md +++ b/.oh/evals/RESULTS.md @@ -6,108 +6,108 @@ probe id; git history is the time series.** Schema and exit-code semantics are i | probe | tier | last-run (UTC) | status | source | |-------|------|----------------|--------|--------| -| advisor-monitored-loop | A | 2026-08-31 02:05 | PASS | conversation 2026-06-19 (single-owner implementation workflow, issue #257) | -| agent-browser-cli | A | 2026-08-31 02:05 | PASS | retro lesson 2026-06-07 (agent-browser 0.8.5 CLI) | -| agents-identity-contract | A | 2026-08-31 02:05 | PASS | issue #854 — T3-style root identity, glossary, and skill-owned procedures | -| artifact-contract-audit | A | 2026-08-31 02:05 | PASS | issue #583/#645 — production /audit implementation Gate 1 behavior | -| audit-dispatcher-contract | A | 2026-08-31 02:05 | PASS | issue #645 — audit consolidation public taxonomy | -| audit-implementation-behavior | A | 2026-08-31 02:05 | PASS | issue #645 — implementation root/repo/browser behavior | -| audit-pr-acquire | A | 2026-08-31 02:05 | PASS | issue #645 — production PR acquisition behavior | -| audit-pr-classifier | A | 2026-08-31 02:05 | PASS | issue #645 — deterministic focused and queue PR classifier | -| audit-run-root-contract | A | 2026-08-31 02:05 | PASS | issue #645 — executable immutable audit root/run correlation | -| audit-shellcheck-coverage | A | 2026-08-31 02:05 | PASS | issue #645 — private audit scripts require release and CI lint coverage | -| audit-stale-references | A | 2026-08-31 02:05 | PASS | issue #645 — clean-breaking audit migration | -| boot-lint-glob | A | 2026-08-31 02:05 | PASS | issue #90, issue #120 | -| builder-skill-consolidation | A | 2026-08-31 02:05 | PASS | issue #643 — consolidate artifact builders behind one /builder dispatcher | -| capability-benchmark-schema | A | 2026-08-31 02:05 | PASS | issue #167 — capability benchmark instrument | -| cc-safety-net-wiring | A | 2026-08-31 02:05 | SKIPPED | .oh/tasks/cc-safety-net/prd.json US-007 2026-07-19 | -| changelog-entry-length | A | 2026-08-31 02:05 | PASS | conversation 2026-08-24 — CHANGELOG.md grew to 259KB of bullet prose because "one line" was unquantified | -| cleanup-tasks-scoped-guard | A | 2026-08-31 02:05 | PASS | issue #85 | -| cleanup-tasks-worktree-grooming | A | 2026-08-31 02:05 | PASS | issue #168; issue #327 | -| cli-publish-typecheck-scope | A | 2026-08-31 02:05 | PASS | release run 33271077312 — v0.5.0 pushed its GHCR image, then failed to publish | -| close-issues-on-development | A | 2026-08-31 02:05 | PASS | issue #841 (closing keywords never fire because the default branch is main) 2026-08-26 | -| codex-stale-response-retry | A | 2026-08-31 02:05 | PASS | issue #506 — Codex previous_response_not_found RCA | -| compose-config-path-parity | A | 2026-08-31 02:05 | PASS | PR #833 (remove harness.yaml — the wrapper and VS Code "Reopen in Container" paths must resolve the same service) 2026-08-26 | -| config-schema-parity | A | 2026-08-31 02:05 | PASS | PR #833 (one schema file — DOCKER_SOCKET, SANDBOX_SSH, OH_SANDBOX_IMAGE, OH_PULL_POLICY, SKIP_PNPM_INSTALL were consumed but undocumented); rewritten for the oh.json/secrets split by PR #887 | -| context-tier-size-budget | A | 2026-08-31 02:05 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-007) — the always-on tier was 85,256 B | -| cron-claude-codex-fallback | A | 2026-08-31 02:05 | PASS | conversation 2026-06-12 (default Codex fallback for crons) | -| cron-watchdog | A | 2026-08-31 02:05 | PASS | issues #130/#453 (cron runtime watchdog + legacy system-cron reaping) 2026-06-19 | -| crons-directory-guide | A | 2026-08-31 02:05 | PASS | issue #874 | -| curl-bash-safe-alternatives | A | 2026-08-31 02:05 | PASS | vet-run/vet integration — public curl|bash examples need review-first alternatives | -| datasets-schema | A | 2026-08-31 02:05 | PASS | issue #196 — .oh/evals/datasets verifiable trajectory corpus (Repo2RLEnv-inspired) | -| debugmcp-availability | A | 2026-08-31 02:05 | SKIPPED | issue #297 — DebugMCP MCP debug-server availability | -| delegate-model-effort-policy | A | 2026-08-31 02:05 | PASS | conversation 2026-07-11 (delegate model inheritance and thinking policy) | -| devtcp-hook | A | 2026-08-31 02:05 | PASS | retro lesson 2026-06-10 (zsh /dev/tcp) | -| docker-inspect-env-guard | A | 2026-08-31 02:05 | PASS | operator directive 2026-08-08 (agents keep the docker socket, but must | -| docs-build-fast-path | A | 2026-08-31 02:05 | PASS | #455 — docs builds must stay out of fast harness/eval/release gates; #536 — docs site externalized to openharness-web; docs markdown relocated to docs/ | -| drift-check-cron-staleness-glob | A | 2026-08-31 02:05 | PASS | issue #98; issue #225 (restart-required cron frontmatter/config drift) | -| entrypoint-pnpm-manifest-fingerprint | A | 2026-08-31 02:05 | PASS | issue #521 (manifest-aware sandbox installs) 2026-07-01 | -| eval-ci-gate | A | 2026-08-31 02:05 | PASS | #103 — eval probe suite gated in CI | -| eval-gate | A | 2026-08-31 02:05 | PASS | retro lesson 2026-06-11 (eval-gate) | -| eval-results-atomic | A | 2026-08-31 02:05 | PASS | issue #83 (eval-results-atomic-write) | -| eval-runner-exit | A | 2026-08-31 02:05 | PASS | retro lesson 2026-06-11 (eval-runner-exit) #29 | -| eval-runs-once-per-cycle | A | 2026-08-31 02:05 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-006) — /eval ran 3x per cycle on the | -| execution-target-contract | A | 2026-08-31 02:05 | PASS | issue #733 (ExecutionTarget contract + Docker Compose adapter) 2026-08-10 | -| get-oh-bootstrap | A | 2026-08-31 02:05 | PASS | get-oh.sh bootstrap — the Node-bootstrapping host-side path to the standalone `oh` CLI (also on npm as @mifune/openharness; see oh-npm-package.sh) | -| git-skill | A | 2026-08-31 02:05 | PASS | conversation 2026-06-15 — rules are not always supported; git workflow must be the /git skill | -| harness-audit-empty-output-gate | A | 2026-08-31 02:05 | PASS | issue #246 — /audit harness must fail closed on empty auditor outputs | -| harness-ci-core-paths | A | 2026-08-31 02:05 | PASS | #165 — core sandbox config files must trigger harness CI | -| harness-ci-hooks-paths | A | 2026-08-31 02:05 | PASS | issue #202 — credential/security hook changes must trigger harness CI | -| harness-home-provisioning | A | 2026-08-31 02:05 | PASS | #902 — `oh harness install` must work from inside the sandbox, where | -| harness-yaml-migration | A | 2026-08-31 02:05 | PASS | PR #833 (migrate-harness-yaml.sh — append / uncomment-in-place / preserve / overwrite, plus a silent no-op second run) 2026-08-26 | -| health-check-docker-stats | A | 2026-08-31 02:05 | PASS | retro lesson 2026-06-10 (docker stats vs ps Size) | -| health-check-socket-degrade | A | 2026-08-31 02:05 | PASS | issue #762 (refs #756) — /health-check degrades to one statement, not nine failures | -| heartbeat-logging-contract | A | 2026-08-31 02:05 | PASS | issue #447 (heartbeat log append hardening) 2026-06-18 | -| image-seed-hygiene | A | 2026-08-31 02:05 | PASS | issue #900 (slim the sandbox image) 2026-08-30 | -| markitdown-wiki-ingest | A | 2026-08-31 02:05 | PASS | issue #649 — pinned local-document normalization contract for /wiki ingest | -| next-dev-prod | A | 2026-08-31 02:05 | SKIPPED | retro lesson 2026-06-04 | -| oh-compose-env-wiring | A | 2026-08-31 02:05 | PASS | issue #880 (oh as the only front door — oh.json is the non-secret config surface) | -| oh-config-surfaces | A | 2026-08-31 02:05 | REGRESSION | PR #887 (config split across two authored surfaces — a tracked oh.json and a secrets-only root dotenv — with nothing left under $HOME) | -| oh-destroy-guard | A | 2026-08-31 02:05 | PASS | issue #879 — `oh` becomes the only front door, so `make destroy` must | -| oh-devcontainer-restructure | A | 2026-08-31 02:05 | PASS | consolidate devcontainer — .oh/devcontainer/ folded back into .devcontainer/ | -| oh-home-mount | A | 2026-08-31 02:05 | PASS | issue #898 (single $HOME mount) 2026-08-30 | -| oh-image-only-deploy | A | 2026-08-31 02:05 | PASS | .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only deploy) | -| oh-init-headless-config | A | 2026-08-31 02:05 | PASS | PR #827 (installer answers landed in the losing config file); retargeted to the .example.env template by PR #833, then to oh.json by PR #887 | -| oh-init-scaffold | A | 2026-08-31 02:05 | PASS | issue #531 Phase 2 | -| oh-lifecycle-surface | A | 2026-08-31 02:05 | PASS | issue #881 — the Makefile is retired and `oh` is the only front door | -| oh-npm-package | A | 2026-08-31 02:05 | PASS | npm publish path for the standalone `oh` CLI (@mifune/openharness) — alternative to get-oh.sh | -| oh-payload-manifest | A | 2026-08-31 02:05 | PASS | issue #531 follow-on (.oh payload manifest — oh update ships a declared allowlist) | -| oh-sandbox-image-mode | A | 2026-08-31 02:05 | PASS | conversation 2026-07-05 (basic Docker deployment — prebuilt-image mode) | -| oh-shipped-repo-overridable | A | 2026-08-31 02:05 | PASS | issue #531 follow-on (de-hardcode residual — shipped .oh shell scripts keep the upstream repo overridable) | -| oh-standalone-lifecycle | A | 2026-08-31 02:05 | PASS | issue #564 | -| oh-update | A | 2026-08-31 02:05 | PASS | issue #531 Phase 3 (oh update — upgrade only the .oh control plane) | -| operator-config-guard | A | 2026-08-31 02:05 | PASS | operator directives 2026-08-06 (.config/ and settings.local.json are operator-only) | -| pnpm-audit-ci-gate | A | 2026-08-31 02:05 | PASS | issue #171 — pnpm security audits must run in CI | -| post-bridge-publish-confirmation | A | 2026-08-31 02:05 | PASS | #523 — post-bridge live publishing requires an explicit final confirmation gate | -| prd-output-path-contract | A | 2026-08-31 02:05 | PASS | retro lesson 2026-06-19 | -| prompt-miner-schema-compat | A | 2026-08-31 02:05 | PASS | issue #253 — prompt-miner JSONL schema-drift guard | -| prompt-miner-symlink-entrypoint | A | 2026-08-31 02:05 | PASS | issue #663 — prompt-miner engine no-ops via the documented .claude/skills symlink | -| prompt-miner-weakness-record | A | 2026-08-31 02:05 | PASS | issue #580 — prompt-miner weakness-record (WH-xxx) cluster output | -| protected-path-deletion | A | 2026-08-31 02:05 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-001) — the critique gate was deleted, | -| protected-paths-resolve | A | 2026-08-31 02:05 | PASS | issue #753 — .claude/protected-paths.txt named 7 paths that did not exist. | -| registry-portability-gate | A | 2026-08-31 02:05 | PASS | issue #758 | -| registry-portability | A | 2026-08-31 02:05 | SKIPPED | issue #758 | -| retro-deterministic-contract | A | 2026-08-31 02:05 | PASS | issue #443 — /retro deterministic output and self-contained helper contract | -| rl-delegation-write-worker | A | 2026-08-31 02:05 | PASS | retro lesson 2026-06-10 (rl-delegation) #57 | -| rlm-context-budget | A | 2026-08-31 02:05 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-006 | -| runtime-preflight-gate | A | 2026-08-31 02:05 | PASS | issue #806 § B1 (open sandbox.substrate vs sandbox.runtime selector); | -| sandbox-boot-guard-ci | A | 2026-08-31 02:05 | PASS | issue #449 (sandbox image build CI guard) 2026-06-19; | -| sandbox-node-base | A | 2026-08-31 02:05 | PASS | openharness#878 — oh as the only front door, T0 sandbox base image | -| skill-paths | A | 2026-08-31 02:05 | PASS | issue #43 — stale path references; extended by issue #69 — apps/->packages/ rename guard; extended by issue #870 — deleted .oh/agents/advisor.md | -| skills-dir-clean | A | 2026-08-31 02:05 | PASS | conversation 2026-06-29 — Pi parses every top-level `.md` in the skills | -| skills-task-tool-coupling | A | 2026-08-31 02:05 | PASS | council review 2026-08-29 (issue #886) — /delegate instructed Claude-Code-only | -| skills-vendored | A | 2026-08-31 02:05 | PASS | absorb .mifune submodule into .oh — the skills/agents/hooks pack is vendored | -| slack-admin-command-surface | A | 2026-08-31 02:05 | PASS | issue #354 — Slack bridge docs must distinguish Pi /msg-bridge commands from Slack DM admin text handlers | -| spec-family-contract | A | 2026-08-31 02:05 | PASS | issue #265; spec-simplification issue #816; workflow authority issue #854 | -| spec-ready-finalization | A | 2026-08-31 02:05 | PASS | issue #134; spec-simplification issue #816; workflow authority issue #854 | -| ste-checker-contract | A | 2026-08-31 02:05 | PASS | issue #750 PR audit — the /ste checker had four fail-open paths (unclosed | -| submitted-by-trailers | A | 2026-08-31 02:05 | PASS | conversation 2026-06-12 (commit attribution trailers); the single-owner | -| sync-skill-contract | A | 2026-08-31 02:05 | PASS | issue #331 — /sync dispatcher skill (bidirectional origin↔upstream sync) | -| tool-catalog-boundary | A | 2026-08-31 02:05 | PASS | agent-browser's exclusion from the harness catalog (#821) and the | -| version-parity | A | 2026-08-31 02:05 | PASS | conversation 2026-08-29 — the oh CLI became the only lifecycle door, so its | -| weigh-scorer-contract | A | 2026-08-31 02:05 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-003 (2026-06-27) | -| wiki-readme-index | A | 2026-08-31 02:05 | PASS | issue #132 — wiki README index drift guard | -| workflow-boundaries | A | 2026-08-31 02:05 | PASS | conversation 2026-06-19 (workflow consolidation, issue #259); authority moved to /spec in issue #854 | -| worktrees-layout | A | 2026-08-31 02:05 | PASS | issue #872 | +| advisor-monitored-loop | A | 2026-08-31 02:42 | PASS | conversation 2026-06-19 (single-owner implementation workflow, issue #257) | +| agent-browser-cli | A | 2026-08-31 02:42 | PASS | retro lesson 2026-06-07 (agent-browser 0.8.5 CLI) | +| agents-identity-contract | A | 2026-08-31 02:42 | PASS | issue #854 — T3-style root identity, glossary, and skill-owned procedures | +| artifact-contract-audit | A | 2026-08-31 02:42 | PASS | issue #583/#645 — production /audit implementation Gate 1 behavior | +| audit-dispatcher-contract | A | 2026-08-31 02:42 | PASS | issue #645 — audit consolidation public taxonomy | +| audit-implementation-behavior | A | 2026-08-31 02:42 | PASS | issue #645 — implementation root/repo/browser behavior | +| audit-pr-acquire | A | 2026-08-31 02:42 | PASS | issue #645 — production PR acquisition behavior | +| audit-pr-classifier | A | 2026-08-31 02:42 | PASS | issue #645 — deterministic focused and queue PR classifier | +| audit-run-root-contract | A | 2026-08-31 02:42 | PASS | issue #645 — executable immutable audit root/run correlation | +| audit-shellcheck-coverage | A | 2026-08-31 02:42 | PASS | issue #645 — private audit scripts require release and CI lint coverage | +| audit-stale-references | A | 2026-08-31 02:42 | PASS | issue #645 — clean-breaking audit migration | +| boot-lint-glob | A | 2026-08-31 02:42 | PASS | issue #90, issue #120 | +| builder-skill-consolidation | A | 2026-08-31 02:42 | PASS | issue #643 — consolidate artifact builders behind one /builder dispatcher | +| capability-benchmark-schema | A | 2026-08-31 02:42 | PASS | issue #167 — capability benchmark instrument | +| cc-safety-net-wiring | A | 2026-08-31 02:42 | SKIPPED | .oh/tasks/cc-safety-net/prd.json US-007 2026-07-19 | +| changelog-entry-length | A | 2026-08-31 02:42 | PASS | conversation 2026-08-24 — CHANGELOG.md grew to 259KB of bullet prose because "one line" was unquantified | +| cleanup-tasks-scoped-guard | A | 2026-08-31 02:42 | PASS | issue #85 | +| cleanup-tasks-worktree-grooming | A | 2026-08-31 02:42 | PASS | issue #168; issue #327 | +| cli-publish-typecheck-scope | A | 2026-08-31 02:42 | PASS | release run 33271077312 — v0.5.0 pushed its GHCR image, then failed to publish | +| close-issues-on-development | A | 2026-08-31 02:42 | PASS | issue #841 (closing keywords never fire because the default branch is main) 2026-08-26 | +| codex-stale-response-retry | A | 2026-08-31 02:42 | PASS | issue #506 — Codex previous_response_not_found RCA | +| compose-config-path-parity | A | 2026-08-31 02:42 | PASS | PR #833 (remove harness.yaml — the wrapper and VS Code "Reopen in Container" paths must resolve the same service) 2026-08-26 | +| config-schema-parity | A | 2026-08-31 02:42 | PASS | PR #833 (one schema file — DOCKER_SOCKET, SANDBOX_SSH, OH_SANDBOX_IMAGE, OH_PULL_POLICY, SKIP_PNPM_INSTALL were consumed but undocumented); rewritten for the oh.json/secrets split by PR #887 | +| context-tier-size-budget | A | 2026-08-31 02:42 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-007) — the always-on tier was 85,256 B | +| cron-claude-codex-fallback | A | 2026-08-31 02:42 | PASS | conversation 2026-06-12 (default Codex fallback for crons) | +| cron-watchdog | A | 2026-08-31 02:42 | PASS | issues #130/#453 (cron runtime watchdog + legacy system-cron reaping) 2026-06-19 | +| crons-directory-guide | A | 2026-08-31 02:42 | PASS | issue #874 | +| curl-bash-safe-alternatives | A | 2026-08-31 02:42 | PASS | vet-run/vet integration — public curl|bash examples need review-first alternatives | +| datasets-schema | A | 2026-08-31 02:42 | PASS | issue #196 — .oh/evals/datasets verifiable trajectory corpus (Repo2RLEnv-inspired) | +| debugmcp-availability | A | 2026-08-31 02:42 | SKIPPED | issue #297 — DebugMCP MCP debug-server availability | +| default-provisioning | A | 2026-08-31 02:42 | PASS | #902 — `oh harness install` must work from inside the sandbox, where | +| delegate-model-effort-policy | A | 2026-08-31 02:42 | PASS | conversation 2026-07-11 (delegate model inheritance and thinking policy) | +| devtcp-hook | A | 2026-08-31 02:42 | PASS | retro lesson 2026-06-10 (zsh /dev/tcp) | +| docker-inspect-env-guard | A | 2026-08-31 02:42 | PASS | operator directive 2026-08-08 (agents keep the docker socket, but must | +| docs-build-fast-path | A | 2026-08-31 02:42 | PASS | #455 — docs builds must stay out of fast harness/eval/release gates; #536 — docs site externalized to openharness-web; docs markdown relocated to docs/ | +| drift-check-cron-staleness-glob | A | 2026-08-31 02:42 | PASS | issue #98; issue #225 (restart-required cron frontmatter/config drift) | +| entrypoint-pnpm-manifest-fingerprint | A | 2026-08-31 02:42 | PASS | issue #521 (manifest-aware sandbox installs) 2026-07-01 | +| eval-ci-gate | A | 2026-08-31 02:42 | PASS | #103 — eval probe suite gated in CI | +| eval-gate | A | 2026-08-31 02:42 | PASS | retro lesson 2026-06-11 (eval-gate) | +| eval-results-atomic | A | 2026-08-31 02:42 | PASS | issue #83 (eval-results-atomic-write) | +| eval-runner-exit | A | 2026-08-31 02:42 | PASS | retro lesson 2026-06-11 (eval-runner-exit) #29 | +| eval-runs-once-per-cycle | A | 2026-08-31 02:42 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-006) — /eval ran 3x per cycle on the | +| execution-target-contract | A | 2026-08-31 02:42 | PASS | issue #733 (ExecutionTarget contract + Docker Compose adapter) 2026-08-10 | +| get-oh-bootstrap | A | 2026-08-31 02:42 | PASS | get-oh.sh bootstrap — the Node-bootstrapping host-side path to the standalone `oh` CLI (also on npm as @mifune/openharness; see oh-npm-package.sh) | +| git-skill | A | 2026-08-31 02:42 | PASS | conversation 2026-06-15 — rules are not always supported; git workflow must be the /git skill | +| harness-audit-empty-output-gate | A | 2026-08-31 02:42 | PASS | issue #246 — /audit harness must fail closed on empty auditor outputs | +| harness-ci-core-paths | A | 2026-08-31 02:42 | PASS | #165 — core sandbox config files must trigger harness CI | +| harness-ci-hooks-paths | A | 2026-08-31 02:42 | PASS | issue #202 — credential/security hook changes must trigger harness CI | +| harness-yaml-migration | A | 2026-08-31 02:42 | PASS | PR #833 (migrate-harness-yaml.sh — append / uncomment-in-place / preserve / overwrite, plus a silent no-op second run) 2026-08-26 | +| health-check-docker-stats | A | 2026-08-31 02:42 | PASS | retro lesson 2026-06-10 (docker stats vs ps Size) | +| health-check-socket-degrade | A | 2026-08-31 02:42 | PASS | issue #762 (refs #756) — /health-check degrades to one statement, not nine failures | +| heartbeat-logging-contract | A | 2026-08-31 02:42 | PASS | issue #447 (heartbeat log append hardening) 2026-06-18 | +| image-seed-hygiene | A | 2026-08-31 02:42 | PASS | issue #900 (slim the sandbox image) 2026-08-30 | +| markitdown-wiki-ingest | A | 2026-08-31 02:42 | PASS | issue #649 — pinned local-document normalization contract for /wiki ingest | +| next-dev-prod | A | 2026-08-31 02:42 | SKIPPED | retro lesson 2026-06-04 | +| oh-compose-env-wiring | A | 2026-08-31 02:42 | PASS | issue #880 (oh as the only front door — oh.json is the non-secret config surface) | +| oh-config-surfaces | A | 2026-08-31 02:42 | REGRESSION | PR #887 (config split across two authored surfaces — a tracked oh.json and a secrets-only root dotenv — with nothing left under $HOME) | +| oh-destroy-guard | A | 2026-08-31 02:42 | PASS | issue #879 — `oh` becomes the only front door, so `make destroy` must | +| oh-devcontainer-restructure | A | 2026-08-31 02:42 | PASS | consolidate devcontainer — .oh/devcontainer/ folded back into .devcontainer/ | +| oh-home-mount | A | 2026-08-31 02:42 | PASS | issue #898 (single $HOME mount) 2026-08-30 | +| oh-image-only-deploy | A | 2026-08-31 02:42 | PASS | .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only deploy) | +| oh-init-headless-config | A | 2026-08-31 02:42 | PASS | PR #827 (installer answers landed in the losing config file); retargeted to the .example.env template by PR #833, then to oh.json by PR #887 | +| oh-init-scaffold | A | 2026-08-31 02:42 | PASS | issue #531 Phase 2 | +| oh-lifecycle-surface | A | 2026-08-31 02:42 | PASS | issue #881 — the Makefile is retired and `oh` is the only front door | +| oh-npm-package | A | 2026-08-31 02:42 | PASS | npm publish path for the standalone `oh` CLI (@mifune/openharness) — alternative to get-oh.sh | +| oh-payload-manifest | A | 2026-08-31 02:42 | PASS | issue #531 follow-on (.oh payload manifest — oh update ships a declared allowlist) | +| oh-sandbox-image-mode | A | 2026-08-31 02:42 | PASS | conversation 2026-07-05 (basic Docker deployment — prebuilt-image mode) | +| oh-shipped-repo-overridable | A | 2026-08-31 02:42 | PASS | issue #531 follow-on (de-hardcode residual — shipped .oh shell scripts keep the upstream repo overridable) | +| oh-standalone-lifecycle | A | 2026-08-31 02:42 | PASS | issue #564 | +| oh-update | A | 2026-08-31 02:42 | PASS | issue #531 Phase 3 (oh update — upgrade only the .oh control plane) | +| operator-config-guard | A | 2026-08-31 02:42 | PASS | operator directives 2026-08-06 (.config/ and settings.local.json are operator-only) | +| pnpm-audit-ci-gate | A | 2026-08-31 02:42 | PASS | issue #171 — pnpm security audits must run in CI | +| post-bridge-publish-confirmation | A | 2026-08-31 02:42 | PASS | #523 — post-bridge live publishing requires an explicit final confirmation gate | +| prd-output-path-contract | A | 2026-08-31 02:42 | PASS | retro lesson 2026-06-19 | +| prompt-miner-schema-compat | A | 2026-08-31 02:42 | PASS | issue #253 — prompt-miner JSONL schema-drift guard | +| prompt-miner-symlink-entrypoint | A | 2026-08-31 02:42 | PASS | issue #663 — prompt-miner engine no-ops via the documented .claude/skills symlink | +| prompt-miner-weakness-record | A | 2026-08-31 02:42 | PASS | issue #580 — prompt-miner weakness-record (WH-xxx) cluster output | +| protected-path-deletion | A | 2026-08-31 02:42 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-001) — the critique gate was deleted, | +| protected-paths-resolve | A | 2026-08-31 02:42 | PASS | issue #753 — .claude/protected-paths.txt named 7 paths that did not exist. | +| registry-portability-gate | A | 2026-08-31 02:42 | PASS | issue #758 | +| registry-portability | A | 2026-08-31 02:42 | SKIPPED | issue #758 | +| retro-deterministic-contract | A | 2026-08-31 02:42 | PASS | issue #443 — /retro deterministic output and self-contained helper contract | +| rl-delegation-write-worker | A | 2026-08-31 02:42 | PASS | retro lesson 2026-06-10 (rl-delegation) #57 | +| rlm-context-budget | A | 2026-08-31 02:42 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-006 | +| runtime-preflight-gate | A | 2026-08-31 02:42 | PASS | issue #806 § B1 (open sandbox.substrate vs sandbox.runtime selector); | +| sandbox-boot-guard-ci | A | 2026-08-31 02:42 | PASS | issue #449 (sandbox image build CI guard) 2026-06-19; | +| sandbox-node-base | A | 2026-08-31 02:42 | PASS | openharness#878 — oh as the only front door, T0 sandbox base image | +| skill-paths | A | 2026-08-31 02:42 | PASS | issue #43 — stale path references; extended by issue #69 — apps/->packages/ rename guard; extended by issue #870 — deleted .oh/agents/advisor.md | +| skills-dir-clean | A | 2026-08-31 02:42 | PASS | conversation 2026-06-29 — Pi parses every top-level `.md` in the skills | +| skills-task-tool-coupling | A | 2026-08-31 02:42 | PASS | council review 2026-08-29 (issue #886) — /delegate instructed Claude-Code-only | +| skills-vendored | A | 2026-08-31 02:42 | PASS | absorb .mifune submodule into .oh — the skills/agents/hooks pack is vendored | +| slack-admin-command-surface | A | 2026-08-31 02:42 | PASS | issue #354 — Slack bridge docs must distinguish Pi /msg-bridge commands from Slack DM admin text handlers | +| spec-family-contract | A | 2026-08-31 02:42 | PASS | issue #265; spec-simplification issue #816; workflow authority issue #854 | +| spec-ready-finalization | A | 2026-08-31 02:42 | PASS | issue #134; spec-simplification issue #816; workflow authority issue #854 | +| ste-checker-contract | A | 2026-08-31 02:42 | PASS | issue #750 PR audit — the /ste checker had four fail-open paths (unclosed | +| submitted-by-trailers | A | 2026-08-31 02:42 | PASS | conversation 2026-06-12 (commit attribution trailers); the single-owner | +| sync-skill-contract | A | 2026-08-31 02:42 | PASS | issue #331 — /sync dispatcher skill (bidirectional origin↔upstream sync) | +| tool-catalog-boundary | A | 2026-08-31 02:42 | PASS | agent-browser's exclusion from the harness catalog (#821) and the | +| version-parity | A | 2026-08-31 02:42 | PASS | conversation 2026-08-29 — the oh CLI became the only lifecycle door, so its | +| weigh-scorer-contract | A | 2026-08-31 02:42 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-003 (2026-06-27) | +| wiki-readme-index | A | 2026-08-31 02:42 | PASS | issue #132 — wiki README index drift guard | +| workflow-boundaries | A | 2026-08-31 02:42 | PASS | conversation 2026-06-19 (workflow consolidation, issue #259); authority moved to /spec in issue #854 | +| worktrees-layout | A | 2026-08-31 02:42 | PASS | issue #872 | diff --git a/.oh/evals/probes/harness-home-provisioning.sh b/.oh/evals/probes/default-provisioning.sh similarity index 52% rename from .oh/evals/probes/harness-home-provisioning.sh rename to .oh/evals/probes/default-provisioning.sh index 0799c9cd..7627c5d0 100755 --- a/.oh/evals/probes/harness-home-provisioning.sh +++ b/.oh/evals/probes/default-provisioning.sh @@ -4,19 +4,22 @@ # sudo has no NOPASSWD, so default harnesses install into the home mount # source: #904 — the image must not bake a default harness, or the boot-time # install path is dead code that CI and a normal boot both skip -# desc: every kind:"default" harness installs as the sandbox user into -# NPM_USER_PREFIX, claude-code keeps its postinstall, no default harness -# package appears in the Dockerfile, and the boot path carries the -# OH_PROVISION_HARNESSES guard and its provisioner. +# source: #906 — herdr and cloudflared are tools the in-sandbox CLI owns, and a +# root-installed default would hit `sudo` with no NOPASSWD +# desc: every kind:"default" harness AND tool installs as the sandbox user into +# NPM_USER_PREFIX, claude-code keeps its postinstall, no default package or +# download URL appears in the Dockerfile, and the boot path carries the +# OH_PROVISION_DEFAULTS guard and its provisioner. set -euo pipefail ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" CATALOG="$ROOT/.oh/cli/src/lib/harnesses/catalog.ts" +TOOLS="$ROOT/.oh/cli/src/lib/tools/catalog.ts" ENTRY="$ROOT/.devcontainer/entrypoint.sh" DOCKERFILE="$ROOT/.devcontainer/Dockerfile" -PROVISIONER="$ROOT/.oh/scripts/provision-harnesses.sh" +PROVISIONER="$ROOT/.oh/scripts/provision-defaults.sh" -for f in "$CATALOG" "$ENTRY" "$DOCKERFILE"; do +for f in "$CATALOG" "$TOOLS" "$ENTRY" "$DOCKERFILE"; do if [[ ! -f $f ]]; then echo "SKIPPED: absent: $f" >&2 exit 2 @@ -58,14 +61,14 @@ if ((defaults == 0)); then exit 2 fi -grep -qF 'OH_PROVISION_HARNESSES' "$ENTRY" \ - || missing+=("entrypoint.sh: no OH_PROVISION_HARNESSES guard — nothing provisions harnesses into the home mount at boot") -grep -qF 'provision-harnesses.sh' "$ENTRY" \ - || missing+=("entrypoint.sh: does not call .oh/scripts/provision-harnesses.sh") -grep -qF 'WARNING: harness provisioning did not complete' "$ENTRY" \ - || missing+=("entrypoint.sh: harness provisioning does not warn-and-continue — an offline sandbox must still come up as a usable shell") +grep -qF 'OH_PROVISION_DEFAULTS' "$ENTRY" \ + || missing+=("entrypoint.sh: no OH_PROVISION_DEFAULTS guard — nothing provisions harnesses into the home mount at boot") +grep -qF 'provision-defaults.sh' "$ENTRY" \ + || missing+=("entrypoint.sh: does not call .oh/scripts/provision-defaults.sh") +grep -qF 'WARNING: default provisioning did not complete' "$ENTRY" \ + || missing+=("entrypoint.sh: default provisioning does not warn-and-continue — an offline sandbox must still come up as a usable shell") [[ -x $PROVISIONER ]] \ - || missing+=(".oh/scripts/provision-harnesses.sh: missing or not executable") + || missing+=(".oh/scripts/provision-defaults.sh: missing or not executable") # #904: the image must not bake any kind:"default" harness. The install target is # the home mount, so a copy under /usr/lib/node_modules shadows it with one no # running sandbox can upgrade — and, worse, makes the boot-time install path @@ -93,7 +96,7 @@ while IFS= read -r entry; do fi pkgs=$((pkgs + 1)) if grep -qF -- "$pkg" <<<"$DOCKERFILE_CODE"; then - missing+=("Dockerfile: names $pkg — default harness \"$id\" is baked into the image again; it belongs to .oh/scripts/provision-harnesses.sh, which installs it into $PREFIX at boot") + missing+=("Dockerfile: names $pkg — default harness \"$id\" is baked into the image again; it belongs to .oh/scripts/provision-defaults.sh, which installs it into $PREFIX at boot") fi done <<<"$entries" @@ -102,8 +105,47 @@ if ((pkgs == 0)); then exit 2 fi -if grep -qE '^ARG (BAKE_HARNESSES|AGENTS)=' <<<"$DOCKERFILE_CODE"; then - missing+=("Dockerfile: ARG BAKE_HARNESSES/AGENTS is back — a build-arg that re-bakes the default harnesses is a dormant path that reintroduces the shadowed install and un-exercises the boot provisioner") +if grep -qE '^ARG (BAKE_HARNESSES|AGENTS|HERDR_VERSION)=' <<<"$DOCKERFILE_CODE"; then + missing+=("Dockerfile: ARG BAKE_HARNESSES/AGENTS/HERDR_VERSION is back — a build arg that re-bakes a default is a dormant path that reintroduces the shadowed install and un-exercises the boot provisioner") +fi + +# #906: the same rule for kind:"default" tools. These install as root nowhere: +# commands/tool.ts passes stdio:"inherit", so local-target.ts selects plain +# `sudo --` for a root install, and /etc/sudoers.d/sandbox has no NOPASSWD — +# an agent in a Herdr pane would hang on a password prompt. +tool_entries=$(awk ' + /^ Object\.freeze\(\{$/ { buf=""; inb=1; next } + /^ \}\),$/ { if (inb) print buf; inb=0; next } + inb { buf = buf $0 " " } +' "$TOOLS") + +tools=0 +while IFS= read -r entry; do + [[ $entry == *'kind: "default"'* ]] || continue + tools=$((tools + 1)) + id=$(sed -n 's/.*id: "\([^"]*\)".*/\1/p' <<<"$entry") + if [[ $entry == *'installUser: "root"'* ]]; then + missing+=("tools/catalog.ts: default tool \"$id\" installs as root — commands/tool.ts uses stdio:\"inherit\", so that becomes an interactive \`sudo\`, and /etc/sudoers.d/sandbox has no NOPASSWD") + fi + if [[ $entry != *'NPM_USER_PREFIX'* ]]; then + missing+=("tools/catalog.ts: default tool \"$id\" does not install into NPM_USER_PREFIX — a system-path install cannot be upgraded by a running sandbox and does not persist in the home mount") + fi + if [[ $entry != *'sha256sum -c -'* ]]; then + missing+=("tools/catalog.ts: default tool \"$id\" downloads without \`sha256sum -c -\` — an unverified binary is installed straight into the agent's PATH") + fi + # Match the pinned project path, not the bare host — the Dockerfile clones + # oh-my-zsh plugins from github.com and that is not a baked tool. + while IFS= read -r origin; do + [[ -n $origin ]] || continue + if grep -qF -- "$origin" <<<"$DOCKERFILE_CODE"; then + missing+=("Dockerfile: names $origin — default tool \"$id\" is baked into the image again; it belongs to .oh/scripts/provision-defaults.sh, which installs it into $PREFIX at boot") + fi + done < <(grep -oE 'https://[a-z0-9.-]+/[A-Za-z0-9._-]+/[A-Za-z0-9._-]+' <<<"$entry" | sort -u) +done <<<"$tool_entries" + +if ((tools == 0)); then + echo "SKIPPED: no kind:\"default\" tool parsed out of $TOOLS, so the tool half would pass vacuously" >&2 + exit 2 fi if ((${#missing[@]})); then @@ -111,4 +153,4 @@ if ((${#missing[@]})); then exit 1 fi -echo "PASS: all $defaults default harnesses install as the sandbox user into $PREFIX, none of the $pkgs packages is baked into the image, and the boot path provisions them" >&2 +echo "PASS: $defaults default harnesses and $tools default tools install as the sandbox user into $PREFIX, none of the $pkgs packages is baked into the image, and the boot path provisions them" >&2 diff --git a/.oh/evals/probes/sandbox-boot-guard-ci.sh b/.oh/evals/probes/sandbox-boot-guard-ci.sh index e603e14b..251d8001 100755 --- a/.oh/evals/probes/sandbox-boot-guard-ci.sh +++ b/.oh/evals/probes/sandbox-boot-guard-ci.sh @@ -76,8 +76,8 @@ fi # #904: boot-time harness provisioning is exercised nowhere else. Turning it off # here to save CI minutes would restore it to untested dead code. -if grep -Eq 'OH_PROVISION_HARNESSES: *"?false' <<<"$text"; then - missing+=("the boot guard disables OH_PROVISION_HARNESSES — this job is the only place the boot-time harness install runs") +if grep -Eq 'OH_PROVISION_DEFAULTS: *"?false' <<<"$text"; then + missing+=("the boot guard disables OH_PROVISION_DEFAULTS — this job is the only place the boot-time harness install runs") fi has 'Sandbox boot guard only' "comment explaining non-release intent" diff --git a/.oh/scripts/__tests__/herdr-default.test.ts b/.oh/scripts/__tests__/herdr-default.test.ts index 1b10150f..9ee4669d 100644 --- a/.oh/scripts/__tests__/herdr-default.test.ts +++ b/.oh/scripts/__tests__/herdr-default.test.ts @@ -7,14 +7,23 @@ const repoRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "../ const readRepoFile = (file: string): string => readFileSync(path.join(repoRoot, file), "utf8"); describe("default Herdr integration", () => { + // #906: the pin moved out of the Dockerfile and into the tool catalog, which + // provisions Herdr into the home mount at boot. The image no longer carries it. it("pins and verifies Herdr for both supported architectures", () => { + const catalog = readRepoFile(".oh/cli/src/lib/tools/catalog.ts"); + + expect(catalog).toContain("version=0.7.4"); + expect(catalog).toContain("bc0fc02d4ba500f9cac2353a43e67fe036785ecca6eb55378e050fac3c103059"); + expect(catalog).toContain("544e0002de42806d1ab64ccdef3a7e7414f24717b0b6b022bc9e57d2eefd26a2"); + expect(catalog).toContain("sha256sum -c -"); + expect(catalog).toContain('test "$("$prefix/bin/herdr" --version)" = "herdr $version"'); + }); + + it("no longer bakes Herdr into the image", () => { const dockerfile = readRepoFile(".devcontainer/Dockerfile"); - expect(dockerfile).toContain("HERDR_VERSION=0.7.4"); - expect(dockerfile).toContain("bc0fc02d4ba500f9cac2353a43e67fe036785ecca6eb55378e050fac3c103059"); - expect(dockerfile).toContain("544e0002de42806d1ab64ccdef3a7e7414f24717b0b6b022bc9e57d2eefd26a2"); - expect(dockerfile).toContain("sha256sum -c -"); - expect(dockerfile).toContain('test "$(herdr --version)" = "herdr ${HERDR_VERSION}"'); + expect(dockerfile).not.toContain("HERDR_VERSION"); + expect(dockerfile).not.toContain("github.com/ogulcancelik/herdr"); }); it.each(["docker-compose.yml", "docker-compose.image-only.yml"])( diff --git a/.oh/scripts/__tests__/sandbox-base-image.test.ts b/.oh/scripts/__tests__/sandbox-base-image.test.ts index c73ca3cd..b50b5041 100644 --- a/.oh/scripts/__tests__/sandbox-base-image.test.ts +++ b/.oh/scripts/__tests__/sandbox-base-image.test.ts @@ -34,29 +34,19 @@ describe("sandbox base image", () => { expect(dockerfile).not.toContain("https://download.docker.com/linux/debian bookworm stable"); }); - it("keeps Cloudflare's apt repository on Bookworm", () => { - expect(dockerfile).toContain("https://pkg.cloudflare.com/cloudflared bookworm main"); - expect(dockerfile).not.toContain("https://pkg.cloudflare.com/cloudflared trixie main"); + // #906: cloudflared moved out of the image into the tool catalog, which + // installs a pinned, checksum-verified binary. That deleted the last reason + // this Dockerfile referenced a non-Trixie apt suite. + it("no longer carries Cloudflare's apt repository", () => { + expect(dockerfile).not.toContain("pkg.cloudflare.com"); }); - it("explains the Cloudflare Bookworm exception next to that repository", () => { - const lines = dockerfile.split("\n"); - const suiteLine = lines.findIndex((line) => line.includes("pkg.cloudflare.com/cloudflared bookworm main")); - expect(suiteLine).toBeGreaterThan(-1); - - const preamble = lines.slice(Math.max(0, suiteLine - 8), suiteLine).join("\n"); - expect(preamble).toMatch(/^#.*trixie/mi); - expect(preamble).toContain("404"); - }); - - it("leaves every other suite reference off Bookworm", () => { + it("leaves no suite reference on Bookworm at all", () => { const bookwormLines = dockerfile .split("\n") .filter((line) => /bookworm/i.test(line)) .filter((line) => !line.trimStart().startsWith("#")); - expect(bookwormLines).toEqual([ - expect.stringContaining("https://pkg.cloudflare.com/cloudflared bookworm main"), - ]); + expect(bookwormLines).toEqual([]); }); }); diff --git a/.oh/scripts/__tests__/sandbox-boot-smoke.test.ts b/.oh/scripts/__tests__/sandbox-boot-smoke.test.ts index 9eceef08..f5d8ac72 100644 --- a/.oh/scripts/__tests__/sandbox-boot-smoke.test.ts +++ b/.oh/scripts/__tests__/sandbox-boot-smoke.test.ts @@ -18,6 +18,7 @@ function fixture( markerOwner?: string; harnessProbeFails?: boolean; noDefaultHarnesses?: boolean; + noDefaultTools?: boolean; } = {}, ) { const runtimeUid = opts.runtimeUid ?? HOST_UID; @@ -86,6 +87,19 @@ ${ { "id": "pi", "title": "Pi", "binary": "pi", "kind": "default", "enabled": null, "installed": true, "docs": "x" } ]` } +JSON + exit 0 + ;; + *"oh tool list --defaults --json"*) + cat <<'JSON' +${ + opts.noDefaultTools + ? "[]" + : `[ + { "id": "herdr", "title": "Herdr", "binary": "herdr", "kind": "default", "enabled": null, "installed": true, "docs": "x" }, + { "id": "cloudflared", "title": "cloudflared", "binary": "cloudflared", "kind": "default", "enabled": null, "installed": true, "docs": "x" } +]` +} JSON exit 0 ;; @@ -166,9 +180,12 @@ describe("sandbox boot smoke", () => { `sandbox user, bind mount, and sandbox-created files all resolve to ${HOST_UID}:${HOST_GID}`, ); expect(dockerCalls).toContain("oh harness list --defaults --json"); + expect(dockerCalls).toContain("oh tool list --defaults --json"); expect(dockerCalls).toContain("type -P"); expect(result.stdout).toContain("claude-code provisioned at boot -> 1.2.3"); expect(result.stdout).toContain("pi provisioned at boot -> 1.2.3"); + expect(result.stdout).toContain("herdr provisioned at boot -> 1.2.3"); + expect(result.stdout).toContain("cloudflared provisioned at boot -> 1.2.3"); }); // #904 deleted the image bake, so this assertion is the only thing standing @@ -186,13 +203,16 @@ describe("sandbox boot smoke", () => { expect(readFileSync(fx.composeLog, "utf8")).toContain("down -v --remove-orphans"); }); - it("refuses to pass vacuously when the catalog reports no default harnesses", () => { - const fx = fixture({ noDefaultHarnesses: true }); + it.each<[string, { noDefaultHarnesses?: boolean; noDefaultTools?: boolean }]>([ + ["harness", { noDefaultHarnesses: true }], + ["tool", { noDefaultTools: true }], + ])("refuses to pass vacuously when the %s catalog reports no defaults", (noun, overrides) => { + const fx = fixture(overrides); const result = runSmoke(fx); expect(result.status).toBe(1); - expect(result.stderr).toContain('reported no kind:"default" harnesses'); + expect(result.stderr).toContain(`the ${noun} catalog reported no kind:"default" entries`); }); it("fails when the runtime sandbox user does not match the checkout owner", () => { diff --git a/.oh/scripts/__tests__/sandbox-healthcheck.test.ts b/.oh/scripts/__tests__/sandbox-healthcheck.test.ts index 27b36ff2..d68b167f 100644 --- a/.oh/scripts/__tests__/sandbox-healthcheck.test.ts +++ b/.oh/scripts/__tests__/sandbox-healthcheck.test.ts @@ -137,7 +137,7 @@ describe("sandbox healthcheck", () => { expect(compose).toContain("healthcheck:"); expect(compose).toContain("/home/sandbox/harness/.oh/scripts/sandbox-healthcheck.sh"); // Boot installs the default harnesses into the home mount (#904), bounded - // by OH_PROVISION_HARNESSES_TIMEOUT (180s). The start period has to cover + // by OH_PROVISION_DEFAULTS_TIMEOUT (240s). The start period has to cover // that plus the rest of boot, so assert the floor rather than a literal // that a reduction could slip past. const startPeriod = /start_period: (\d+)s/.exec(compose); diff --git a/.oh/scripts/__tests__/verify-sandbox-image.test.ts b/.oh/scripts/__tests__/verify-sandbox-image.test.ts index 0ce05a8d..17dafbc2 100644 --- a/.oh/scripts/__tests__/verify-sandbox-image.test.ts +++ b/.oh/scripts/__tests__/verify-sandbox-image.test.ts @@ -7,24 +7,21 @@ import { describe, expect, it } from "vitest"; const ROOT = join(import.meta.dirname, "../../.."); const SCRIPT = join(ROOT, ".oh", "scripts", "verify-sandbox-image.sh"); -const AMD64_SHA = "bc0fc02d4ba500f9cac2353a43e67fe036785ecca6eb55378e050fac3c103059"; - type Overrides = Partial<{ architecture: string; codename: string; dockerSuite: string; - cloudflareSuite: string; uid: string; gid: string; node: string; pnpm: string; - herdr: string; - herdrSha: string; missingTool: string; nonVersionTool: string; platformWarning: string; bakedHarnesses: boolean; + bakedTools: boolean; noDefaultHarnesses: boolean; + noDefaultTools: boolean; harnessCatalogFails: boolean; }>; @@ -37,18 +34,17 @@ function fixture(o: Overrides = {}) { architecture: "amd64", codename: "trixie", dockerSuite: "trixie", - cloudflareSuite: "bookworm", uid: "1000", gid: "1000", node: "v22.14.0", pnpm: "10.33.0", - herdr: "herdr 0.7.4", - herdrSha: AMD64_SHA, missingTool: "", nonVersionTool: "", platformWarning: "", bakedHarnesses: false, + bakedTools: false, noDefaultHarnesses: false, + noDefaultTools: false, harnessCatalogFails: false, ...o, }; @@ -62,12 +58,9 @@ cmd="\${@: -1}" case "$cmd" in *VERSION_CODENAME*) printf '%s' ${JSON.stringify(v.codename)} ;; *docker.list*) printf 'deb [arch=amd64] https://download.docker.com/linux/debian %s stable\\n' ${JSON.stringify(v.dockerSuite)} ;; - *cloudflared.list*) printf 'deb [arch=amd64] https://pkg.cloudflare.com/cloudflared %s main\\n' ${JSON.stringify(v.cloudflareSuite)} ;; *"id -u sandbox"*) printf '%s\\n%s\\n' ${JSON.stringify(v.uid)} ${JSON.stringify(v.gid)} ;; "node --version") printf '%s\\n' ${JSON.stringify(v.node)} ;; "pnpm --version") printf '%s\\n' ${JSON.stringify(v.pnpm)} ;; - "herdr --version") printf '%s\\n' ${JSON.stringify(v.herdr)} ;; - *sha256sum*) printf '%s /usr/local/bin/herdr\\n' ${JSON.stringify(v.herdrSha)} ;; *"oh harness list --defaults --json"*) if [ "${v.harnessCatalogFails ? "1" : "0"}" = "1" ]; then echo 'not an OpenHarness-equipped repo' >&2 @@ -82,6 +75,18 @@ ${ { "id": "pi", "binary": "pi", "kind": "default", "installed": false } ]` } +JSON + ;; + *"oh tool list --defaults --json"*) + cat <<'JSON' +${ + v.noDefaultTools + ? "[]" + : `[ + { "id": "herdr", "binary": "herdr", "kind": "default", "installed": ${v.bakedTools} }, + { "id": "cloudflared", "binary": "cloudflared", "kind": "default", "installed": false } +]` +} JSON ;; *) @@ -122,12 +127,11 @@ describe("verify-sandbox-image", () => { expect(result.status).toBe(0); expect(result.stdout).toContain("base distribution is Debian trixie"); expect(result.stdout).toContain("Docker apt suite is trixie"); - expect(result.stdout).toContain("Cloudflare apt suite is bookworm"); expect(result.stdout).toContain("built-in sandbox user is 1000:1000"); expect(result.stdout).toContain("node is major 22"); expect(result.stdout).toContain("pnpm is exactly 10.33.0"); - expect(result.stdout).toContain("herdr is 0.7.4"); - expect(result.stdout).toContain("matches the amd64 (x86_64) Dockerfile checksum pin"); + expect(result.stdout).toContain("no default harness is baked into the image"); + expect(result.stdout).toContain("no default tool is baked into the image"); expect(result.stdout).toContain("all checks passed"); }); @@ -141,12 +145,9 @@ describe("verify-sandbox-image", () => { it.each<[string, Overrides, string]>([ ["a Bookworm base", { codename: "bookworm" }, "base distribution codename is 'bookworm'"], ["a Bookworm Docker suite", { dockerSuite: "bookworm" }, "Docker apt suite is not trixie"], - ["a Trixie Cloudflare suite", { cloudflareSuite: "trixie" }, "Cloudflare apt suite is not bookworm"], ["a shifted sandbox UID", { uid: "1001" }, "built-in sandbox user is 1001:1000"], ["a wrong Node major", { node: "v20.19.0" }, "node major is not 22"], ["a drifted pnpm version", { pnpm: "10.34.0" }, "pnpm is 10.34.0"], - ["a drifted Herdr version", { herdr: "herdr 0.7.3" }, "herdr is 'herdr 0.7.3'"], - ["a Herdr binary that misses its pin", { herdrSha: "deadbeef" }, "does not match the amd64 pin"], ["a missing required tool", { missingTool: "uv --version" }, "uv --version produced no version output"], ])("rejects %s", (_label, overrides, expected) => { const result = run(fixture(overrides)); @@ -159,7 +160,6 @@ describe("verify-sandbox-image", () => { "gh --version", "docker --version", "docker compose version", - "cloudflared --version", "bun --version", "uv --version", ])("rejects clean but non-version output from %s", (tool) => { @@ -186,37 +186,43 @@ describe("verify-sandbox-image", () => { expect(result.stderr).toContain("unsupported image architecture: riscv64"); }); - it("resolves the arm64 checksum pin from the Dockerfile", () => { - const result = run( - fixture({ - architecture: "arm64", - herdrSha: "544e0002de42806d1ab64ccdef3a7e7414f24717b0b6b022bc9e57d2eefd26a2", - }), - ); + it("accepts an arm64 image", () => { + const result = run(fixture({ architecture: "arm64" })); expect(result.status).toBe(0); - expect(result.stdout).toContain("matches the arm64 (aarch64) Dockerfile checksum pin"); }); - // #904: the default harnesses moved out of the image and into the boot path. - // A baked copy under /usr/lib/node_modules shadows the home-mount install and - // silently un-exercises the provisioner, so the image must not carry one. - it("passes an image that bakes no default harness", () => { + // #904/#906: the default harnesses AND the default tools moved out of the + // image and into the boot path. A baked copy in a system path shadows the + // home-mount install and silently un-exercises the provisioner, so the image + // must not carry either. + it("passes an image that bakes no default harness or tool", () => { const result = run(fixture()); expect(result.status).toBe(0); expect(result.stdout).toContain("no default harness is baked into the image"); + expect(result.stdout).toContain("no default tool is baked into the image"); }); it("rejects an image that bakes a default harness", () => { const result = run(fixture({ bakedHarnesses: true })); expect(result.status).toBe(1); - expect(result.stderr).toContain("the image ships baked default harnesses: claude-code (claude)"); + expect(result.stderr).toContain("the image ships baked default harnesss: claude-code (claude)"); + }); + + it("rejects an image that bakes a default tool", () => { + const result = run(fixture({ bakedTools: true })); + + expect(result.status).toBe(1); + expect(result.stderr).toContain("the image ships baked default tools: herdr (herdr)"); }); - it("refuses to pass vacuously when the image catalog lists no default harness", () => { - const result = run(fixture({ noDefaultHarnesses: true })); + it.each<[string, Overrides]>([ + ["harness", { noDefaultHarnesses: true }], + ["tool", { noDefaultTools: true }], + ])("refuses to pass vacuously when the image lists no default %s", (_noun, overrides) => { + const result = run(fixture(overrides)); expect(result.status).toBe(1); expect(result.stderr).toContain("would pass vacuously"); diff --git a/.oh/scripts/provision-harnesses.sh b/.oh/scripts/provision-defaults.sh similarity index 59% rename from .oh/scripts/provision-harnesses.sh rename to .oh/scripts/provision-defaults.sh index b1f9f2cf..bb029361 100755 --- a/.oh/scripts/provision-harnesses.sh +++ b/.oh/scripts/provision-defaults.sh @@ -12,12 +12,12 @@ case "${1:-}" in *) echo "usage: $(basename "$0") [--verify]" >&2; exit 2 ;; esac -log() { echo "[provision-harnesses] $*"; } +log() { echo "[provision-defaults] $*"; } die() { - echo "[provision-harnesses] ERROR: $1" >&2 + echo "[provision-defaults] ERROR: $1" >&2 shift - for line in "$@"; do echo "[provision-harnesses] $line" >&2; done + for line in "$@"; do echo "[provision-defaults] $line" >&2; done exit 1 } @@ -33,7 +33,7 @@ inside_sandbox || die \ "this provisions /home/$SANDBOX_USER/.local inside the sandbox and must not run on the host" \ "open a sandbox shell first:" \ " oh shell" \ - " bash .oh/scripts/provision-harnesses.sh" + " bash .oh/scripts/provision-defaults.sh" export OH_EXECUTION_TARGET=local @@ -103,56 +103,76 @@ command -v jq >/dev/null 2>&1 || die \ "the image installs it with apt; rebuild the sandbox image:" \ " oh sandbox" -STATES="" -if ! STATES="$("$OH_BIN" harness list --defaults --json 2>/dev/null)" || [ -z "$STATES" ]; then - die "'$OH_BIN harness list --defaults --json' produced no catalog" \ - "the CLI at $(command -v "$OH_BIN") predates \`oh harness\`; the harness catalog" \ - "is the only source of truth for what to install, so there is nothing to provision." \ - "rebuild the sandbox image from this control plane:" \ - " oh sandbox" -fi - -DEFAULTS="$(jq -r '.[] | select(.kind == "default") | "\(.id)\t\(.installed)"' <<<"$STATES")" -[ -n "$DEFAULTS" ] || die \ - "the harness catalog declares no default harnesses" \ - "check .oh/cli/src/lib/harnesses/catalog.ts" - +# Both catalogs answer the same question — "what does a working sandbox need that +# is not in the image?" — and get the same policy: install what is missing, never +# replace what is already there. The catalogs are the only source of truth for +# the list, so this script never names a package. missing=() failed=() +provisioned=0 -while IFS=$'\t' read -r id installed; do - [ -n "$id" ] || continue - if [ "$installed" = "true" ]; then - log "OK $id present (unpinned — an existing install is never replaced)" - continue - fi - if [ "$MODE" = "verify" ]; then - missing+=("$id") - continue - fi - log "installing $id into $NPM_USER_PREFIX" - if "$OH_BIN" harness install "$id" --no-persist /dev/null)" || [ -z "$states" ]; then + die "'$OH_BIN $cmd list --defaults --json' produced no catalog" \ + "the CLI at $(command -v "$OH_BIN") predates \`oh $cmd --defaults\`; the catalog" \ + "is the only source of truth for what to install, so there is nothing to provision." \ + "rebuild the sandbox image from this control plane:" \ + " oh sandbox" fi -done <<<"$DEFAULTS" + + defaults="$(jq -r '.[] | select(.kind == "default") | "\(.id)\t\(.installed)"' <<<"$states")" + [ -n "$defaults" ] || die \ + "the $noun catalog declares no defaults" \ + "check $catalog" + + while IFS=$'\t' read -r id installed; do + [ -n "$id" ] || continue + provisioned=$((provisioned + 1)) + if [ "$installed" = "true" ]; then + log "OK $id present (unpinned — an existing install is never replaced)" + continue + fi + if [ "$MODE" = "verify" ]; then + missing+=("$id") + continue + fi + log "installing $id into $NPM_USER_PREFIX" + if "$OH_BIN" "$cmd" install "$id" --no-persist /tmp/sandbox-boot-smoke-harness.err); then - echo "sandbox boot smoke failed: 'oh harness list --defaults --json' did not run in the booted sandbox" >&2 - cat /tmp/sandbox-boot-smoke-harness.err >&2 || true + if ! states=$(docker exec -u sandbox "$cid" bash -lc "oh $cmd list --defaults --json" 2>/tmp/sandbox-boot-smoke-catalog.err); then + echo "sandbox boot smoke failed: 'oh $cmd list --defaults --json' did not run in the booted sandbox" >&2 + cat /tmp/sandbox-boot-smoke-catalog.err >&2 || true return 1 fi ids=$(jq -r '.[] | select(.kind == "default") | .id' <<<"$states") if [ -z "$ids" ]; then - echo "sandbox boot smoke failed: the harness catalog reported no kind:\"default\" harnesses, so this check would pass vacuously" >&2 + echo "sandbox boot smoke failed: the $noun catalog reported no kind:\"default\" entries, so this check would pass vacuously" >&2 return 1 fi @@ -121,7 +122,7 @@ verify_default_harnesses() { [ -n "$id" ] || continue binary=$(jq -r --arg id "$id" '.[] | select(.id == $id) | .binary' <<<"$states") if [ -z "$binary" ] || [ "$binary" = "null" ]; then - echo "sandbox boot smoke failed: default harness '$id' declares no binary to check" >&2 + echo "sandbox boot smoke failed: default $noun '$id' declares no binary to check" >&2 failed=1 continue fi @@ -136,7 +137,7 @@ verify_default_harnesses() { [ \"\$owner\" = '$sandbox_uid' ] || { echo \"binary is owned by uid \$owner, not sandbox ($sandbox_uid)\" >&2; exit 1; } \"\$path\" --version " 2>&1); then - echo "sandbox boot smoke failed: default harness '$id' was not provisioned into the home mount at boot" >&2 + echo "sandbox boot smoke failed: default $noun '$id' was not provisioned into the home mount at boot" >&2 printf ' %s\n' "$out" >&2 failed=1 continue @@ -169,7 +170,7 @@ while [ "$(date +%s)" -le "$end" ]; do # shellcheck disable=SC2086 # HEALTH_CMD intentionally splits into command argv. if docker exec "$cid" $HEALTH_CMD >/tmp/sandbox-boot-smoke-health.out 2>/tmp/sandbox-boot-smoke-health.err; then if ! docker exec -u sandbox "$cid" sh -lc \ - 'test "$(herdr --version)" = "herdr 0.7.4" && test -w "$HOME/.config" && test -w "$HOME/.herdr" && command -v lsof >/dev/null && lsof -v >/dev/null 2>&1 && command -v htop >/dev/null && htop --version >/dev/null && command -v telnet >/dev/null && telnet --version >/dev/null'; then + 'test -w "$HOME/.config" && test -w "$HOME/.herdr" && command -v lsof >/dev/null && lsof -v >/dev/null 2>&1 && command -v htop >/dev/null && htop --version >/dev/null && command -v telnet >/dev/null && telnet --version >/dev/null'; then echo "sandbox boot smoke failed: required utilities, Herdr runtime, or writable state is unavailable" >&2 status_diagnostics "$cid" exit 1 @@ -178,11 +179,15 @@ while [ "$(date +%s)" -le "$end" ]; do status_diagnostics "$cid" exit 1 fi - if ! verify_default_harnesses "$cid"; then + if ! verify_default_catalog "$cid" harness harness; then + status_diagnostics "$cid" + exit 1 + fi + if ! verify_default_catalog "$cid" tool tool; then status_diagnostics "$cid" exit 1 fi - echo "sandbox boot smoke ok: $SERVICE ($cid) passed $HEALTH_CMD, Herdr runtime, bind-ownership, and boot-provisioned harness checks" + echo "sandbox boot smoke ok: $SERVICE ($cid) passed $HEALTH_CMD, Herdr runtime, bind-ownership, and boot-provisioned harness and tool checks" exit 0 fi last_status=$(docker inspect -f '{{if .State.Health}}{{.State.Health.Status}}{{else}}no-healthcheck{{end}}' "$cid" 2>/dev/null || echo "inspect-failed") diff --git a/.oh/scripts/verify-sandbox-image.sh b/.oh/scripts/verify-sandbox-image.sh index 18342b2a..c3ff9996 100755 --- a/.oh/scripts/verify-sandbox-image.sh +++ b/.oh/scripts/verify-sandbox-image.sh @@ -1,23 +1,17 @@ #!/usr/bin/env bash # Verify a built sandbox image: base distribution, apt suites, the sandbox -# UID/GID contract, the Node/pnpm pins, the Herdr checksum, and version output -# from every required default tool, and that no kind:"default" harness is baked -# into it. Usage: verify-sandbox-image.sh +# UID/GID contract, the Node/pnpm pins, and version output from every baked-in +# tool, and that no kind:"default" harness or tool is baked into it. +# Usage: verify-sandbox-image.sh set -euo pipefail -SCRIPT_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd) -REPO_ROOT=$(cd "$SCRIPT_DIR/../.." && pwd) -DOCKERFILE=${VERIFY_IMAGE_DOCKERFILE:-$REPO_ROOT/.devcontainer/Dockerfile} - EXPECTED_CODENAME=trixie EXPECTED_DOCKER_SUITE=trixie -EXPECTED_CLOUDFLARE_SUITE=bookworm EXPECTED_UID=1000 EXPECTED_GID=1000 EXPECTED_NODE_MAJOR=22 EXPECTED_PNPM=10.33.0 -EXPECTED_HERDR=0.7.4 usage() { echo "usage: ${0##*/} " >&2 @@ -39,20 +33,10 @@ arch=$(docker image inspect -f '{{.Architecture}}' "$IMAGE") echo "verifying $IMAGE (architecture: $arch)" case "$arch" in - amd64) herdr_arch=x86_64 ;; - arm64) herdr_arch=aarch64 ;; + amd64|arm64) ;; *) echo "FAIL: unsupported image architecture: $arch" >&2; exit 1 ;; esac -expected_sha=$( - awk -v a="$arch" '$0 ~ a"\\)" && /herdr_sha=/ { - for (i = 1; i <= NF; i++) if ($i ~ /^herdr_sha=/) { sub(/^herdr_sha=/, "", $i); print $i; exit } - }' "$DOCKERFILE" -) -if [ -z "$expected_sha" ]; then - fail "no herdr_sha pinned for $arch in ${DOCKERFILE#"$REPO_ROOT"/}" -fi - codename=$(run '. /etc/os-release && printf "%s" "${VERSION_CODENAME:-}"') if [ "$codename" = "$EXPECTED_CODENAME" ]; then ok "base distribution is Debian $EXPECTED_CODENAME" @@ -67,13 +51,6 @@ else fail "Docker apt suite is not $EXPECTED_DOCKER_SUITE: $docker_suite" fi -cf_suite=$(run 'cat /etc/apt/sources.list.d/cloudflared.list') -if grep -qF "cloudflared $EXPECTED_CLOUDFLARE_SUITE main" <<<"$cf_suite"; then - ok "Cloudflare apt suite is $EXPECTED_CLOUDFLARE_SUITE (no Trixie suite is published)" -else - fail "Cloudflare apt suite is not $EXPECTED_CLOUDFLARE_SUITE: $cf_suite" -fi - ids=$(run 'id -u sandbox; id -g sandbox') built_uid=$(sed -n 1p <<<"$ids") built_gid=$(sed -n 2p <<<"$ids") @@ -97,22 +74,6 @@ else fail "pnpm is $pnpm_version, expected exactly $EXPECTED_PNPM" fi -herdr_version=$(run 'herdr --version') -if [ "$herdr_version" = "herdr $EXPECTED_HERDR" ]; then - ok "herdr is $EXPECTED_HERDR" -else - fail "herdr is '$herdr_version', expected 'herdr $EXPECTED_HERDR'" -fi - -if [ -n "$expected_sha" ]; then - actual_sha=$(run 'sha256sum /usr/local/bin/herdr' | awk '{print $1}') - if [ "$actual_sha" = "$expected_sha" ]; then - ok "installed herdr matches the $arch ($herdr_arch) Dockerfile checksum pin" - else - fail "installed herdr checksum $actual_sha does not match the $arch pin $expected_sha" - fi -fi - # Under emulation `docker run` prefixes its output with a platform-mismatch # warning on stderr. Drop it so the reported line is the tool's own version, # not the runner's complaint about the architecture. @@ -125,7 +86,7 @@ has_numeric_dotted_version() { } for tool in "gh --version" "docker --version" "docker compose version" \ - "cloudflared --version" "bun --version" "uv --version"; do + "bun --version" "uv --version"; do if out=$(run "$tool" 2>&1); then line=$(first_real_line <<<"$out") if has_numeric_dotted_version <<<"$line"; then @@ -138,30 +99,39 @@ for tool in "gh --version" "docker --version" "docker compose version" \ fi done -# The image must NOT ship the default harnesses (#904). They are the in-sandbox -# CLI's responsibility and are installed into the home mount at boot, so a -# default harness found here means the bake came back and the home mount's copy -# is shadowed by an unupgradable one under /usr/lib/node_modules. The catalog in -# the image is the source of truth for which ids are default, so this cannot -# drift from harnesses/catalog.ts. -if defaults_json=$(run 'cd /opt/oh-seed && OH_EXECUTION_TARGET=local oh harness list --defaults --json' 2>/tmp/verify-sandbox-defaults.err); then - if command -v jq >/dev/null 2>&1; then - default_ids=$(jq -r '.[] | select(.kind == "default") | .id' <<<"$defaults_json") - if [ -z "$default_ids" ]; then - fail "the image's harness catalog reports no kind:\"default\" harnesses — the unbaked-image check would pass vacuously" - else - baked=$(jq -r '.[] | select(.kind == "default" and .installed == true) | "\(.id) (\(.binary))"' <<<"$defaults_json") - if [ -n "$baked" ]; then - fail "the image ships baked default harnesses: $(tr '\n' ' ' <<<"$baked")— these must be provisioned into /home/sandbox/.local at boot, not baked" - else - ok "no default harness is baked into the image ($(tr '\n' ' ' <<<"$default_ids"))" - fi - fi +# The image must NOT ship any kind:"default" harness (#904) or tool (#906). +# Both are installed into /home/sandbox/.local at boot: a copy baked into a +# system path shadows the home-mount install with one no running sandbox can +# upgrade, and makes the boot install dead code that never runs and never gets +# tested. The catalogs inside the image are the source of truth for which ids +# are default, so this cannot drift from the TypeScript. +check_no_baked_defaults() { + local noun="$1" cmd="$2" json ids baked + + if ! json=$(run "cd /opt/oh-seed && OH_EXECUTION_TARGET=local oh $cmd list --defaults --json" 2>/tmp/verify-sandbox-defaults.err); then + fail "could not read the $noun catalog from the image: $(head -3 /tmp/verify-sandbox-defaults.err 2>/dev/null)" + return + fi + + ids=$(jq -r '.[] | select(.kind == "default") | .id' <<<"$json") + if [ -z "$ids" ]; then + fail "the image's $noun catalog reports no kind:\"default\" entries — the unbaked-image check would pass vacuously" + return + fi + + baked=$(jq -r '.[] | select(.kind == "default" and .installed == true) | "\(.id) (\(.binary))"' <<<"$json") + if [ -n "$baked" ]; then + fail "the image ships baked default ${noun}s: $(tr '\n' ' ' <<<"$baked")— these must be provisioned into /home/sandbox/.local at boot, not baked" else - fail "jq is required to read the image's harness catalog JSON" + ok "no default $noun is baked into the image ($(tr '\n' ' ' <<<"$ids"))" fi +} + +if command -v jq >/dev/null 2>&1; then + check_no_baked_defaults harness harness + check_no_baked_defaults tool tool else - fail "could not read the harness catalog from the image: $(cat /tmp/verify-sandbox-defaults.err 2>/dev/null | head -3)" + fail "jq is required to read the image's harness and tool catalogs" fi if ((${#failures[@]})); then diff --git a/CHANGELOG.md b/CHANGELOG.md index 6d2b6452..1394c495 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,15 +12,18 @@ Update policy and release automation live in [`/git`](.claude/skills/git/SKILL.m - **BREAKING:** Persist the sandbox home through one `/home/sandbox` mount, not eleven per-tool volumes; set `storage.homePath` for a host path, else `_workspace` ([#898](https://github.com/mifunedev/openharness/issues/898)). - Shrink the sandbox image ~540 MB: drop build caches from the baked home seed, stage the seed once via a builder stage, and keep untracked build output out of the build context ([#900](https://github.com/mifunedev/openharness/issues/900)). - **BREAKING:** Stop baking Claude Code, Codex, and Pi into the image; boot installs them into the home mount, so a first boot needs network and runs 60-180s longer ([#904](https://github.com/mifunedev/openharness/issues/904)). +- **BREAKING:** Stop baking Herdr and cloudflared into the image; both become `kind: "default"` tools installed into `~/.local/bin` at boot from a pinned, checksum-verified binary ([#906](https://github.com/mifunedev/openharness/issues/906)). ### Removed - Remove the `BAKE_HARNESSES` and `AGENTS` build args along with the image bake they gated; the harness catalog is the only source of truth for what gets installed ([#904](https://github.com/mifunedev/openharness/issues/904)). +- Remove Cloudflare's apt repository and its bookworm-suite pin from the image; Docker's is now the only third-party apt source ([#906](https://github.com/mifunedev/openharness/issues/906)). - **BREAKING:** Retire the `projectRoot` / `OH_PROJECT_ROOT` config knob — the checkout is fixed at `/home/sandbox/harness`, nested inside the home mount ([#898](https://github.com/mifunedev/openharness/issues/898)). ### Added - Provision the default harnesses into `/home/sandbox/.local` at boot, gated by `OH_PROVISION_HARNESSES`, so `oh harness install` also works from inside the sandbox ([#902](https://github.com/mifunedev/openharness/issues/902)). - Add `oh-home-mount.sh`, a tier-A probe holding the single-`$HOME`-mount contract: one mount per compose file, the baked `/opt/home-seed`, and the checkout prune that replaces `-xdev` ([#898](https://github.com/mifunedev/openharness/issues/898)). - Assert boot-provisioned harnesses in the boot smoke and reject a baked default harness in `verify-sandbox-image.sh`, so CI exercises the install path ([#904](https://github.com/mifunedev/openharness/issues/904)). +- Add `oh tool list --defaults` and generalize the boot provisioner over both catalogs as `provision-defaults.sh` (`OH_PROVISION_DEFAULTS`) ([#906](https://github.com/mifunedev/openharness/issues/906)). - Add `skills-task-tool-coupling.sh`, a tier-A probe holding the canonical skill pack and the sandbox in agreement about the Claude-Code-only task tools ([#886](https://github.com/mifunedev/openharness/issues/886)). ### Fixed diff --git a/docs/deployment-prebuilt-image.md b/docs/deployment-prebuilt-image.md index b934d978..6c2d4378 100644 --- a/docs/deployment-prebuilt-image.md +++ b/docs/deployment-prebuilt-image.md @@ -239,7 +239,7 @@ into `/home/sandbox/.local`; they are not baked into the image. Expect the boot to run 60–180s longer than the `sleep 8` above and to need network — check with `docker exec "$NAME" bash -lc 'oh harness list --defaults'`. If the registry was unreachable the container still comes up; re-run -`docker exec "$NAME" bash -lc 'bash /home/sandbox/harness/.oh/scripts/provision-harnesses.sh'`. +`docker exec "$NAME" bash -lc 'bash /home/sandbox/harness/.oh/scripts/provision-defaults.sh'`. ```bash # ── 4. Attach an interactive shell (once the container is stable) ── diff --git a/docs/installation.md b/docs/installation.md index 2223e93a..c9b45a9f 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -276,28 +276,32 @@ Project-local Pi packages are loaded from `.pi/settings.json`; the defaults incl Debian Trixie (slim), the current Debian stable. The `sandbox` user has passwordless sudo. -Docker's apt repository tracks the `trixie` suite. Cloudflare's stays on `bookworm`: Cloudflare publishes no Trixie suite (`pkg.cloudflare.com/cloudflared/dists/trixie` returns HTTP 404) and its Bookworm `cloudflared` package runs on Trixie. +Docker's apt repository tracks the `trixie` suite, and it is now the only third-party apt source in the image. cloudflared used to force a `bookworm` suite here because Cloudflare publishes no Trixie suite (`pkg.cloudflare.com/cloudflared/dists/trixie` returns HTTP 404); moving it to a pinned, checksum-verified binary in the tool catalog removed that exception. ### AI agent CLIs Default CLIs are not baked into the image. The entrypoint runs -`.oh/scripts/provision-harnesses.sh` on every boot, which installs any missing -default harness into `~/.local` — inside the home mount — as the `sandbox` user. -That is what makes `oh harness install ` able to upgrade one in place: a copy -under `/usr/lib/node_modules` would be root-owned and unwritable from a running -sandbox. Consequences worth knowing: +`.oh/scripts/provision-defaults.sh` on every boot, which installs any missing +default **harness** (Claude Code, Codex, Pi) and default **tool** (Herdr, +cloudflared) into `~/.local` — inside the home mount — as the `sandbox` user. +That is what makes `oh harness install ` and `oh tool install ` able to +upgrade one in place: a copy in a root-owned system path is unwritable from a +running sandbox. Consequences worth knowing: - A **first boot on a fresh home mount needs network**. Measured at 21s on a GitHub Actions runner; budget 60–180s on a slower link. The compose healthcheck's `start_period` is 600s to cover it. -- If the registry is unreachable the sandbox still comes up as a usable shell, - with a warning and no agent CLIs. Re-run - `bash .oh/scripts/provision-harnesses.sh` once you have network. +- If the network is unreachable the sandbox still comes up as a usable shell, + with a warning and no agent CLIs — **and no Herdr**, so `oh shell` lands you in + a plain shell and `tmux` is the fallback multiplexer. Re-run + `bash .oh/scripts/provision-defaults.sh` once you have network. - An existing install is never replaced, so the provisioner is a no-op on every - boot after the first. Upgrade deliberately with `oh harness install `. + boot after the first. Upgrade deliberately with `oh harness install ` or + `oh tool install `. +- Every download is pinned and `sha256sum`-verified before it is installed. - npm's cache now lives in the home mount at `~/.npm` and grows across upgrades. `npm cache clean --force` reclaims it. -- Set `OH_PROVISION_HARNESSES=false` to skip the step entirely. +- Set `OH_PROVISION_DEFAULTS=false` to skip the step entirely. Optional CLIs are excluded from the default image; `oh harness install ` flips the matching `install.*` field in `oh.json` and installs it. @@ -324,15 +328,17 @@ Optional CLIs are excluded from the default image; `oh harness install ` f ### DevOps & infrastructure `oh tool list` reports which of these are present, and `oh tool status ` -adds a version where the tool has a verified version flag. Unlike the agent -CLIs above, these are baked into the image, so there is nothing to install. +adds a version where the tool has a verified version flag. Herdr and cloudflared +are `kind: "default"` — provisioned into `~/.local/bin` at boot from a pinned, +checksum-verified binary, and upgradeable in place. The rest are baked into the +image, so there is nothing to install. | Tool | Purpose | |------|---------| -| Herdr (`herdr`) | Default multi-agent terminal workspace; state persists across rebuilds in dedicated volumes | +| Herdr (`herdr`) | Default multi-agent terminal workspace; provisioned at boot, state and binary both persist in the home mount | | Docker CLI + Compose | Container management from inside the sandbox (host docker socket bind-mounted by the base compose) | | GitHub CLI (`gh`) | PRs, issues, releases from the terminal | -| cloudflared | Cloudflare Tunnel client, for exposing a sandbox port (see the `/cloudflared` skill) | +| cloudflared | Cloudflare Tunnel client, for exposing a sandbox port (see the `/cloudflared` skill); provisioned at boot | | tmux | Detachable terminal sessions for long-running agents | | croner | Markdown-frontmatter cron scheduler for autonomous agent tasks | diff --git a/docs/integrations/herdr.md b/docs/integrations/herdr.md index f5d90481..485c8234 100644 --- a/docs/integrations/herdr.md +++ b/docs/integrations/herdr.md @@ -58,6 +58,6 @@ herdr server stop # end a broken Herdr server herdr --no-session # run Herdr without its server/client session ``` -Herdr is pinned in the Open Harness image. Upgrade it by rebuilding against a reviewed Open Harness release rather than self-updating `/usr/local/bin/herdr`. +Herdr is pinned in the tool catalog (`.oh/cli/src/lib/tools/catalog.ts`) and provisioned into `~/.local/bin/herdr` at boot from a checksum-verified binary. Upgrade it by bumping that pin and running `oh tool install herdr`, not by self-updating the binary in place. See the upstream [quick start](https://herdr.dev/docs/quick-start/), [agents guide](https://herdr.dev/docs/agents/), and [configuration reference](https://herdr.dev/docs/configuration/). diff --git a/docs/quickstart.md b/docs/quickstart.md index 024d73cd..5f89d65d 100644 --- a/docs/quickstart.md +++ b/docs/quickstart.md @@ -119,12 +119,13 @@ continue to run independently under tmux. ## Set up agents inside Herdr -The sandbox provisions Claude Code, Codex, and Pi into `~/.local` on first boot — +The sandbox provisions Claude Code, Codex, and Pi — plus the Herdr and cloudflared +tools — into `~/.local` on first boot — they live in the home mount, not the image, so `oh harness install ` upgrades them in place without a rebuild. A first boot on a fresh home mount therefore needs network access and takes a minute or two longer; the sandbox still comes up as a usable shell if the registry is unreachable, and you can retry with -`bash .oh/scripts/provision-harnesses.sh`. OpenCode, DeepAgents, Hermes, and Grok +`bash .oh/scripts/provision-defaults.sh`. OpenCode, DeepAgents, Hermes, and Grok Build are optional image-level installs; T3 Code runs on demand via the `/t3` skill or direct `npx`. Authenticate at least one harness before use. From ecfda4c6e9d80eb7b1c7fb983889282580a454e7 Mon Sep 17 00:00:00 2001 From: Ryan Eggleston Date: Mon, 31 Aug 2026 14:40:37 -0600 Subject: [PATCH 08/23] FROM task/908-cli-first-class TO development (#909) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * task: provision the default harnesses into the home mount oh is meant to split by execution target -- on the host it provisions the host or the sandbox, and inside the sandbox it provisions the sandbox with harnesses and tools. The second half did not work for the two harnesses most people use. claude-code and codex carried installUser: "root", which against the local execution target becomes sudo -n -- npm install -g, and /etc/sudoers.d/sandbox grants sandbox ALL=(ALL) ALL with no NOPASSWD. sudo -n true returns "a password is required". Both now match the pi entry directly above them: installUser "sandbox", npm --prefix /home/sandbox/.local install -g. That lands them inside the home mount, so they also survive container recreate and can be upgraded in place in a running remote sandbox rather than requiring an image rebuild. claude-code deliberately does not get --ignore-scripts. Its postinstall copies the native binary over a placeholder; with the flag the install succeeds and claude --version then fails with "claude native binary not installed". Verified both ways against a scratch prefix. provision-harnesses.sh follows provision-python.sh: the same mode flag, the same root to gosu sandbox re-exec, the same ownership diagnostics, the same die-with- the-command-to-re-run style. --print-env is absent because this provisioner exports nothing downstream. It reads the catalog through oh harness list --json and installs through oh harness install, so the shell knows no ids, packages, prefixes, or argv, and the TypeScript catalog stays the only description. No default harness carries a version pin today, so an existing install is never replaced and the script says so in its own output rather than implying it refreshes. The entrypoint hook runs after link-providers.sh, not before. link-providers' only binary dependency is cc-safety-net, which stays baked, and it is the boot-critical hard gate; a network-dependent best-effort step does not belong in front of the step that decides whether the boot is viable. Provisioning warns and continues, so an offline sandbox still comes up as a usable shell. BAKE_HARNESSES defaults to true and gates only the $AGENTS loop, not the whole RUN. INSTALL_OPENCODE and INSTALL_GROK_BUILD are separate opt-ins and turning them off as a side effect would be a silent regression. Nothing leaves the image in this change. * fix: bound the boot path and close four more provisioning defects An adversarial audit of #903 found five defects that six green checks missed. The serious one is a boot hang. oh harness list --json probes every entry in the catalog, not just the three defaults, and one of them is t3code, whose verifyArgv is npx --no-install t3 --version. npx contacts the registry, and probeInstalled passed no timeoutMs, so spawnSync waited without bound. Against an unreachable registry the auditor's run was still going at 2m30 when their own timeout killed it. On any boot where DNS resolves but the registry does not answer, the entrypoint blocks before sleep infinity, exceeds the 300s start_period, and never goes healthy -- and restart: unless-stopped does not rescue an unhealthy-but-alive container. Warn-and-continue cannot help, because a hang never reaches the if !. It hangs while listing, before any install, so BAKE_HARNESSES=true did not avoid it either. Bounded at three layers, because each fails differently: a 15s timeoutMs on the probe spawn, reported as unknown rather than a crash; a --defaults filter on oh harness list so the boot path probes three entries instead of nine and never runs npx; and a timeout wrapper on the entrypoint call so the boot is bounded whatever the CLI does. Measured against the auditor's exact command: 2m30 and killed, to 17.0s full-catalog and 1.55s with --defaults. The install loop read from a herestring while installs run with stdio inherit, so an installer that reads stdin consumed the rest of the loop. Reproduced with a stub: three missing harnesses, one installed, exit 0, success printed. Latent with npm, live the moment a default uses the curl | bash shape two catalog entries already use. Installs now read from /dev/null. The script force-exported OH_EXECUTION_TARGET=local, which short-circuits the in-container check, while the prefix is hardcoded to /home/sandbox/.local, and every error told the operator to re-run with no mention of where. On the host that provisioned the host. It now refuses unless inside the sandbox, reusing the CLI's own runningInsideSandbox predicate rather than inventing a check, and the entrypoint asserts the local target explicitly -- the documented raw docker run recipe never passes SANDBOX_NAME, so the guard would otherwise have silently skipped provisioning for the prebuilt-image flavor. The probe asserted that ARG BAKE_HARNESSES was declared, not that anything used it: deleting the gate left it green. It now checks the ARG is referenced by the RUN that installs $AGENTS and by the one that bakes pi, and that both stages declare it. Deleting either declaration also used to pass. ARG is stage-scoped, so BAKE_HARNESSES=false unbaked claude-code and codex but left pi baked in the home stage while the else-branch claimed otherwise. The home stage now declares and honors the flag. Also: OH_SANDBOX_USER was advertised but illusory, since the catalog hardcodes the user and prefix; it is gone. The final log line no longer claims to have provisioned anything in --verify mode. * task: stop baking the default harnesses into the sandbox image PR #903 wired provision-harnesses.sh into the boot path but shipped it behind ARG BAKE_HARNESSES=true, so every default harness was already present when the provisioner ran and the install path never executed. All four defects that PR's audit found lived in code a green CI run and a normal boot both skip. Delete the bake rather than flip its default: remove ARG BAKE_HARNESSES, ARG AGENTS, the PKG map and the $AGENTS loop in `base`, and the gated pi install in `home`. A build arg that can re-bake is a dormant path that would restore both the shadowed /usr/lib/node_modules copy and the untested boot install. Make the install path CI-visible, since it is now load-bearing on every boot: - The boot smoke asserts the outcome — each default harness resolves under NPM_USER_PREFIX via `type -P`, is owned by the reconciled sandbox uid, and prints its own version — and refuses to pass when the catalog reports no defaults. It boots on a fresh home volume, so this runs real npm work. - verify-sandbox-image.sh gains the negative: reading the catalog out of the image itself, no kind:"default" harness may be installed. - start_period goes 300s -> 600s in both compose files to cover the install, and the boot-guard probe now derives the smoke deadline from the healthcheck window instead of pinning a literal that a start_period bump could invert. - The probe and unit assertions invert from "the bake is gated" to "no default harness package appears in the Dockerfile", reading the package names out of installArgv so they cannot drift from the catalog. cc-safety-net stays baked. Opt-in INSTALL_* harnesses are untouched. Costs this accepts, documented in installation.md: a first boot on a fresh home mount needs network and runs 60-180s longer; an offline first boot yields a usable shell with no agent CLIs; ~/.npm now lives in the home mount. Closes #904 * task: cite the measured first-boot provisioning time * task: move herdr and cloudflared out of the image into the tool catalog Per the ownership boundary — the in-sandbox CLI provisions harnesses and tools — herdr and cloudflared are tools, so the image should not carry them. #905 did this for the default harnesses; this does it for the default tools. The obvious template does not work. #897's tailscale entry root-installs to /usr/local/bin, and commands/tool.ts:309 passes stdio:"inherit", so local-target.ts:113-116 selects the INTERACTIVE branch — plain `sudo --`, no -n. /etc/sudoers.d/sandbox grants `sandbox ALL=(ALL) ALL` with no NOPASSWD, so `oh tool install ` hangs on a password prompt no agent can answer. Verified in a running sandbox: `sudo -n -- true` → "a password is required". (#897's tailscale has the same defect; flagged there, not fixed here.) So install to ~/.local/bin as the sandbox user instead, the same correction #900 made for the harnesses. No sudo, survives container recreation in the home mount, and upgradeable in place by a running sandbox. - ToolKind gains "default". herdr 0.7.4 and cloudflared 2026.8.2 become kind:"default", installUser:"sandbox", with per-arch pinned URLs and sha256 verification into $NPM_USER_PREFIX/bin. Checksums measured by downloading both arches, not copied from anywhere. - provision-harnesses.sh generalizes over both catalogs and becomes provision-defaults.sh (OH_PROVISION_DEFAULTS, timeout 180s → 240s). It dies rather than reporting success when neither catalog yields a default. - The Dockerfile loses the herdr RUN, ARG HERDR_VERSION, and the whole cloudflared apt block — with it the bookworm-suite workaround that existed only because Cloudflare publishes no trixie suite. Docker's is now the only third-party apt source. - Both oracles generalize: verify-sandbox-image.sh rejects a baked default harness OR tool, reading each catalog out of the image; the boot smoke asserts every default in both catalogs resolves under NPM_USER_PREFIX, is owned by the sandbox uid, and prints a version. - The herdr version+checksum pin moves from the Dockerfile to the catalog, and herdr-default.test.ts follows it. Costs, documented in installation.md: an offline first boot on a fresh home mount now has no herdr, so `oh shell` lands in a plain shell with tmux as the fallback multiplexer. The entrypoint says so explicitly on failure. Closes #906 * task: remove optional-harness build args and make the CLI the install path #905 and #907 moved the default harnesses and tools out of the image but left the four optional harnesses behind. The boundary — inside the sandbox the CLI provisions harnesses and tools — has no carve-out for optional ones. They were not merely leftover. opencode, grok-build, and hermes are all installUser:"root", and harness.ts:256 installs with stdio:"inherit", so local-target.ts selects the INTERACTIVE branch: plain `sudo --`, no -n. /etc/sudoers.d/sandbox has no NOPASSWD, so `oh harness install opencode` hangs on a password prompt no agent can answer. The build arg was the only working path, which is why the Dockerfile blocks could not simply be deleted. All four relocate to the sandbox user, verified by reading the upstream installers rather than guessing: opencode takes an npm --prefix like claude-code; grok's installer honours GROK_BIN_DIR; hermes honours HERMES_INSTALL_DIR and its get_command_link_dir() already picks ~/.local/bin for a non-root install; deepagents was already sandbox-installed via uv. So no sudoers change is needed and no security posture moves. - Delete all four ARG/RUN pairs, the compose build.args block, and the dead /opt/grok-build and /usr/local/lib/hermes-agent chowns. INSTALL_HERMES keeps its RUNTIME life — link-providers.sh vendors the Hermes skill pack from it and entrypoint.sh wires auth.json — so only its build-arg role goes. - Remove `buildArg` from HarnessEntry entirely. It was dead metadata: declared, set four times, read by nothing. tool-catalog-boundary.sh already banned the same field in the tool catalog. - provision-defaults.sh now reads the full catalog and also installs any non-default entry whose install. is true, so declared intent survives a fresh home mount. isInstallFlagEnabled already reads oh.json, so this needs no new env plumbing. - verify-sandbox-image.sh widens to "no harness of any kind is baked", and gains the inverse for tools: every kind:"baked-in" tool must be present, or the check passes on an image missing everything. - sandbox-compatibility.yml's optional-installer job loses its subject. It now boots the image and runs `oh harness install` for each optional harness, asserting the binary lands under /home/sandbox/.local — the path operators actually use, instead of one that no longer exists. harness.test.ts had a case asserting `cmd === "sudo"`, codifying the very defect this fixes. It now asserts no install shells out to sudo at all. Closes #908 * fix: retry an optional-harness install once before failing the job The new compatibility job reaches four third-party endpoints. Hermes' own installer hard-fails the whole install when its internal `npm install` step blips, which took the job down on a commit that was correct — the rerun passed unchanged, on the same SHA. A vendor's transient error must not block this repo's merges. One retry absorbs it. The contract is unchanged: a genuine break — wrong user, wrong path, a sudo prompt — fails both attempts and still fails the job. The probe now asserts both halves, so neither the retry nor the hard failure after it can be dropped silently. * fix(test): drop a template-literal scan that fired on catalog order The source-level regex I added in #906 cannot distinguish a JS backtick from a backtick inside prose — notInstallableReason has several. It passed only because every ${...} in the catalog happened to precede the first prose backtick. Adding a tool below them flips it to a false failure, which is exactly what happened on the #858 branch. The per-token ban, with the bash -lc body exempted, covers what is actually checkable. --- .devcontainer/Dockerfile | 34 --- .devcontainer/docker-compose.yml | 5 - .github/workflows/sandbox-compatibility.yml | 80 +++++-- .oh/cli/src/__tests__/harness-catalog.test.ts | 69 +++--- .oh/cli/src/__tests__/harness.test.ts | 19 +- .oh/cli/src/__tests__/tool-catalog.test.ts | 13 +- .oh/cli/src/lib/harnesses/catalog.ts | 28 ++- .oh/evals/RESULTS.md | 206 +++++++++--------- .oh/evals/probes/default-provisioning.sh | 40 +++- .oh/evals/probes/sandbox-boot-guard-ci.sh | 22 +- .oh/install/banner.sh | 8 +- .../__tests__/verify-sandbox-image.test.ts | 71 +++--- .oh/scripts/provision-defaults.sh | 17 +- .oh/scripts/verify-sandbox-image.sh | 55 +++-- CHANGELOG.md | 3 + docs/configuration.md | 8 +- docs/harnesses/deepagents.md | 11 +- docs/harnesses/grok-build.md | 10 +- docs/harnesses/hermes.md | 24 +- docs/harnesses/opencode.md | 8 +- docs/harnesses/overview.md | 12 +- docs/harnesses/prime-agent.md | 2 +- docs/quickstart.md | 2 +- 23 files changed, 432 insertions(+), 315 deletions(-) diff --git a/.devcontainer/Dockerfile b/.devcontainer/Dockerfile index 1207ca0e..1f0f7162 100644 --- a/.devcontainer/Dockerfile +++ b/.devcontainer/Dockerfile @@ -34,25 +34,6 @@ RUN curl -LsSf https://astral.sh/uv/install.sh | env INSTALLER_NO_MODIFY_PATH=1 && cp /root/.local/bin/uv /usr/local/bin/uv \ && cp /root/.local/bin/uvx /usr/local/bin/uvx -ARG INSTALL_HERMES=false -RUN if [ "${INSTALL_HERMES}" = "true" ]; then \ - curl -fsSL https://hermes-agent.nousresearch.com/install.sh \ - | bash -s -- --skip-setup --skip-browser; \ - rm -rf /root/.hermes; \ - uv pip install --python /usr/local/lib/hermes-agent/venv/bin/python 'hermes-agent[slack,teams,web,pty]'; \ - else \ - echo "Skipping Hermes Agent CLI install (INSTALL_HERMES=false)"; \ - fi - -ARG INSTALL_DEEPAGENTS=false -RUN export UV_TOOL_DIR=/opt/uv/tools UV_TOOL_BIN_DIR=/usr/local/bin \ - && mkdir -p "$UV_TOOL_DIR" \ - && if [ "${INSTALL_DEEPAGENTS}" = "true" ]; then \ - uv tool install deepagents-cli; \ - else \ - echo "Skipping DeepAgents CLI install (INSTALL_DEEPAGENTS=false)"; \ - fi - ENV PNPM_HOME="/usr/local/share/pnpm" ENV NPM_USER_PREFIX="/home/sandbox/.local" ENV PATH="$NPM_USER_PREFIX/bin:$PNPM_HOME:$PATH" @@ -62,18 +43,6 @@ RUN corepack enable && corepack prepare pnpm@10.33.0 --activate \ SHELL ["/bin/bash", "-c"] -ARG INSTALL_OPENCODE=false -ARG INSTALL_GROK_BUILD=false - -RUN set -e; \ - if [ "${INSTALL_OPENCODE}" = "true" ]; then npm install -g opencode-ai; \ - else echo "Skipping OpenCode CLI install (INSTALL_OPENCODE=false)"; fi; \ - if [ "${INSTALL_GROK_BUILD}" = "true" ]; then \ - curl -fsSL https://x.ai/cli/install.sh | HOME=/opt/grok-build GROK_BIN_DIR=/opt/grok-build/bin bash -s 0.2.39; \ - ln -sf /opt/grok-build/bin/grok /usr/local/bin/grok; \ - rm -f /usr/local/bin/agent; \ - else echo "Skipping Grok Build CLI install (INSTALL_GROK_BUILD=false)"; fi - RUN npm install -g cc-safety-net@1.0.6 COPY .oh/cli/ /opt/oh/ @@ -146,9 +115,6 @@ RUN if [ "${INSTALL_PYTHON_KERNEL}" = "true" ]; then \ && rm -rf /home/sandbox/.cache/uv FROM base AS final -RUN chown -R sandbox:sandbox /opt/uv 2>/dev/null || true \ - && if [ -d /usr/local/lib/hermes-agent ]; then chown -R sandbox:sandbox /usr/local/lib/hermes-agent; fi - RUN mkdir -p /.devcontainer && echo '{"workspaceFolder":"/home/sandbox/harness","remoteUser":"sandbox"}' > /.devcontainer/devcontainer.json LABEL devcontainer.metadata='[{"remoteUser":"sandbox","workspaceFolder":"/home/sandbox/harness"}]' diff --git a/.devcontainer/docker-compose.yml b/.devcontainer/docker-compose.yml index 8a99d972..796f73b3 100644 --- a/.devcontainer/docker-compose.yml +++ b/.devcontainer/docker-compose.yml @@ -39,11 +39,6 @@ services: build: context: .. dockerfile: .devcontainer/Dockerfile - args: - INSTALL_OPENCODE: ${INSTALL_OPENCODE:-false} - INSTALL_GROK_BUILD: ${INSTALL_GROK_BUILD:-false} - INSTALL_DEEPAGENTS: ${INSTALL_DEEPAGENTS:-false} - INSTALL_HERMES: ${INSTALL_HERMES:-false} volumes: - ${OH_HOME_MOUNT:-workspace}:/home/sandbox - ..:/home/sandbox/harness diff --git a/.github/workflows/sandbox-compatibility.yml b/.github/workflows/sandbox-compatibility.yml index a09f0785..d43a69e3 100644 --- a/.github/workflows/sandbox-compatibility.yml +++ b/.github/workflows/sandbox-compatibility.yml @@ -50,8 +50,8 @@ jobs: node:22-bookworm-slim \ node:22-trixie-slim - optional-installers-image: - name: Build one amd64 image with every optional installer + optional-harness-install: + name: Install every optional harness through the CLI runs-on: ${{ vars.CI_RUNNER || 'ubuntu-latest' }} steps: @@ -64,31 +64,81 @@ jobs: - name: Ensure Mifune submodule run: bash .oh/scripts/link-providers.sh --init - - name: Build with all INSTALL_* paths enabled + # #908 removed the INSTALL_* build args, so there is no longer an image to + # build with every optional installer enabled. The path that replaced it — + # `oh harness install ` into the home mount, as the sandbox user — is + # what this job must exercise instead. Building and then INSTALLING is + # strictly closer to what an operator does than the old build-arg matrix. + - name: Build the sandbox image run: | docker build \ --file .devcontainer/Dockerfile \ - --build-arg INSTALL_HERMES=true \ - --build-arg INSTALL_DEEPAGENTS=true \ - --build-arg INSTALL_OPENCODE=true \ - --build-arg INSTALL_GROK_BUILD=true \ --tag openharness-sandbox-optional:${{ github.sha }} \ . - - name: Verify the optional image still satisfies the default contract + - name: Verify the image satisfies the default contract run: bash .oh/scripts/verify-sandbox-image.sh openharness-sandbox-optional:${{ github.sha }} - - name: Verify a version from every optional CLI + - name: Install each optional harness through the CLI and verify its version run: | set -euo pipefail - for tool in hermes deepagents opencode grok; do - echo "--- $tool --version" - output="$(docker run --rm --entrypoint /bin/bash \ - openharness-sandbox-optional:${{ github.sha }} \ - -lc "$tool --version" 2>&1)" + cid=$(docker run -d --rm \ + -e SANDBOX_NAME=openharness-optional-${{ github.run_id }} \ + -e OH_PROVISION_DEFAULTS=false \ + --entrypoint sleep \ + openharness-sandbox-optional:${{ github.sha }} infinity) + trap 'docker rm -f "$cid" >/dev/null 2>&1 || true' EXIT + + # The image ships an empty /home/sandbox; seed it the way the + # entrypoint would so ~/.local exists and is sandbox-owned. + docker exec "$cid" bash -lc 'cp -a -n /opt/home-seed/. /home/sandbox/ 2>/dev/null || true; chown -R sandbox:sandbox /home/sandbox' + + ids=$(docker exec -u sandbox "$cid" bash -lc \ + 'cd /opt/oh-seed && OH_EXECUTION_TARGET=local oh harness list --json' \ + | jq -r '.[] | select(.kind == "optional") | .id') + if [ -z "$ids" ]; then + echo "ERROR: the harness catalog declares no optional harnesses — this job would pass vacuously" >&2 + exit 1 + fi + + # These four installs reach four third-party endpoints. A transient + # upstream failure must not block this repo's merges: Hermes' own + # installer hard-fails the whole install when its `npm install` step + # blips, which took this job down once on a commit that was correct + # (the rerun passed unchanged). One retry absorbs that. It does NOT + # weaken the contract below — a genuine break (wrong user, wrong + # path, a sudo prompt) fails both attempts and still fails the job. + for id in $ids; do + for attempt in 1 2; do + echo "--- oh harness install $id (attempt $attempt)" + if docker exec -u sandbox "$cid" bash -lc \ + "cd /opt/oh-seed && OH_EXECUTION_TARGET=local oh harness install '$id' --no-persist"; then + break + fi + if [ "$attempt" = 2 ]; then + echo "ERROR: $id failed to install twice — this is not a transient upstream blip" >&2 + exit 1 + fi + echo "$id install failed; retrying once after 15s in case the upstream endpoint blipped" >&2 + sleep 15 + done + done + + for id in $ids; do + binary=$(docker exec -u sandbox "$cid" bash -lc \ + 'cd /opt/oh-seed && OH_EXECUTION_TARGET=local oh harness list --json' \ + | jq -r --arg id "$id" '.[] | select(.id == $id) | .binary') + echo "--- $binary --version" + output=$(docker exec -u sandbox "$cid" bash -lc " + path=\$(type -P '$binary') + case \"\$path\" in + /home/sandbox/.local/*) ;; + *) echo \"$id resolved to '\$path', not under /home/sandbox/.local\" >&2; exit 1 ;; + esac + \"\$path\" --version" 2>&1) printf '%s\n' "$output" if ! grep -Eq '(^|[^[:alnum:]])v?[0-9]+([.][0-9]+)+([^[:alnum:]]|$)' <<<"$output"; then - echo "ERROR: $tool --version did not output a numeric dotted version" >&2 + echo "ERROR: $binary --version did not output a numeric dotted version" >&2 exit 1 fi done diff --git a/.oh/cli/src/__tests__/harness-catalog.test.ts b/.oh/cli/src/__tests__/harness-catalog.test.ts index 47a62bd8..2b3290c0 100644 --- a/.oh/cli/src/__tests__/harness-catalog.test.ts +++ b/.oh/cli/src/__tests__/harness-catalog.test.ts @@ -42,12 +42,6 @@ describe("harness catalog", () => { } }); - it("pairs harnessKey and buildArg — never one without the other", () => { - for (const h of HARNESS_CATALOG) { - expect(Boolean(h.harnessKey)).toBe(Boolean(h.buildArg)); - } - }); - it("gives every optional harness a flag, and no other kind one", () => { for (const h of HARNESS_CATALOG) { if (h.kind === "optional") expect(h.harnessKey).toBeDefined(); @@ -67,11 +61,13 @@ describe("harness catalog", () => { } }); - describe("does not drift from the image build", () => { - const flagged = HARNESS_CATALOG.filter((h) => h.buildArg !== undefined); + // #908: the INSTALL_* build args are gone. The catalog no longer mirrors the + // Dockerfile — it replaces it, and `oh harness install` is the only path. + describe("owns the install, and the image no longer does", () => { + const optional = HARNESS_CATALOG.filter((h) => h.kind === "optional"); it("covers all four optional harnesses", () => { - expect(flagged.map((h) => h.id).sort()).toEqual([ + expect(optional.map((h) => h.id).sort()).toEqual([ "deepagents", "grok-build", "hermes", @@ -79,48 +75,55 @@ describe("harness catalog", () => { ]); }); - it.each(flagged.map((h) => [h.id, h] as const))( - "%s: build arg is in the Dockerfile", + it("declares no buildArg anywhere — the field itself is gone", () => { + expect(read(".oh/cli/src/lib/harnesses/catalog.ts")).not.toContain("buildArg"); + }); + + it.each(optional.map((h) => [h.id, h] as const))( + "%s: its INSTALL_* build arg is absent from the Dockerfile", (_id, h) => { - expect(DOCKERFILE).toContain(h.buildArg as string); + const arg = `INSTALL_${(h.harnessKey as string).toUpperCase()}`; + expect(DOCKERFILE).not.toMatch(new RegExp(`^ARG ${arg}`, "m")); + expect(COMPOSE_YML).not.toContain(`${arg}: \${${arg}:-false}`); }, ); - it.each(flagged.map((h) => [h.id, h] as const))( - "%s: the INSTALL_* key derived from harnessKey IS the build arg, and compose forwards it", + it.each(optional.map((h) => [h.id, h] as const))( + "%s: installs as the sandbox user into the home mount", (_id, h) => { - expect(`INSTALL_${(h.harnessKey as string).toUpperCase()}`).toBe(h.buildArg); - expect(COMPOSE_YML).toContain(`${h.buildArg}: \${${h.buildArg}:-false}`); + expect(h.installUser).toBe("sandbox"); + expect(h.installArgv.join("\n")).toMatch(/\/home\/sandbox\/\.local|\$HOME\/\.local|uv/); }, ); - it.each(flagged.map((h) => [h.id, h] as const))( - "%s: key ships documented in docs/configuration.md, the oh.json field reference", + it.each(optional.map((h) => [h.id, h] as const))( + "%s: its oh.json key stays documented in docs/configuration.md", (_id, h) => { + const arg = `INSTALL_${(h.harnessKey as string).toUpperCase()}`; expect(CONFIG_DOC).toMatch( - new RegExp(`^\\| \`install\\.[A-Za-z]+\` \\|.*\`${h.buildArg}\``, "m"), + new RegExp(`^\\| \`install\\.[A-Za-z]+\` \\|.*\`${arg}\``, "m"), ); }, ); - it.each(flagged.map((h) => [h.id, h] as const))( - "%s: every pinned version appears verbatim in the Dockerfile", - (_id, h) => { - for (const pin of versionPins(h.installArgv)) { - expect(DOCKERFILE).toContain(pin); - } - }, - ); - - it("grok-build keeps the Dockerfile's exact pin", () => { + it("keeps the grok-build pin in the catalog, now that the Dockerfile has none", () => { const grok = findHarness("grok-build"); expect(versionPins(grok!.installArgv)).toEqual(["0.2.39"]); - expect(DOCKERFILE).toContain("bash -s 0.2.39"); + expect(DOCKERFILE).not.toContain("bash -s 0.2.39"); + }); + + // INSTALL_HERMES survives as a RUNTIME flag: link-providers.sh vendors the + // Hermes skill pack from it and entrypoint.sh wires auth.json. Only its + // build-arg role is gone. + it("keeps INSTALL_HERMES as a container environment variable", () => { + expect(COMPOSE_YML).toContain("- INSTALL_HERMES=${INSTALL_HERMES:-false}"); + expect(DOCKERFILE).not.toContain("INSTALL_HERMES"); }); - it("installs deepagents and pi as the sandbox user, not root", () => { - expect(findHarness("deepagents")!.installUser).toBe("sandbox"); - expect(findHarness("pi")!.installUser).toBe("sandbox"); + it("installs every harness as the sandbox user, never root", () => { + for (const h of HARNESS_CATALOG) { + expect(h.installUser, h.id).toBe("sandbox"); + } expect(DOCKERFILE).toContain("UV_TOOL_DIR=/home/sandbox"); }); }); diff --git a/.oh/cli/src/__tests__/harness.test.ts b/.oh/cli/src/__tests__/harness.test.ts index 06b9840f..acbde334 100644 --- a/.oh/cli/src/__tests__/harness.test.ts +++ b/.oh/cli/src/__tests__/harness.test.ts @@ -272,8 +272,17 @@ describe("runHarnessInstall against the container", () => { const install = execCalls(calls).find((c) => c.args.includes("opencode-ai")); expect(install).toBeDefined(); expect(install!.args).toContain("-u"); - expect(install!.args).toContain("root"); - expect(install!.args.slice(-4)).toEqual(["npm", "install", "-g", "opencode-ai"]); + // #908: every harness installs as the sandbox user into the home mount. + expect(install!.args).toContain("sandbox"); + expect(install!.args).not.toContain("root"); + expect(install!.args.slice(-6)).toEqual([ + "npm", + "--prefix", + "/home/sandbox/.local", + "install", + "-g", + "opencode-ai", + ]); expect(text(out)).toContain("installed"); expect(text(out)).toContain( "https://github.com/mifunedev/openharness/blob/main/docs/harnesses/opencode.md", @@ -508,7 +517,11 @@ describe("oh harness — inside the sandbox", () => { const { io, out } = makeIo(); expect(await runHarnessInstall("opencode", { cwd: root, run, env: INSIDE }, io)).toBe(0); expect(text(out)).not.toContain("skipping the live install"); - expect(calls.some((c) => c.cmd === "sudo" && c.args.includes("opencode-ai"))).toBe(true); + // #908: this previously asserted `cmd === "sudo"`, codifying the very defect + // that made `oh harness install opencode` hang inside the sandbox — + // stdio:"inherit" selects plain `sudo --`, and sandbox has no NOPASSWD. + expect(calls.some((c) => c.cmd === "sudo")).toBe(false); + expect(calls.some((c) => c.args.includes("opencode-ai"))).toBe(true); expect(installFlag(root, "opencode")).toBe(true); }); diff --git a/.oh/cli/src/__tests__/tool-catalog.test.ts b/.oh/cli/src/__tests__/tool-catalog.test.ts index 1e72346a..3057999b 100644 --- a/.oh/cli/src/__tests__/tool-catalog.test.ts +++ b/.oh/cli/src/__tests__/tool-catalog.test.ts @@ -75,14 +75,11 @@ describe("tool catalog shape", () => { }); it("passes argv arrays with no interpolation this process performs", () => { - // The hazard is a JS template literal that Node expands before the argv - // ever reaches a shell. A `bash -lc` script body legitimately contains - // ${...} for the shell IN the container to expand, so exempt that one - // token and forbid backticks in the catalog source instead. - expect( - read(".oh/cli/src/lib/tools/catalog.ts"), - "a template literal with ${...} would be expanded by Node before any shell sees it", - ).not.toMatch(/`[^`]*\$\{/s); + // A `bash -lc` script body legitimately contains ${...} for the shell IN the + // container to expand, so that one token is exempt. A source-level scan for + // an interpolating template literal was tried and removed: it cannot tell a + // JS backtick from a backtick inside prose (`notInstallableReason` has + // several), so whether it fired depended on catalog ORDER, not the hazard. for (const t of TOOL_CATALOG) { for (const argv of [t.installArgv, t.verifyArgv, t.versionArgv]) { if (!argv) continue; diff --git a/.oh/cli/src/lib/harnesses/catalog.ts b/.oh/cli/src/lib/harnesses/catalog.ts index b6c0f408..c677d52c 100644 --- a/.oh/cli/src/lib/harnesses/catalog.ts +++ b/.oh/cli/src/lib/harnesses/catalog.ts @@ -9,7 +9,6 @@ export interface HarnessEntry { readonly title: string; readonly binary: string; readonly harnessKey?: string; - readonly buildArg?: string; readonly installArgv: readonly string[]; readonly installUser: "root" | "sandbox"; readonly verifyArgv: readonly string[]; @@ -75,9 +74,15 @@ export const HARNESS_CATALOG: readonly HarnessEntry[] = [ title: "OpenCode", binary: "opencode", harnessKey: "opencode", - buildArg: "INSTALL_OPENCODE", - installArgv: ["npm", "install", "-g", "opencode-ai"], - installUser: "root", + installArgv: [ + "npm", + "--prefix", + "/home/sandbox/.local", + "install", + "-g", + "opencode-ai", + ], + installUser: "sandbox", verifyArgv: ["opencode", "--version"], docsPath: "docs/harnesses/opencode.md", kind: "optional", @@ -87,13 +92,12 @@ export const HARNESS_CATALOG: readonly HarnessEntry[] = [ title: "Grok Build", binary: "grok", harnessKey: "grok_build", - buildArg: "INSTALL_GROK_BUILD", installArgv: [ "bash", "-lc", - "curl -fsSL https://x.ai/cli/install.sh | HOME=/opt/grok-build GROK_BIN_DIR=/opt/grok-build/bin bash -s 0.2.39 && ln -sf /opt/grok-build/bin/grok /usr/local/bin/grok && rm -f /usr/local/bin/agent", + "curl -fsSL https://x.ai/cli/install.sh | GROK_BIN_DIR=\"$HOME/.local/bin\" bash -s 0.2.39 && rm -f \"$HOME/.local/bin/agent\"", ], - installUser: "root", + installUser: "sandbox", verifyArgv: ["grok", "--version"], docsPath: "docs/harnesses/grok-build.md", kind: "optional", @@ -103,7 +107,6 @@ export const HARNESS_CATALOG: readonly HarnessEntry[] = [ title: "DeepAgents", binary: "deepagents", harnessKey: "deepagents", - buildArg: "INSTALL_DEEPAGENTS", installArgv: ["uv", "tool", "install", "deepagents-cli"], installUser: "sandbox", verifyArgv: ["deepagents", "--version"], @@ -115,13 +118,12 @@ export const HARNESS_CATALOG: readonly HarnessEntry[] = [ title: "Hermes", binary: "hermes", harnessKey: "hermes", - buildArg: "INSTALL_HERMES", installArgv: [ "bash", "-lc", - "curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash -s -- --skip-setup --skip-browser && uv pip install --python /usr/local/lib/hermes-agent/venv/bin/python 'hermes-agent[slack,teams,web,pty]'", + "curl -fsSL https://hermes-agent.nousresearch.com/install.sh | HERMES_INSTALL_DIR=\"$HOME/.local/lib/hermes-agent\" bash -s -- --skip-setup --skip-browser && uv pip install --python \"$HOME/.local/lib/hermes-agent/venv/bin/python\" 'hermes-agent[slack,teams,web,pty]'", ], - installUser: "root", + installUser: "sandbox", verifyArgv: ["hermes", "--version"], docsPath: "docs/harnesses/hermes.md", kind: "optional", @@ -152,6 +154,10 @@ export const HARNESS_CATALOG: readonly HarnessEntry[] = [ }, ]; +export function optionalHarnesses(): readonly HarnessEntry[] { + return HARNESS_CATALOG.filter((h) => h.kind === "optional"); +} + export function defaultHarnesses(): readonly HarnessEntry[] { return HARNESS_CATALOG.filter((h) => h.kind === "default"); } diff --git a/.oh/evals/RESULTS.md b/.oh/evals/RESULTS.md index 319a405a..880cf73a 100644 --- a/.oh/evals/RESULTS.md +++ b/.oh/evals/RESULTS.md @@ -6,108 +6,108 @@ probe id; git history is the time series.** Schema and exit-code semantics are i | probe | tier | last-run (UTC) | status | source | |-------|------|----------------|--------|--------| -| advisor-monitored-loop | A | 2026-08-31 02:42 | PASS | conversation 2026-06-19 (single-owner implementation workflow, issue #257) | -| agent-browser-cli | A | 2026-08-31 02:42 | PASS | retro lesson 2026-06-07 (agent-browser 0.8.5 CLI) | -| agents-identity-contract | A | 2026-08-31 02:42 | PASS | issue #854 — T3-style root identity, glossary, and skill-owned procedures | -| artifact-contract-audit | A | 2026-08-31 02:42 | PASS | issue #583/#645 — production /audit implementation Gate 1 behavior | -| audit-dispatcher-contract | A | 2026-08-31 02:42 | PASS | issue #645 — audit consolidation public taxonomy | -| audit-implementation-behavior | A | 2026-08-31 02:42 | PASS | issue #645 — implementation root/repo/browser behavior | -| audit-pr-acquire | A | 2026-08-31 02:42 | PASS | issue #645 — production PR acquisition behavior | -| audit-pr-classifier | A | 2026-08-31 02:42 | PASS | issue #645 — deterministic focused and queue PR classifier | -| audit-run-root-contract | A | 2026-08-31 02:42 | PASS | issue #645 — executable immutable audit root/run correlation | -| audit-shellcheck-coverage | A | 2026-08-31 02:42 | PASS | issue #645 — private audit scripts require release and CI lint coverage | -| audit-stale-references | A | 2026-08-31 02:42 | PASS | issue #645 — clean-breaking audit migration | -| boot-lint-glob | A | 2026-08-31 02:42 | PASS | issue #90, issue #120 | -| builder-skill-consolidation | A | 2026-08-31 02:42 | PASS | issue #643 — consolidate artifact builders behind one /builder dispatcher | -| capability-benchmark-schema | A | 2026-08-31 02:42 | PASS | issue #167 — capability benchmark instrument | -| cc-safety-net-wiring | A | 2026-08-31 02:42 | SKIPPED | .oh/tasks/cc-safety-net/prd.json US-007 2026-07-19 | -| changelog-entry-length | A | 2026-08-31 02:42 | PASS | conversation 2026-08-24 — CHANGELOG.md grew to 259KB of bullet prose because "one line" was unquantified | -| cleanup-tasks-scoped-guard | A | 2026-08-31 02:42 | PASS | issue #85 | -| cleanup-tasks-worktree-grooming | A | 2026-08-31 02:42 | PASS | issue #168; issue #327 | -| cli-publish-typecheck-scope | A | 2026-08-31 02:42 | PASS | release run 33271077312 — v0.5.0 pushed its GHCR image, then failed to publish | -| close-issues-on-development | A | 2026-08-31 02:42 | PASS | issue #841 (closing keywords never fire because the default branch is main) 2026-08-26 | -| codex-stale-response-retry | A | 2026-08-31 02:42 | PASS | issue #506 — Codex previous_response_not_found RCA | -| compose-config-path-parity | A | 2026-08-31 02:42 | PASS | PR #833 (remove harness.yaml — the wrapper and VS Code "Reopen in Container" paths must resolve the same service) 2026-08-26 | -| config-schema-parity | A | 2026-08-31 02:42 | PASS | PR #833 (one schema file — DOCKER_SOCKET, SANDBOX_SSH, OH_SANDBOX_IMAGE, OH_PULL_POLICY, SKIP_PNPM_INSTALL were consumed but undocumented); rewritten for the oh.json/secrets split by PR #887 | -| context-tier-size-budget | A | 2026-08-31 02:42 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-007) — the always-on tier was 85,256 B | -| cron-claude-codex-fallback | A | 2026-08-31 02:42 | PASS | conversation 2026-06-12 (default Codex fallback for crons) | -| cron-watchdog | A | 2026-08-31 02:42 | PASS | issues #130/#453 (cron runtime watchdog + legacy system-cron reaping) 2026-06-19 | -| crons-directory-guide | A | 2026-08-31 02:42 | PASS | issue #874 | -| curl-bash-safe-alternatives | A | 2026-08-31 02:42 | PASS | vet-run/vet integration — public curl|bash examples need review-first alternatives | -| datasets-schema | A | 2026-08-31 02:42 | PASS | issue #196 — .oh/evals/datasets verifiable trajectory corpus (Repo2RLEnv-inspired) | -| debugmcp-availability | A | 2026-08-31 02:42 | SKIPPED | issue #297 — DebugMCP MCP debug-server availability | -| default-provisioning | A | 2026-08-31 02:42 | PASS | #902 — `oh harness install` must work from inside the sandbox, where | -| delegate-model-effort-policy | A | 2026-08-31 02:42 | PASS | conversation 2026-07-11 (delegate model inheritance and thinking policy) | -| devtcp-hook | A | 2026-08-31 02:42 | PASS | retro lesson 2026-06-10 (zsh /dev/tcp) | -| docker-inspect-env-guard | A | 2026-08-31 02:42 | PASS | operator directive 2026-08-08 (agents keep the docker socket, but must | -| docs-build-fast-path | A | 2026-08-31 02:42 | PASS | #455 — docs builds must stay out of fast harness/eval/release gates; #536 — docs site externalized to openharness-web; docs markdown relocated to docs/ | -| drift-check-cron-staleness-glob | A | 2026-08-31 02:42 | PASS | issue #98; issue #225 (restart-required cron frontmatter/config drift) | -| entrypoint-pnpm-manifest-fingerprint | A | 2026-08-31 02:42 | PASS | issue #521 (manifest-aware sandbox installs) 2026-07-01 | -| eval-ci-gate | A | 2026-08-31 02:42 | PASS | #103 — eval probe suite gated in CI | -| eval-gate | A | 2026-08-31 02:42 | PASS | retro lesson 2026-06-11 (eval-gate) | -| eval-results-atomic | A | 2026-08-31 02:42 | PASS | issue #83 (eval-results-atomic-write) | -| eval-runner-exit | A | 2026-08-31 02:42 | PASS | retro lesson 2026-06-11 (eval-runner-exit) #29 | -| eval-runs-once-per-cycle | A | 2026-08-31 02:42 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-006) — /eval ran 3x per cycle on the | -| execution-target-contract | A | 2026-08-31 02:42 | PASS | issue #733 (ExecutionTarget contract + Docker Compose adapter) 2026-08-10 | -| get-oh-bootstrap | A | 2026-08-31 02:42 | PASS | get-oh.sh bootstrap — the Node-bootstrapping host-side path to the standalone `oh` CLI (also on npm as @mifune/openharness; see oh-npm-package.sh) | -| git-skill | A | 2026-08-31 02:42 | PASS | conversation 2026-06-15 — rules are not always supported; git workflow must be the /git skill | -| harness-audit-empty-output-gate | A | 2026-08-31 02:42 | PASS | issue #246 — /audit harness must fail closed on empty auditor outputs | -| harness-ci-core-paths | A | 2026-08-31 02:42 | PASS | #165 — core sandbox config files must trigger harness CI | -| harness-ci-hooks-paths | A | 2026-08-31 02:42 | PASS | issue #202 — credential/security hook changes must trigger harness CI | -| harness-yaml-migration | A | 2026-08-31 02:42 | PASS | PR #833 (migrate-harness-yaml.sh — append / uncomment-in-place / preserve / overwrite, plus a silent no-op second run) 2026-08-26 | -| health-check-docker-stats | A | 2026-08-31 02:42 | PASS | retro lesson 2026-06-10 (docker stats vs ps Size) | -| health-check-socket-degrade | A | 2026-08-31 02:42 | PASS | issue #762 (refs #756) — /health-check degrades to one statement, not nine failures | -| heartbeat-logging-contract | A | 2026-08-31 02:42 | PASS | issue #447 (heartbeat log append hardening) 2026-06-18 | -| image-seed-hygiene | A | 2026-08-31 02:42 | PASS | issue #900 (slim the sandbox image) 2026-08-30 | -| markitdown-wiki-ingest | A | 2026-08-31 02:42 | PASS | issue #649 — pinned local-document normalization contract for /wiki ingest | -| next-dev-prod | A | 2026-08-31 02:42 | SKIPPED | retro lesson 2026-06-04 | -| oh-compose-env-wiring | A | 2026-08-31 02:42 | PASS | issue #880 (oh as the only front door — oh.json is the non-secret config surface) | -| oh-config-surfaces | A | 2026-08-31 02:42 | REGRESSION | PR #887 (config split across two authored surfaces — a tracked oh.json and a secrets-only root dotenv — with nothing left under $HOME) | -| oh-destroy-guard | A | 2026-08-31 02:42 | PASS | issue #879 — `oh` becomes the only front door, so `make destroy` must | -| oh-devcontainer-restructure | A | 2026-08-31 02:42 | PASS | consolidate devcontainer — .oh/devcontainer/ folded back into .devcontainer/ | -| oh-home-mount | A | 2026-08-31 02:42 | PASS | issue #898 (single $HOME mount) 2026-08-30 | -| oh-image-only-deploy | A | 2026-08-31 02:42 | PASS | .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only deploy) | -| oh-init-headless-config | A | 2026-08-31 02:42 | PASS | PR #827 (installer answers landed in the losing config file); retargeted to the .example.env template by PR #833, then to oh.json by PR #887 | -| oh-init-scaffold | A | 2026-08-31 02:42 | PASS | issue #531 Phase 2 | -| oh-lifecycle-surface | A | 2026-08-31 02:42 | PASS | issue #881 — the Makefile is retired and `oh` is the only front door | -| oh-npm-package | A | 2026-08-31 02:42 | PASS | npm publish path for the standalone `oh` CLI (@mifune/openharness) — alternative to get-oh.sh | -| oh-payload-manifest | A | 2026-08-31 02:42 | PASS | issue #531 follow-on (.oh payload manifest — oh update ships a declared allowlist) | -| oh-sandbox-image-mode | A | 2026-08-31 02:42 | PASS | conversation 2026-07-05 (basic Docker deployment — prebuilt-image mode) | -| oh-shipped-repo-overridable | A | 2026-08-31 02:42 | PASS | issue #531 follow-on (de-hardcode residual — shipped .oh shell scripts keep the upstream repo overridable) | -| oh-standalone-lifecycle | A | 2026-08-31 02:42 | PASS | issue #564 | -| oh-update | A | 2026-08-31 02:42 | PASS | issue #531 Phase 3 (oh update — upgrade only the .oh control plane) | -| operator-config-guard | A | 2026-08-31 02:42 | PASS | operator directives 2026-08-06 (.config/ and settings.local.json are operator-only) | -| pnpm-audit-ci-gate | A | 2026-08-31 02:42 | PASS | issue #171 — pnpm security audits must run in CI | -| post-bridge-publish-confirmation | A | 2026-08-31 02:42 | PASS | #523 — post-bridge live publishing requires an explicit final confirmation gate | -| prd-output-path-contract | A | 2026-08-31 02:42 | PASS | retro lesson 2026-06-19 | -| prompt-miner-schema-compat | A | 2026-08-31 02:42 | PASS | issue #253 — prompt-miner JSONL schema-drift guard | -| prompt-miner-symlink-entrypoint | A | 2026-08-31 02:42 | PASS | issue #663 — prompt-miner engine no-ops via the documented .claude/skills symlink | -| prompt-miner-weakness-record | A | 2026-08-31 02:42 | PASS | issue #580 — prompt-miner weakness-record (WH-xxx) cluster output | -| protected-path-deletion | A | 2026-08-31 02:42 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-001) — the critique gate was deleted, | -| protected-paths-resolve | A | 2026-08-31 02:42 | PASS | issue #753 — .claude/protected-paths.txt named 7 paths that did not exist. | -| registry-portability-gate | A | 2026-08-31 02:42 | PASS | issue #758 | -| registry-portability | A | 2026-08-31 02:42 | SKIPPED | issue #758 | -| retro-deterministic-contract | A | 2026-08-31 02:42 | PASS | issue #443 — /retro deterministic output and self-contained helper contract | -| rl-delegation-write-worker | A | 2026-08-31 02:42 | PASS | retro lesson 2026-06-10 (rl-delegation) #57 | -| rlm-context-budget | A | 2026-08-31 02:42 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-006 | -| runtime-preflight-gate | A | 2026-08-31 02:42 | PASS | issue #806 § B1 (open sandbox.substrate vs sandbox.runtime selector); | -| sandbox-boot-guard-ci | A | 2026-08-31 02:42 | PASS | issue #449 (sandbox image build CI guard) 2026-06-19; | -| sandbox-node-base | A | 2026-08-31 02:42 | PASS | openharness#878 — oh as the only front door, T0 sandbox base image | -| skill-paths | A | 2026-08-31 02:42 | PASS | issue #43 — stale path references; extended by issue #69 — apps/->packages/ rename guard; extended by issue #870 — deleted .oh/agents/advisor.md | -| skills-dir-clean | A | 2026-08-31 02:42 | PASS | conversation 2026-06-29 — Pi parses every top-level `.md` in the skills | -| skills-task-tool-coupling | A | 2026-08-31 02:42 | PASS | council review 2026-08-29 (issue #886) — /delegate instructed Claude-Code-only | -| skills-vendored | A | 2026-08-31 02:42 | PASS | absorb .mifune submodule into .oh — the skills/agents/hooks pack is vendored | -| slack-admin-command-surface | A | 2026-08-31 02:42 | PASS | issue #354 — Slack bridge docs must distinguish Pi /msg-bridge commands from Slack DM admin text handlers | -| spec-family-contract | A | 2026-08-31 02:42 | PASS | issue #265; spec-simplification issue #816; workflow authority issue #854 | -| spec-ready-finalization | A | 2026-08-31 02:42 | PASS | issue #134; spec-simplification issue #816; workflow authority issue #854 | -| ste-checker-contract | A | 2026-08-31 02:42 | PASS | issue #750 PR audit — the /ste checker had four fail-open paths (unclosed | -| submitted-by-trailers | A | 2026-08-31 02:42 | PASS | conversation 2026-06-12 (commit attribution trailers); the single-owner | -| sync-skill-contract | A | 2026-08-31 02:42 | PASS | issue #331 — /sync dispatcher skill (bidirectional origin↔upstream sync) | -| tool-catalog-boundary | A | 2026-08-31 02:42 | PASS | agent-browser's exclusion from the harness catalog (#821) and the | -| version-parity | A | 2026-08-31 02:42 | PASS | conversation 2026-08-29 — the oh CLI became the only lifecycle door, so its | -| weigh-scorer-contract | A | 2026-08-31 02:42 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-003 (2026-06-27) | -| wiki-readme-index | A | 2026-08-31 02:42 | PASS | issue #132 — wiki README index drift guard | -| workflow-boundaries | A | 2026-08-31 02:42 | PASS | conversation 2026-06-19 (workflow consolidation, issue #259); authority moved to /spec in issue #854 | -| worktrees-layout | A | 2026-08-31 02:42 | PASS | issue #872 | +| advisor-monitored-loop | A | 2026-08-31 17:17 | PASS | conversation 2026-06-19 (single-owner implementation workflow, issue #257) | +| agent-browser-cli | A | 2026-08-31 17:17 | PASS | retro lesson 2026-06-07 (agent-browser 0.8.5 CLI) | +| agents-identity-contract | A | 2026-08-31 17:17 | PASS | issue #854 — T3-style root identity, glossary, and skill-owned procedures | +| artifact-contract-audit | A | 2026-08-31 17:17 | PASS | issue #583/#645 — production /audit implementation Gate 1 behavior | +| audit-dispatcher-contract | A | 2026-08-31 17:17 | PASS | issue #645 — audit consolidation public taxonomy | +| audit-implementation-behavior | A | 2026-08-31 17:17 | PASS | issue #645 — implementation root/repo/browser behavior | +| audit-pr-acquire | A | 2026-08-31 17:17 | PASS | issue #645 — production PR acquisition behavior | +| audit-pr-classifier | A | 2026-08-31 17:17 | PASS | issue #645 — deterministic focused and queue PR classifier | +| audit-run-root-contract | A | 2026-08-31 17:17 | PASS | issue #645 — executable immutable audit root/run correlation | +| audit-shellcheck-coverage | A | 2026-08-31 17:17 | PASS | issue #645 — private audit scripts require release and CI lint coverage | +| audit-stale-references | A | 2026-08-31 17:17 | PASS | issue #645 — clean-breaking audit migration | +| boot-lint-glob | A | 2026-08-31 17:17 | PASS | issue #90, issue #120 | +| builder-skill-consolidation | A | 2026-08-31 17:17 | PASS | issue #643 — consolidate artifact builders behind one /builder dispatcher | +| capability-benchmark-schema | A | 2026-08-31 17:17 | PASS | issue #167 — capability benchmark instrument | +| cc-safety-net-wiring | A | 2026-08-31 17:17 | SKIPPED | .oh/tasks/cc-safety-net/prd.json US-007 2026-07-19 | +| changelog-entry-length | A | 2026-08-31 17:17 | PASS | conversation 2026-08-24 — CHANGELOG.md grew to 259KB of bullet prose because "one line" was unquantified | +| cleanup-tasks-scoped-guard | A | 2026-08-31 17:17 | PASS | issue #85 | +| cleanup-tasks-worktree-grooming | A | 2026-08-31 17:17 | PASS | issue #168; issue #327 | +| cli-publish-typecheck-scope | A | 2026-08-31 17:17 | PASS | release run 33271077312 — v0.5.0 pushed its GHCR image, then failed to publish | +| close-issues-on-development | A | 2026-08-31 17:17 | PASS | issue #841 (closing keywords never fire because the default branch is main) 2026-08-26 | +| codex-stale-response-retry | A | 2026-08-31 17:17 | PASS | issue #506 — Codex previous_response_not_found RCA | +| compose-config-path-parity | A | 2026-08-31 17:17 | PASS | PR #833 (remove harness.yaml — the wrapper and VS Code "Reopen in Container" paths must resolve the same service) 2026-08-26 | +| config-schema-parity | A | 2026-08-31 17:17 | PASS | PR #833 (one schema file — DOCKER_SOCKET, SANDBOX_SSH, OH_SANDBOX_IMAGE, OH_PULL_POLICY, SKIP_PNPM_INSTALL were consumed but undocumented); rewritten for the oh.json/secrets split by PR #887 | +| context-tier-size-budget | A | 2026-08-31 17:17 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-007) — the always-on tier was 85,256 B | +| cron-claude-codex-fallback | A | 2026-08-31 17:17 | PASS | conversation 2026-06-12 (default Codex fallback for crons) | +| cron-watchdog | A | 2026-08-31 17:17 | PASS | issues #130/#453 (cron runtime watchdog + legacy system-cron reaping) 2026-06-19 | +| crons-directory-guide | A | 2026-08-31 17:17 | PASS | issue #874 | +| curl-bash-safe-alternatives | A | 2026-08-31 17:17 | PASS | vet-run/vet integration — public curl|bash examples need review-first alternatives | +| datasets-schema | A | 2026-08-31 17:17 | PASS | issue #196 — .oh/evals/datasets verifiable trajectory corpus (Repo2RLEnv-inspired) | +| debugmcp-availability | A | 2026-08-31 17:17 | SKIPPED | issue #297 — DebugMCP MCP debug-server availability | +| default-provisioning | A | 2026-08-31 17:17 | PASS | #902 — `oh harness install` must work from inside the sandbox, where | +| delegate-model-effort-policy | A | 2026-08-31 17:17 | PASS | conversation 2026-07-11 (delegate model inheritance and thinking policy) | +| devtcp-hook | A | 2026-08-31 17:17 | PASS | retro lesson 2026-06-10 (zsh /dev/tcp) | +| docker-inspect-env-guard | A | 2026-08-31 17:17 | PASS | operator directive 2026-08-08 (agents keep the docker socket, but must | +| docs-build-fast-path | A | 2026-08-31 17:17 | PASS | #455 — docs builds must stay out of fast harness/eval/release gates; #536 — docs site externalized to openharness-web; docs markdown relocated to docs/ | +| drift-check-cron-staleness-glob | A | 2026-08-31 17:17 | PASS | issue #98; issue #225 (restart-required cron frontmatter/config drift) | +| entrypoint-pnpm-manifest-fingerprint | A | 2026-08-31 17:17 | PASS | issue #521 (manifest-aware sandbox installs) 2026-07-01 | +| eval-ci-gate | A | 2026-08-31 17:17 | PASS | #103 — eval probe suite gated in CI | +| eval-gate | A | 2026-08-31 17:17 | PASS | retro lesson 2026-06-11 (eval-gate) | +| eval-results-atomic | A | 2026-08-31 17:17 | PASS | issue #83 (eval-results-atomic-write) | +| eval-runner-exit | A | 2026-08-31 17:17 | PASS | retro lesson 2026-06-11 (eval-runner-exit) #29 | +| eval-runs-once-per-cycle | A | 2026-08-31 17:17 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-006) — /eval ran 3x per cycle on the | +| execution-target-contract | A | 2026-08-31 17:17 | PASS | issue #733 (ExecutionTarget contract + Docker Compose adapter) 2026-08-10 | +| get-oh-bootstrap | A | 2026-08-31 17:17 | PASS | get-oh.sh bootstrap — the Node-bootstrapping host-side path to the standalone `oh` CLI (also on npm as @mifune/openharness; see oh-npm-package.sh) | +| git-skill | A | 2026-08-31 17:17 | PASS | conversation 2026-06-15 — rules are not always supported; git workflow must be the /git skill | +| harness-audit-empty-output-gate | A | 2026-08-31 17:17 | PASS | issue #246 — /audit harness must fail closed on empty auditor outputs | +| harness-ci-core-paths | A | 2026-08-31 17:17 | PASS | #165 — core sandbox config files must trigger harness CI | +| harness-ci-hooks-paths | A | 2026-08-31 17:17 | PASS | issue #202 — credential/security hook changes must trigger harness CI | +| harness-yaml-migration | A | 2026-08-31 17:17 | PASS | PR #833 (migrate-harness-yaml.sh — append / uncomment-in-place / preserve / overwrite, plus a silent no-op second run) 2026-08-26 | +| health-check-docker-stats | A | 2026-08-31 17:17 | PASS | retro lesson 2026-06-10 (docker stats vs ps Size) | +| health-check-socket-degrade | A | 2026-08-31 17:17 | PASS | issue #762 (refs #756) — /health-check degrades to one statement, not nine failures | +| heartbeat-logging-contract | A | 2026-08-31 17:17 | PASS | issue #447 (heartbeat log append hardening) 2026-06-18 | +| image-seed-hygiene | A | 2026-08-31 17:17 | PASS | issue #900 (slim the sandbox image) 2026-08-30 | +| markitdown-wiki-ingest | A | 2026-08-31 17:17 | PASS | issue #649 — pinned local-document normalization contract for /wiki ingest | +| next-dev-prod | A | 2026-08-31 17:17 | SKIPPED | retro lesson 2026-06-04 | +| oh-compose-env-wiring | A | 2026-08-31 17:17 | PASS | issue #880 (oh as the only front door — oh.json is the non-secret config surface) | +| oh-config-surfaces | A | 2026-08-31 17:17 | REGRESSION | PR #887 (config split across two authored surfaces — a tracked oh.json and a secrets-only root dotenv — with nothing left under $HOME) | +| oh-destroy-guard | A | 2026-08-31 17:17 | PASS | issue #879 — `oh` becomes the only front door, so `make destroy` must | +| oh-devcontainer-restructure | A | 2026-08-31 17:17 | PASS | consolidate devcontainer — .oh/devcontainer/ folded back into .devcontainer/ | +| oh-home-mount | A | 2026-08-31 17:17 | PASS | issue #898 (single $HOME mount) 2026-08-30 | +| oh-image-only-deploy | A | 2026-08-31 17:17 | PASS | .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only deploy) | +| oh-init-headless-config | A | 2026-08-31 17:17 | PASS | PR #827 (installer answers landed in the losing config file); retargeted to the .example.env template by PR #833, then to oh.json by PR #887 | +| oh-init-scaffold | A | 2026-08-31 17:17 | PASS | issue #531 Phase 2 | +| oh-lifecycle-surface | A | 2026-08-31 17:17 | PASS | issue #881 — the Makefile is retired and `oh` is the only front door | +| oh-npm-package | A | 2026-08-31 17:17 | PASS | npm publish path for the standalone `oh` CLI (@mifune/openharness) — alternative to get-oh.sh | +| oh-payload-manifest | A | 2026-08-31 17:17 | PASS | issue #531 follow-on (.oh payload manifest — oh update ships a declared allowlist) | +| oh-sandbox-image-mode | A | 2026-08-31 17:17 | PASS | conversation 2026-07-05 (basic Docker deployment — prebuilt-image mode) | +| oh-shipped-repo-overridable | A | 2026-08-31 17:17 | PASS | issue #531 follow-on (de-hardcode residual — shipped .oh shell scripts keep the upstream repo overridable) | +| oh-standalone-lifecycle | A | 2026-08-31 17:17 | PASS | issue #564 | +| oh-update | A | 2026-08-31 17:17 | PASS | issue #531 Phase 3 (oh update — upgrade only the .oh control plane) | +| operator-config-guard | A | 2026-08-31 17:17 | PASS | operator directives 2026-08-06 (.config/ and settings.local.json are operator-only) | +| pnpm-audit-ci-gate | A | 2026-08-31 17:17 | PASS | issue #171 — pnpm security audits must run in CI | +| post-bridge-publish-confirmation | A | 2026-08-31 17:17 | PASS | #523 — post-bridge live publishing requires an explicit final confirmation gate | +| prd-output-path-contract | A | 2026-08-31 17:17 | PASS | retro lesson 2026-06-19 | +| prompt-miner-schema-compat | A | 2026-08-31 17:17 | PASS | issue #253 — prompt-miner JSONL schema-drift guard | +| prompt-miner-symlink-entrypoint | A | 2026-08-31 17:17 | PASS | issue #663 — prompt-miner engine no-ops via the documented .claude/skills symlink | +| prompt-miner-weakness-record | A | 2026-08-31 17:17 | PASS | issue #580 — prompt-miner weakness-record (WH-xxx) cluster output | +| protected-path-deletion | A | 2026-08-31 17:17 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-001) — the critique gate was deleted, | +| protected-paths-resolve | A | 2026-08-31 17:17 | PASS | issue #753 — .claude/protected-paths.txt named 7 paths that did not exist. | +| registry-portability-gate | A | 2026-08-31 17:17 | PASS | issue #758 | +| registry-portability | A | 2026-08-31 17:17 | SKIPPED | issue #758 | +| retro-deterministic-contract | A | 2026-08-31 17:17 | PASS | issue #443 — /retro deterministic output and self-contained helper contract | +| rl-delegation-write-worker | A | 2026-08-31 17:17 | PASS | retro lesson 2026-06-10 (rl-delegation) #57 | +| rlm-context-budget | A | 2026-08-31 17:17 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-006 | +| runtime-preflight-gate | A | 2026-08-31 17:17 | PASS | issue #806 § B1 (open sandbox.substrate vs sandbox.runtime selector); | +| sandbox-boot-guard-ci | A | 2026-08-31 17:17 | PASS | issue #449 (sandbox image build CI guard) 2026-06-19; | +| sandbox-node-base | A | 2026-08-31 17:17 | PASS | openharness#878 — oh as the only front door, T0 sandbox base image | +| skill-paths | A | 2026-08-31 17:17 | PASS | issue #43 — stale path references; extended by issue #69 — apps/->packages/ rename guard; extended by issue #870 — deleted .oh/agents/advisor.md | +| skills-dir-clean | A | 2026-08-31 17:17 | PASS | conversation 2026-06-29 — Pi parses every top-level `.md` in the skills | +| skills-task-tool-coupling | A | 2026-08-31 17:17 | PASS | council review 2026-08-29 (issue #886) — /delegate instructed Claude-Code-only | +| skills-vendored | A | 2026-08-31 17:17 | PASS | absorb .mifune submodule into .oh — the skills/agents/hooks pack is vendored | +| slack-admin-command-surface | A | 2026-08-31 17:17 | PASS | issue #354 — Slack bridge docs must distinguish Pi /msg-bridge commands from Slack DM admin text handlers | +| spec-family-contract | A | 2026-08-31 17:17 | PASS | issue #265; spec-simplification issue #816; workflow authority issue #854 | +| spec-ready-finalization | A | 2026-08-31 17:17 | PASS | issue #134; spec-simplification issue #816; workflow authority issue #854 | +| ste-checker-contract | A | 2026-08-31 17:17 | PASS | issue #750 PR audit — the /ste checker had four fail-open paths (unclosed | +| submitted-by-trailers | A | 2026-08-31 17:17 | PASS | conversation 2026-06-12 (commit attribution trailers); the single-owner | +| sync-skill-contract | A | 2026-08-31 17:17 | PASS | issue #331 — /sync dispatcher skill (bidirectional origin↔upstream sync) | +| tool-catalog-boundary | A | 2026-08-31 17:17 | PASS | agent-browser's exclusion from the harness catalog (#821) and the | +| version-parity | A | 2026-08-31 17:17 | PASS | conversation 2026-08-29 — the oh CLI became the only lifecycle door, so its | +| weigh-scorer-contract | A | 2026-08-31 17:17 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-003 (2026-06-27) | +| wiki-readme-index | A | 2026-08-31 17:17 | PASS | issue #132 — wiki README index drift guard | +| workflow-boundaries | A | 2026-08-31 17:17 | PASS | conversation 2026-06-19 (workflow consolidation, issue #259); authority moved to /spec in issue #854 | +| worktrees-layout | A | 2026-08-31 17:17 | PASS | issue #872 | diff --git a/.oh/evals/probes/default-provisioning.sh b/.oh/evals/probes/default-provisioning.sh index 7627c5d0..c8b4dc3b 100755 --- a/.oh/evals/probes/default-provisioning.sh +++ b/.oh/evals/probes/default-provisioning.sh @@ -40,16 +40,28 @@ entries=$(awk ' inb { buf = buf $0 " " } ' "$CATALOG") +# #908: every harness, not just kind:"default". commands/harness.ts installs +# with stdio:"inherit", so local-target.ts picks plain `sudo --` for a root +# install and /etc/sudoers.d/sandbox has no NOPASSWD — an agent would hang on a +# password prompt. No harness of any kind belongs in the image either. defaults=0 while IFS= read -r entry; do - [[ $entry == *'kind: "default"'* ]] || continue - defaults=$((defaults + 1)) id=$(sed -n 's/.*id: "\([^"]*\)".*/\1/p' <<<"$entry") + [[ -n $id ]] || continue + defaults=$((defaults + 1)) if [[ $entry == *'installUser: "root"'* ]]; then - missing+=("harnesses/catalog.ts: default harness \"$id\" installs as root — inside the sandbox that becomes \`sudo -n\`, and /etc/sudoers.d/sandbox has no NOPASSWD") + missing+=("harnesses/catalog.ts: harness \"$id\" installs as root — commands/harness.ts uses stdio:\"inherit\", so that becomes an interactive \`sudo\`, and /etc/sudoers.d/sandbox has no NOPASSWD") fi - if [[ $entry != *"$PREFIX"* ]]; then - missing+=("harnesses/catalog.ts: default harness \"$id\" does not install into $PREFIX — a baked install under /usr/lib/node_modules cannot be upgraded by a running sandbox") + if [[ $entry == *'buildArg:'* ]]; then + missing+=("harnesses/catalog.ts: harness \"$id\" declares buildArg — that field carries a Dockerfile invariant this catalog cannot satisfy; the tool catalog already bans it") + fi + # on-demand entries (t3code) are fetched per invocation via npx and install + # nowhere, so the prefix rule does not apply to them. + if [[ $entry != *'kind: "on-demand"'* \ + && $entry != *"$PREFIX"* \ + && $entry != *'$HOME/.local'* \ + && $entry != *'uv", "tool", "install'* ]]; then + missing+=("harnesses/catalog.ts: harness \"$id\" does not install into $PREFIX — a system-path install cannot be upgraded by a running sandbox and does not persist in the home mount") fi if [[ $id == "claude-code" && $entry == *"--ignore-scripts"* ]]; then missing+=("harnesses/catalog.ts: claude-code uses --ignore-scripts — its postinstall copies the native binary over the placeholder, so \`claude --version\` fails with 'claude native binary not installed'") @@ -57,7 +69,7 @@ while IFS= read -r entry; do done <<<"$entries" if ((defaults == 0)); then - echo "SKIPPED: no kind:\"default\" harness parsed out of $CATALOG" >&2 + echo "SKIPPED: no harness parsed out of $CATALOG" >&2 exit 2 fi @@ -82,7 +94,9 @@ DOCKERFILE_CODE=$(strip_dockerfile_comments) pkgs=0 while IFS= read -r entry; do - [[ $entry == *'kind: "default"'* ]] || continue + # on-demand entries (t3code, prime-agent) are fetched per invocation and were + # never baked; skip them rather than assert against an npx incantation. + [[ $entry == *'kind: "default"'* || $entry == *'kind: "optional"'* ]] || continue id=$(sed -n 's/.*id: "\([^"]*\)".*/\1/p' <<<"$entry") # The package specifier is the last element of installArgv. Read it from that # array alone — `binary` and `verifyArgv` also hold bare names, and matching @@ -96,7 +110,7 @@ while IFS= read -r entry; do fi pkgs=$((pkgs + 1)) if grep -qF -- "$pkg" <<<"$DOCKERFILE_CODE"; then - missing+=("Dockerfile: names $pkg — default harness \"$id\" is baked into the image again; it belongs to .oh/scripts/provision-defaults.sh, which installs it into $PREFIX at boot") + missing+=("Dockerfile: names $pkg — harness \"$id\" is baked into the image again; it belongs to \`oh harness install\`, which installs it into $PREFIX") fi done <<<"$entries" @@ -109,6 +123,14 @@ if grep -qE '^ARG (BAKE_HARNESSES|AGENTS|HERDR_VERSION)=' <<<"$DOCKERFILE_CODE"; missing+=("Dockerfile: ARG BAKE_HARNESSES/AGENTS/HERDR_VERSION is back — a build arg that re-bakes a default is a dormant path that reintroduces the shadowed install and un-exercises the boot provisioner") fi +# #908: no harness may have a Dockerfile build arg at all. INSTALL_HERMES keeps +# its RUNTIME life (link-providers.sh vendors the Hermes skill pack from it, +# entrypoint.sh wires auth.json), so only an `ARG` declaration is a regression. +while IFS= read -r arg; do + [[ -n $arg ]] || continue + missing+=("Dockerfile: $arg is back — optional harnesses install through \`oh harness install\`, and a build arg makes the image the install path again") +done < <(grep -oE '^ARG INSTALL_(HERMES|OPENCODE|GROK_BUILD|DEEPAGENTS)' <<<"$DOCKERFILE_CODE" | sort -u) + # #906: the same rule for kind:"default" tools. These install as root nowhere: # commands/tool.ts passes stdio:"inherit", so local-target.ts selects plain # `sudo --` for a root install, and /etc/sudoers.d/sandbox has no NOPASSWD — @@ -153,4 +175,4 @@ if ((${#missing[@]})); then exit 1 fi -echo "PASS: $defaults default harnesses and $tools default tools install as the sandbox user into $PREFIX, none of the $pkgs packages is baked into the image, and the boot path provisions them" >&2 +echo "PASS: all $defaults harnesses and $tools default tools install as the sandbox user into $PREFIX, none of the $pkgs packages is baked into the image, and the boot path provisions them" >&2 diff --git a/.oh/evals/probes/sandbox-boot-guard-ci.sh b/.oh/evals/probes/sandbox-boot-guard-ci.sh index 251d8001..9d20f969 100755 --- a/.oh/evals/probes/sandbox-boot-guard-ci.sh +++ b/.oh/evals/probes/sandbox-boot-guard-ci.sh @@ -124,19 +124,29 @@ else if grep -Eq 'arm64-default-image|linux/arm64|docker/setup-qemu-action|CI_RUNNER_ARM64' <<<"$compat"; then missing+=("compatibility workflow: retains the removed permanent arm64 build") fi + # #908 deleted the INSTALL_* build args, so a build-arg matrix can no longer + # exercise the optional harnesses. The job must install them the way an + # operator does instead, and must not reintroduce the args. for arg in INSTALL_HERMES INSTALL_DEEPAGENTS INSTALL_OPENCODE INSTALL_GROK_BUILD; do - chas "--build-arg $arg=true" "does not build with $arg=true" + if grep -Fq -- "--build-arg $arg" <<<"$compat"; then + missing+=("compatibility workflow: still builds with $arg — that build arg no longer exists; install through \`oh harness install\`") + fi done optional=$(awk ' - /^ optional-installers-image:$/ { found=1 } - found && /^ [[:alnum:]_-]+:$/ && !/^ optional-installers-image:$/ { exit } + /^ optional-harness-install:$/ { found=1 } + found && /^ [[:alnum:]_-]+:$/ && !/^ optional-harness-install:$/ { exit } found { print } ' <<<"$compat") if [[ -z "$optional" ]]; then - missing+=("compatibility workflow: no optional installer job") + missing+=("compatibility workflow: no optional-harness-install job") else - ohas() { grep -Fq -- "$1" <<<"$optional" || missing+=("compatibility optional installer job: $2"); } - ohas 'for tool in hermes deepagents opencode grok; do' "does not check every optional tool in one guarded loop" + ohas() { grep -Fq -- "$1" <<<"$optional" || missing+=("compatibility optional harness job: $2"); } + ohas 'oh harness install' "does not install through the CLI — the path #908 made the only one" + ohas 'select(.kind == "optional") | .id' "does not read the optional set from the catalog, so it can drift" + ohas 'would pass vacuously' "does not fail closed when the catalog yields no optional harness" + ohas '/home/sandbox/.local/*)' "does not assert the install landed in the home mount" + ohas 'for attempt in 1 2; do' "does not retry a transient upstream failure — four third-party endpoints can each block a merge" + ohas 'this is not a transient upstream blip' "retries without ever failing hard, so a real break would pass" ohas "if ! grep -Eq '(^|[^[:alnum:]])v?[0-9]+([.][0-9]+)+" "does not require numeric dotted versions" ohas 'did not output a numeric dotted version' "does not fail false-positive output" fi diff --git a/.oh/install/banner.sh b/.oh/install/banner.sh index 9115870f..544f267f 100755 --- a/.oh/install/banner.sh +++ b/.oh/install/banner.sh @@ -81,7 +81,7 @@ if [ -s "${HOME}/.pi/agent/auth.json" ]; then fi opencode_status="$status_x" -opencode_detail="not installed — set INSTALL_OPENCODE=true and rebuild" +opencode_detail="not installed — run: oh harness install opencode" if command -v opencode >/dev/null 2>&1; then if [ -s "${HOME}/.local/share/opencode/auth.json" ]; then opencode_status="$status_ok" @@ -93,7 +93,7 @@ if command -v opencode >/dev/null 2>&1; then fi grok_status="$status_x" -grok_detail="not installed — enable via install.grok_build / INSTALL_GROK_BUILD" +grok_detail="not installed — run: oh harness install grok-build" if command -v grok >/dev/null 2>&1; then if [ -s "${HOME}/.grok/auth.json" ]; then grok_status="$status_ok" @@ -108,7 +108,7 @@ if command -v grok >/dev/null 2>&1; then fi deepagents_status="$status_x" -deepagents_detail="not installed — set INSTALL_DEEPAGENTS=true and rebuild" +deepagents_detail="not installed — run: oh harness install deepagents" if command -v deepagents >/dev/null 2>&1; then if [ -s "${HOME}/.deepagents/.env" ] || [ -s "${HOME}/.deepagents/config.toml" ]; then deepagents_status="$status_ok" @@ -120,7 +120,7 @@ if command -v deepagents >/dev/null 2>&1; then fi hermes_status="$status_x" -hermes_detail="not installed — set INSTALL_HERMES=true and rebuild" +hermes_detail="not installed — run: oh harness install hermes" if command -v hermes >/dev/null 2>&1; then if [ -s "${HERMES_HOME:-${OH_PROJECT_ROOT:-/home/sandbox/harness}/.hermes}/auth.json" ]; then hermes_status="$status_ok" diff --git a/.oh/scripts/__tests__/verify-sandbox-image.test.ts b/.oh/scripts/__tests__/verify-sandbox-image.test.ts index 17dafbc2..5ec4ccaa 100644 --- a/.oh/scripts/__tests__/verify-sandbox-image.test.ts +++ b/.oh/scripts/__tests__/verify-sandbox-image.test.ts @@ -20,8 +20,10 @@ type Overrides = Partial<{ platformWarning: string; bakedHarnesses: boolean; bakedTools: boolean; - noDefaultHarnesses: boolean; + noHarnesses: boolean; noDefaultTools: boolean; + noBakedInTools: boolean; + missingBakedInTool: boolean; harnessCatalogFails: boolean; }>; @@ -43,8 +45,10 @@ function fixture(o: Overrides = {}) { platformWarning: "", bakedHarnesses: false, bakedTools: false, - noDefaultHarnesses: false, + noHarnesses: false, noDefaultTools: false, + noBakedInTools: false, + missingBakedInTool: false, harnessCatalogFails: false, ...o, }; @@ -61,32 +65,35 @@ case "$cmd" in *"id -u sandbox"*) printf '%s\\n%s\\n' ${JSON.stringify(v.uid)} ${JSON.stringify(v.gid)} ;; "node --version") printf '%s\\n' ${JSON.stringify(v.node)} ;; "pnpm --version") printf '%s\\n' ${JSON.stringify(v.pnpm)} ;; - *"oh harness list --defaults --json"*) + *"oh harness list --json"*) if [ "${v.harnessCatalogFails ? "1" : "0"}" = "1" ]; then echo 'not an OpenHarness-equipped repo' >&2 exit 1 fi cat <<'JSON' ${ - v.noDefaultHarnesses + v.noHarnesses ? "[]" : `[ { "id": "claude-code", "binary": "claude", "kind": "default", "installed": ${v.bakedHarnesses} }, - { "id": "pi", "binary": "pi", "kind": "default", "installed": false } + { "id": "hermes", "binary": "hermes", "kind": "optional", "installed": false } ]` } JSON ;; - *"oh tool list --defaults --json"*) + *"oh tool list --json"*) cat <<'JSON' -${ - v.noDefaultTools - ? "[]" - : `[ - { "id": "herdr", "binary": "herdr", "kind": "default", "installed": ${v.bakedTools} }, - { "id": "cloudflared", "binary": "cloudflared", "kind": "default", "installed": false } -]` -} +${(() => { + const rows: string[] = []; + if (!v.noDefaultTools) { + rows.push(` { "id": "herdr", "binary": "herdr", "kind": "default", "installed": ${v.bakedTools} }`); + rows.push(' { "id": "cloudflared", "binary": "cloudflared", "kind": "default", "installed": false }'); + } + if (!v.noBakedInTools) { + rows.push(` { "id": "gh", "binary": "gh", "kind": "baked-in", "installed": ${!v.missingBakedInTool} }`); + } + return `[\n${rows.join(",\n")}\n]`; +})()} JSON ;; *) @@ -130,7 +137,7 @@ describe("verify-sandbox-image", () => { expect(result.stdout).toContain("built-in sandbox user is 1000:1000"); expect(result.stdout).toContain("node is major 22"); expect(result.stdout).toContain("pnpm is exactly 10.33.0"); - expect(result.stdout).toContain("no default harness is baked into the image"); + expect(result.stdout).toContain("no harness is baked into the image"); expect(result.stdout).toContain("no default tool is baked into the image"); expect(result.stdout).toContain("all checks passed"); }); @@ -192,23 +199,23 @@ describe("verify-sandbox-image", () => { expect(result.status).toBe(0); }); - // #904/#906: the default harnesses AND the default tools moved out of the - // image and into the boot path. A baked copy in a system path shadows the - // home-mount install and silently un-exercises the provisioner, so the image - // must not carry either. - it("passes an image that bakes no default harness or tool", () => { + // #904/#906/#908: no harness of any kind, and no kind:"default" tool, may be + // baked into the image. kind:"baked-in" tools are the image-level half and + // must be present, or the checks above pass on an image missing everything. + it("passes an image that bakes no harness and no default tool", () => { const result = run(fixture()); expect(result.status).toBe(0); - expect(result.stdout).toContain("no default harness is baked into the image"); + expect(result.stdout).toContain("no harness is baked into the image"); expect(result.stdout).toContain("no default tool is baked into the image"); + expect(result.stdout).toContain("every baked-in tool is present"); }); it("rejects an image that bakes a default harness", () => { const result = run(fixture({ bakedHarnesses: true })); expect(result.status).toBe(1); - expect(result.stderr).toContain("the image ships baked default harnesss: claude-code (claude)"); + expect(result.stderr).toContain("the image ships baked harnesss: claude-code (claude)"); }); it("rejects an image that bakes a default tool", () => { @@ -218,16 +225,30 @@ describe("verify-sandbox-image", () => { expect(result.stderr).toContain("the image ships baked default tools: herdr (herdr)"); }); + it("rejects an image whose baked-in tool is missing", () => { + const result = run(fixture({ missingBakedInTool: true })); + + expect(result.status).toBe(1); + expect(result.stderr).toContain("baked-in tools are missing from the image: gh"); + }); + it.each<[string, Overrides]>([ - ["harness", { noDefaultHarnesses: true }], - ["tool", { noDefaultTools: true }], - ])("refuses to pass vacuously when the image lists no default %s", (_noun, overrides) => { + ["harness", { noHarnesses: true }], + ["default tool", { noDefaultTools: true }], + ])("refuses to pass vacuously when the image lists no %s", (_noun, overrides) => { const result = run(fixture(overrides)); expect(result.status).toBe(1); expect(result.stderr).toContain("would pass vacuously"); }); + it("refuses to pass vacuously when the image declares no baked-in tool", () => { + const result = run(fixture({ noBakedInTools: true })); + + expect(result.status).toBe(1); + expect(result.stderr).toContain('declares no kind:"baked-in" tool'); + }); + it("fails loudly when the harness catalog cannot be read out of the image", () => { const result = run(fixture({ harnessCatalogFails: true })); diff --git a/.oh/scripts/provision-defaults.sh b/.oh/scripts/provision-defaults.sh index bb029361..181f9359 100755 --- a/.oh/scripts/provision-defaults.sh +++ b/.oh/scripts/provision-defaults.sh @@ -113,11 +113,14 @@ provisioned=0 provision_catalog() { local noun="$1" cmd="$2" catalog="$3" - local states defaults id installed - - if ! states="$("$OH_BIN" "$cmd" list --defaults --json 2>/dev/null)" || [ -z "$states" ]; then - die "'$OH_BIN $cmd list --defaults --json' produced no catalog" \ - "the CLI at $(command -v "$OH_BIN") predates \`oh $cmd --defaults\`; the catalog" \ + local states defaults wanted id installed + + # The full listing, not --defaults: an operator who set install. in + # oh.json declared intent that a fresh home mount must honour too, and + # `enabled` is computed from oh.json rather than the environment. + if ! states="$("$OH_BIN" "$cmd" list --json 2>/dev/null)" || [ -z "$states" ]; then + die "'$OH_BIN $cmd list --json' produced no catalog" \ + "the CLI at $(command -v "$OH_BIN") predates \`oh $cmd\`; the catalog" \ "is the only source of truth for what to install, so there is nothing to provision." \ "rebuild the sandbox image from this control plane:" \ " oh sandbox" @@ -128,6 +131,10 @@ provision_catalog() { "the $noun catalog declares no defaults" \ "check $catalog" + # Opted-in extras ride along; absent any, this is empty and nothing changes. + wanted="$(jq -r '.[] | select(.kind != "default" and .enabled == true) | "\(.id)\t\(.installed)"' <<<"$states")" + [ -n "$wanted" ] && defaults="$defaults"$'\n'"$wanted" + while IFS=$'\t' read -r id installed; do [ -n "$id" ] || continue provisioned=$((provisioned + 1)) diff --git a/.oh/scripts/verify-sandbox-image.sh b/.oh/scripts/verify-sandbox-image.sh index c3ff9996..cdc85b91 100755 --- a/.oh/scripts/verify-sandbox-image.sh +++ b/.oh/scripts/verify-sandbox-image.sh @@ -1,7 +1,8 @@ #!/usr/bin/env bash # Verify a built sandbox image: base distribution, apt suites, the sandbox # UID/GID contract, the Node/pnpm pins, and version output from every baked-in -# tool, and that no kind:"default" harness or tool is baked into it. +# tool; that no harness and no kind:"default" tool is baked into it; and that +# every kind:"baked-in" tool actually is. # Usage: verify-sandbox-image.sh set -euo pipefail @@ -99,41 +100,61 @@ for tool in "gh --version" "docker --version" "docker compose version" \ fi done -# The image must NOT ship any kind:"default" harness (#904) or tool (#906). -# Both are installed into /home/sandbox/.local at boot: a copy baked into a +# The image must ship NO harness at all (#904, #908) and no kind:"default" tool +# (#906). Every one of them is installed into /home/sandbox/.local at boot or on +# demand through `oh harness install` / `oh tool install`: a copy baked into a # system path shadows the home-mount install with one no running sandbox can -# upgrade, and makes the boot install dead code that never runs and never gets -# tested. The catalogs inside the image are the source of truth for which ids -# are default, so this cannot drift from the TypeScript. -check_no_baked_defaults() { - local noun="$1" cmd="$2" json ids baked - - if ! json=$(run "cd /opt/oh-seed && OH_EXECUTION_TARGET=local oh $cmd list --defaults --json" 2>/tmp/verify-sandbox-defaults.err); then +# upgrade, and makes the install path dead code that never runs and never gets +# tested. The catalogs inside the image are the source of truth, so this cannot +# drift from the TypeScript. Tools that are kind:"baked-in" — the Docker CLI, +# gh — are genuinely image-level and are exempt. +check_nothing_baked() { + local noun="$1" cmd="$2" filter="$3" json ids baked + + if ! json=$(run "cd /opt/oh-seed && OH_EXECUTION_TARGET=local oh $cmd list --json" 2>/tmp/verify-sandbox-defaults.err); then fail "could not read the $noun catalog from the image: $(head -3 /tmp/verify-sandbox-defaults.err 2>/dev/null)" return fi - ids=$(jq -r '.[] | select(.kind == "default") | .id' <<<"$json") + ids=$(jq -r "$filter | .id" <<<"$json") if [ -z "$ids" ]; then - fail "the image's $noun catalog reports no kind:\"default\" entries — the unbaked-image check would pass vacuously" + fail "the image's $noun catalog matched no entry for '$filter' — the unbaked-image check would pass vacuously" return fi - baked=$(jq -r '.[] | select(.kind == "default" and .installed == true) | "\(.id) (\(.binary))"' <<<"$json") + baked=$(jq -r "$filter | select(.installed == true) | \"\(.id) (\(.binary))\"" <<<"$json") if [ -n "$baked" ]; then - fail "the image ships baked default ${noun}s: $(tr '\n' ' ' <<<"$baked")— these must be provisioned into /home/sandbox/.local at boot, not baked" + fail "the image ships baked ${noun}s: $(tr '\n' ' ' <<<"$baked")— these must be installed into /home/sandbox/.local by the CLI, not baked" else - ok "no default $noun is baked into the image ($(tr '\n' ' ' <<<"$ids"))" + ok "no $noun is baked into the image ($(tr '\n' ' ' <<<"$ids"))" fi } if command -v jq >/dev/null 2>&1; then - check_no_baked_defaults harness harness - check_no_baked_defaults tool tool + # Every harness, whatever its kind — none belongs in the image. + check_nothing_baked harness harness '.[]' + # Tools split: kind:"default" is provisioned, kind:"baked-in" is image-level. + check_nothing_baked "default tool" tool '.[] | select(.kind == "default")' else fail "jq is required to read the image's harness and tool catalogs" fi +# The inverse for tools: a kind:"baked-in" tool must actually be present, or the +# check above is passing because the image is simply missing everything. +if command -v jq >/dev/null 2>&1; then + if baked_json=$(run "cd /opt/oh-seed && OH_EXECUTION_TARGET=local oh tool list --json" 2>/dev/null); then + absent=$(jq -r '.[] | select(.kind == "baked-in" and .installed != true) | .id' <<<"$baked_json") + present=$(jq -r '.[] | select(.kind == "baked-in") | .id' <<<"$baked_json") + if [ -z "$present" ]; then + fail "the image's tool catalog declares no kind:\"baked-in\" tool — nothing anchors the image-level half" + elif [ -n "$absent" ]; then + fail "baked-in tools are missing from the image: $(tr '\n' ' ' <<<"$absent")" + else + ok "every baked-in tool is present ($(tr '\n' ' ' <<<"$present"))" + fi + fi +fi + if ((${#failures[@]})); then printf '\nverify-sandbox-image: %d check(s) failed\n' "${#failures[@]}" >&2 printf ' - %s\n' "${failures[@]}" >&2 diff --git a/CHANGELOG.md b/CHANGELOG.md index 1394c495..b827f1de 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,10 +13,12 @@ Update policy and release automation live in [`/git`](.claude/skills/git/SKILL.m - Shrink the sandbox image ~540 MB: drop build caches from the baked home seed, stage the seed once via a builder stage, and keep untracked build output out of the build context ([#900](https://github.com/mifunedev/openharness/issues/900)). - **BREAKING:** Stop baking Claude Code, Codex, and Pi into the image; boot installs them into the home mount, so a first boot needs network and runs 60-180s longer ([#904](https://github.com/mifunedev/openharness/issues/904)). - **BREAKING:** Stop baking Herdr and cloudflared into the image; both become `kind: "default"` tools installed into `~/.local/bin` at boot from a pinned, checksum-verified binary ([#906](https://github.com/mifunedev/openharness/issues/906)). +- **BREAKING:** Stop baking OpenCode, DeepAgents, Hermes, and Grok Build into the image; `oh harness install ` installs them into `~/.local` as the sandbox user ([#908](https://github.com/mifunedev/openharness/issues/908)). ### Removed - Remove the `BAKE_HARNESSES` and `AGENTS` build args along with the image bake they gated; the harness catalog is the only source of truth for what gets installed ([#904](https://github.com/mifunedev/openharness/issues/904)). - Remove Cloudflare's apt repository and its bookworm-suite pin from the image; Docker's is now the only third-party apt source ([#906](https://github.com/mifunedev/openharness/issues/906)). +- Remove the four optional-harness build args and the dead `buildArg` catalog field; the `install.*` keys keep working and now drive boot provisioning ([#908](https://github.com/mifunedev/openharness/issues/908)). - **BREAKING:** Retire the `projectRoot` / `OH_PROJECT_ROOT` config knob — the checkout is fixed at `/home/sandbox/harness`, nested inside the home mount ([#898](https://github.com/mifunedev/openharness/issues/898)). ### Added @@ -24,6 +26,7 @@ Update policy and release automation live in [`/git`](.claude/skills/git/SKILL.m - Add `oh-home-mount.sh`, a tier-A probe holding the single-`$HOME`-mount contract: one mount per compose file, the baked `/opt/home-seed`, and the checkout prune that replaces `-xdev` ([#898](https://github.com/mifunedev/openharness/issues/898)). - Assert boot-provisioned harnesses in the boot smoke and reject a baked default harness in `verify-sandbox-image.sh`, so CI exercises the install path ([#904](https://github.com/mifunedev/openharness/issues/904)). - Add `oh tool list --defaults` and generalize the boot provisioner over both catalogs as `provision-defaults.sh` (`OH_PROVISION_DEFAULTS`) ([#906](https://github.com/mifunedev/openharness/issues/906)). +- Fix `oh harness install` hanging on a sudo password prompt inside the sandbox: every harness now installs as the sandbox user, so no install path needs root ([#908](https://github.com/mifunedev/openharness/issues/908)). - Add `skills-task-tool-coupling.sh`, a tier-A probe holding the canonical skill pack and the sandbox in agreement about the Claude-Code-only task tools ([#886](https://github.com/mifunedev/openharness/issues/886)). ### Fixed diff --git a/docs/configuration.md b/docs/configuration.md index 199c967e..25babe7e 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -61,10 +61,10 @@ to `oh harness` names: `opencode`, `grok-build`, `deepagents`, `hermes`. | Field | Type | Default | Compose variable | What it does | | --- | --- | --- | --- | --- | -| `install.opencode` | boolean | `false` | `INSTALL_OPENCODE` | Build the OpenCode CLI into the image. | -| `install.grokBuild` | boolean | `false` | `INSTALL_GROK_BUILD` | Build the Grok Build CLI into the image. | -| `install.deepagents` | boolean | `false` | `INSTALL_DEEPAGENTS` | Build the DeepAgents CLI into the image. | -| `install.hermes` | boolean | `false` | `INSTALL_HERMES` | Build the Hermes CLI into the image and enable its runtime wiring. | +| `install.opencode` | boolean | `false` | `INSTALL_OPENCODE` | Install the OpenCode CLI into `~/.local` at boot. `oh harness install opencode` sets it and installs now. | +| `install.grokBuild` | boolean | `false` | `INSTALL_GROK_BUILD` | Install the Grok Build CLI into `~/.local` at boot. `oh harness install grok-build` sets it and installs now. | +| `install.deepagents` | boolean | `false` | `INSTALL_DEEPAGENTS` | Install the DeepAgents CLI into `~/.local` at boot. `oh harness install deepagents` sets it and installs now. | +| `install.hermes` | boolean | `false` | `INSTALL_HERMES` | Install the Hermes CLI into `~/.local` at boot and enable its runtime wiring (skill vendoring, `auth.json`). | | `install.agentBrowser` | boolean | `false` | `INSTALL_AGENT_BROWSER` | Install agent-browser and Chromium (about 1 GB). | ### Access diff --git a/docs/harnesses/deepagents.md b/docs/harnesses/deepagents.md index 9f74fee7..cb924d64 100644 --- a/docs/harnesses/deepagents.md +++ b/docs/harnesses/deepagents.md @@ -44,14 +44,14 @@ install: Or set `INSTALL_DEEPAGENTS=true` in `.devcontainer/.env` (legacy). -Then rebuild/restart the sandbox: +Either way the boot provisioner installs it on the next start — no rebuild: ```bash oh stop && oh sandbox ``` -Open Harness installs the upstream CLI during image build via `uv tool -install` into image-level paths: +Open Harness installs the upstream CLI with `uv tool install` as the `sandbox` +user, which lands it in `~/.local/bin` inside the home mount: ```bash uv tool install deepagents-cli @@ -66,8 +66,9 @@ Verify the install inside the sandbox: deepagents -v ``` -If the command is not found, confirm `INSTALL_DEEPAGENTS=true` is set in -`.devcontainer/.env`, then rebuild with `oh stop && oh sandbox`. +If the command is not found, run `oh harness install deepagents` — it installs +into `~/.local/bin` in the running sandbox and sets `install.deepagents` so a +fresh home mount reinstalls it at boot. No rebuild is involved. ## Authentication and provider keys diff --git a/docs/harnesses/grok-build.md b/docs/harnesses/grok-build.md index c31552c1..d3044d41 100644 --- a/docs/harnesses/grok-build.md +++ b/docs/harnesses/grok-build.md @@ -29,22 +29,22 @@ install: grok_build: true ``` -Or set the legacy build flag in `.devcontainer/.env`: +Or set the legacy flag in `.devcontainer/.env`: ```bash INSTALL_GROK_BUILD=true ``` -Then rebuild/restart the sandbox: +Either way the boot provisioner installs it on the next start — no rebuild: ```bash oh stop && oh sandbox ``` -Open Harness uses the upstream installer during image build, pinned to the version verified when this support was added: +Open Harness uses the upstream installer as the `sandbox` user, pinned to the version verified when this support was added, with the binary directed into the home mount: ```bash -curl -fsSL https://x.ai/cli/install.sh | bash -s 0.2.39 +curl -fsSL https://x.ai/cli/install.sh | GROK_BIN_DIR="$HOME/.local/bin" bash -s 0.2.39 ``` Review-first equivalent for manual inspection: @@ -63,7 +63,7 @@ Verify the install inside the sandbox: grok --version ``` -If `grok` is not found, confirm `INSTALL_GROK_BUILD=true` is set in `.devcontainer/.env` and rebuild. +If `grok` is not found, run `oh harness install grok-build` — it installs into `~/.local/bin` and sets `install.grokBuild` so a fresh home mount reinstalls it at boot. ## Authentication diff --git a/docs/harnesses/hermes.md b/docs/harnesses/hermes.md index e386ddbb..f217257c 100644 --- a/docs/harnesses/hermes.md +++ b/docs/harnesses/hermes.md @@ -10,7 +10,7 @@ skills from experience, scheduled task automation, sub-agent delegation, container sandboxing across multiple backends, and bridges to chat platforms (Telegram, Discord, Slack, WhatsApp, Signal, Email). -Hermes is an **optional image-level runtime** in Open Harness. When enabled (set `INSTALL_HERMES=true` in `.devcontainer/.env`), it sits alongside `claude`, `codex`, +Hermes is an **optional harness** in Open Harness. Install it with `oh harness install hermes` (or set `install.hermes` / `INSTALL_HERMES=true`, which the boot provisioner honours); it then sits alongside `claude`, `codex`, `pi`, `opencode`, and `deepagents` as a sandbox CLI primitive. See the upstream documentation below for canonical facts about Hermes. @@ -54,19 +54,20 @@ Then rebuild/restart the sandbox: oh stop && oh sandbox ``` -The executable is installed during image build, not at container boot, so -an enabled sandbox has `hermes` on PATH immediately: +The executable is installed by `oh harness install hermes`, or at boot on a +fresh home mount when `install.hermes` is true. Once installed it persists in +the home mount, so later boots find it on PATH immediately: ```bash hermes --version ``` -At image build time, Open Harness runs the official installer with setup -and browser installation disabled: +Open Harness runs the official installer as the `sandbox` user with setup and +browser installation disabled, directing it into the home mount: ```bash curl -fsSL https://hermes-agent.nousresearch.com/install.sh \ - | bash -s -- --skip-setup --skip-browser + | HERMES_INSTALL_DIR="$HOME/.local/lib/hermes-agent" bash -s -- --skip-setup --skip-browser ``` Review-first equivalent for manual inspection: @@ -122,10 +123,11 @@ secrets from this directory. the bind-mounted `.hermes/` directory from the checkout. Remove that directory manually if you want a full Hermes project-state reset. -The Hermes binary itself is installed in the image when -`INSTALL_HERMES=true` is set in `.devcontainer/.env`, under the installer's root Linux FHS layout -(`/usr/local/lib/hermes-agent` with a `/usr/local/bin/hermes` launcher). -Disabling the flag on a future rebuild omits the executable; project-local +The Hermes binary is installed into the home mount by `oh harness install hermes` +— at `~/.local/lib/hermes-agent` with a `~/.local/bin/hermes` launcher — and the +boot provisioner reinstalls it on a fresh home mount whenever `install.hermes` is +true. Nothing about it lives in the image. Disabling the flag stops the +reinstall but leaves an existing install in place; project-local state remains in `.hermes/` until removed. ## Common usage @@ -276,7 +278,7 @@ vars — see upstream Hermes documentation for the full list. The sandbox onboarding banner reports Hermes as: -- `❌ not installed` — set `INSTALL_HERMES=true` in `.devcontainer/.env` and rebuild — when the binary is absent from PATH. +- `❌ not installed` — run `oh harness install hermes` — when the binary is absent from PATH. - `✅ installed — run: hermes setup` — when the binary is on PATH but `~/.hermes/auth.json` is absent or empty. - `✅ authenticated` — when `~/.hermes/auth.json` exists and is diff --git a/docs/harnesses/opencode.md b/docs/harnesses/opencode.md index c6bc5bfb..9122304f 100644 --- a/docs/harnesses/opencode.md +++ b/docs/harnesses/opencode.md @@ -4,7 +4,7 @@ title: "OpenCode" # OpenCode -OpenCode is a terminal coding agent that can run interactively or execute one-shot tasks. It is an optional image-level runtime in Open Harness; the default harnesses — Claude Code, Codex, and Pi — are provisioned into `~/.local` at boot instead of being baked into the image. +OpenCode is a terminal coding agent that can run interactively or execute one-shot tasks. It is an optional harness in Open Harness. Like every other harness it installs into `~/.local` rather than the image — `oh harness install opencode`. ## Install (optional) @@ -29,16 +29,16 @@ install: Or set `INSTALL_OPENCODE=true` in `.devcontainer/.env` (legacy). -Then rebuild/restart the sandbox: +Either way the boot provisioner installs it on the next start — no rebuild: ```bash oh stop && oh sandbox ``` -Open Harness installs the upstream npm package globally during image build: +Open Harness installs the upstream npm package into the home mount as the `sandbox` user: ```bash -npm install -g opencode-ai +npm --prefix /home/sandbox/.local install -g opencode-ai ``` Verify the install inside the sandbox: diff --git a/docs/harnesses/overview.md b/docs/harnesses/overview.md index 23bb0eba..a5849e68 100644 --- a/docs/harnesses/overview.md +++ b/docs/harnesses/overview.md @@ -4,16 +4,16 @@ title: "Harnesses Overview" # Harnesses Overview -Open Harness ships with three agent CLIs in the default sandbox image: **Claude Code** (default), **Codex**, and **Pi**. **OpenCode**, **DeepAgents**, **Hermes**, and **Grok Build** are optional image-level installs controlled by the `INSTALL_*` keys in `.devcontainer/.env`. **T3 Code** runs on demand via the `/t3` skill (or directly with `npx t3`) as a browser UI on port 3773, and **Prime Agent** installs on demand with `oh harness install prime-agent` — neither has an `INSTALL_*` key, because neither is ever baked into the image. Inside the sandbox, run `herdr` first, then launch whichever agent you prefer from its panes and switch between them at any time. Reserve tmux for Open Harness's managed/headless cron, gateway, and watchdog infrastructure. +Open Harness provisions three agent CLIs into `~/.local` on first boot: **Claude Code** (default), **Codex**, and **Pi**. **OpenCode**, **DeepAgents**, **Hermes**, and **Grok Build** are optional — install one with `oh harness install `, which also sets its `install.*` key so a fresh home mount reinstalls it at boot. No harness is baked into the image. **T3 Code** runs on demand via the `/t3` skill (or directly with `npx t3`) as a browser UI on port 3773, and **Prime Agent** installs on demand with `oh harness install prime-agent` — neither has an `INSTALL_*` key, because neither is ever baked into the image. Inside the sandbox, run `herdr` first, then launch whichever agent you prefer from its panes and switch between them at any time. Reserve tmux for Open Harness's managed/headless cron, gateway, and watchdog infrastructure. Open Harness is the harness; the **agent** is your call. To go beyond the preinstalled options, install via `npm` / `pip` / `cargo` inside the sandbox or edit the Dockerfile. For Pi+Slack specifically, the recommended path is the `pi-messenger-bridge` npm package — see [Slack integration](../integrations/slack.md). The product surface is one developer, one project, one agent — not racing or stacking multiple CLIs against each other. ## Installing a harness -`oh harness` is the shortest path. It does both halves in one command: it sets -the `.devcontainer/.env` `INSTALL_*` flag so the choice survives the next image build, -**and** installs the CLI into the already-running container so it is usable now. -It never rebuilds or restarts the sandbox. +`oh harness` is the only path, and it does both halves in one command: it sets +the `install.*` key in `oh.json` so a fresh home mount reinstalls the harness at +boot, **and** installs the CLI into the already-running container so it is usable +now. It never rebuilds or restarts the sandbox. ```bash oh harness list # what exists, what is enabled, what is installed @@ -64,7 +64,7 @@ claude --version codex --version pi --version -# Optional image-level CLIs, present only when enabled in .devcontainer/.env: +# Optional CLIs, present only after `oh harness install ` (or its install.* key): opencode --version # install.opencode: true deepagents -v # install.deepagents: true hermes --version # install.hermes: true diff --git a/docs/harnesses/prime-agent.md b/docs/harnesses/prime-agent.md index 0b459caa..01d3426b 100644 --- a/docs/harnesses/prime-agent.md +++ b/docs/harnesses/prime-agent.md @@ -108,7 +108,7 @@ stops all of them. These are deliberate gaps, not oversights: -- **No image-level install.** There is no `INSTALL_PRIME_AGENT` build arg, no compose auth +- **No install.\* key.** There is no `INSTALL_PRIME_AGENT` key, no compose auth volume, and no `install.*` field in `oh.json`. The harness is `on-demand` only. - **No `/spec execute` provider integration.** `/spec execute` uses the active Advisor session and does not launch a provider-specific wrapper; run Prime Agent directly only diff --git a/docs/quickstart.md b/docs/quickstart.md index 5f89d65d..f7d4c20c 100644 --- a/docs/quickstart.md +++ b/docs/quickstart.md @@ -126,7 +126,7 @@ them in place without a rebuild. A first boot on a fresh home mount therefore ne network access and takes a minute or two longer; the sandbox still comes up as a usable shell if the registry is unreachable, and you can retry with `bash .oh/scripts/provision-defaults.sh`. OpenCode, DeepAgents, Hermes, and Grok -Build are optional image-level installs; T3 Code runs on demand via the `/t3` skill +Build are optional installs via `oh harness install `; T3 Code runs on demand via the `/t3` skill or direct `npx`. Authenticate at least one harness before use. > **Simplest cross-provider login — device mode via `/login`.** The most straightforward path From 3a00bb05013fc3206403f13444e4b02998680d3a Mon Sep 17 00:00:00 2001 From: Ryan Eggleston Date: Mon, 31 Aug 2026 14:57:37 -0600 Subject: [PATCH 09/23] FROM task/910-retire-deepagents TO development (#911) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * task: provision the default harnesses into the home mount oh is meant to split by execution target -- on the host it provisions the host or the sandbox, and inside the sandbox it provisions the sandbox with harnesses and tools. The second half did not work for the two harnesses most people use. claude-code and codex carried installUser: "root", which against the local execution target becomes sudo -n -- npm install -g, and /etc/sudoers.d/sandbox grants sandbox ALL=(ALL) ALL with no NOPASSWD. sudo -n true returns "a password is required". Both now match the pi entry directly above them: installUser "sandbox", npm --prefix /home/sandbox/.local install -g. That lands them inside the home mount, so they also survive container recreate and can be upgraded in place in a running remote sandbox rather than requiring an image rebuild. claude-code deliberately does not get --ignore-scripts. Its postinstall copies the native binary over a placeholder; with the flag the install succeeds and claude --version then fails with "claude native binary not installed". Verified both ways against a scratch prefix. provision-harnesses.sh follows provision-python.sh: the same mode flag, the same root to gosu sandbox re-exec, the same ownership diagnostics, the same die-with- the-command-to-re-run style. --print-env is absent because this provisioner exports nothing downstream. It reads the catalog through oh harness list --json and installs through oh harness install, so the shell knows no ids, packages, prefixes, or argv, and the TypeScript catalog stays the only description. No default harness carries a version pin today, so an existing install is never replaced and the script says so in its own output rather than implying it refreshes. The entrypoint hook runs after link-providers.sh, not before. link-providers' only binary dependency is cc-safety-net, which stays baked, and it is the boot-critical hard gate; a network-dependent best-effort step does not belong in front of the step that decides whether the boot is viable. Provisioning warns and continues, so an offline sandbox still comes up as a usable shell. BAKE_HARNESSES defaults to true and gates only the $AGENTS loop, not the whole RUN. INSTALL_OPENCODE and INSTALL_GROK_BUILD are separate opt-ins and turning them off as a side effect would be a silent regression. Nothing leaves the image in this change. * fix: bound the boot path and close four more provisioning defects An adversarial audit of #903 found five defects that six green checks missed. The serious one is a boot hang. oh harness list --json probes every entry in the catalog, not just the three defaults, and one of them is t3code, whose verifyArgv is npx --no-install t3 --version. npx contacts the registry, and probeInstalled passed no timeoutMs, so spawnSync waited without bound. Against an unreachable registry the auditor's run was still going at 2m30 when their own timeout killed it. On any boot where DNS resolves but the registry does not answer, the entrypoint blocks before sleep infinity, exceeds the 300s start_period, and never goes healthy -- and restart: unless-stopped does not rescue an unhealthy-but-alive container. Warn-and-continue cannot help, because a hang never reaches the if !. It hangs while listing, before any install, so BAKE_HARNESSES=true did not avoid it either. Bounded at three layers, because each fails differently: a 15s timeoutMs on the probe spawn, reported as unknown rather than a crash; a --defaults filter on oh harness list so the boot path probes three entries instead of nine and never runs npx; and a timeout wrapper on the entrypoint call so the boot is bounded whatever the CLI does. Measured against the auditor's exact command: 2m30 and killed, to 17.0s full-catalog and 1.55s with --defaults. The install loop read from a herestring while installs run with stdio inherit, so an installer that reads stdin consumed the rest of the loop. Reproduced with a stub: three missing harnesses, one installed, exit 0, success printed. Latent with npm, live the moment a default uses the curl | bash shape two catalog entries already use. Installs now read from /dev/null. The script force-exported OH_EXECUTION_TARGET=local, which short-circuits the in-container check, while the prefix is hardcoded to /home/sandbox/.local, and every error told the operator to re-run with no mention of where. On the host that provisioned the host. It now refuses unless inside the sandbox, reusing the CLI's own runningInsideSandbox predicate rather than inventing a check, and the entrypoint asserts the local target explicitly -- the documented raw docker run recipe never passes SANDBOX_NAME, so the guard would otherwise have silently skipped provisioning for the prebuilt-image flavor. The probe asserted that ARG BAKE_HARNESSES was declared, not that anything used it: deleting the gate left it green. It now checks the ARG is referenced by the RUN that installs $AGENTS and by the one that bakes pi, and that both stages declare it. Deleting either declaration also used to pass. ARG is stage-scoped, so BAKE_HARNESSES=false unbaked claude-code and codex but left pi baked in the home stage while the else-branch claimed otherwise. The home stage now declares and honors the flag. Also: OH_SANDBOX_USER was advertised but illusory, since the catalog hardcodes the user and prefix; it is gone. The final log line no longer claims to have provisioned anything in --verify mode. * task: stop baking the default harnesses into the sandbox image PR #903 wired provision-harnesses.sh into the boot path but shipped it behind ARG BAKE_HARNESSES=true, so every default harness was already present when the provisioner ran and the install path never executed. All four defects that PR's audit found lived in code a green CI run and a normal boot both skip. Delete the bake rather than flip its default: remove ARG BAKE_HARNESSES, ARG AGENTS, the PKG map and the $AGENTS loop in `base`, and the gated pi install in `home`. A build arg that can re-bake is a dormant path that would restore both the shadowed /usr/lib/node_modules copy and the untested boot install. Make the install path CI-visible, since it is now load-bearing on every boot: - The boot smoke asserts the outcome — each default harness resolves under NPM_USER_PREFIX via `type -P`, is owned by the reconciled sandbox uid, and prints its own version — and refuses to pass when the catalog reports no defaults. It boots on a fresh home volume, so this runs real npm work. - verify-sandbox-image.sh gains the negative: reading the catalog out of the image itself, no kind:"default" harness may be installed. - start_period goes 300s -> 600s in both compose files to cover the install, and the boot-guard probe now derives the smoke deadline from the healthcheck window instead of pinning a literal that a start_period bump could invert. - The probe and unit assertions invert from "the bake is gated" to "no default harness package appears in the Dockerfile", reading the package names out of installArgv so they cannot drift from the catalog. cc-safety-net stays baked. Opt-in INSTALL_* harnesses are untouched. Costs this accepts, documented in installation.md: a first boot on a fresh home mount needs network and runs 60-180s longer; an offline first boot yields a usable shell with no agent CLIs; ~/.npm now lives in the home mount. Closes #904 * task: cite the measured first-boot provisioning time * task: move herdr and cloudflared out of the image into the tool catalog Per the ownership boundary — the in-sandbox CLI provisions harnesses and tools — herdr and cloudflared are tools, so the image should not carry them. #905 did this for the default harnesses; this does it for the default tools. The obvious template does not work. #897's tailscale entry root-installs to /usr/local/bin, and commands/tool.ts:309 passes stdio:"inherit", so local-target.ts:113-116 selects the INTERACTIVE branch — plain `sudo --`, no -n. /etc/sudoers.d/sandbox grants `sandbox ALL=(ALL) ALL` with no NOPASSWD, so `oh tool install ` hangs on a password prompt no agent can answer. Verified in a running sandbox: `sudo -n -- true` → "a password is required". (#897's tailscale has the same defect; flagged there, not fixed here.) So install to ~/.local/bin as the sandbox user instead, the same correction #900 made for the harnesses. No sudo, survives container recreation in the home mount, and upgradeable in place by a running sandbox. - ToolKind gains "default". herdr 0.7.4 and cloudflared 2026.8.2 become kind:"default", installUser:"sandbox", with per-arch pinned URLs and sha256 verification into $NPM_USER_PREFIX/bin. Checksums measured by downloading both arches, not copied from anywhere. - provision-harnesses.sh generalizes over both catalogs and becomes provision-defaults.sh (OH_PROVISION_DEFAULTS, timeout 180s → 240s). It dies rather than reporting success when neither catalog yields a default. - The Dockerfile loses the herdr RUN, ARG HERDR_VERSION, and the whole cloudflared apt block — with it the bookworm-suite workaround that existed only because Cloudflare publishes no trixie suite. Docker's is now the only third-party apt source. - Both oracles generalize: verify-sandbox-image.sh rejects a baked default harness OR tool, reading each catalog out of the image; the boot smoke asserts every default in both catalogs resolves under NPM_USER_PREFIX, is owned by the sandbox uid, and prints a version. - The herdr version+checksum pin moves from the Dockerfile to the catalog, and herdr-default.test.ts follows it. Costs, documented in installation.md: an offline first boot on a fresh home mount now has no herdr, so `oh shell` lands in a plain shell with tmux as the fallback multiplexer. The entrypoint says so explicitly on failure. Closes #906 * task: remove optional-harness build args and make the CLI the install path #905 and #907 moved the default harnesses and tools out of the image but left the four optional harnesses behind. The boundary — inside the sandbox the CLI provisions harnesses and tools — has no carve-out for optional ones. They were not merely leftover. opencode, grok-build, and hermes are all installUser:"root", and harness.ts:256 installs with stdio:"inherit", so local-target.ts selects the INTERACTIVE branch: plain `sudo --`, no -n. /etc/sudoers.d/sandbox has no NOPASSWD, so `oh harness install opencode` hangs on a password prompt no agent can answer. The build arg was the only working path, which is why the Dockerfile blocks could not simply be deleted. All four relocate to the sandbox user, verified by reading the upstream installers rather than guessing: opencode takes an npm --prefix like claude-code; grok's installer honours GROK_BIN_DIR; hermes honours HERMES_INSTALL_DIR and its get_command_link_dir() already picks ~/.local/bin for a non-root install; deepagents was already sandbox-installed via uv. So no sudoers change is needed and no security posture moves. - Delete all four ARG/RUN pairs, the compose build.args block, and the dead /opt/grok-build and /usr/local/lib/hermes-agent chowns. INSTALL_HERMES keeps its RUNTIME life — link-providers.sh vendors the Hermes skill pack from it and entrypoint.sh wires auth.json — so only its build-arg role goes. - Remove `buildArg` from HarnessEntry entirely. It was dead metadata: declared, set four times, read by nothing. tool-catalog-boundary.sh already banned the same field in the tool catalog. - provision-defaults.sh now reads the full catalog and also installs any non-default entry whose install. is true, so declared intent survives a fresh home mount. isInstallFlagEnabled already reads oh.json, so this needs no new env plumbing. - verify-sandbox-image.sh widens to "no harness of any kind is baked", and gains the inverse for tools: every kind:"baked-in" tool must be present, or the check passes on an image missing everything. - sandbox-compatibility.yml's optional-installer job loses its subject. It now boots the image and runs `oh harness install` for each optional harness, asserting the binary lands under /home/sandbox/.local — the path operators actually use, instead of one that no longer exists. harness.test.ts had a case asserting `cmd === "sudo"`, codifying the very defect this fixes. It now asserts no install shells out to sudo at all. Closes #908 * fix: retry an optional-harness install once before failing the job The new compatibility job reaches four third-party endpoints. Hermes' own installer hard-fails the whole install when its internal `npm install` step blips, which took the job down on a commit that was correct — the rerun passed unchanged, on the same SHA. A vendor's transient error must not block this repo's merges. One retry absorbs it. The contract is unchanged: a genuine break — wrong user, wrong path, a sudo prompt — fails both attempts and still fails the job. The probe now asserts both halves, so neither the retry nor the hard failure after it can be dropped silently. * fix(test): drop a template-literal scan that fired on catalog order The source-level regex I added in #906 cannot distinguish a JS backtick from a backtick inside prose — notInstallableReason has several. It passed only because every ${...} in the catalog happened to precede the first prose backtick. Adding a tool below them flips it to a false failure, which is exactly what happened on the #858 branch. The per-token ban, with the bash -lc body exempted, covers what is actually checkable. * task: retire the DeepAgents harness deepagents-cli is deprecated upstream. Its own --version output now leads with: DeprecationWarning: deepagents-cli is deprecated and will not receive further releases. Use the `managed-deepagents` package (uv tool install managed-deepagents, then mda) instead. Following it is not a rename — managed-deepagents ships a different binary (mda), so binary, verifyArgv, the banner tile, and the docs would all move. Operator ruling: retire the entry rather than chase a product that is moving away from a local CLI. Removed: the catalog entry; install.deepagents from OhConfig, its default, its validation loop and its OH_CONFIG_FIELDS row; the config-render put(); the env-file INSTALL_FIELDS/CONFIG_FIELD_BY_ENV_KEY entries; the init wizard option and its ENV_TO_CONFIG mapping; the banner tile and shortcut; install.sh and migrate-harness-yaml.sh entries; docs/harnesses/deepagents.md and every reference to it; the oh.json key. Kept deliberately: - INSTALL_DEEPAGENTS in config-render's RETIRED_KEYS, so anything that tries to render it again fails loudly instead of emitting a dead variable. - INSTALL_DEEPAGENTS in the two probes that assert no harness build arg returns to the Dockerfile. Dropping it there would let the arg come back unnoticed. `oh config set install.deepagents true` now exits 1 with `unknown field`, because the OH_CONFIG_FIELDS row is gone. A stale key already present in an oh.json is ignored rather than rejected — that is the pre-existing behaviour for any unrecognised install.* key, not something this change introduces. Closes #910 --- .oh/cli/src/__tests__/harness-catalog.test.ts | 1 - .oh/cli/src/__tests__/harness.test.ts | 15 +- .oh/cli/src/__tests__/init.test.ts | 1 - .oh/cli/src/commands/init.ts | 4 - .../src/lib/__tests__/config-render.test.ts | 3 +- .oh/cli/src/lib/config-render.ts | 9 +- .oh/cli/src/lib/env-file.ts | 2 - .oh/cli/src/lib/harnesses/catalog.ts | 11 - .oh/cli/src/lib/oh-config.ts | 5 +- .oh/evals/RESULTS.md | 206 +++++++++--------- .oh/evals/probes/oh-home-mount.sh | 2 +- .oh/install/banner.sh | 14 -- .oh/scripts/install.sh | 4 +- .oh/scripts/migrate-harness-yaml.sh | 2 - CHANGELOG.md | 3 +- README.md | 3 +- docs/README.md | 1 - docs/configuration.md | 3 +- docs/harnesses/deepagents.md | 187 ---------------- docs/harnesses/hermes.md | 2 +- docs/harnesses/overview.md | 5 +- docs/installation.md | 1 - docs/quickstart.md | 5 +- oh.json | 1 - 24 files changed, 123 insertions(+), 367 deletions(-) delete mode 100644 docs/harnesses/deepagents.md diff --git a/.oh/cli/src/__tests__/harness-catalog.test.ts b/.oh/cli/src/__tests__/harness-catalog.test.ts index 2b3290c0..b35a49ce 100644 --- a/.oh/cli/src/__tests__/harness-catalog.test.ts +++ b/.oh/cli/src/__tests__/harness-catalog.test.ts @@ -68,7 +68,6 @@ describe("harness catalog", () => { it("covers all four optional harnesses", () => { expect(optional.map((h) => h.id).sort()).toEqual([ - "deepagents", "grok-build", "hermes", "opencode", diff --git a/.oh/cli/src/__tests__/harness.test.ts b/.oh/cli/src/__tests__/harness.test.ts index acbde334..91918d75 100644 --- a/.oh/cli/src/__tests__/harness.test.ts +++ b/.oh/cli/src/__tests__/harness.test.ts @@ -166,7 +166,7 @@ describe("help", () => { it("names every installable harness so `` is discoverable", () => { const help = captureStdout(printHarnessHelp); - for (const id of ["claude-code", "codex", "pi", "opencode", "grok-build", "deepagents", "hermes", "t3code"]) { + for (const id of ["claude-code", "codex", "pi", "opencode", "grok-build", "hermes", "t3code"]) { expect(help).toContain(id); } }); @@ -289,19 +289,6 @@ describe("runHarnessInstall against the container", () => { ); }); - it("installs deepagents as the sandbox user, not root", async () => { - const root = makeRepo(); - const { calls, run } = makeRunner((c, a) => { - if (isInspect(c, a)) return running; - if (isExecOf(c, a, "--version")) return { status: 1, stdout: "", stderr: "" }; - return undefined; - }); - - await runHarnessInstall("deepagents", { cwd: root, run }, makeIo().io); - const install = execCalls(calls).find((c) => c.args.includes("deepagents-cli")); - expect(install!.args[install!.args.indexOf("-u") + 1]).toBe("sandbox"); - }); - it("is a no-op when the binary is already present", async () => { const root = makeRepo(); const { calls, run } = makeRunner((c, a) => (isInspect(c, a) ? running : undefined)); diff --git a/.oh/cli/src/__tests__/init.test.ts b/.oh/cli/src/__tests__/init.test.ts index f74bf689..a06ab547 100644 --- a/.oh/cli/src/__tests__/init.test.ts +++ b/.oh/cli/src/__tests__/init.test.ts @@ -476,7 +476,6 @@ describe("runInit", () => { expect(config.install).toEqual({ opencode: false, grokBuild: false, - deepagents: false, hermes: false, agentBrowser: false, }); diff --git a/.oh/cli/src/commands/init.ts b/.oh/cli/src/commands/init.ts index c1db4fd7..d313a55b 100644 --- a/.oh/cli/src/commands/init.ts +++ b/.oh/cli/src/commands/init.ts @@ -653,9 +653,6 @@ const ENV_TO_CONFIG: Record = { INSTALL_GROK_BUILD: (c, v) => { section(c, "install").grokBuild = asBool(v); }, - INSTALL_DEEPAGENTS: (c, v) => { - section(c, "install").deepagents = asBool(v); - }, INSTALL_HERMES: (c, v) => { section(c, "install").hermes = asBool(v); }, @@ -777,7 +774,6 @@ async function runWizard( prompt.step(2, 5, "Optional installs"); const installs: { key: string; field: string; desc: string }[] = [ { key: "opencode", field: "opencode", desc: "OpenCode TUI coding agent" }, - { key: "deepagents", field: "deepagents", desc: "DeepAgents multi-agent runtime" }, { key: "hermes", field: "hermes", desc: "Hermes CLI + runtime (build arg + runtime)" }, { key: "grok_build", field: "grokBuild", desc: "Grok build tooling" }, { key: "agent_browser", field: "agentBrowser", desc: "agent-browser + Chromium (~1 GB)" }, diff --git a/.oh/cli/src/lib/__tests__/config-render.test.ts b/.oh/cli/src/lib/__tests__/config-render.test.ts index ce0d3a0e..a8df288f 100644 --- a/.oh/cli/src/lib/__tests__/config-render.test.ts +++ b/.oh/cli/src/lib/__tests__/config-render.test.ts @@ -57,7 +57,8 @@ describe("renderComposeEnv", () => { expect(text).toContain("GIT_USER_EMAIL=ada@example.com"); expect(text).toContain("INSTALL_OPENCODE=false"); expect(text).toContain("INSTALL_GROK_BUILD=false"); - expect(text).toContain("INSTALL_DEEPAGENTS=false"); + // #910: deepagents is retired; the key must no longer be rendered. + expect(text).not.toContain("INSTALL_DEEPAGENTS"); expect(text).toContain("INSTALL_HERMES=false"); expect(text).toContain("INSTALL_AGENT_BROWSER=false"); expect(text).toContain("DOCKER_SOCKET=true"); diff --git a/.oh/cli/src/lib/config-render.ts b/.oh/cli/src/lib/config-render.ts index 276b28b3..44a1f590 100644 --- a/.oh/cli/src/lib/config-render.ts +++ b/.oh/cli/src/lib/config-render.ts @@ -1,7 +1,13 @@ import type { OhConfig } from "./oh-config.js"; import { isSecretKey } from "./secrets.js"; -const RETIRED_KEYS = ["WORKTREES_DIR", "PROJECTS_DIR", "CRONS_DIR", "OH_PROJECT_ROOT"] as const; +const RETIRED_KEYS = [ + "WORKTREES_DIR", + "PROJECTS_DIR", + "CRONS_DIR", + "OH_PROJECT_ROOT", + "INSTALL_DEEPAGENTS", +] as const; export interface RenderedVar { key: string; @@ -24,7 +30,6 @@ export function renderComposeVars(config: OhConfig): RenderedVar[] { put("INSTALL_OPENCODE", config.install?.opencode); put("INSTALL_GROK_BUILD", config.install?.grokBuild); - put("INSTALL_DEEPAGENTS", config.install?.deepagents); put("INSTALL_HERMES", config.install?.hermes); put("INSTALL_AGENT_BROWSER", config.install?.agentBrowser); diff --git a/.oh/cli/src/lib/env-file.ts b/.oh/cli/src/lib/env-file.ts index d06293a5..e1e09d75 100644 --- a/.oh/cli/src/lib/env-file.ts +++ b/.oh/cli/src/lib/env-file.ts @@ -25,7 +25,6 @@ function stripQuotes(s: string): string { export const INSTALL_FIELDS: Record = { opencode: "install.opencode", grok_build: "install.grokBuild", - deepagents: "install.deepagents", hermes: "install.hermes", agent_browser: "install.agentBrowser", }; @@ -34,7 +33,6 @@ export const CONFIG_FIELD_BY_ENV_KEY: Record = { DOCKER_SOCKET: "access.dockerSocket", INSTALL_OPENCODE: INSTALL_FIELDS.opencode, INSTALL_GROK_BUILD: INSTALL_FIELDS.grok_build, - INSTALL_DEEPAGENTS: INSTALL_FIELDS.deepagents, INSTALL_HERMES: INSTALL_FIELDS.hermes, INSTALL_AGENT_BROWSER: INSTALL_FIELDS.agent_browser, }; diff --git a/.oh/cli/src/lib/harnesses/catalog.ts b/.oh/cli/src/lib/harnesses/catalog.ts index c677d52c..3975e173 100644 --- a/.oh/cli/src/lib/harnesses/catalog.ts +++ b/.oh/cli/src/lib/harnesses/catalog.ts @@ -102,17 +102,6 @@ export const HARNESS_CATALOG: readonly HarnessEntry[] = [ docsPath: "docs/harnesses/grok-build.md", kind: "optional", }, - { - id: "deepagents", - title: "DeepAgents", - binary: "deepagents", - harnessKey: "deepagents", - installArgv: ["uv", "tool", "install", "deepagents-cli"], - installUser: "sandbox", - verifyArgv: ["deepagents", "--version"], - docsPath: "docs/harnesses/deepagents.md", - kind: "optional", - }, { id: "hermes", title: "Hermes", diff --git a/.oh/cli/src/lib/oh-config.ts b/.oh/cli/src/lib/oh-config.ts index f09caac4..4014f8d4 100644 --- a/.oh/cli/src/lib/oh-config.ts +++ b/.oh/cli/src/lib/oh-config.ts @@ -36,7 +36,6 @@ export interface GitIdentity { export interface InstallFlags { opencode?: boolean; grokBuild?: boolean; - deepagents?: boolean; hermes?: boolean; agentBrowser?: boolean; } @@ -126,7 +125,6 @@ export function defaultOhConfig(name: string): OhConfig { install: { opencode: false, grokBuild: false, - deepagents: false, hermes: false, agentBrowser: false, }, @@ -222,7 +220,7 @@ export function validateOhConfig(value: unknown): OhConfig { const install = expectSection(record, "install"); if (install) { - for (const key of ["opencode", "grokBuild", "deepagents", "hermes", "agentBrowser"]) { + for (const key of ["opencode", "grokBuild", "hermes", "agentBrowser"]) { expectBoolean(install, key, "install."); } } @@ -349,7 +347,6 @@ export const OH_CONFIG_FIELDS: readonly OhConfigField[] = [ { path: "git.userEmail", type: "string" }, { path: "install.opencode", type: "boolean" }, { path: "install.grokBuild", type: "boolean" }, - { path: "install.deepagents", type: "boolean" }, { path: "install.hermes", type: "boolean" }, { path: "install.agentBrowser", type: "boolean" }, { path: "access.ssh", type: "boolean" }, diff --git a/.oh/evals/RESULTS.md b/.oh/evals/RESULTS.md index 880cf73a..25105ddf 100644 --- a/.oh/evals/RESULTS.md +++ b/.oh/evals/RESULTS.md @@ -6,108 +6,108 @@ probe id; git history is the time series.** Schema and exit-code semantics are i | probe | tier | last-run (UTC) | status | source | |-------|------|----------------|--------|--------| -| advisor-monitored-loop | A | 2026-08-31 17:17 | PASS | conversation 2026-06-19 (single-owner implementation workflow, issue #257) | -| agent-browser-cli | A | 2026-08-31 17:17 | PASS | retro lesson 2026-06-07 (agent-browser 0.8.5 CLI) | -| agents-identity-contract | A | 2026-08-31 17:17 | PASS | issue #854 — T3-style root identity, glossary, and skill-owned procedures | -| artifact-contract-audit | A | 2026-08-31 17:17 | PASS | issue #583/#645 — production /audit implementation Gate 1 behavior | -| audit-dispatcher-contract | A | 2026-08-31 17:17 | PASS | issue #645 — audit consolidation public taxonomy | -| audit-implementation-behavior | A | 2026-08-31 17:17 | PASS | issue #645 — implementation root/repo/browser behavior | -| audit-pr-acquire | A | 2026-08-31 17:17 | PASS | issue #645 — production PR acquisition behavior | -| audit-pr-classifier | A | 2026-08-31 17:17 | PASS | issue #645 — deterministic focused and queue PR classifier | -| audit-run-root-contract | A | 2026-08-31 17:17 | PASS | issue #645 — executable immutable audit root/run correlation | -| audit-shellcheck-coverage | A | 2026-08-31 17:17 | PASS | issue #645 — private audit scripts require release and CI lint coverage | -| audit-stale-references | A | 2026-08-31 17:17 | PASS | issue #645 — clean-breaking audit migration | -| boot-lint-glob | A | 2026-08-31 17:17 | PASS | issue #90, issue #120 | -| builder-skill-consolidation | A | 2026-08-31 17:17 | PASS | issue #643 — consolidate artifact builders behind one /builder dispatcher | -| capability-benchmark-schema | A | 2026-08-31 17:17 | PASS | issue #167 — capability benchmark instrument | -| cc-safety-net-wiring | A | 2026-08-31 17:17 | SKIPPED | .oh/tasks/cc-safety-net/prd.json US-007 2026-07-19 | -| changelog-entry-length | A | 2026-08-31 17:17 | PASS | conversation 2026-08-24 — CHANGELOG.md grew to 259KB of bullet prose because "one line" was unquantified | -| cleanup-tasks-scoped-guard | A | 2026-08-31 17:17 | PASS | issue #85 | -| cleanup-tasks-worktree-grooming | A | 2026-08-31 17:17 | PASS | issue #168; issue #327 | -| cli-publish-typecheck-scope | A | 2026-08-31 17:17 | PASS | release run 33271077312 — v0.5.0 pushed its GHCR image, then failed to publish | -| close-issues-on-development | A | 2026-08-31 17:17 | PASS | issue #841 (closing keywords never fire because the default branch is main) 2026-08-26 | -| codex-stale-response-retry | A | 2026-08-31 17:17 | PASS | issue #506 — Codex previous_response_not_found RCA | -| compose-config-path-parity | A | 2026-08-31 17:17 | PASS | PR #833 (remove harness.yaml — the wrapper and VS Code "Reopen in Container" paths must resolve the same service) 2026-08-26 | -| config-schema-parity | A | 2026-08-31 17:17 | PASS | PR #833 (one schema file — DOCKER_SOCKET, SANDBOX_SSH, OH_SANDBOX_IMAGE, OH_PULL_POLICY, SKIP_PNPM_INSTALL were consumed but undocumented); rewritten for the oh.json/secrets split by PR #887 | -| context-tier-size-budget | A | 2026-08-31 17:17 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-007) — the always-on tier was 85,256 B | -| cron-claude-codex-fallback | A | 2026-08-31 17:17 | PASS | conversation 2026-06-12 (default Codex fallback for crons) | -| cron-watchdog | A | 2026-08-31 17:17 | PASS | issues #130/#453 (cron runtime watchdog + legacy system-cron reaping) 2026-06-19 | -| crons-directory-guide | A | 2026-08-31 17:17 | PASS | issue #874 | -| curl-bash-safe-alternatives | A | 2026-08-31 17:17 | PASS | vet-run/vet integration — public curl|bash examples need review-first alternatives | -| datasets-schema | A | 2026-08-31 17:17 | PASS | issue #196 — .oh/evals/datasets verifiable trajectory corpus (Repo2RLEnv-inspired) | -| debugmcp-availability | A | 2026-08-31 17:17 | SKIPPED | issue #297 — DebugMCP MCP debug-server availability | -| default-provisioning | A | 2026-08-31 17:17 | PASS | #902 — `oh harness install` must work from inside the sandbox, where | -| delegate-model-effort-policy | A | 2026-08-31 17:17 | PASS | conversation 2026-07-11 (delegate model inheritance and thinking policy) | -| devtcp-hook | A | 2026-08-31 17:17 | PASS | retro lesson 2026-06-10 (zsh /dev/tcp) | -| docker-inspect-env-guard | A | 2026-08-31 17:17 | PASS | operator directive 2026-08-08 (agents keep the docker socket, but must | -| docs-build-fast-path | A | 2026-08-31 17:17 | PASS | #455 — docs builds must stay out of fast harness/eval/release gates; #536 — docs site externalized to openharness-web; docs markdown relocated to docs/ | -| drift-check-cron-staleness-glob | A | 2026-08-31 17:17 | PASS | issue #98; issue #225 (restart-required cron frontmatter/config drift) | -| entrypoint-pnpm-manifest-fingerprint | A | 2026-08-31 17:17 | PASS | issue #521 (manifest-aware sandbox installs) 2026-07-01 | -| eval-ci-gate | A | 2026-08-31 17:17 | PASS | #103 — eval probe suite gated in CI | -| eval-gate | A | 2026-08-31 17:17 | PASS | retro lesson 2026-06-11 (eval-gate) | -| eval-results-atomic | A | 2026-08-31 17:17 | PASS | issue #83 (eval-results-atomic-write) | -| eval-runner-exit | A | 2026-08-31 17:17 | PASS | retro lesson 2026-06-11 (eval-runner-exit) #29 | -| eval-runs-once-per-cycle | A | 2026-08-31 17:17 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-006) — /eval ran 3x per cycle on the | -| execution-target-contract | A | 2026-08-31 17:17 | PASS | issue #733 (ExecutionTarget contract + Docker Compose adapter) 2026-08-10 | -| get-oh-bootstrap | A | 2026-08-31 17:17 | PASS | get-oh.sh bootstrap — the Node-bootstrapping host-side path to the standalone `oh` CLI (also on npm as @mifune/openharness; see oh-npm-package.sh) | -| git-skill | A | 2026-08-31 17:17 | PASS | conversation 2026-06-15 — rules are not always supported; git workflow must be the /git skill | -| harness-audit-empty-output-gate | A | 2026-08-31 17:17 | PASS | issue #246 — /audit harness must fail closed on empty auditor outputs | -| harness-ci-core-paths | A | 2026-08-31 17:17 | PASS | #165 — core sandbox config files must trigger harness CI | -| harness-ci-hooks-paths | A | 2026-08-31 17:17 | PASS | issue #202 — credential/security hook changes must trigger harness CI | -| harness-yaml-migration | A | 2026-08-31 17:17 | PASS | PR #833 (migrate-harness-yaml.sh — append / uncomment-in-place / preserve / overwrite, plus a silent no-op second run) 2026-08-26 | -| health-check-docker-stats | A | 2026-08-31 17:17 | PASS | retro lesson 2026-06-10 (docker stats vs ps Size) | -| health-check-socket-degrade | A | 2026-08-31 17:17 | PASS | issue #762 (refs #756) — /health-check degrades to one statement, not nine failures | -| heartbeat-logging-contract | A | 2026-08-31 17:17 | PASS | issue #447 (heartbeat log append hardening) 2026-06-18 | -| image-seed-hygiene | A | 2026-08-31 17:17 | PASS | issue #900 (slim the sandbox image) 2026-08-30 | -| markitdown-wiki-ingest | A | 2026-08-31 17:17 | PASS | issue #649 — pinned local-document normalization contract for /wiki ingest | -| next-dev-prod | A | 2026-08-31 17:17 | SKIPPED | retro lesson 2026-06-04 | -| oh-compose-env-wiring | A | 2026-08-31 17:17 | PASS | issue #880 (oh as the only front door — oh.json is the non-secret config surface) | -| oh-config-surfaces | A | 2026-08-31 17:17 | REGRESSION | PR #887 (config split across two authored surfaces — a tracked oh.json and a secrets-only root dotenv — with nothing left under $HOME) | -| oh-destroy-guard | A | 2026-08-31 17:17 | PASS | issue #879 — `oh` becomes the only front door, so `make destroy` must | -| oh-devcontainer-restructure | A | 2026-08-31 17:17 | PASS | consolidate devcontainer — .oh/devcontainer/ folded back into .devcontainer/ | -| oh-home-mount | A | 2026-08-31 17:17 | PASS | issue #898 (single $HOME mount) 2026-08-30 | -| oh-image-only-deploy | A | 2026-08-31 17:17 | PASS | .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only deploy) | -| oh-init-headless-config | A | 2026-08-31 17:17 | PASS | PR #827 (installer answers landed in the losing config file); retargeted to the .example.env template by PR #833, then to oh.json by PR #887 | -| oh-init-scaffold | A | 2026-08-31 17:17 | PASS | issue #531 Phase 2 | -| oh-lifecycle-surface | A | 2026-08-31 17:17 | PASS | issue #881 — the Makefile is retired and `oh` is the only front door | -| oh-npm-package | A | 2026-08-31 17:17 | PASS | npm publish path for the standalone `oh` CLI (@mifune/openharness) — alternative to get-oh.sh | -| oh-payload-manifest | A | 2026-08-31 17:17 | PASS | issue #531 follow-on (.oh payload manifest — oh update ships a declared allowlist) | -| oh-sandbox-image-mode | A | 2026-08-31 17:17 | PASS | conversation 2026-07-05 (basic Docker deployment — prebuilt-image mode) | -| oh-shipped-repo-overridable | A | 2026-08-31 17:17 | PASS | issue #531 follow-on (de-hardcode residual — shipped .oh shell scripts keep the upstream repo overridable) | -| oh-standalone-lifecycle | A | 2026-08-31 17:17 | PASS | issue #564 | -| oh-update | A | 2026-08-31 17:17 | PASS | issue #531 Phase 3 (oh update — upgrade only the .oh control plane) | -| operator-config-guard | A | 2026-08-31 17:17 | PASS | operator directives 2026-08-06 (.config/ and settings.local.json are operator-only) | -| pnpm-audit-ci-gate | A | 2026-08-31 17:17 | PASS | issue #171 — pnpm security audits must run in CI | -| post-bridge-publish-confirmation | A | 2026-08-31 17:17 | PASS | #523 — post-bridge live publishing requires an explicit final confirmation gate | -| prd-output-path-contract | A | 2026-08-31 17:17 | PASS | retro lesson 2026-06-19 | -| prompt-miner-schema-compat | A | 2026-08-31 17:17 | PASS | issue #253 — prompt-miner JSONL schema-drift guard | -| prompt-miner-symlink-entrypoint | A | 2026-08-31 17:17 | PASS | issue #663 — prompt-miner engine no-ops via the documented .claude/skills symlink | -| prompt-miner-weakness-record | A | 2026-08-31 17:17 | PASS | issue #580 — prompt-miner weakness-record (WH-xxx) cluster output | -| protected-path-deletion | A | 2026-08-31 17:17 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-001) — the critique gate was deleted, | -| protected-paths-resolve | A | 2026-08-31 17:17 | PASS | issue #753 — .claude/protected-paths.txt named 7 paths that did not exist. | -| registry-portability-gate | A | 2026-08-31 17:17 | PASS | issue #758 | -| registry-portability | A | 2026-08-31 17:17 | SKIPPED | issue #758 | -| retro-deterministic-contract | A | 2026-08-31 17:17 | PASS | issue #443 — /retro deterministic output and self-contained helper contract | -| rl-delegation-write-worker | A | 2026-08-31 17:17 | PASS | retro lesson 2026-06-10 (rl-delegation) #57 | -| rlm-context-budget | A | 2026-08-31 17:17 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-006 | -| runtime-preflight-gate | A | 2026-08-31 17:17 | PASS | issue #806 § B1 (open sandbox.substrate vs sandbox.runtime selector); | -| sandbox-boot-guard-ci | A | 2026-08-31 17:17 | PASS | issue #449 (sandbox image build CI guard) 2026-06-19; | -| sandbox-node-base | A | 2026-08-31 17:17 | PASS | openharness#878 — oh as the only front door, T0 sandbox base image | -| skill-paths | A | 2026-08-31 17:17 | PASS | issue #43 — stale path references; extended by issue #69 — apps/->packages/ rename guard; extended by issue #870 — deleted .oh/agents/advisor.md | -| skills-dir-clean | A | 2026-08-31 17:17 | PASS | conversation 2026-06-29 — Pi parses every top-level `.md` in the skills | -| skills-task-tool-coupling | A | 2026-08-31 17:17 | PASS | council review 2026-08-29 (issue #886) — /delegate instructed Claude-Code-only | -| skills-vendored | A | 2026-08-31 17:17 | PASS | absorb .mifune submodule into .oh — the skills/agents/hooks pack is vendored | -| slack-admin-command-surface | A | 2026-08-31 17:17 | PASS | issue #354 — Slack bridge docs must distinguish Pi /msg-bridge commands from Slack DM admin text handlers | -| spec-family-contract | A | 2026-08-31 17:17 | PASS | issue #265; spec-simplification issue #816; workflow authority issue #854 | -| spec-ready-finalization | A | 2026-08-31 17:17 | PASS | issue #134; spec-simplification issue #816; workflow authority issue #854 | -| ste-checker-contract | A | 2026-08-31 17:17 | PASS | issue #750 PR audit — the /ste checker had four fail-open paths (unclosed | -| submitted-by-trailers | A | 2026-08-31 17:17 | PASS | conversation 2026-06-12 (commit attribution trailers); the single-owner | -| sync-skill-contract | A | 2026-08-31 17:17 | PASS | issue #331 — /sync dispatcher skill (bidirectional origin↔upstream sync) | -| tool-catalog-boundary | A | 2026-08-31 17:17 | PASS | agent-browser's exclusion from the harness catalog (#821) and the | -| version-parity | A | 2026-08-31 17:17 | PASS | conversation 2026-08-29 — the oh CLI became the only lifecycle door, so its | -| weigh-scorer-contract | A | 2026-08-31 17:17 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-003 (2026-06-27) | -| wiki-readme-index | A | 2026-08-31 17:17 | PASS | issue #132 — wiki README index drift guard | -| workflow-boundaries | A | 2026-08-31 17:17 | PASS | conversation 2026-06-19 (workflow consolidation, issue #259); authority moved to /spec in issue #854 | -| worktrees-layout | A | 2026-08-31 17:17 | PASS | issue #872 | +| advisor-monitored-loop | A | 2026-08-31 20:04 | PASS | conversation 2026-06-19 (single-owner implementation workflow, issue #257) | +| agent-browser-cli | A | 2026-08-31 20:04 | PASS | retro lesson 2026-06-07 (agent-browser 0.8.5 CLI) | +| agents-identity-contract | A | 2026-08-31 20:04 | PASS | issue #854 — T3-style root identity, glossary, and skill-owned procedures | +| artifact-contract-audit | A | 2026-08-31 20:04 | PASS | issue #583/#645 — production /audit implementation Gate 1 behavior | +| audit-dispatcher-contract | A | 2026-08-31 20:04 | PASS | issue #645 — audit consolidation public taxonomy | +| audit-implementation-behavior | A | 2026-08-31 20:04 | PASS | issue #645 — implementation root/repo/browser behavior | +| audit-pr-acquire | A | 2026-08-31 20:04 | PASS | issue #645 — production PR acquisition behavior | +| audit-pr-classifier | A | 2026-08-31 20:04 | PASS | issue #645 — deterministic focused and queue PR classifier | +| audit-run-root-contract | A | 2026-08-31 20:04 | PASS | issue #645 — executable immutable audit root/run correlation | +| audit-shellcheck-coverage | A | 2026-08-31 20:04 | PASS | issue #645 — private audit scripts require release and CI lint coverage | +| audit-stale-references | A | 2026-08-31 20:04 | PASS | issue #645 — clean-breaking audit migration | +| boot-lint-glob | A | 2026-08-31 20:04 | PASS | issue #90, issue #120 | +| builder-skill-consolidation | A | 2026-08-31 20:04 | PASS | issue #643 — consolidate artifact builders behind one /builder dispatcher | +| capability-benchmark-schema | A | 2026-08-31 20:04 | PASS | issue #167 — capability benchmark instrument | +| cc-safety-net-wiring | A | 2026-08-31 20:04 | SKIPPED | .oh/tasks/cc-safety-net/prd.json US-007 2026-07-19 | +| changelog-entry-length | A | 2026-08-31 20:04 | PASS | conversation 2026-08-24 — CHANGELOG.md grew to 259KB of bullet prose because "one line" was unquantified | +| cleanup-tasks-scoped-guard | A | 2026-08-31 20:04 | PASS | issue #85 | +| cleanup-tasks-worktree-grooming | A | 2026-08-31 20:04 | PASS | issue #168; issue #327 | +| cli-publish-typecheck-scope | A | 2026-08-31 20:04 | PASS | release run 33271077312 — v0.5.0 pushed its GHCR image, then failed to publish | +| close-issues-on-development | A | 2026-08-31 20:04 | PASS | issue #841 (closing keywords never fire because the default branch is main) 2026-08-26 | +| codex-stale-response-retry | A | 2026-08-31 20:04 | PASS | issue #506 — Codex previous_response_not_found RCA | +| compose-config-path-parity | A | 2026-08-31 20:04 | PASS | PR #833 (remove harness.yaml — the wrapper and VS Code "Reopen in Container" paths must resolve the same service) 2026-08-26 | +| config-schema-parity | A | 2026-08-31 20:04 | PASS | PR #833 (one schema file — DOCKER_SOCKET, SANDBOX_SSH, OH_SANDBOX_IMAGE, OH_PULL_POLICY, SKIP_PNPM_INSTALL were consumed but undocumented); rewritten for the oh.json/secrets split by PR #887 | +| context-tier-size-budget | A | 2026-08-31 20:04 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-007) — the always-on tier was 85,256 B | +| cron-claude-codex-fallback | A | 2026-08-31 20:04 | PASS | conversation 2026-06-12 (default Codex fallback for crons) | +| cron-watchdog | A | 2026-08-31 20:04 | PASS | issues #130/#453 (cron runtime watchdog + legacy system-cron reaping) 2026-06-19 | +| crons-directory-guide | A | 2026-08-31 20:04 | PASS | issue #874 | +| curl-bash-safe-alternatives | A | 2026-08-31 20:04 | PASS | vet-run/vet integration — public curl|bash examples need review-first alternatives | +| datasets-schema | A | 2026-08-31 20:04 | PASS | issue #196 — .oh/evals/datasets verifiable trajectory corpus (Repo2RLEnv-inspired) | +| debugmcp-availability | A | 2026-08-31 20:04 | SKIPPED | issue #297 — DebugMCP MCP debug-server availability | +| default-provisioning | A | 2026-08-31 20:04 | PASS | #902 — `oh harness install` must work from inside the sandbox, where | +| delegate-model-effort-policy | A | 2026-08-31 20:04 | PASS | conversation 2026-07-11 (delegate model inheritance and thinking policy) | +| devtcp-hook | A | 2026-08-31 20:04 | PASS | retro lesson 2026-06-10 (zsh /dev/tcp) | +| docker-inspect-env-guard | A | 2026-08-31 20:04 | PASS | operator directive 2026-08-08 (agents keep the docker socket, but must | +| docs-build-fast-path | A | 2026-08-31 20:04 | PASS | #455 — docs builds must stay out of fast harness/eval/release gates; #536 — docs site externalized to openharness-web; docs markdown relocated to docs/ | +| drift-check-cron-staleness-glob | A | 2026-08-31 20:04 | PASS | issue #98; issue #225 (restart-required cron frontmatter/config drift) | +| entrypoint-pnpm-manifest-fingerprint | A | 2026-08-31 20:04 | PASS | issue #521 (manifest-aware sandbox installs) 2026-07-01 | +| eval-ci-gate | A | 2026-08-31 20:04 | PASS | #103 — eval probe suite gated in CI | +| eval-gate | A | 2026-08-31 20:04 | PASS | retro lesson 2026-06-11 (eval-gate) | +| eval-results-atomic | A | 2026-08-31 20:04 | PASS | issue #83 (eval-results-atomic-write) | +| eval-runner-exit | A | 2026-08-31 20:04 | PASS | retro lesson 2026-06-11 (eval-runner-exit) #29 | +| eval-runs-once-per-cycle | A | 2026-08-31 20:04 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-006) — /eval ran 3x per cycle on the | +| execution-target-contract | A | 2026-08-31 20:04 | PASS | issue #733 (ExecutionTarget contract + Docker Compose adapter) 2026-08-10 | +| get-oh-bootstrap | A | 2026-08-31 20:04 | PASS | get-oh.sh bootstrap — the Node-bootstrapping host-side path to the standalone `oh` CLI (also on npm as @mifune/openharness; see oh-npm-package.sh) | +| git-skill | A | 2026-08-31 20:04 | PASS | conversation 2026-06-15 — rules are not always supported; git workflow must be the /git skill | +| harness-audit-empty-output-gate | A | 2026-08-31 20:04 | PASS | issue #246 — /audit harness must fail closed on empty auditor outputs | +| harness-ci-core-paths | A | 2026-08-31 20:04 | PASS | #165 — core sandbox config files must trigger harness CI | +| harness-ci-hooks-paths | A | 2026-08-31 20:04 | PASS | issue #202 — credential/security hook changes must trigger harness CI | +| harness-yaml-migration | A | 2026-08-31 20:04 | PASS | PR #833 (migrate-harness-yaml.sh — append / uncomment-in-place / preserve / overwrite, plus a silent no-op second run) 2026-08-26 | +| health-check-docker-stats | A | 2026-08-31 20:04 | PASS | retro lesson 2026-06-10 (docker stats vs ps Size) | +| health-check-socket-degrade | A | 2026-08-31 20:04 | PASS | issue #762 (refs #756) — /health-check degrades to one statement, not nine failures | +| heartbeat-logging-contract | A | 2026-08-31 20:04 | PASS | issue #447 (heartbeat log append hardening) 2026-06-18 | +| image-seed-hygiene | A | 2026-08-31 20:04 | PASS | issue #900 (slim the sandbox image) 2026-08-30 | +| markitdown-wiki-ingest | A | 2026-08-31 20:04 | PASS | issue #649 — pinned local-document normalization contract for /wiki ingest | +| next-dev-prod | A | 2026-08-31 20:04 | SKIPPED | retro lesson 2026-06-04 | +| oh-compose-env-wiring | A | 2026-08-31 20:04 | PASS | issue #880 (oh as the only front door — oh.json is the non-secret config surface) | +| oh-config-surfaces | A | 2026-08-31 20:04 | REGRESSION | PR #887 (config split across two authored surfaces — a tracked oh.json and a secrets-only root dotenv — with nothing left under $HOME) | +| oh-destroy-guard | A | 2026-08-31 20:04 | PASS | issue #879 — `oh` becomes the only front door, so `make destroy` must | +| oh-devcontainer-restructure | A | 2026-08-31 20:04 | PASS | consolidate devcontainer — .oh/devcontainer/ folded back into .devcontainer/ | +| oh-home-mount | A | 2026-08-31 20:04 | PASS | issue #898 (single $HOME mount) 2026-08-30 | +| oh-image-only-deploy | A | 2026-08-31 20:04 | PASS | .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only deploy) | +| oh-init-headless-config | A | 2026-08-31 20:04 | PASS | PR #827 (installer answers landed in the losing config file); retargeted to the .example.env template by PR #833, then to oh.json by PR #887 | +| oh-init-scaffold | A | 2026-08-31 20:04 | PASS | issue #531 Phase 2 | +| oh-lifecycle-surface | A | 2026-08-31 20:04 | PASS | issue #881 — the Makefile is retired and `oh` is the only front door | +| oh-npm-package | A | 2026-08-31 20:04 | PASS | npm publish path for the standalone `oh` CLI (@mifune/openharness) — alternative to get-oh.sh | +| oh-payload-manifest | A | 2026-08-31 20:04 | PASS | issue #531 follow-on (.oh payload manifest — oh update ships a declared allowlist) | +| oh-sandbox-image-mode | A | 2026-08-31 20:04 | PASS | conversation 2026-07-05 (basic Docker deployment — prebuilt-image mode) | +| oh-shipped-repo-overridable | A | 2026-08-31 20:04 | PASS | issue #531 follow-on (de-hardcode residual — shipped .oh shell scripts keep the upstream repo overridable) | +| oh-standalone-lifecycle | A | 2026-08-31 20:04 | PASS | issue #564 | +| oh-update | A | 2026-08-31 20:04 | PASS | issue #531 Phase 3 (oh update — upgrade only the .oh control plane) | +| operator-config-guard | A | 2026-08-31 20:04 | PASS | operator directives 2026-08-06 (.config/ and settings.local.json are operator-only) | +| pnpm-audit-ci-gate | A | 2026-08-31 20:04 | PASS | issue #171 — pnpm security audits must run in CI | +| post-bridge-publish-confirmation | A | 2026-08-31 20:04 | PASS | #523 — post-bridge live publishing requires an explicit final confirmation gate | +| prd-output-path-contract | A | 2026-08-31 20:04 | PASS | retro lesson 2026-06-19 | +| prompt-miner-schema-compat | A | 2026-08-31 20:04 | PASS | issue #253 — prompt-miner JSONL schema-drift guard | +| prompt-miner-symlink-entrypoint | A | 2026-08-31 20:04 | PASS | issue #663 — prompt-miner engine no-ops via the documented .claude/skills symlink | +| prompt-miner-weakness-record | A | 2026-08-31 20:04 | PASS | issue #580 — prompt-miner weakness-record (WH-xxx) cluster output | +| protected-path-deletion | A | 2026-08-31 20:04 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-001) — the critique gate was deleted, | +| protected-paths-resolve | A | 2026-08-31 20:04 | PASS | issue #753 — .claude/protected-paths.txt named 7 paths that did not exist. | +| registry-portability-gate | A | 2026-08-31 20:04 | PASS | issue #758 | +| registry-portability | A | 2026-08-31 20:04 | SKIPPED | issue #758 | +| retro-deterministic-contract | A | 2026-08-31 20:04 | PASS | issue #443 — /retro deterministic output and self-contained helper contract | +| rl-delegation-write-worker | A | 2026-08-31 20:04 | PASS | retro lesson 2026-06-10 (rl-delegation) #57 | +| rlm-context-budget | A | 2026-08-31 20:04 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-006 | +| runtime-preflight-gate | A | 2026-08-31 20:04 | PASS | issue #806 § B1 (open sandbox.substrate vs sandbox.runtime selector); | +| sandbox-boot-guard-ci | A | 2026-08-31 20:04 | PASS | issue #449 (sandbox image build CI guard) 2026-06-19; | +| sandbox-node-base | A | 2026-08-31 20:04 | PASS | openharness#878 — oh as the only front door, T0 sandbox base image | +| skill-paths | A | 2026-08-31 20:04 | PASS | issue #43 — stale path references; extended by issue #69 — apps/->packages/ rename guard; extended by issue #870 — deleted .oh/agents/advisor.md | +| skills-dir-clean | A | 2026-08-31 20:04 | PASS | conversation 2026-06-29 — Pi parses every top-level `.md` in the skills | +| skills-task-tool-coupling | A | 2026-08-31 20:04 | PASS | council review 2026-08-29 (issue #886) — /delegate instructed Claude-Code-only | +| skills-vendored | A | 2026-08-31 20:04 | PASS | absorb .mifune submodule into .oh — the skills/agents/hooks pack is vendored | +| slack-admin-command-surface | A | 2026-08-31 20:04 | PASS | issue #354 — Slack bridge docs must distinguish Pi /msg-bridge commands from Slack DM admin text handlers | +| spec-family-contract | A | 2026-08-31 20:04 | PASS | issue #265; spec-simplification issue #816; workflow authority issue #854 | +| spec-ready-finalization | A | 2026-08-31 20:04 | PASS | issue #134; spec-simplification issue #816; workflow authority issue #854 | +| ste-checker-contract | A | 2026-08-31 20:04 | PASS | issue #750 PR audit — the /ste checker had four fail-open paths (unclosed | +| submitted-by-trailers | A | 2026-08-31 20:04 | PASS | conversation 2026-06-12 (commit attribution trailers); the single-owner | +| sync-skill-contract | A | 2026-08-31 20:04 | PASS | issue #331 — /sync dispatcher skill (bidirectional origin↔upstream sync) | +| tool-catalog-boundary | A | 2026-08-31 20:04 | PASS | agent-browser's exclusion from the harness catalog (#821) and the | +| version-parity | A | 2026-08-31 20:04 | PASS | conversation 2026-08-29 — the oh CLI became the only lifecycle door, so its | +| weigh-scorer-contract | A | 2026-08-31 20:04 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-003 (2026-06-27) | +| wiki-readme-index | A | 2026-08-31 20:04 | PASS | issue #132 — wiki README index drift guard | +| workflow-boundaries | A | 2026-08-31 20:04 | PASS | conversation 2026-06-19 (workflow consolidation, issue #259); authority moved to /spec in issue #854 | +| worktrees-layout | A | 2026-08-31 20:04 | PASS | issue #872 | diff --git a/.oh/evals/probes/oh-home-mount.sh b/.oh/evals/probes/oh-home-mount.sh index a529c2e0..25682d0d 100755 --- a/.oh/evals/probes/oh-home-mount.sh +++ b/.oh/evals/probes/oh-home-mount.sh @@ -17,7 +17,7 @@ done fails=() RETIRED_VOLUMES=(claude-auth codex-auth pi-auth opencode-auth grok-auth - deepagents-auth herdr-data cloudflared-auth ssh-config config-dir + herdr-data cloudflared-auth ssh-config config-dir cc-safety-net oh_workspace) for compose in "$COMPOSE_PRIMARY" "$COMPOSE_IO"; do diff --git a/.oh/install/banner.sh b/.oh/install/banner.sh index 544f267f..fdf45196 100755 --- a/.oh/install/banner.sh +++ b/.oh/install/banner.sh @@ -107,18 +107,6 @@ if command -v grok >/dev/null 2>&1; then fi fi -deepagents_status="$status_x" -deepagents_detail="not installed — run: oh harness install deepagents" -if command -v deepagents >/dev/null 2>&1; then - if [ -s "${HOME}/.deepagents/.env" ] || [ -s "${HOME}/.deepagents/config.toml" ]; then - deepagents_status="$status_ok" - deepagents_detail="configured" - else - deepagents_status="$status_ok" - deepagents_detail="installed — configure ~/.deepagents/.env or run: deepagents" - fi -fi - hermes_status="$status_x" hermes_detail="not installed — run: oh harness install hermes" if command -v hermes >/dev/null 2>&1; then @@ -171,7 +159,6 @@ printf ' %-6s %-11s %s\n' "$codex_status" "codex" "$codex_detail" printf ' %-6s %-11s %s\n' "$opencode_status" "opencode" "$opencode_detail" printf ' %-6s %-11s %s\n' "$grok_status" "grok" "$grok_detail" printf ' %-6s %-11s %s\n' "$pi_status" "pi" "$pi_detail" -printf ' %-6s %-11s %s\n' "$deepagents_status" "deepagents" "$deepagents_detail" printf ' %-6s %-11s %s\n' "$hermes_status" "hermes" "$hermes_detail" [ -n "$dashboard_status" ] && printf ' %-6s %-11s %s\n' "$dashboard_status" "dashboard" "$dashboard_detail" printf ' %-6s %-11s %s\n' "$oh_status" "oh" "$oh_detail" @@ -179,7 +166,6 @@ printf '\n' shortcuts="claude · codex · pi" command -v opencode >/dev/null 2>&1 && shortcuts="$shortcuts · opencode" command -v grok >/dev/null 2>&1 && shortcuts="$shortcuts · grok" -command -v deepagents >/dev/null 2>&1 && shortcuts="$shortcuts · deepagents" command -v hermes >/dev/null 2>&1 && shortcuts="$shortcuts · hermes" printf ' Recovery commands: %s · tmux attach -t cron-system\n' "$shortcuts" printf '\n' diff --git a/.oh/scripts/install.sh b/.oh/scripts/install.sh index b8cf7050..765835fd 100644 --- a/.oh/scripts/install.sh +++ b/.oh/scripts/install.sh @@ -118,7 +118,7 @@ Env vars: same name is already running (default: refuse, so a live sandbox is never overwritten) INSTALL_HERMES=true Enable an optional agent non-interactively. Also: - INSTALL_OPENCODE, INSTALL_DEEPAGENTS, INSTALL_GROK_BUILD, + INSTALL_OPENCODE, INSTALL_GROK_BUILD, INSTALL_AGENT_BROWSER DOCKER_SOCKET=true Mount the host Docker socket into the sandbox non-interactively. OFF by default (socket access is @@ -438,7 +438,6 @@ _opt_install() { } _opt_install HERMES install.hermes "Hermes — Nous self-improving agent CLI" _opt_install OPENCODE install.opencode "OpenCode — OpenAI-OAuth terminal agent" -_opt_install DEEPAGENTS install.deepagents "DeepAgents — LangChain multi-provider agent" _opt_install GROK_BUILD install.grokBuild "Grok Build — xAI terminal agent" _opt_install AGENT_BROWSER install.agentBrowser "agent-browser + Chromium (~1 GB)" @@ -492,7 +491,6 @@ printf " ${CYAN}Optional capabilities${NC} (installed live — no rebuild)\n" printf " ──────────────────────────────────────\n" printf " oh harness install hermes — Hermes agent (then 'hermes setup'; optional dashboard)\n" printf " oh harness install opencode — OpenCode terminal agent\n" -printf " oh harness install deepagents — LangChain DeepAgents\n" printf " oh harness install grok-build — xAI Grok Build\n" printf " oh tool install agent-browser — headless Chromium for screenshots / previews (~1 GB)\n" printf " (each flips the matching install.* flag in oh.json)\n" diff --git a/.oh/scripts/migrate-harness-yaml.sh b/.oh/scripts/migrate-harness-yaml.sh index ba30a3c2..c45b073b 100755 --- a/.oh/scripts/migrate-harness-yaml.sh +++ b/.oh/scripts/migrate-harness-yaml.sh @@ -26,7 +26,6 @@ BEGIN { envmap["git.user_email"] = "GIT_USER_EMAIL" envmap["install.opencode"] = "INSTALL_OPENCODE" envmap["install.grok_build"] = "INSTALL_GROK_BUILD" - envmap["install.deepagents"] = "INSTALL_DEEPAGENTS" envmap["install.hermes"] = "INSTALL_HERMES" envmap["install.agent_browser"] = "INSTALL_AGENT_BROWSER" envmap["hermes.dashboard"] = "HERMES_DASHBOARD" @@ -140,7 +139,6 @@ _field_for() { GIT_USER_EMAIL) printf 'git.userEmail string\n' ;; INSTALL_OPENCODE) printf 'install.opencode boolean\n' ;; INSTALL_GROK_BUILD) printf 'install.grokBuild boolean\n' ;; - INSTALL_DEEPAGENTS) printf 'install.deepagents boolean\n' ;; INSTALL_HERMES) printf 'install.hermes boolean\n' ;; INSTALL_AGENT_BROWSER) printf 'install.agentBrowser boolean\n' ;; HERMES_DASHBOARD) printf 'hermesDashboard.enabled boolean\n' ;; diff --git a/CHANGELOG.md b/CHANGELOG.md index b827f1de..f79e816f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,12 +13,13 @@ Update policy and release automation live in [`/git`](.claude/skills/git/SKILL.m - Shrink the sandbox image ~540 MB: drop build caches from the baked home seed, stage the seed once via a builder stage, and keep untracked build output out of the build context ([#900](https://github.com/mifunedev/openharness/issues/900)). - **BREAKING:** Stop baking Claude Code, Codex, and Pi into the image; boot installs them into the home mount, so a first boot needs network and runs 60-180s longer ([#904](https://github.com/mifunedev/openharness/issues/904)). - **BREAKING:** Stop baking Herdr and cloudflared into the image; both become `kind: "default"` tools installed into `~/.local/bin` at boot from a pinned, checksum-verified binary ([#906](https://github.com/mifunedev/openharness/issues/906)). -- **BREAKING:** Stop baking OpenCode, DeepAgents, Hermes, and Grok Build into the image; `oh harness install ` installs them into `~/.local` as the sandbox user ([#908](https://github.com/mifunedev/openharness/issues/908)). +- **BREAKING:** Stop baking OpenCode, Hermes, and Grok Build into the image; `oh harness install ` installs them into `~/.local` as the sandbox user ([#908](https://github.com/mifunedev/openharness/issues/908)). ### Removed - Remove the `BAKE_HARNESSES` and `AGENTS` build args along with the image bake they gated; the harness catalog is the only source of truth for what gets installed ([#904](https://github.com/mifunedev/openharness/issues/904)). - Remove Cloudflare's apt repository and its bookworm-suite pin from the image; Docker's is now the only third-party apt source ([#906](https://github.com/mifunedev/openharness/issues/906)). - Remove the four optional-harness build args and the dead `buildArg` catalog field; the `install.*` keys keep working and now drive boot provisioning ([#908](https://github.com/mifunedev/openharness/issues/908)). +- **BREAKING:** Retire the DeepAgents harness — `deepagents-cli` is deprecated upstream. `install.deepagents` is no longer a settable oh.json field ([#910](https://github.com/mifunedev/openharness/issues/910)). - **BREAKING:** Retire the `projectRoot` / `OH_PROJECT_ROOT` config knob — the checkout is fixed at `/home/sandbox/harness`, nested inside the home mount ([#898](https://github.com/mifunedev/openharness/issues/898)). ### Added diff --git a/README.md b/README.md index 6f9dc420..79bac09b 100644 --- a/README.md +++ b/README.md @@ -174,7 +174,6 @@ herdr # first command: open the primary interactive workspace # codex # OpenAI Codex CLI # pi # Pi Coding Agent # opencode # OpenCode (optional: oh harness install opencode) -# deepagents # LangChain DeepAgents (optional: oh harness install deepagents) # hermes # Nous Research Hermes (optional: oh harness install hermes) # grok # xAI Grok Build (optional: oh harness install grok-build) oh stop # stop the sandbox, keeping volumes @@ -216,7 +215,7 @@ the image-mode recipe. | | | |---|---| -| **Core agents** | Defaults: Claude Code, Codex, Pi. Optional: OpenCode, DeepAgents, Hermes, Grok Build | +| **Core agents** | Defaults: Claude Code, Codex, Pi. Optional: OpenCode, Hermes, Grok Build | | **Runtimes** | Node 22, pnpm, Bun, uv (Python) | | **DevOps** | Herdr, Docker CLI + Compose, GitHub CLI, cloudflared, tmux, croner | | **Browser** | agent-browser + Chromium (headless) | diff --git a/docs/README.md b/docs/README.md index 07f48165..29877e29 100644 --- a/docs/README.md +++ b/docs/README.md @@ -46,7 +46,6 @@ Open Harness vendors the shared skills/agents/hooks primitive pack directly into - [Codex](harnesses/codex.md) - [Pi](harnesses/pi.md) - [OpenCode](harnesses/opencode.md) -- [DeepAgents](harnesses/deepagents.md) - [Hermes](harnesses/hermes.md) - [Grok Build](harnesses/grok-build.md) - [T3 Code](harnesses/t3code.md) diff --git a/docs/configuration.md b/docs/configuration.md index 25babe7e..a4def2e1 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -56,14 +56,13 @@ to; `—` means the field is consumed by the `oh` CLI itself and never rendered. All off by default. `oh harness install ` flips the matching field and installs into the running sandbox with no rebuild. The four harness fields map -to `oh harness` names: `opencode`, `grok-build`, `deepagents`, `hermes`. +to `oh harness` names: `opencode`, `grok-build`, `hermes`. `agentBrowser` is not a harness — `oh tool install agent-browser` manages it. | Field | Type | Default | Compose variable | What it does | | --- | --- | --- | --- | --- | | `install.opencode` | boolean | `false` | `INSTALL_OPENCODE` | Install the OpenCode CLI into `~/.local` at boot. `oh harness install opencode` sets it and installs now. | | `install.grokBuild` | boolean | `false` | `INSTALL_GROK_BUILD` | Install the Grok Build CLI into `~/.local` at boot. `oh harness install grok-build` sets it and installs now. | -| `install.deepagents` | boolean | `false` | `INSTALL_DEEPAGENTS` | Install the DeepAgents CLI into `~/.local` at boot. `oh harness install deepagents` sets it and installs now. | | `install.hermes` | boolean | `false` | `INSTALL_HERMES` | Install the Hermes CLI into `~/.local` at boot and enable its runtime wiring (skill vendoring, `auth.json`). | | `install.agentBrowser` | boolean | `false` | `INSTALL_AGENT_BROWSER` | Install agent-browser and Chromium (about 1 GB). | diff --git a/docs/harnesses/deepagents.md b/docs/harnesses/deepagents.md deleted file mode 100644 index cb924d64..00000000 --- a/docs/harnesses/deepagents.md +++ /dev/null @@ -1,187 +0,0 @@ ---- -title: "DeepAgents" ---- - -# DeepAgents - -DeepAgents is LangChain's terminal coding agent. It runs interactive -conversational sessions and one-shot non-interactive tasks against multiple -LLM providers, with optional shell tool use gated by an explicit allow list. - -DeepAgents is an **optional supported runtime** in Open Harness. The default -agent set is Claude Code, Codex, and Pi; enable DeepAgents when its provider -mix or non-interactive shell-allow-list model is the right fit for a task. - -## Purpose - -- Multi-provider agent (Anthropic, OpenAI, etc.) configurable from a single - `~/.deepagents/.env` file. -- Non-interactive mode (`-n "$task"`) with explicit shell-allow-list gating, - suitable for bounded `/spec execute` worker tasks with a constrained default tool surface. -- Project-aware: optionally reads memory and skills from a repo-local - `.deepagents/` directory at the workspace root. - -## Install (optional) - -The shortest path is the CLI, which sets the `.devcontainer/.env` flag **and** -installs into the already-running sandbox without a rebuild: - -```bash -oh harness install deepagents -``` - -See [Harnesses Overview](./overview.md#installing-a-harness) for `--persist-only`, -`--no-persist`, and what happens when the sandbox is not running. - -### Manual path - -Enable DeepAgents in `.devcontainer/.env`: - -```yaml -install: - deepagents: true -``` - -Or set `INSTALL_DEEPAGENTS=true` in `.devcontainer/.env` (legacy). - -Either way the boot provisioner installs it on the next start — no rebuild: - -```bash -oh stop && oh sandbox -``` - -Open Harness installs the upstream CLI with `uv tool install` as the `sandbox` -user, which lands it in `~/.local/bin` inside the home mount: - -```bash -uv tool install deepagents-cli -``` - -The `deepagents` shim lands on `/usr/local/bin` so it is on PATH for the -`sandbox` user when the flag is enabled. - -Verify the install inside the sandbox: - -```bash -deepagents -v -``` - -If the command is not found, run `oh harness install deepagents` — it installs -into `~/.local/bin` in the running sandbox and sets `install.deepagents` so a -fresh home mount reinstalls it at boot. No rebuild is involved. - -## Authentication and provider keys - -DeepAgents reads provider API keys from `~/.deepagents/.env` and CLI -defaults from `~/.deepagents/config.toml`. The directory is persisted by -the single `/home/sandbox` mount by default, so credentials survive -container rebuilds. - -Create the env file on first use: - -```bash -mkdir -p ~/.deepagents -cat > ~/.deepagents/.env <<'EOF' -ANTHROPIC_API_KEY=sk-ant-... -OPENAI_API_KEY=sk-... -EOF -chmod 600 ~/.deepagents/.env -``` - -The sandbox banner reports DeepAgents as **installed** when `deepagents` is -on PATH and **configured** only when `~/.deepagents/.env` or -`~/.deepagents/config.toml` is non-empty — an empty mounted directory is -not treated as authenticated. - -## State persistence and repo-local `.deepagents/` - -Two separate directories carry DeepAgents state, with very different -durability and review semantics: - -| Path | Scope | Persistence | Notes | -|---|---|---|---| -| `~/.deepagents/` | Per-sandbox user state | Survives rebuilds via the `/home/sandbox` mount | Provider keys, model defaults, memory, skills, sessions live here. **Only place secrets here.** | -| `/.deepagents/` | Per-project | Whatever git decides | Project memory and skills the agent may load from the workspace root. **Treat as project data — follows normal `.gitignore` and code-review rules.** | - -A repo-local `.deepagents/` directory may be read by the agent and may be -committed to the repository — keep secrets and provider keys **only** in -`~/.deepagents/` or in ignored local files. Never commit a `.env` to the -repo-local `.deepagents/`. - -v1 of Open Harness persists only `/home/sandbox/.deepagents`; the -repo-local `.deepagents/` is project data subject to your repository's -ordinary git rules. - -## Common usage - -### Interactive - -Launch a conversational session: - -```bash -deepagents -``` - -Run inside a dedicated tmux session so the conversation survives shell -disconnects (per `.claude/rules/sandbox-processes.md`): - -```bash -tmux new-session -d -s agent-deepagents 'deepagents' -tmux attach -t agent-deepagents -``` - -### Non-interactive - -Pass a task with `-n` for a single execution. By default DeepAgents -**disables shell execution in non-interactive mode** unless an allow list -is configured via `-S`/`--shell-allow-list` or -`DEEPAGENTS_CLI_SHELL_ALLOW_LIST`. Open Harness defaults pick the -`recommended` allow list, never `all`: - -```bash -deepagents -y --shell-allow-list recommended -n "Summarize the changes on this branch" -q --no-stream -``` - -Flags: - -- `-y` — assume "yes" to confirmation prompts (the sandbox is the trust - boundary). -- `--shell-allow-list recommended` — allow only the curated safe shell - command set. **Do not default to `all`** — see the warning below. -- `-n "$task"` — non-interactive single task. -- `-q --no-stream` — quiet, buffered output for clean log capture. - -### `/spec execute` usage - -`/spec execute` owns implementation in one Advisor session. If DeepAgents is the chosen -provider for a bounded worker, run it directly with a task prompt; do not add a second -workflow or a provider-specific executor wrapper: - -```bash -deepagents -y --shell-allow-list recommended -q --no-stream --max-turns 25 -n "$task" -``` - -DeepAgents is never auto-selected. Keep its shell allow-list explicit and let the Advisor -validate the worker's result against the story acceptance criteria. - -> **`--shell-allow-list all` warning.** Choosing `--shell-allow-list all` -> grants unrestricted non-interactive shell -> execution. Combined with the mounted Docker socket (enabled by default -> in the base compose file), this can affect sibling containers or the -> host Docker daemon. Only use `all` for trusted tasks where -> you have accepted that risk explicitly. - -## Tips - -- Keep provider keys in `~/.deepagents/.env`. Never commit a repo-local - `.deepagents/.env`. -- Pair DeepAgents with a git worktree so its branch is isolated. -- Inspect non-interactive runs with `tmux attach -t agent-deepagents` (or - the Advisor-owned task session) to see live progress. - -## Upstream documentation - -- [DeepAgents documentation](https://docs.langchain.com/oss/python/deepagents/overview) -- [DeepAgents CLI overview](https://docs.langchain.com/oss/python/deepagents/cli/overview) -- [DeepAgents CLI configuration](https://docs.langchain.com/oss/python/deepagents/cli/configuration) -- [`langchain-ai/deepagents` on GitHub](https://github.com/langchain-ai/deepagents/tree/main/libs/cli) diff --git a/docs/harnesses/hermes.md b/docs/harnesses/hermes.md index f217257c..05940088 100644 --- a/docs/harnesses/hermes.md +++ b/docs/harnesses/hermes.md @@ -11,7 +11,7 @@ container sandboxing across multiple backends, and bridges to chat platforms (Telegram, Discord, Slack, WhatsApp, Signal, Email). Hermes is an **optional harness** in Open Harness. Install it with `oh harness install hermes` (or set `install.hermes` / `INSTALL_HERMES=true`, which the boot provisioner honours); it then sits alongside `claude`, `codex`, -`pi`, `opencode`, and `deepagents` as a sandbox CLI primitive. See the +`pi`, and `opencode` as a sandbox CLI primitive. See the upstream documentation below for canonical facts about Hermes. ## Purpose diff --git a/docs/harnesses/overview.md b/docs/harnesses/overview.md index a5849e68..16b7bc15 100644 --- a/docs/harnesses/overview.md +++ b/docs/harnesses/overview.md @@ -4,7 +4,7 @@ title: "Harnesses Overview" # Harnesses Overview -Open Harness provisions three agent CLIs into `~/.local` on first boot: **Claude Code** (default), **Codex**, and **Pi**. **OpenCode**, **DeepAgents**, **Hermes**, and **Grok Build** are optional — install one with `oh harness install `, which also sets its `install.*` key so a fresh home mount reinstalls it at boot. No harness is baked into the image. **T3 Code** runs on demand via the `/t3` skill (or directly with `npx t3`) as a browser UI on port 3773, and **Prime Agent** installs on demand with `oh harness install prime-agent` — neither has an `INSTALL_*` key, because neither is ever baked into the image. Inside the sandbox, run `herdr` first, then launch whichever agent you prefer from its panes and switch between them at any time. Reserve tmux for Open Harness's managed/headless cron, gateway, and watchdog infrastructure. +Open Harness provisions three agent CLIs into `~/.local` on first boot: **Claude Code** (default), **Codex**, and **Pi**. **OpenCode**, **Hermes**, and **Grok Build** are optional — install one with `oh harness install `, which also sets its `install.*` key so a fresh home mount reinstalls it at boot. No harness is baked into the image. **T3 Code** runs on demand via the `/t3` skill (or directly with `npx t3`) as a browser UI on port 3773, and **Prime Agent** installs on demand with `oh harness install prime-agent` — neither has an `INSTALL_*` key, because neither is ever baked into the image. Inside the sandbox, run `herdr` first, then launch whichever agent you prefer from its panes and switch between them at any time. Reserve tmux for Open Harness's managed/headless cron, gateway, and watchdog infrastructure. Open Harness is the harness; the **agent** is your call. To go beyond the preinstalled options, install via `npm` / `pip` / `cargo` inside the sandbox or edit the Dockerfile. For Pi+Slack specifically, the recommended path is the `pi-messenger-bridge` npm package — see [Slack integration](../integrations/slack.md). The product surface is one developer, one project, one agent — not racing or stacking multiple CLIs against each other. @@ -51,7 +51,6 @@ The manual path still works: uncomment the key in `.devcontainer/.env` (or expor | [Codex](./codex.md) | OpenAI's CLI coding agent | `codex` | preinstalled | | [OpenCode](./opencode.md) | Terminal coding agent with OpenAI OAuth support | `opencode` | optional: `oh harness install opencode` | | [Pi](./pi.md) | Lightweight, customizable agent | `pi` | default | -| [DeepAgents](./deepagents.md) | LangChain's multi-provider terminal agent | `deepagents` | optional: `oh harness install deepagents` | | [Hermes](./hermes.md) | Nous Research's self-improving terminal agent | `hermes` | optional: `oh harness install hermes` | | [Grok Build](./grok-build.md) | xAI's proprietary Grok Build terminal agent | `grok` | optional: `oh harness install grok-build` | | [T3 Code](./t3code.md) | Browser UI over Claude/Codex/OpenCode (port 3773) | `/t3` or `npx t3` | on-demand | @@ -66,7 +65,6 @@ pi --version # Optional CLIs, present only after `oh harness install ` (or its install.* key): opencode --version # install.opencode: true -deepagents -v # install.deepagents: true hermes --version # install.hermes: true grok --version # install.grok_build: true @@ -82,7 +80,6 @@ Open Harness provisions Claude Code, Codex, and Pi into `~/.local` on first boot - **Codex**: run `codex login` (see [Codex](./codex.md)). - **OpenCode**: run `opencode auth login` (see [OpenCode](./opencode.md)). - **Pi**: configure provider keys via environment variables (see [Pi](./pi.md)). -- **DeepAgents**: write provider keys to `~/.deepagents/.env` (see [DeepAgents](./deepagents.md)). - **Hermes**: run `hermes setup` (see [Hermes](./hermes.md)). - **Grok Build**: run `grok login --device-auth` for headless/remote auth, `grok login` for interactive OAuth, or set `XAI_API_KEY` as a fallback (see [Grok Build](./grok-build.md)). Cached `~/.grok/auth.json` takes precedence over `XAI_API_KEY`. - **Prime Agent**: run `prime-agent`, then `/login` — OAuth for ChatGPT Plus/Pro (Codex), Claude Pro/Max, or GitHub Copilot; API keys via environment variables or the same `/login` flow. Credentials land in `~/.prime/agent/auth.json` (see [Prime Agent](./prime-agent.md)). diff --git a/docs/installation.md b/docs/installation.md index c9b45a9f..9de2ea32 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -311,7 +311,6 @@ Optional CLIs are excluded from the default image; `oh harness install ` f | OpenAI Codex | `codex` | OpenAI's coding agent (aliased to `codex --dangerously-bypass-approvals-and-sandbox`) | default | | Pi | `pi` | `@earendil-works/pi-coding-agent` — local-first coding agent (was `@mariozechner/pi-coding-agent`, now deprecated) | default | | OpenCode | `opencode` | `opencode-ai` — terminal coding agent with OpenAI OAuth support | optional: `oh harness install opencode` | -| DeepAgents | `deepagents` | LangChain's multi-provider terminal agent (`deepagents-cli` via `uv tool install`) | optional: `oh harness install deepagents` | | Hermes | `hermes` | Nous Research's self-improving agent CLI | optional: `oh harness install hermes` | | Grok Build | `grok` | xAI's proprietary Grok Build CLI (`@xai-official/grok@0.2.39`, Node >=20) | optional: `oh harness install grok-build` | | agent-browser | `agent-browser` | Headless Chromium for web-capable agents | optional: `oh tool install agent-browser` | diff --git a/docs/quickstart.md b/docs/quickstart.md index f7d4c20c..144cbf31 100644 --- a/docs/quickstart.md +++ b/docs/quickstart.md @@ -125,7 +125,7 @@ they live in the home mount, not the image, so `oh harness install ` upgrade them in place without a rebuild. A first boot on a fresh home mount therefore needs network access and takes a minute or two longer; the sandbox still comes up as a usable shell if the registry is unreachable, and you can retry with -`bash .oh/scripts/provision-defaults.sh`. OpenCode, DeepAgents, Hermes, and Grok +`bash .oh/scripts/provision-defaults.sh`. OpenCode, Hermes, and Grok Build are optional installs via `oh harness install `; T3 Code runs on demand via the `/t3` skill or direct `npx`. Authenticate at least one harness before use. @@ -142,7 +142,6 @@ or direct `npx`. Authenticate at least one harness before use. - **[Codex](./harnesses/codex.md)**: `codex login --device-auth` (device mode; or `/login` in-session) - **[OpenCode](./harnesses/opencode.md)**: `oh harness install opencode`, then run `opencode auth login` - **[Pi](./harnesses/pi.md)**: configure provider keys via environment variables -- **[DeepAgents](./harnesses/deepagents.md)**: `oh harness install deepagents`, then write provider keys to `~/.deepagents/.env` - **[Hermes](./harnesses/hermes.md)**: `oh harness install hermes`, then run `hermes setup` - **[Grok Build](./harnesses/grok-build.md)**: `oh harness install grok-build`, verify `grok --version`, then run `grok login --device-auth` (headless/remote) or `grok login` - **[T3 Code](./harnesses/t3code.md)**: authenticate one of Claude / Codex / OpenCode, then `/t3` or `npx t3` (browser UI on port 3773) @@ -189,7 +188,6 @@ lifecycle command.) "git": { "userName": "your-name", "userEmail": "you@example.com" }, "install": { "opencode": false, - "deepagents": false, "hermes": false, "grokBuild": false, "agentBrowser": false @@ -221,7 +219,6 @@ full field reference, and `oh config set ` to edit one field. | `git.userEmail` | Commit author email | | `install.agentBrowser` | Set `true` to install Chromium (~1 GB) | | `install.opencode` | Set `true` to include OpenCode in the sandbox image | -| `install.deepagents` | Set `true` to include DeepAgents in the sandbox image | | `install.hermes` | Set `true` to include Hermes in the sandbox image; state defaults to `~/harness/.hermes`, auth lives in `~/.hermes` | | `install.grokBuild` | Set `true` to include Grok Build in the sandbox image; all Grok user state lives in the persisted `~/.grok` volume | diff --git a/oh.json b/oh.json index c17aad65..91f571b9 100644 --- a/oh.json +++ b/oh.json @@ -7,7 +7,6 @@ "install": { "opencode": false, "grokBuild": false, - "deepagents": false, "hermes": false, "agentBrowser": false }, From a679a048de9e26b5617efdfcb655f96a39e2e807 Mon Sep 17 00:00:00 2001 From: Ryan Eggleston Date: Mon, 31 Aug 2026 15:04:46 -0600 Subject: [PATCH 10/23] feat(cli): support T3 Code mobile access over Tailscale (#897) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(cli): support T3 Code mobile access over Tailscale (#858) Add Tailscale as an opt-in `oh tool` and give T3 Code a headless, tailnet-private mobile access path. Tailscale runs inside the sandbox in userspace-networking mode as the unprivileged sandbox user. T3 Code stays on container loopback and `t3 serve --tailscale-serve` advertises it over tailnet HTTPS. This adds no capabilities, no /dev/net/tun, and no published port; the compose surface gains only an INSTALL_TAILSCALE passthrough and a named state volume. The entrypoint installs a pinned, sha256-verified tarball and never starts the daemon or handles an auth key. The /t3 skill gains serve, pair, url, and doctor. Headless launches use `t3 serve` under the existing tmux convention, `t3 pair` adds a device without restarting the server, and doctor fails with actionable guidance on Node range, tailscaled state, tailnet login, and port reachability. Two probes guard the result: tailscale-tool-boundary asserts the no-capability, no-port, no-Funnel, pinned-and-hash-matched boundary, and t3-headless-launch asserts the serve-only launch surface. Closes #858 * fix(cli): use the default tailscaled socket for T3 mobile (#858) The documented daemon ran on $HOME/.tailscale/tailscaled.sock while every consumer — /t3 doctor, the doc'd tailscale status/serve/up/logout, and T3 Code's own --tailscale-serve — called the CLI bare, which only ever reaches /var/run/tailscale/tailscaled.sock. There is no TS_SOCKET override, so `/t3 start --tailscale` could never pass preflight. The entrypoint and the tool catalog now create /var/run/tailscale owned by sandbox, and every documented daemon and CLI invocation drops --socket. --statedir and the tailscale-state volume are unchanged. The Node-floor assertion in t3-headless-launch.sh becomes behavioral: a stub node printing v22.15.0 must make doctor fail and name the range. /t3 start creates its tee log under umask 077 so the one-time pairing URL is not world-readable. * test(cli): guard the tailscale socket-directory step against drift (#858) The /var/run/tailscale pre-creation is duplicated between the entrypoint and the catalog installArgv but was the only install step with no parity assertion. Version, both sha256 literals, the download base, and the /usr/local/bin paths were already guarded. * fix(cli): install Tailscale as the sandbox user, not root (#858) The tailscale tool entry was installUser:"root", writing both binaries into /usr/local/bin. #908 proved that shape is unusable from inside the sandbox: commands/tool.ts execs with stdio:"inherit", so local-target.ts selects the INTERACTIVE branch — plain `sudo --`, no -n — and /etc/sudoers.d/sandbox grants `sandbox ALL=(ALL) ALL` with no NOPASSWD. `oh tool install tailscale` therefore hung on a password prompt no agent can answer. t3-code.sh's doctor already printed that exact command as its remedy. Nothing here needed root. Verified by running it: with --tun=userspace-networking, tailscaled starts as the sandbox user, creates its socket, and answers `tailscale status`. The only "operation not permitted" lines are UDP buffer sizing, which Tailscale itself labels throughput-only warnings. Both install paths — the catalog entry and the entrypoint's boot-time install — now install into $NPM_USER_PREFIX/bin as the sandbox user. That also fixes a second defect: /usr/local/bin is an image-layer path, so the old location was discarded on every container recreate and re-downloaded 33 MB + 42 MB each time, while leaving a root-owned binary no running sandbox could upgrade in place. /var/run/tailscale still needs root to create, since tailscaled defaults its control socket there and t3-code.sh calls a bare `tailscale status`. The entrypoint keeps creating it, but no longer behind the INSTALL_TAILSCALE guard: `oh tool install tailscale` promises the tool is usable in the already-running container, and gating the socket directory would make that wait for a reboot. The probe now holds all of it: no root install, no /usr/local/bin in either path, and the socket directory created outside the guard. * fix(test): restore the newline escape lost resolving the tool-catalog merge * eval: refresh the scoreboard after merging development --- .claude/protected-paths.txt | 1 + .devcontainer/docker-compose.yml | 1 + .devcontainer/entrypoint.sh | 59 +++++ .oh/cli/package-lock.json | 4 + .oh/cli/src/__tests__/env-file.test.ts | 19 ++ .oh/cli/src/__tests__/init.test.ts | 4 + .../install-flag-persistence.test.ts | 14 ++ .oh/cli/src/__tests__/tool-catalog.test.ts | 103 ++++++++- .oh/cli/src/__tests__/tool.test.ts | 103 ++++++++- .oh/cli/src/commands/init.ts | 8 + .../src/lib/__tests__/config-render.test.ts | 1 + .oh/cli/src/lib/__tests__/oh-config.test.ts | 5 + .oh/cli/src/lib/config-render.ts | 1 + .oh/cli/src/lib/env-file.ts | 2 + .oh/cli/src/lib/oh-config.ts | 11 +- .oh/cli/src/lib/tools/catalog.ts | 17 ++ .oh/evals/RESULTS.md | 208 +++++++++--------- .oh/evals/probes/t3-headless-launch.sh | 129 +++++++++++ .oh/evals/probes/tailscale-tool-boundary.sh | 154 +++++++++++++ .oh/scripts/install.sh | 4 +- .oh/scripts/migrate-harness-yaml.sh | 2 + .oh/skills/t3/SKILL.md | 112 +++++++--- .oh/skills/t3/references/sandbox-processes.md | 2 +- .oh/skills/t3/references/tailscale-mobile.md | 98 +++++++++ .oh/skills/t3/scripts/t3-code.sh | 185 ++++++++++++++-- CHANGELOG.md | 4 + docs/configuration.md | 4 +- docs/connecting.md | 187 +++++++++++++++- docs/harnesses/overview.md | 4 +- docs/harnesses/t3code.md | 95 ++++++-- docs/installation.md | 18 ++ docs/security-considerations.md | 36 +++ oh.json | 3 +- 33 files changed, 1415 insertions(+), 183 deletions(-) create mode 100755 .oh/evals/probes/t3-headless-launch.sh create mode 100755 .oh/evals/probes/tailscale-tool-boundary.sh create mode 100644 .oh/skills/t3/references/tailscale-mobile.md diff --git a/.claude/protected-paths.txt b/.claude/protected-paths.txt index 6dcb0c45..ba458893 100644 --- a/.claude/protected-paths.txt +++ b/.claude/protected-paths.txt @@ -30,6 +30,7 @@ spec eval health-check retro +t3 # --- Orchestrator scripts (.oh/scripts/) --- .oh/scripts/cron-runtime.ts .oh/scripts/sandbox-healthcheck.sh diff --git a/.devcontainer/docker-compose.yml b/.devcontainer/docker-compose.yml index 796f73b3..72ddf66a 100644 --- a/.devcontainer/docker-compose.yml +++ b/.devcontainer/docker-compose.yml @@ -60,6 +60,7 @@ services: - GH_TOKEN=${GH_TOKEN:-} - XAI_API_KEY=${XAI_API_KEY:-} - INSTALL_AGENT_BROWSER=${INSTALL_AGENT_BROWSER:-false} + - INSTALL_TAILSCALE=${INSTALL_TAILSCALE:-false} - SKIP_PNPM_INSTALL=${SKIP_PNPM_INSTALL:-0} - INSTALL_HERMES=${INSTALL_HERMES:-false} - HERMES_HOME=/home/sandbox/harness/.hermes diff --git a/.devcontainer/entrypoint.sh b/.devcontainer/entrypoint.sh index f1d13501..5b0346fa 100644 --- a/.devcontainer/entrypoint.sh +++ b/.devcontainer/entrypoint.sh @@ -550,6 +550,65 @@ if [ "${INSTALL_AGENT_BROWSER:-false}" = "true" ] && ! command -v agent-browser || echo "[entrypoint] agent-browser install failed — skipping" fi +# tailscaled defaults its control socket to /var/run/tailscale/tailscaled.sock, +# and t3-code.sh calls a bare `tailscale status` that expects exactly that path. +# Only root can create it, so the entrypoint must — unconditionally, not behind +# the guard below: `oh tool install tailscale` promises the tool is usable in the +# already-running container, and gating this on INSTALL_TAILSCALE would make an +# install-now/use-now flow wait for a reboot. An empty directory costs nothing. +install -d -o sandbox -g sandbox -m 0755 /var/run/tailscale 2>/dev/null || true + +if [ "${INSTALL_TAILSCALE:-false}" = "true" ]; then + install -d -o sandbox -g sandbox -m 0700 /home/sandbox/.tailscale 2>/dev/null || true + + if ! gosu sandbox bash -lc 'command -v tailscale' >/dev/null 2>&1; then + case "$(dpkg --print-architecture)" in + amd64) + ts_tarball=tailscale_1.102.3_amd64.tgz + ts_sha=36ddd9b51be57ffc2990cf76323cfa13643bfbb1b8a969f6183fa164741cdef5 + ;; + arm64) + ts_tarball=tailscale_1.102.3_arm64.tgz + ts_sha=a0fa1b154af8c61f862a2259f559f7396d96c0225f4a863eae2333e1546bbe25 + ;; + *) + ts_tarball="" + ts_sha="" + ;; + esac + + if [ -z "$ts_tarball" ]; then + echo "[entrypoint] WARNING: no pinned Tailscale build for $(dpkg --print-architecture) — skipping" >&2 + else + echo "[entrypoint] Installing ${ts_tarball%.tgz} (INSTALL_TAILSCALE=true)..." + # Install into the home mount as the sandbox user, matching the tool + # catalog. /usr/local/bin is an image-layer path: it is lost on every + # container recreate, so the old location re-downloaded Tailscale on every + # fresh container, and left a root-owned binary no running sandbox could + # upgrade in place. + ts_tmp="$(mktemp -d)" + chown sandbox:sandbox "$ts_tmp" + if gosu sandbox bash -lc " + set -e + prefix=\"\${NPM_USER_PREFIX:-\$HOME/.local}\" + curl -fsSL 'https://pkgs.tailscale.com/stable/${ts_tarball}' -o '$ts_tmp/$ts_tarball' + echo '${ts_sha} $ts_tmp/$ts_tarball' | sha256sum -c - + tar -xzf '$ts_tmp/$ts_tarball' -C '$ts_tmp' + install -d \"\$prefix/bin\" + install -m 0755 '$ts_tmp/${ts_tarball%.tgz}/tailscale' \"\$prefix/bin/tailscale\" + install -m 0755 '$ts_tmp/${ts_tarball%.tgz}/tailscaled' \"\$prefix/bin/tailscaled\" + "; then + echo "[entrypoint] ${ts_tarball%.tgz} installed into the home mount" + else + echo "[entrypoint] WARNING: Tailscale install failed — skipping" >&2 + fi + rm -rf "$ts_tmp" + unset ts_tmp + fi + unset ts_tarball ts_sha + fi +fi + for hook in /usr/local/bin/*-entrypoint-hook.sh; do [ -x "$hook" ] && "$hook" done diff --git a/.oh/cli/package-lock.json b/.oh/cli/package-lock.json index 7d96d6b9..7783093d 100644 --- a/.oh/cli/package-lock.json +++ b/.oh/cli/package-lock.json @@ -7,6 +7,7 @@ "": { "name": "@mifune/openharness", "version": "0.5.1", + "license": "Apache-2.0", "bin": { "oh": "dist/oh.js" }, @@ -14,6 +15,9 @@ "@types/node": "^22.0.0", "esbuild": "^0.28.1", "typescript": "^5.6.0" + }, + "engines": { + "node": ">=20" } }, "node_modules/@esbuild/aix-ppc64": { diff --git a/.oh/cli/src/__tests__/env-file.test.ts b/.oh/cli/src/__tests__/env-file.test.ts index 3998bc75..d90097df 100644 --- a/.oh/cli/src/__tests__/env-file.test.ts +++ b/.oh/cli/src/__tests__/env-file.test.ts @@ -104,6 +104,7 @@ describe("install flags", () => { expect(installFieldPath("opencode")).toBe("install.opencode"); expect(installFieldPath("grok_build")).toBe("install.grokBuild"); expect(installFieldPath("agent_browser")).toBe("install.agentBrowser"); + expect(installFieldPath("tailscale")).toBe("install.tailscale"); }); it("writes the flag to oh.json and never to a dotenv", () => { @@ -117,6 +118,17 @@ describe("install flags", () => { expectNoDotenv(root); }); + it("writes install.tailscale from the tailscale flag", () => { + const root = makeRepo(); + expect(isInstallFlagEnabled(root, "tailscale")).toBe(false); + + expect(setInstallFlag(root, "tailscale")).toBe("updated"); + + expect(readConfig(root)).toMatchObject({ install: { tailscale: true } }); + expect(isInstallFlagEnabled(root, "tailscale")).toBe(true); + expectNoDotenv(root); + }); + it("is idempotent — a second call rewrites nothing", () => { const root = makeRepo(); setInstallFlag(root, "opencode"); @@ -139,6 +151,13 @@ describe("setEnvValue", () => { expectNoDotenv(root); }); + it("routes INSTALL_TAILSCALE to install.tailscale", () => { + const root = makeRepo(); + expect(setEnvValue(root, "INSTALL_TAILSCALE", "true")).toBe("updated"); + expect(readConfig(root)).toMatchObject({ install: { tailscale: true } }); + expectNoDotenv(root); + }); + it("refuses a key that has no oh.json field rather than falling back to a dotenv", () => { const root = makeRepo(); expect(() => setEnvValue(root, "GH_TOKEN", "ghp_example")).toThrow(/oh secret set/); diff --git a/.oh/cli/src/__tests__/init.test.ts b/.oh/cli/src/__tests__/init.test.ts index a06ab547..05cea50b 100644 --- a/.oh/cli/src/__tests__/init.test.ts +++ b/.oh/cli/src/__tests__/init.test.ts @@ -394,6 +394,7 @@ describe("runInit", () => { if (q.includes("Git user name")) return "Ada Lovelace"; if (q.includes("Git user email")) return "ada@example.com"; if (q.includes("agent_browser")) return "y"; + if (q.includes("tailscale")) return "y"; return ""; }; const askSecret = async (q: string): Promise => @@ -416,6 +417,7 @@ describe("runInit", () => { expect(config.timezone).toBe("America/New_York"); expect(config.git).toEqual({ userName: "Ada Lovelace", userEmail: "ada@example.com" }); expect(config.install.agentBrowser).toBe(true); + expect(config.install.tailscale).toBe(true); expect(config.install.hermes).toBe(false); expect(config.access.ssh).toBe(false); expect(config.access.dockerSocket).toBe(false); @@ -432,6 +434,7 @@ describe("runInit", () => { "GIT_USER_NAME", "GIT_USER_EMAIL", "INSTALL_AGENT_BROWSER", + "INSTALL_TAILSCALE", "DOCKER_SOCKET", ]) { expect(dotenv).not.toContain(nonSecret); @@ -478,6 +481,7 @@ describe("runInit", () => { grokBuild: false, hermes: false, agentBrowser: false, + tailscale: false, }); expect(existsSync(join(t, ".env"))).toBe(false); expect(lstatSync(join(t, ".devcontainer/.env")).isSymbolicLink()).toBe(true); diff --git a/.oh/cli/src/__tests__/install-flag-persistence.test.ts b/.oh/cli/src/__tests__/install-flag-persistence.test.ts index 34d61240..5bfe1d2e 100644 --- a/.oh/cli/src/__tests__/install-flag-persistence.test.ts +++ b/.oh/cli/src/__tests__/install-flag-persistence.test.ts @@ -65,6 +65,20 @@ describe("install flags never reach the secrets dotenv", () => { expect(lstatSync(link).isSymbolicLink()).toBe(true); }); + it("`oh tool install tailscale --persist-only` writes oh.json and leaves .env byte-identical", async () => { + const root = makeRepo(); + const before = readFileSync(secretsFilePath(root), "utf8"); + const { io, out } = makeIo(); + + expect(await runToolInstall("tailscale", { cwd: root, persistOnly: true }, io)).toBe(0); + + expect(readConfig(root)).toMatchObject({ install: { tailscale: true } }); + expect(readFileSync(secretsFilePath(root), "utf8")).toBe(before); + expect(before).not.toMatch(/INSTALL_/); + expect(out.join("")).toContain("oh.json: set install.tailscale=true"); + expect(out.join("")).not.toContain(".devcontainer/.env"); + }); + it("`oh harness install --persist-only` writes oh.json and leaves .env byte-identical", async () => { const root = makeRepo(); const before = readFileSync(secretsFilePath(root), "utf8"); diff --git a/.oh/cli/src/__tests__/tool-catalog.test.ts b/.oh/cli/src/__tests__/tool-catalog.test.ts index 3057999b..66d0adbd 100644 --- a/.oh/cli/src/__tests__/tool-catalog.test.ts +++ b/.oh/cli/src/__tests__/tool-catalog.test.ts @@ -15,18 +15,24 @@ const REPO_ROOT = join(dirname(fileURLToPath(import.meta.url)), "..", "..", ".." const read = (p: string): string => readFileSync(join(REPO_ROOT, p), "utf8"); describe("tool catalog shape", () => { - it("lists the five known tools", () => { + it("lists the six known tools", () => { expect(toolIds()).toEqual([ "agent-browser", "herdr", "cloudflared", "docker-cli", "gh", + "tailscale", ]); }); it("makes exactly the default and opt-in tools installable", () => { - expect(installableToolIds()).toEqual(["agent-browser", "herdr", "cloudflared"]); + expect(installableToolIds()).toEqual([ + "agent-browser", + "herdr", + "cloudflared", + "tailscale", + ]); for (const t of TOOL_CATALOG) { // A kind:"default" tool is provisioned at boot through `oh tool install`, // so it MUST be installable; a baked-in one must not be. @@ -68,7 +74,13 @@ describe("tool catalog shape", () => { it("declares a version probe only where the flag is a safe standard", () => { const withVersion = TOOL_CATALOG.filter((t) => t.versionArgv !== undefined).map((t) => t.id); - expect(withVersion).toEqual(["herdr", "cloudflared", "docker-cli", "gh"]); + expect(withVersion).toEqual([ + "herdr", + "cloudflared", + "docker-cli", + "gh", + "tailscale", + ]); for (const t of TOOL_CATALOG) { if (t.versionArgv) expect(t.versionArgv, t.id).toEqual([t.binary, "--version"]); } @@ -169,6 +181,91 @@ describe("agent-browser matches the entrypoint that really installs it", () => { }); }); +describe("tailscale matches the entrypoint that really installs it", () => { + const ts = findTool("tailscale")!; + const ENTRYPOINT = read(".devcontainer/entrypoint.sh"); + const VERSION = "1.102.3"; + const SHA_AMD64 = "36ddd9b51be57ffc2990cf76323cfa13643bfbb1b8a969f6183fa164741cdef5"; + const SHA_ARM64 = "a0fa1b154af8c61f862a2259f559f7396d96c0225f4a863eae2333e1546bbe25"; + + it("carries the entrypoint guard, not a build arg", () => { + expect(ts.entrypointGuard).toBe("INSTALL_TAILSCALE"); + expect(Object.keys(ts)).not.toContain("buildArg"); + expect(ts.toolKey).toBe("tailscale"); + expect(ts.kind).toBe("opt-in"); + }); + + it("is installed by the entrypoint and is ABSENT from the Dockerfile", () => { + expect(ENTRYPOINT).toContain("INSTALL_TAILSCALE"); + expect(read(".devcontainer/Dockerfile")).not.toContain("INSTALL_TAILSCALE"); + }); + + it("pins the same version the entrypoint pins", () => { + expect(ts.installArgv!.join(" ")).toContain(`tailscale_${VERSION}_`); + expect(ENTRYPOINT).toContain(`tailscale_${VERSION}_`); + }); + + it("verifies the same per-arch sha256 the entrypoint verifies", () => { + const argv = ts.installArgv!.join(" "); + for (const sha of [SHA_AMD64, SHA_ARM64]) { + expect(argv, sha).toContain(sha); + expect(ENTRYPOINT, sha).toContain(sha); + } + expect(argv).toContain("sha256sum -c -"); + expect(ENTRYPOINT).toContain("sha256sum -c -"); + }); + + it("downloads from the pinned stable base the entrypoint uses", () => { + const base = "https://pkgs.tailscale.com/stable/"; + expect(ts.installArgv!.join(" ")).toContain(base); + expect(ENTRYPOINT).toContain(base); + }); + + // tailscaled runs unprivileged under --tun=userspace-networking, so nothing + // here needs root. A root install would hang `oh tool install tailscale` on a + // sudo password prompt (commands/tool.ts uses stdio:"inherit", and + // /etc/sudoers.d/sandbox has no NOPASSWD), and would put the binaries in an + // image-layer path discarded on every container recreate. + it("installs as the sandbox user into the home mount", () => { + expect(ts.installUser).toBe("sandbox"); + const argv = ts.installArgv!.join(" "); + expect(argv).toContain("NPM_USER_PREFIX"); + expect(argv).not.toContain("/usr/local/bin/tailscale"); + expect(argv).not.toContain("/usr/local/bin/tailscaled"); + }); + + // /var/run/tailscale is tailscaled's default socket directory and only root + // can create it, so it belongs to the entrypoint, not to an install that runs + // as the sandbox user. + it("leaves the root-owned socket directory to the entrypoint", () => { + expect(ts.installArgv!.join(" ")).not.toContain("/var/run/tailscale"); + expect(ENTRYPOINT).toContain("/var/run/tailscale"); + }); + + it("never joins a tailnet — installation is not authentication", () => { + const argv = ts.installArgv!.join(" "); + expect(argv).not.toContain("tailscale up"); + expect(argv).not.toMatch(/(^|[^d])tailscaled\s+--tun/); + }); + + it("drops the entrypoint's log cosmetics, which would eat the exit code", () => { + const argv = ts.installArgv!.join(" "); + expect(argv).not.toContain("[entrypoint]"); + expect(argv).not.toContain("tail -"); + }); + + it("arms no download gate — the tarball is small", () => { + expect(ts.downloadSize).toBeUndefined(); + }); + + it("keeps the env plumbing wired end to end", () => { + expect(read(".devcontainer/docker-compose.yml")).toContain("INSTALL_TAILSCALE"); + expect(read("docs/configuration.md")).toMatch( + /^\| `install\.tailscale` \|.*`INSTALL_TAILSCALE`/m, + ); + }); +}); + describe("baked-in tools", () => { it("declare no install key — the installer must not invent one", () => { for (const t of TOOL_CATALOG) { diff --git a/.oh/cli/src/__tests__/tool.test.ts b/.oh/cli/src/__tests__/tool.test.ts index d135b98f..f2369d4b 100644 --- a/.oh/cli/src/__tests__/tool.test.ts +++ b/.oh/cli/src/__tests__/tool.test.ts @@ -107,6 +107,19 @@ const agentBrowserFlag = (root: string): unknown => (JSON.parse(readFileSync(ohConfigPath(root), "utf8")) as { install?: Record }) .install?.agentBrowser; +const tailscaleFlag = (root: string): unknown => + (JSON.parse(readFileSync(ohConfigPath(root), "utf8")) as { install?: Record }) + .install?.tailscale; + +const absentTailscale = (cmd: string, args: string[]): RunResult | undefined => + isExecOf(cmd, args, "command -v tailscale") ? { status: 1, stdout: "", stderr: "" } : undefined; + +const isTailscaleVersionExec = (cmd: string, args: string[]): boolean => + cmd === "docker" && args[0] === "exec" && args.join(" ").includes("tailscale --version"); + +const isTailscaleInstallCall = (c: RecordedCall): boolean => + c.cmd === "docker" && c.args[0] === "exec" && c.args.some((a) => a.includes("sha256sum -c -")); + describe("oh tool — argument parsing", () => { it("shows help with no args", () => { const r = parseToolArgs([]); @@ -163,7 +176,7 @@ describe("oh tool list / status", () => { const { io, out } = makeIo(); expect(await runToolList({ cwd: root, run: liveHost().run }, io)).toBe(0); const text = out.join(""); - for (const id of ["agent-browser", "herdr", "cloudflared", "docker-cli", "gh"]) { + for (const id of ["agent-browser", "herdr", "cloudflared", "docker-cli", "gh", "tailscale"]) { expect(text, id).toContain(id); } expect(text).toContain("baked-in"); @@ -342,6 +355,94 @@ describe("oh tool install — the other exits", () => { }); }); +describe("oh tool install tailscale", () => { + it("--persist-only writes the flag and execs nothing", async () => { + const root = makeRepo(); + const { calls, run } = liveHost(absentTailscale); + const { io, out } = makeIo(); + expect(await runToolInstall("tailscale", { cwd: root, run, persistOnly: true }, io)).toBe(0); + expect(tailscaleFlag(root)).toBe(true); + expect(calls.length).toBe(0); + expect(out.join("")).toContain("oh.json: set install.tailscale=true"); + }); + + it("execs the pinned install argv as the sandbox user, with no download prompt", async () => { + const root = makeRepo(); + const { calls, run } = liveHost(absentTailscale); + const { io, asked, out } = makeIo(true); + expect(await runToolInstall("tailscale", { cwd: root, run }, io)).toBe(0); + expect(asked).toEqual([]); + const install = calls.find(isTailscaleInstallCall); + expect(install).toBeDefined(); + // #858/#908: a root install becomes an interactive `sudo` inside the sandbox. + expect(install!.args.join(" ")).toContain("-u sandbox"); + expect(install!.args.join(" ")).not.toContain("-u root"); + expect(install!.args.join(" ")).toContain("pkgs.tailscale.com/stable/"); + expect(tailscaleFlag(root)).toBe(true); + expect(out.join("")).toContain("installed"); + }); + + it("is idempotent — an already-present binary short-circuits", async () => { + const root = makeRepo(); + const { calls, run } = liveHost((cmd, args) => + isExecOf(cmd, args, "command -v tailscale") + ? { status: 0, stdout: "", stderr: "" } + : undefined, + ); + const { io, out } = makeIo(true); + expect(await runToolInstall("tailscale", { cwd: root, run }, io)).toBe(0); + expect(calls.some(isTailscaleInstallCall)).toBe(false); + expect(out.join("")).toContain("already installed"); + }); + + it("keeps the flag set when the installer fails", async () => { + const root = makeRepo(); + const { run } = liveHost((cmd, args) => { + if (isExecOf(cmd, args, "sha256sum -c -")) return { status: 9, stdout: "", stderr: "" }; + return absentTailscale(cmd, args); + }); + const { io, err } = makeIo(true); + expect(await runToolInstall("tailscale", { cwd: root, run }, io)).toBe(9); + expect(tailscaleFlag(root)).toBe(true); + expect(err.join("")).toContain("next container start"); + }); +}); + +describe("oh tool status tailscale", () => { + it("reports enabled, installed and version as JSON", async () => { + const root = makeRepo(); + const { run } = liveHost((cmd, args) => + isTailscaleVersionExec(cmd, args) + ? { status: 0, stdout: "1.102.3\n tailscale commit: abc\n", stderr: "" } + : undefined, + ); + const { io: persistIo } = makeIo(); + await runToolInstall("tailscale", { cwd: root, run, persistOnly: true }, persistIo); + + const { io, out } = makeIo(); + expect(await runToolStatus("tailscale", { cwd: root, run, json: true }, io)).toBe(0); + const status = JSON.parse(out.join("")); + expect(status.id).toBe("tailscale"); + expect(status.kind).toBe("opt-in"); + expect(status.enabled).toBe(true); + expect(status.installed).toBe(true); + expect(status.version).toBe("1.102.3"); + expect(status.installable).toBe(true); + }); + + it("reports not-installed and no version when the binary is absent", async () => { + const root = makeRepo(); + const { calls, run } = liveHost(absentTailscale); + const { io, out } = makeIo(); + await runToolStatus("tailscale", { cwd: root, run, json: true }, io); + const status = JSON.parse(out.join("")); + expect(status.enabled).toBe(false); + expect(status.installed).toBe(false); + expect(status.version).toBeNull(); + expect(calls.some((c) => isTailscaleVersionExec(c.cmd, c.args))).toBe(false); + }); +}); + describe("oh tool — inside the sandbox", () => { const INSIDE: NodeJS.ProcessEnv = { OH_EXECUTION_TARGET: "local" }; diff --git a/.oh/cli/src/commands/init.ts b/.oh/cli/src/commands/init.ts index d313a55b..40a8cab3 100644 --- a/.oh/cli/src/commands/init.ts +++ b/.oh/cli/src/commands/init.ts @@ -659,6 +659,9 @@ const ENV_TO_CONFIG: Record = { INSTALL_AGENT_BROWSER: (c, v) => { section(c, "install").agentBrowser = asBool(v); }, + INSTALL_TAILSCALE: (c, v) => { + section(c, "install").tailscale = asBool(v); + }, SANDBOX_SSH: (c, v) => { section(c, "access").ssh = asBool(v); }, @@ -777,6 +780,11 @@ async function runWizard( { key: "hermes", field: "hermes", desc: "Hermes CLI + runtime (build arg + runtime)" }, { key: "grok_build", field: "grokBuild", desc: "Grok build tooling" }, { key: "agent_browser", field: "agentBrowser", desc: "agent-browser + Chromium (~1 GB)" }, + { + key: "tailscale", + field: "tailscale", + desc: "Tailscale (userspace) — private remote access for T3 Code", + }, ]; for (const inst of installs) { const yes = await confirmWith(askFn, `Install ${inst.key} — ${inst.desc}?`, false); diff --git a/.oh/cli/src/lib/__tests__/config-render.test.ts b/.oh/cli/src/lib/__tests__/config-render.test.ts index a8df288f..7bd4b70e 100644 --- a/.oh/cli/src/lib/__tests__/config-render.test.ts +++ b/.oh/cli/src/lib/__tests__/config-render.test.ts @@ -61,6 +61,7 @@ describe("renderComposeEnv", () => { expect(text).not.toContain("INSTALL_DEEPAGENTS"); expect(text).toContain("INSTALL_HERMES=false"); expect(text).toContain("INSTALL_AGENT_BROWSER=false"); + expect(text).toContain("INSTALL_TAILSCALE=false"); expect(text).toContain("DOCKER_SOCKET=true"); expect(text).toContain("SANDBOX_SSH=true"); expect(text).toContain("SANDBOX_SSH_PORT=2022"); diff --git a/.oh/cli/src/lib/__tests__/oh-config.test.ts b/.oh/cli/src/lib/__tests__/oh-config.test.ts index d0e571f4..ca25e80d 100644 --- a/.oh/cli/src/lib/__tests__/oh-config.test.ts +++ b/.oh/cli/src/lib/__tests__/oh-config.test.ts @@ -128,6 +128,11 @@ describe("validateOhConfig", () => { { install: { agentBrowser: 1 } }, /^oh\.json: install\.agentBrowser must be a boolean$/, ], + [ + "install.tailscale", + { install: { tailscale: 1 } }, + /^oh\.json: install\.tailscale must be a boolean$/, + ], ["access.ssh", { access: { ssh: "yes" } }, /^oh\.json: access\.ssh must be a boolean$/], ["access.sshPort", { access: { sshPort: "2222" } }, /^oh\.json: access\.sshPort must be a number$/], [ diff --git a/.oh/cli/src/lib/config-render.ts b/.oh/cli/src/lib/config-render.ts index 44a1f590..3e789790 100644 --- a/.oh/cli/src/lib/config-render.ts +++ b/.oh/cli/src/lib/config-render.ts @@ -32,6 +32,7 @@ export function renderComposeVars(config: OhConfig): RenderedVar[] { put("INSTALL_GROK_BUILD", config.install?.grokBuild); put("INSTALL_HERMES", config.install?.hermes); put("INSTALL_AGENT_BROWSER", config.install?.agentBrowser); + put("INSTALL_TAILSCALE", config.install?.tailscale); put("DOCKER_SOCKET", config.access?.dockerSocket); put("SANDBOX_SSH", config.access?.ssh); diff --git a/.oh/cli/src/lib/env-file.ts b/.oh/cli/src/lib/env-file.ts index e1e09d75..1e9805ee 100644 --- a/.oh/cli/src/lib/env-file.ts +++ b/.oh/cli/src/lib/env-file.ts @@ -27,6 +27,7 @@ export const INSTALL_FIELDS: Record = { grok_build: "install.grokBuild", hermes: "install.hermes", agent_browser: "install.agentBrowser", + tailscale: "install.tailscale", }; export const CONFIG_FIELD_BY_ENV_KEY: Record = { @@ -35,6 +36,7 @@ export const CONFIG_FIELD_BY_ENV_KEY: Record = { INSTALL_GROK_BUILD: INSTALL_FIELDS.grok_build, INSTALL_HERMES: INSTALL_FIELDS.hermes, INSTALL_AGENT_BROWSER: INSTALL_FIELDS.agent_browser, + INSTALL_TAILSCALE: INSTALL_FIELDS.tailscale, }; export function installFieldPath(key: string): string { diff --git a/.oh/cli/src/lib/oh-config.ts b/.oh/cli/src/lib/oh-config.ts index 4014f8d4..7783f203 100644 --- a/.oh/cli/src/lib/oh-config.ts +++ b/.oh/cli/src/lib/oh-config.ts @@ -38,6 +38,7 @@ export interface InstallFlags { grokBuild?: boolean; hermes?: boolean; agentBrowser?: boolean; + tailscale?: boolean; } export interface AccessSettings { @@ -127,6 +128,7 @@ export function defaultOhConfig(name: string): OhConfig { grokBuild: false, hermes: false, agentBrowser: false, + tailscale: false, }, access: { ssh: false, @@ -220,7 +222,13 @@ export function validateOhConfig(value: unknown): OhConfig { const install = expectSection(record, "install"); if (install) { - for (const key of ["opencode", "grokBuild", "hermes", "agentBrowser"]) { + for (const key of [ + "opencode", + "grokBuild", + "hermes", + "agentBrowser", + "tailscale", + ]) { expectBoolean(install, key, "install."); } } @@ -349,6 +357,7 @@ export const OH_CONFIG_FIELDS: readonly OhConfigField[] = [ { path: "install.grokBuild", type: "boolean" }, { path: "install.hermes", type: "boolean" }, { path: "install.agentBrowser", type: "boolean" }, + { path: "install.tailscale", type: "boolean" }, { path: "access.ssh", type: "boolean" }, { path: "access.sshPort", type: "port" }, { path: "access.sshPasswordAuth", type: "boolean" }, diff --git a/.oh/cli/src/lib/tools/catalog.ts b/.oh/cli/src/lib/tools/catalog.ts index fac0d638..5c5b1916 100644 --- a/.oh/cli/src/lib/tools/catalog.ts +++ b/.oh/cli/src/lib/tools/catalog.ts @@ -125,6 +125,23 @@ export const TOOL_CATALOG: readonly ToolEntry[] = Object.freeze([ "The GitHub CLI is installed in the base image. Run `gh auth login` inside the sandbox to authenticate it.", docsPath: TOOLS_DOC, }), + Object.freeze({ + id: "tailscale", + title: "Tailscale", + kind: "opt-in", + binary: "tailscale", + verifyArgv: Object.freeze(["bash", "-lc", "command -v tailscale >/dev/null"]), + versionArgv: Object.freeze(["tailscale", "--version"]), + toolKey: "tailscale", + entrypointGuard: "INSTALL_TAILSCALE", + installArgv: Object.freeze([ + "bash", + "-lc", + "set -e\narch=\"$(dpkg --print-architecture)\"\ncase \"$arch\" in\n amd64) tarball=tailscale_1.102.3_amd64.tgz; sha=36ddd9b51be57ffc2990cf76323cfa13643bfbb1b8a969f6183fa164741cdef5 ;;\n arm64) tarball=tailscale_1.102.3_arm64.tgz; sha=a0fa1b154af8c61f862a2259f559f7396d96c0225f4a863eae2333e1546bbe25 ;;\n *) echo \"no pinned Tailscale build for $arch\" >&2; exit 1 ;;\nesac\nprefix=\"${NPM_USER_PREFIX:-$HOME/.local}\"\ntmp=\"$(mktemp -d)\"\ntrap 'rm -rf \"$tmp\"' EXIT\ncurl -fsSL \"https://pkgs.tailscale.com/stable/$tarball\" -o \"$tmp/$tarball\"\necho \"$sha $tmp/$tarball\" | sha256sum -c -\ntar -xzf \"$tmp/$tarball\" -C \"$tmp\"\ninstall -d \"$prefix/bin\"\ninstall -m 0755 \"$tmp/tailscale_1.102.3_$arch/tailscale\" \"$prefix/bin/tailscale\"\ninstall -m 0755 \"$tmp/tailscale_1.102.3_$arch/tailscaled\" \"$prefix/bin/tailscaled\"\ninstall -d -m 0700 \"$HOME/.tailscale\"", + ]), + installUser: "sandbox", + docsPath: TOOLS_DOC, + }), ]); export function findTool(id: string): ToolEntry | undefined { diff --git a/.oh/evals/RESULTS.md b/.oh/evals/RESULTS.md index 25105ddf..5275d737 100644 --- a/.oh/evals/RESULTS.md +++ b/.oh/evals/RESULTS.md @@ -6,108 +6,110 @@ probe id; git history is the time series.** Schema and exit-code semantics are i | probe | tier | last-run (UTC) | status | source | |-------|------|----------------|--------|--------| -| advisor-monitored-loop | A | 2026-08-31 20:04 | PASS | conversation 2026-06-19 (single-owner implementation workflow, issue #257) | -| agent-browser-cli | A | 2026-08-31 20:04 | PASS | retro lesson 2026-06-07 (agent-browser 0.8.5 CLI) | -| agents-identity-contract | A | 2026-08-31 20:04 | PASS | issue #854 — T3-style root identity, glossary, and skill-owned procedures | -| artifact-contract-audit | A | 2026-08-31 20:04 | PASS | issue #583/#645 — production /audit implementation Gate 1 behavior | -| audit-dispatcher-contract | A | 2026-08-31 20:04 | PASS | issue #645 — audit consolidation public taxonomy | -| audit-implementation-behavior | A | 2026-08-31 20:04 | PASS | issue #645 — implementation root/repo/browser behavior | -| audit-pr-acquire | A | 2026-08-31 20:04 | PASS | issue #645 — production PR acquisition behavior | -| audit-pr-classifier | A | 2026-08-31 20:04 | PASS | issue #645 — deterministic focused and queue PR classifier | -| audit-run-root-contract | A | 2026-08-31 20:04 | PASS | issue #645 — executable immutable audit root/run correlation | -| audit-shellcheck-coverage | A | 2026-08-31 20:04 | PASS | issue #645 — private audit scripts require release and CI lint coverage | -| audit-stale-references | A | 2026-08-31 20:04 | PASS | issue #645 — clean-breaking audit migration | -| boot-lint-glob | A | 2026-08-31 20:04 | PASS | issue #90, issue #120 | -| builder-skill-consolidation | A | 2026-08-31 20:04 | PASS | issue #643 — consolidate artifact builders behind one /builder dispatcher | -| capability-benchmark-schema | A | 2026-08-31 20:04 | PASS | issue #167 — capability benchmark instrument | -| cc-safety-net-wiring | A | 2026-08-31 20:04 | SKIPPED | .oh/tasks/cc-safety-net/prd.json US-007 2026-07-19 | -| changelog-entry-length | A | 2026-08-31 20:04 | PASS | conversation 2026-08-24 — CHANGELOG.md grew to 259KB of bullet prose because "one line" was unquantified | -| cleanup-tasks-scoped-guard | A | 2026-08-31 20:04 | PASS | issue #85 | -| cleanup-tasks-worktree-grooming | A | 2026-08-31 20:04 | PASS | issue #168; issue #327 | -| cli-publish-typecheck-scope | A | 2026-08-31 20:04 | PASS | release run 33271077312 — v0.5.0 pushed its GHCR image, then failed to publish | -| close-issues-on-development | A | 2026-08-31 20:04 | PASS | issue #841 (closing keywords never fire because the default branch is main) 2026-08-26 | -| codex-stale-response-retry | A | 2026-08-31 20:04 | PASS | issue #506 — Codex previous_response_not_found RCA | -| compose-config-path-parity | A | 2026-08-31 20:04 | PASS | PR #833 (remove harness.yaml — the wrapper and VS Code "Reopen in Container" paths must resolve the same service) 2026-08-26 | -| config-schema-parity | A | 2026-08-31 20:04 | PASS | PR #833 (one schema file — DOCKER_SOCKET, SANDBOX_SSH, OH_SANDBOX_IMAGE, OH_PULL_POLICY, SKIP_PNPM_INSTALL were consumed but undocumented); rewritten for the oh.json/secrets split by PR #887 | -| context-tier-size-budget | A | 2026-08-31 20:04 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-007) — the always-on tier was 85,256 B | -| cron-claude-codex-fallback | A | 2026-08-31 20:04 | PASS | conversation 2026-06-12 (default Codex fallback for crons) | -| cron-watchdog | A | 2026-08-31 20:04 | PASS | issues #130/#453 (cron runtime watchdog + legacy system-cron reaping) 2026-06-19 | -| crons-directory-guide | A | 2026-08-31 20:04 | PASS | issue #874 | -| curl-bash-safe-alternatives | A | 2026-08-31 20:04 | PASS | vet-run/vet integration — public curl|bash examples need review-first alternatives | -| datasets-schema | A | 2026-08-31 20:04 | PASS | issue #196 — .oh/evals/datasets verifiable trajectory corpus (Repo2RLEnv-inspired) | -| debugmcp-availability | A | 2026-08-31 20:04 | SKIPPED | issue #297 — DebugMCP MCP debug-server availability | -| default-provisioning | A | 2026-08-31 20:04 | PASS | #902 — `oh harness install` must work from inside the sandbox, where | -| delegate-model-effort-policy | A | 2026-08-31 20:04 | PASS | conversation 2026-07-11 (delegate model inheritance and thinking policy) | -| devtcp-hook | A | 2026-08-31 20:04 | PASS | retro lesson 2026-06-10 (zsh /dev/tcp) | -| docker-inspect-env-guard | A | 2026-08-31 20:04 | PASS | operator directive 2026-08-08 (agents keep the docker socket, but must | -| docs-build-fast-path | A | 2026-08-31 20:04 | PASS | #455 — docs builds must stay out of fast harness/eval/release gates; #536 — docs site externalized to openharness-web; docs markdown relocated to docs/ | -| drift-check-cron-staleness-glob | A | 2026-08-31 20:04 | PASS | issue #98; issue #225 (restart-required cron frontmatter/config drift) | -| entrypoint-pnpm-manifest-fingerprint | A | 2026-08-31 20:04 | PASS | issue #521 (manifest-aware sandbox installs) 2026-07-01 | -| eval-ci-gate | A | 2026-08-31 20:04 | PASS | #103 — eval probe suite gated in CI | -| eval-gate | A | 2026-08-31 20:04 | PASS | retro lesson 2026-06-11 (eval-gate) | -| eval-results-atomic | A | 2026-08-31 20:04 | PASS | issue #83 (eval-results-atomic-write) | -| eval-runner-exit | A | 2026-08-31 20:04 | PASS | retro lesson 2026-06-11 (eval-runner-exit) #29 | -| eval-runs-once-per-cycle | A | 2026-08-31 20:04 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-006) — /eval ran 3x per cycle on the | -| execution-target-contract | A | 2026-08-31 20:04 | PASS | issue #733 (ExecutionTarget contract + Docker Compose adapter) 2026-08-10 | -| get-oh-bootstrap | A | 2026-08-31 20:04 | PASS | get-oh.sh bootstrap — the Node-bootstrapping host-side path to the standalone `oh` CLI (also on npm as @mifune/openharness; see oh-npm-package.sh) | -| git-skill | A | 2026-08-31 20:04 | PASS | conversation 2026-06-15 — rules are not always supported; git workflow must be the /git skill | -| harness-audit-empty-output-gate | A | 2026-08-31 20:04 | PASS | issue #246 — /audit harness must fail closed on empty auditor outputs | -| harness-ci-core-paths | A | 2026-08-31 20:04 | PASS | #165 — core sandbox config files must trigger harness CI | -| harness-ci-hooks-paths | A | 2026-08-31 20:04 | PASS | issue #202 — credential/security hook changes must trigger harness CI | -| harness-yaml-migration | A | 2026-08-31 20:04 | PASS | PR #833 (migrate-harness-yaml.sh — append / uncomment-in-place / preserve / overwrite, plus a silent no-op second run) 2026-08-26 | -| health-check-docker-stats | A | 2026-08-31 20:04 | PASS | retro lesson 2026-06-10 (docker stats vs ps Size) | -| health-check-socket-degrade | A | 2026-08-31 20:04 | PASS | issue #762 (refs #756) — /health-check degrades to one statement, not nine failures | -| heartbeat-logging-contract | A | 2026-08-31 20:04 | PASS | issue #447 (heartbeat log append hardening) 2026-06-18 | -| image-seed-hygiene | A | 2026-08-31 20:04 | PASS | issue #900 (slim the sandbox image) 2026-08-30 | -| markitdown-wiki-ingest | A | 2026-08-31 20:04 | PASS | issue #649 — pinned local-document normalization contract for /wiki ingest | -| next-dev-prod | A | 2026-08-31 20:04 | SKIPPED | retro lesson 2026-06-04 | -| oh-compose-env-wiring | A | 2026-08-31 20:04 | PASS | issue #880 (oh as the only front door — oh.json is the non-secret config surface) | -| oh-config-surfaces | A | 2026-08-31 20:04 | REGRESSION | PR #887 (config split across two authored surfaces — a tracked oh.json and a secrets-only root dotenv — with nothing left under $HOME) | -| oh-destroy-guard | A | 2026-08-31 20:04 | PASS | issue #879 — `oh` becomes the only front door, so `make destroy` must | -| oh-devcontainer-restructure | A | 2026-08-31 20:04 | PASS | consolidate devcontainer — .oh/devcontainer/ folded back into .devcontainer/ | -| oh-home-mount | A | 2026-08-31 20:04 | PASS | issue #898 (single $HOME mount) 2026-08-30 | -| oh-image-only-deploy | A | 2026-08-31 20:04 | PASS | .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only deploy) | -| oh-init-headless-config | A | 2026-08-31 20:04 | PASS | PR #827 (installer answers landed in the losing config file); retargeted to the .example.env template by PR #833, then to oh.json by PR #887 | -| oh-init-scaffold | A | 2026-08-31 20:04 | PASS | issue #531 Phase 2 | -| oh-lifecycle-surface | A | 2026-08-31 20:04 | PASS | issue #881 — the Makefile is retired and `oh` is the only front door | -| oh-npm-package | A | 2026-08-31 20:04 | PASS | npm publish path for the standalone `oh` CLI (@mifune/openharness) — alternative to get-oh.sh | -| oh-payload-manifest | A | 2026-08-31 20:04 | PASS | issue #531 follow-on (.oh payload manifest — oh update ships a declared allowlist) | -| oh-sandbox-image-mode | A | 2026-08-31 20:04 | PASS | conversation 2026-07-05 (basic Docker deployment — prebuilt-image mode) | -| oh-shipped-repo-overridable | A | 2026-08-31 20:04 | PASS | issue #531 follow-on (de-hardcode residual — shipped .oh shell scripts keep the upstream repo overridable) | -| oh-standalone-lifecycle | A | 2026-08-31 20:04 | PASS | issue #564 | -| oh-update | A | 2026-08-31 20:04 | PASS | issue #531 Phase 3 (oh update — upgrade only the .oh control plane) | -| operator-config-guard | A | 2026-08-31 20:04 | PASS | operator directives 2026-08-06 (.config/ and settings.local.json are operator-only) | -| pnpm-audit-ci-gate | A | 2026-08-31 20:04 | PASS | issue #171 — pnpm security audits must run in CI | -| post-bridge-publish-confirmation | A | 2026-08-31 20:04 | PASS | #523 — post-bridge live publishing requires an explicit final confirmation gate | -| prd-output-path-contract | A | 2026-08-31 20:04 | PASS | retro lesson 2026-06-19 | -| prompt-miner-schema-compat | A | 2026-08-31 20:04 | PASS | issue #253 — prompt-miner JSONL schema-drift guard | -| prompt-miner-symlink-entrypoint | A | 2026-08-31 20:04 | PASS | issue #663 — prompt-miner engine no-ops via the documented .claude/skills symlink | -| prompt-miner-weakness-record | A | 2026-08-31 20:04 | PASS | issue #580 — prompt-miner weakness-record (WH-xxx) cluster output | -| protected-path-deletion | A | 2026-08-31 20:04 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-001) — the critique gate was deleted, | -| protected-paths-resolve | A | 2026-08-31 20:04 | PASS | issue #753 — .claude/protected-paths.txt named 7 paths that did not exist. | -| registry-portability-gate | A | 2026-08-31 20:04 | PASS | issue #758 | -| registry-portability | A | 2026-08-31 20:04 | SKIPPED | issue #758 | -| retro-deterministic-contract | A | 2026-08-31 20:04 | PASS | issue #443 — /retro deterministic output and self-contained helper contract | -| rl-delegation-write-worker | A | 2026-08-31 20:04 | PASS | retro lesson 2026-06-10 (rl-delegation) #57 | -| rlm-context-budget | A | 2026-08-31 20:04 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-006 | -| runtime-preflight-gate | A | 2026-08-31 20:04 | PASS | issue #806 § B1 (open sandbox.substrate vs sandbox.runtime selector); | -| sandbox-boot-guard-ci | A | 2026-08-31 20:04 | PASS | issue #449 (sandbox image build CI guard) 2026-06-19; | -| sandbox-node-base | A | 2026-08-31 20:04 | PASS | openharness#878 — oh as the only front door, T0 sandbox base image | -| skill-paths | A | 2026-08-31 20:04 | PASS | issue #43 — stale path references; extended by issue #69 — apps/->packages/ rename guard; extended by issue #870 — deleted .oh/agents/advisor.md | -| skills-dir-clean | A | 2026-08-31 20:04 | PASS | conversation 2026-06-29 — Pi parses every top-level `.md` in the skills | -| skills-task-tool-coupling | A | 2026-08-31 20:04 | PASS | council review 2026-08-29 (issue #886) — /delegate instructed Claude-Code-only | -| skills-vendored | A | 2026-08-31 20:04 | PASS | absorb .mifune submodule into .oh — the skills/agents/hooks pack is vendored | -| slack-admin-command-surface | A | 2026-08-31 20:04 | PASS | issue #354 — Slack bridge docs must distinguish Pi /msg-bridge commands from Slack DM admin text handlers | -| spec-family-contract | A | 2026-08-31 20:04 | PASS | issue #265; spec-simplification issue #816; workflow authority issue #854 | -| spec-ready-finalization | A | 2026-08-31 20:04 | PASS | issue #134; spec-simplification issue #816; workflow authority issue #854 | -| ste-checker-contract | A | 2026-08-31 20:04 | PASS | issue #750 PR audit — the /ste checker had four fail-open paths (unclosed | -| submitted-by-trailers | A | 2026-08-31 20:04 | PASS | conversation 2026-06-12 (commit attribution trailers); the single-owner | -| sync-skill-contract | A | 2026-08-31 20:04 | PASS | issue #331 — /sync dispatcher skill (bidirectional origin↔upstream sync) | -| tool-catalog-boundary | A | 2026-08-31 20:04 | PASS | agent-browser's exclusion from the harness catalog (#821) and the | -| version-parity | A | 2026-08-31 20:04 | PASS | conversation 2026-08-29 — the oh CLI became the only lifecycle door, so its | -| weigh-scorer-contract | A | 2026-08-31 20:04 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-003 (2026-06-27) | -| wiki-readme-index | A | 2026-08-31 20:04 | PASS | issue #132 — wiki README index drift guard | -| workflow-boundaries | A | 2026-08-31 20:04 | PASS | conversation 2026-06-19 (workflow consolidation, issue #259); authority moved to /spec in issue #854 | -| worktrees-layout | A | 2026-08-31 20:04 | PASS | issue #872 | +| advisor-monitored-loop | A | 2026-08-31 21:00 | PASS | conversation 2026-06-19 (single-owner implementation workflow, issue #257) | +| agent-browser-cli | A | 2026-08-31 21:00 | PASS | retro lesson 2026-06-07 (agent-browser 0.8.5 CLI) | +| agents-identity-contract | A | 2026-08-31 21:00 | PASS | issue #854 — T3-style root identity, glossary, and skill-owned procedures | +| artifact-contract-audit | A | 2026-08-31 21:00 | PASS | issue #583/#645 — production /audit implementation Gate 1 behavior | +| audit-dispatcher-contract | A | 2026-08-31 21:00 | PASS | issue #645 — audit consolidation public taxonomy | +| audit-implementation-behavior | A | 2026-08-31 21:00 | PASS | issue #645 — implementation root/repo/browser behavior | +| audit-pr-acquire | A | 2026-08-31 21:00 | PASS | issue #645 — production PR acquisition behavior | +| audit-pr-classifier | A | 2026-08-31 21:00 | PASS | issue #645 — deterministic focused and queue PR classifier | +| audit-run-root-contract | A | 2026-08-31 21:00 | PASS | issue #645 — executable immutable audit root/run correlation | +| audit-shellcheck-coverage | A | 2026-08-31 21:00 | PASS | issue #645 — private audit scripts require release and CI lint coverage | +| audit-stale-references | A | 2026-08-31 21:00 | PASS | issue #645 — clean-breaking audit migration | +| boot-lint-glob | A | 2026-08-31 21:00 | PASS | issue #90, issue #120 | +| builder-skill-consolidation | A | 2026-08-31 21:00 | PASS | issue #643 — consolidate artifact builders behind one /builder dispatcher | +| capability-benchmark-schema | A | 2026-08-31 21:00 | PASS | issue #167 — capability benchmark instrument | +| cc-safety-net-wiring | A | 2026-08-31 21:00 | SKIPPED | .oh/tasks/cc-safety-net/prd.json US-007 2026-07-19 | +| changelog-entry-length | A | 2026-08-31 21:00 | PASS | conversation 2026-08-24 — CHANGELOG.md grew to 259KB of bullet prose because "one line" was unquantified | +| cleanup-tasks-scoped-guard | A | 2026-08-31 21:00 | PASS | issue #85 | +| cleanup-tasks-worktree-grooming | A | 2026-08-31 21:00 | PASS | issue #168; issue #327 | +| cli-publish-typecheck-scope | A | 2026-08-31 21:00 | PASS | release run 33271077312 — v0.5.0 pushed its GHCR image, then failed to publish | +| close-issues-on-development | A | 2026-08-31 21:00 | PASS | issue #841 (closing keywords never fire because the default branch is main) 2026-08-26 | +| codex-stale-response-retry | A | 2026-08-31 21:00 | PASS | issue #506 — Codex previous_response_not_found RCA | +| compose-config-path-parity | A | 2026-08-31 21:00 | PASS | PR #833 (remove harness.yaml — the wrapper and VS Code "Reopen in Container" paths must resolve the same service) 2026-08-26 | +| config-schema-parity | A | 2026-08-31 21:00 | PASS | PR #833 (one schema file — DOCKER_SOCKET, SANDBOX_SSH, OH_SANDBOX_IMAGE, OH_PULL_POLICY, SKIP_PNPM_INSTALL were consumed but undocumented); rewritten for the oh.json/secrets split by PR #887 | +| context-tier-size-budget | A | 2026-08-31 21:00 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-007) — the always-on tier was 85,256 B | +| cron-claude-codex-fallback | A | 2026-08-31 21:00 | PASS | conversation 2026-06-12 (default Codex fallback for crons) | +| cron-watchdog | A | 2026-08-31 21:00 | PASS | issues #130/#453 (cron runtime watchdog + legacy system-cron reaping) 2026-06-19 | +| crons-directory-guide | A | 2026-08-31 21:00 | PASS | issue #874 | +| curl-bash-safe-alternatives | A | 2026-08-31 21:00 | PASS | vet-run/vet integration — public curl|bash examples need review-first alternatives | +| datasets-schema | A | 2026-08-31 21:00 | PASS | issue #196 — .oh/evals/datasets verifiable trajectory corpus (Repo2RLEnv-inspired) | +| debugmcp-availability | A | 2026-08-31 21:00 | SKIPPED | issue #297 — DebugMCP MCP debug-server availability | +| default-provisioning | A | 2026-08-31 21:00 | PASS | #902 — `oh harness install` must work from inside the sandbox, where | +| delegate-model-effort-policy | A | 2026-08-31 21:00 | PASS | conversation 2026-07-11 (delegate model inheritance and thinking policy) | +| devtcp-hook | A | 2026-08-31 21:00 | PASS | retro lesson 2026-06-10 (zsh /dev/tcp) | +| docker-inspect-env-guard | A | 2026-08-31 21:00 | PASS | operator directive 2026-08-08 (agents keep the docker socket, but must | +| docs-build-fast-path | A | 2026-08-31 21:00 | PASS | #455 — docs builds must stay out of fast harness/eval/release gates; #536 — docs site externalized to openharness-web; docs markdown relocated to docs/ | +| drift-check-cron-staleness-glob | A | 2026-08-31 21:00 | PASS | issue #98; issue #225 (restart-required cron frontmatter/config drift) | +| entrypoint-pnpm-manifest-fingerprint | A | 2026-08-31 21:00 | PASS | issue #521 (manifest-aware sandbox installs) 2026-07-01 | +| eval-ci-gate | A | 2026-08-31 21:00 | PASS | #103 — eval probe suite gated in CI | +| eval-gate | A | 2026-08-31 21:00 | PASS | retro lesson 2026-06-11 (eval-gate) | +| eval-results-atomic | A | 2026-08-31 21:00 | PASS | issue #83 (eval-results-atomic-write) | +| eval-runner-exit | A | 2026-08-31 21:00 | PASS | retro lesson 2026-06-11 (eval-runner-exit) #29 | +| eval-runs-once-per-cycle | A | 2026-08-31 21:00 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-006) — /eval ran 3x per cycle on the | +| execution-target-contract | A | 2026-08-31 21:00 | PASS | issue #733 (ExecutionTarget contract + Docker Compose adapter) 2026-08-10 | +| get-oh-bootstrap | A | 2026-08-31 21:00 | PASS | get-oh.sh bootstrap — the Node-bootstrapping host-side path to the standalone `oh` CLI (also on npm as @mifune/openharness; see oh-npm-package.sh) | +| git-skill | A | 2026-08-31 21:00 | PASS | conversation 2026-06-15 — rules are not always supported; git workflow must be the /git skill | +| harness-audit-empty-output-gate | A | 2026-08-31 21:00 | PASS | issue #246 — /audit harness must fail closed on empty auditor outputs | +| harness-ci-core-paths | A | 2026-08-31 21:00 | PASS | #165 — core sandbox config files must trigger harness CI | +| harness-ci-hooks-paths | A | 2026-08-31 21:00 | PASS | issue #202 — credential/security hook changes must trigger harness CI | +| harness-yaml-migration | A | 2026-08-31 21:00 | PASS | PR #833 (migrate-harness-yaml.sh — append / uncomment-in-place / preserve / overwrite, plus a silent no-op second run) 2026-08-26 | +| health-check-docker-stats | A | 2026-08-31 21:00 | PASS | retro lesson 2026-06-10 (docker stats vs ps Size) | +| health-check-socket-degrade | A | 2026-08-31 21:00 | PASS | issue #762 (refs #756) — /health-check degrades to one statement, not nine failures | +| heartbeat-logging-contract | A | 2026-08-31 21:00 | PASS | issue #447 (heartbeat log append hardening) 2026-06-18 | +| image-seed-hygiene | A | 2026-08-31 21:00 | PASS | issue #900 (slim the sandbox image) 2026-08-30 | +| markitdown-wiki-ingest | A | 2026-08-31 21:00 | PASS | issue #649 — pinned local-document normalization contract for /wiki ingest | +| next-dev-prod | A | 2026-08-31 21:00 | SKIPPED | retro lesson 2026-06-04 | +| oh-compose-env-wiring | A | 2026-08-31 21:00 | PASS | issue #880 (oh as the only front door — oh.json is the non-secret config surface) | +| oh-config-surfaces | A | 2026-08-31 21:00 | PASS | PR #887 (config split across two authored surfaces — a tracked oh.json and a secrets-only root dotenv — with nothing left under $HOME) | +| oh-destroy-guard | A | 2026-08-31 21:00 | PASS | issue #879 — `oh` becomes the only front door, so `make destroy` must | +| oh-devcontainer-restructure | A | 2026-08-31 21:00 | PASS | consolidate devcontainer — .oh/devcontainer/ folded back into .devcontainer/ | +| oh-home-mount | A | 2026-08-31 21:00 | PASS | issue #898 (single $HOME mount) 2026-08-30 | +| oh-image-only-deploy | A | 2026-08-31 21:00 | PASS | .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only deploy) | +| oh-init-headless-config | A | 2026-08-31 21:00 | PASS | PR #827 (installer answers landed in the losing config file); retargeted to the .example.env template by PR #833, then to oh.json by PR #887 | +| oh-init-scaffold | A | 2026-08-31 21:00 | PASS | issue #531 Phase 2 | +| oh-lifecycle-surface | A | 2026-08-31 21:00 | PASS | issue #881 — the Makefile is retired and `oh` is the only front door | +| oh-npm-package | A | 2026-08-31 21:00 | PASS | npm publish path for the standalone `oh` CLI (@mifune/openharness) — alternative to get-oh.sh | +| oh-payload-manifest | A | 2026-08-31 21:00 | PASS | issue #531 follow-on (.oh payload manifest — oh update ships a declared allowlist) | +| oh-sandbox-image-mode | A | 2026-08-31 21:00 | PASS | conversation 2026-07-05 (basic Docker deployment — prebuilt-image mode) | +| oh-shipped-repo-overridable | A | 2026-08-31 21:00 | PASS | issue #531 follow-on (de-hardcode residual — shipped .oh shell scripts keep the upstream repo overridable) | +| oh-standalone-lifecycle | A | 2026-08-31 21:00 | PASS | issue #564 | +| oh-update | A | 2026-08-31 21:00 | PASS | issue #531 Phase 3 (oh update — upgrade only the .oh control plane) | +| operator-config-guard | A | 2026-08-31 21:00 | PASS | operator directives 2026-08-06 (.config/ and settings.local.json are operator-only) | +| pnpm-audit-ci-gate | A | 2026-08-31 21:00 | PASS | issue #171 — pnpm security audits must run in CI | +| post-bridge-publish-confirmation | A | 2026-08-31 21:00 | PASS | #523 — post-bridge live publishing requires an explicit final confirmation gate | +| prd-output-path-contract | A | 2026-08-31 21:00 | PASS | retro lesson 2026-06-19 | +| prompt-miner-schema-compat | A | 2026-08-31 21:00 | PASS | issue #253 — prompt-miner JSONL schema-drift guard | +| prompt-miner-symlink-entrypoint | A | 2026-08-31 21:00 | PASS | issue #663 — prompt-miner engine no-ops via the documented .claude/skills symlink | +| prompt-miner-weakness-record | A | 2026-08-31 21:00 | PASS | issue #580 — prompt-miner weakness-record (WH-xxx) cluster output | +| protected-path-deletion | A | 2026-08-31 21:00 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-001) — the critique gate was deleted, | +| protected-paths-resolve | A | 2026-08-31 21:00 | PASS | issue #753 — .claude/protected-paths.txt named 7 paths that did not exist. | +| registry-portability-gate | A | 2026-08-31 21:00 | PASS | issue #758 | +| registry-portability | A | 2026-08-31 21:00 | SKIPPED | issue #758 | +| retro-deterministic-contract | A | 2026-08-31 21:00 | PASS | issue #443 — /retro deterministic output and self-contained helper contract | +| rl-delegation-write-worker | A | 2026-08-31 21:00 | PASS | retro lesson 2026-06-10 (rl-delegation) #57 | +| rlm-context-budget | A | 2026-08-31 21:00 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-006 | +| runtime-preflight-gate | A | 2026-08-31 21:00 | PASS | issue #806 § B1 (open sandbox.substrate vs sandbox.runtime selector); | +| sandbox-boot-guard-ci | A | 2026-08-31 21:00 | PASS | issue #449 (sandbox image build CI guard) 2026-06-19; | +| sandbox-node-base | A | 2026-08-31 21:00 | PASS | openharness#878 — oh as the only front door, T0 sandbox base image | +| skill-paths | A | 2026-08-31 21:00 | PASS | issue #43 — stale path references; extended by issue #69 — apps/->packages/ rename guard; extended by issue #870 — deleted .oh/agents/advisor.md | +| skills-dir-clean | A | 2026-08-31 21:00 | PASS | conversation 2026-06-29 — Pi parses every top-level `.md` in the skills | +| skills-task-tool-coupling | A | 2026-08-31 21:00 | PASS | council review 2026-08-29 (issue #886) — /delegate instructed Claude-Code-only | +| skills-vendored | A | 2026-08-31 21:00 | PASS | absorb .mifune submodule into .oh — the skills/agents/hooks pack is vendored | +| slack-admin-command-surface | A | 2026-08-31 21:00 | PASS | issue #354 — Slack bridge docs must distinguish Pi /msg-bridge commands from Slack DM admin text handlers | +| spec-family-contract | A | 2026-08-31 21:00 | PASS | issue #265; spec-simplification issue #816; workflow authority issue #854 | +| spec-ready-finalization | A | 2026-08-31 21:00 | PASS | issue #134; spec-simplification issue #816; workflow authority issue #854 | +| ste-checker-contract | A | 2026-08-31 21:00 | PASS | issue #750 PR audit — the /ste checker had four fail-open paths (unclosed | +| submitted-by-trailers | A | 2026-08-31 21:00 | PASS | conversation 2026-06-12 (commit attribution trailers); the single-owner | +| sync-skill-contract | A | 2026-08-31 21:00 | PASS | issue #331 — /sync dispatcher skill (bidirectional origin↔upstream sync) | +| t3-headless-launch | A | 2026-08-31 21:00 | PASS | issue #858 — /t3 launched a bare `npx --yes t3`, which is the local GUI and | +| tailscale-tool-boundary | A | 2026-08-31 21:00 | PASS | issue #858 — Tailscale mobile access for T3 Code. There is no tailnet, no | +| tool-catalog-boundary | A | 2026-08-31 21:00 | PASS | agent-browser's exclusion from the harness catalog (#821) and the | +| version-parity | A | 2026-08-31 21:00 | PASS | conversation 2026-08-29 — the oh CLI became the only lifecycle door, so its | +| weigh-scorer-contract | A | 2026-08-31 21:00 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-003 (2026-06-27) | +| wiki-readme-index | A | 2026-08-31 21:00 | PASS | issue #132 — wiki README index drift guard | +| workflow-boundaries | A | 2026-08-31 21:00 | PASS | conversation 2026-06-19 (workflow consolidation, issue #259); authority moved to /spec in issue #854 | +| worktrees-layout | A | 2026-08-31 21:00 | PASS | issue #872 | diff --git a/.oh/evals/probes/t3-headless-launch.sh b/.oh/evals/probes/t3-headless-launch.sh new file mode 100755 index 00000000..8d51ad6e --- /dev/null +++ b/.oh/evals/probes/t3-headless-launch.sh @@ -0,0 +1,129 @@ +#!/usr/bin/env bash +# tier: A +# source: issue #858 — /t3 launched a bare `npx --yes t3`, which is the local GUI and +# never prints a pairing URL, and docs claimed T3 listens on 0.0.0.0:3773. +# Mobile access has to come from the tailnet, not from a wide bind, and the +# server has to survive the operator's terminal going away. +# desc: .oh/skills/t3/scripts/t3-code.sh launches the headless `t3 serve` inside +# `tmux new-session -d`, maps --tailscale to --tailscale-serve, offers `t3 pair`, +# checks the Node floor, never binds 0.0.0.0, and refuses the tailscale path with +# an actionable error when the tailscale binary is absent. +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)"; cd "$ROOT" +SCRIPT=".oh/skills/t3/scripts/t3-code.sh" + +[ -f "$SCRIPT" ] || { echo "SKIPPED: $SCRIPT absent" >&2; exit 2; } + +BASH_BIN="$(command -v bash)" +WORK="$(mktemp -d)" +trap 'rm -rf "$WORK"' EXIT + +missing=() + +mapfile -t invocations < <(grep -oE 'npx --yes t3([[:space:]]+[a-z-]+)?' "$SCRIPT" | sort -u) +if ((${#invocations[@]} == 0)); then + missing+=("$SCRIPT: no 'npx --yes t3' invocation at all") +fi +for inv in "${invocations[@]}"; do + case "$inv" in + "npx --yes t3 serve"|"npx --yes t3 pair") ;; + *) missing+=("$SCRIPT: '$inv' is not a headless subcommand — bare 't3' is the local GUI and never prints a pairing URL") ;; + esac +done +grep -qF 'npx --yes t3 serve' "$SCRIPT" \ + || missing+=("$SCRIPT: no 'npx --yes t3 serve' — the headless server is how a phone pairs") +grep -qF 'npx --yes t3 pair' "$SCRIPT" \ + || missing+=("$SCRIPT: no 'npx --yes t3 pair' — a second device cannot be added without a restart") +grep -qF -- '--tailscale-serve' "$SCRIPT" \ + || missing+=("$SCRIPT: --tailscale is not mapped to t3's --tailscale-serve") +grep -qE 'tmux new-session -d' "$SCRIPT" \ + || missing+=("$SCRIPT: the server is not launched with 'tmux new-session -d' — it would die with the terminal") +grep -qF '0.0.0.0' "$SCRIPT" \ + && missing+=("$SCRIPT: mentions 0.0.0.0 — T3 Code must stay on loopback and be reached through the tailnet") + +stub_absent="$WORK/stub-absent" +mkdir -p "$stub_absent" +printf '#!/bin/sh\necho v22.16.0\n' > "$stub_absent/node" +printf '#!/bin/sh\nexit 0\n' > "$stub_absent/npx" +printf '#!/bin/sh\nexit 1\n' > "$stub_absent/tmux" +chmod 0755 "$stub_absent"/* + +if command -v tailscale >/dev/null 2>&1 && PATH="$stub_absent" command -v tailscale >/dev/null 2>&1; then + echo "SKIPPED: a real tailscale binary is unavoidable on PATH; the absent-binary branch cannot be exercised" >&2 + exit 2 +fi + +set +e +doctor_out="$(env -i PATH="$stub_absent" "$BASH_BIN" "$SCRIPT" doctor --tailscale 2>&1)" +doctor_code=$? +set -e +if ((doctor_code == 0)); then + missing+=("$SCRIPT: 'doctor --tailscale' succeeded with no tailscale binary on PATH — the preflight is not load-bearing") +fi +grep -qF 'oh tool install tailscale' <<<"$doctor_out" \ + || missing+=("$SCRIPT: 'doctor --tailscale' does not name 'oh tool install tailscale' as the fix (got: ${doctor_out//$'\n'/ })") + +stub_oldnode="$WORK/stub-oldnode" +mkdir -p "$stub_oldnode" +printf '#!/bin/sh\necho v22.15.0\n' > "$stub_oldnode/node" +printf '#!/bin/sh\nexit 0\n' > "$stub_oldnode/npx" +printf '#!/bin/sh\nexit 0\n' > "$stub_oldnode/tmux" +chmod 0755 "$stub_oldnode"/* + +set +e +oldnode_out="$(env -i PATH="$stub_oldnode" "$BASH_BIN" "$SCRIPT" doctor 2>&1)" +oldnode_code=$? +set -e +if ((oldnode_code == 0)); then + missing+=("$SCRIPT: 'doctor' accepted Node v22.15.0 — the Node floor is not enforced at runtime") +fi +grep -qF 'v22.15.0' <<<"$oldnode_out" \ + || missing+=("$SCRIPT: 'doctor' on Node v22.15.0 does not report the offending version (got: ${oldnode_out//$'\n'/ })") +grep -qF '^22.16 || ^23.11 || >=24.10' <<<"$oldnode_out" \ + || missing+=("$SCRIPT: 'doctor' on Node v22.15.0 does not name the supported range (got: ${oldnode_out//$'\n'/ })") + +stub_present="$WORK/stub-present" +mkdir -p "$stub_present" +printf '#!/usr/bin/env bash\necho v22.16.0\n' > "$stub_present/node" +printf '#!/usr/bin/env bash\nexit 0\n' > "$stub_present/npx" +printf '#!/usr/bin/env bash\necho "{\\"BackendState\\":\\"Running\\"}"\n' > "$stub_present/tailscale" +cat > "$stub_present/tmux" <<'STUB' +#!/usr/bin/env bash +case "${1:-}" in + has-session) [ -f "$T3_PROBE_STATE/session" ] ;; + new-session) shift; printf '%s\n' "$*" > "$T3_PROBE_STATE/launch"; : > "$T3_PROBE_STATE/session" ;; + capture-pane) echo "pairing url: https://box.example-tailnet.ts.net/pair?token=probe" ;; + kill-session) rm -f "$T3_PROBE_STATE/session" ;; + *) : ;; +esac +STUB +chmod 0755 "$stub_present"/* + +state="$WORK/state" +mkdir -p "$state" +set +e +T3_PROBE_STATE="$state" PATH="$stub_present:$PATH" "$BASH_BIN" "$SCRIPT" start --tailscale \ + --session t3-probe --log "$WORK/t3-probe.log" >"$WORK/start.out" 2>&1 +start_code=$? +set -e +if ((start_code != 0)); then + missing+=("$SCRIPT: 'start --tailscale' failed under stubbed tmux/node/npx/tailscale (${start_code}): $(tr '\n' ' ' < "$WORK/start.out")") +elif [ ! -f "$state/launch" ]; then + missing+=("$SCRIPT: 'start --tailscale' never reached 'tmux new-session -d'") +else + launch="$(cat "$state/launch")" + grep -qE '^-d -s t3-probe ' <<<"$launch" \ + || missing+=("$SCRIPT: the tmux session is not detached and named (got: $launch)") + grep -qF 'npx --yes t3 serve --tailscale-serve' <<<"$launch" \ + || missing+=("$SCRIPT: 'start --tailscale' does not launch 'npx --yes t3 serve --tailscale-serve' (got: $launch)") + grep -qF '0.0.0.0' <<<"$launch" \ + && missing+=("$SCRIPT: the launch command binds 0.0.0.0 (got: $launch)") +fi + +if ((${#missing[@]})); then + printf 'REGRESSION: %s\n' "${missing[@]}" >&2 + exit 1 +fi + +echo "PASS: /t3 launches detached 't3 serve', maps --tailscale to --tailscale-serve, offers 't3 pair', and never binds 0.0.0.0" >&2 diff --git a/.oh/evals/probes/tailscale-tool-boundary.sh b/.oh/evals/probes/tailscale-tool-boundary.sh new file mode 100755 index 00000000..66c78f96 --- /dev/null +++ b/.oh/evals/probes/tailscale-tool-boundary.sh @@ -0,0 +1,154 @@ +#!/usr/bin/env bash +# tier: A +# source: issue #858 — Tailscale mobile access for T3 Code. There is no tailnet, no +# auth key, and no phone in CI, so the acceptance criterion "a phone outside +# the tailnet cannot reach the backend" cannot be executed. It is discharged +# structurally instead: the sandbox gains no capability, no tun device and no +# published port, the boot path installs a pinned checksummed binary without +# ever joining a tailnet, and no Funnel command or reusable auth key ships. +# #908 additionally proved a root-installed tool is unusable from inside the +# sandbox: commands/tool.ts uses stdio:"inherit", so a root install becomes an +# interactive `sudo` and /etc/sudoers.d/sandbox has no NOPASSWD. +# desc: the Tailscale optional tool stays a zero-privilege, zero-exposure install — +# entrypointGuard (not buildArg) ground truth, version and both sha256 pins +# agreeing between the entrypoint and the tool catalog, no cap_add/devices/ +# privileged/3773 in any compose file, no tailscaled or `tailscale up` on boot, +# no Funnel, no committed auth key. +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)"; cd "$ROOT" +ENTRY=".devcontainer/entrypoint.sh" +DOCKERFILE=".devcontainer/Dockerfile" +CATALOG=".oh/cli/src/lib/tools/catalog.ts" + +for f in "$ENTRY" "$DOCKERFILE" "$CATALOG"; do + [ -f "$f" ] || { echo "SKIPPED: $f absent" >&2; exit 2; } +done + +shopt -s nullglob +COMPOSE=(.devcontainer/docker-compose*.yml) +if ((${#COMPOSE[@]} == 0)); then + echo "SKIPPED: no .devcontainer/docker-compose*.yml to check" >&2 + exit 2 +fi + +missing=() + +grep -qF 'INSTALL_TAILSCALE' "$ENTRY" \ + || missing+=("$ENTRY: no INSTALL_TAILSCALE guard — the tool catalog's ground truth moved") +grep -qF 'INSTALL_TAILSCALE' "$DOCKERFILE" \ + && missing+=("$DOCKERFILE: INSTALL_TAILSCALE appeared — a Dockerfile guard means the catalog field must be buildArg, not entrypointGuard") + +mapfile -t pins < <(grep -oE 'tailscale_[0-9]+\.[0-9]+\.[0-9]+_' "$ENTRY" | sed 's/^tailscale_//; s/_$//' | sort -u) +if ((${#pins[@]} == 0)); then + missing+=("$ENTRY: no pinned tailscale__ tarball — the install is unpinned") +elif ((${#pins[@]} > 1)); then + missing+=("$ENTRY: per-architecture version pins disagree (${pins[*]})") +else + grep -qF "tailscale_${pins[0]}_" "$CATALOG" \ + || missing+=("$CATALOG: version pin disagrees with $ENTRY (${pins[0]})") +fi + +grep -qF 'sha256sum -c' "$ENTRY" \ + || missing+=("$ENTRY: no 'sha256sum -c' verification of the Tailscale tarball") + +mapfile -t entry_shas < <(grep -iE 'tailscale|ts_sha' "$ENTRY" | grep -oE '\b[0-9a-f]{64}\b' | sort -u) +mapfile -t catalog_shas < <(grep -iE 'tailscale' "$CATALOG" | grep -oE '\b[0-9a-f]{64}\b' | sort -u) +if ((${#entry_shas[@]} < 2)); then + missing+=("$ENTRY: expected a sha256 literal per supported architecture, found ${#entry_shas[@]}") +elif [ "${entry_shas[*]}" != "${catalog_shas[*]}" ]; then + missing+=("$CATALOG: sha256 literals disagree with $ENTRY (entrypoint: ${entry_shas[*]:-none} / catalog: ${catalog_shas[*]:-none})") +fi + +if grep -qE '(^|[;&|]|&&|\|\||\bthen |\bdo |\bexec |\bnohup |\bsudo )[[:space:]]*("?[^[:space:]"]*/)?tailscaled\b' "$ENTRY"; then + missing+=("$ENTRY: invokes tailscaled — the boot path installs the binary, it never starts the daemon") +fi +if grep -qE '\btailscale[[:space:]]+(-[^[:space:]]+[[:space:]]+)*up([[:space:]]|$)' "$ENTRY"; then + missing+=("$ENTRY: runs 'tailscale up' — joining a tailnet must stay an explicit human act") +fi +if grep -qE 'TS_AUTHKEY|--authkey' "$ENTRY"; then + missing+=("$ENTRY: reads an auth key — the documented path is interactive 'tailscale up'") +fi + +for f in "${COMPOSE[@]}"; do + if grep -qE '^[[:space:]]*cap_add:' "$f"; then + missing+=("$f: cap_add — userspace networking needs no capability grant") + fi + if grep -qE '^[[:space:]]*devices:' "$f"; then + missing+=("$f: devices: — /dev/net/tun must never be handed to the sandbox") + fi + if grep -qE '^[[:space:]]*privileged:[[:space:]]*true' "$f"; then + missing+=("$f: privileged: true — the Tailscale path grants no privilege") + fi + published=$(awk ' + /^[[:space:]]*ports:[[:space:]]*$/ { indent = match($0, /[^ ]/); inports = 1; next } + inports { + if ($0 ~ /^[[:space:]]*$/) next + if (match($0, /[^ ]/) <= indent) { inports = 0; next } + if ($0 ~ /3773/) print + } + ' "$f") + if [ -n "$published" ]; then + missing+=("$f: publishes 3773 — T3 Code must stay on container loopback and be reachable only through the tailnet") + fi +done + +if ! grep -qE 'id:[[:space:]]*"tailscale"' "$CATALOG"; then + missing+=("$CATALOG: no tool entry with id \"tailscale\"") +else + entry_block=$(awk '/id:[[:space:]]*"tailscale"/{found=1} found{print; if (/\}\)/) exit}' "$CATALOG") + grep -qE 'kind:[[:space:]]*"opt-in"' <<<"$entry_block" \ + || missing+=("$CATALOG: the tailscale entry is not kind \"opt-in\" — it must never install by default") + grep -qE 'toolKey:[[:space:]]*"tailscale"' <<<"$entry_block" \ + || missing+=("$CATALOG: the tailscale entry has no toolKey \"tailscale\" — the oh.json opt-in is not wired") + grep -qE 'entrypointGuard:[[:space:]]*"INSTALL_TAILSCALE"' <<<"$entry_block" \ + || missing+=("$CATALOG: the tailscale entry has no entrypointGuard \"INSTALL_TAILSCALE\"") + # tailscaled runs fine unprivileged with --tun=userspace-networking, so nothing + # here needs root. A root install would hang `oh tool install tailscale` on a + # sudo password prompt no agent can answer, and would put the binary in an + # image-layer path that no running sandbox can upgrade and every container + # recreate discards. + grep -qE 'installUser:[[:space:]]*"root"' <<<"$entry_block" \ + && missing+=("$CATALOG: the tailscale entry installs as root — commands/tool.ts uses stdio:\"inherit\", so that becomes an interactive \`sudo\`, and /etc/sudoers.d/sandbox has no NOPASSWD") + grep -qE 'installUser:[[:space:]]*"sandbox"' <<<"$entry_block" \ + || missing+=("$CATALOG: the tailscale entry does not declare installUser \"sandbox\"") + grep -qF 'NPM_USER_PREFIX' <<<"$entry_block" \ + || missing+=("$CATALOG: the tailscale entry does not install into NPM_USER_PREFIX — the binary must land in the home mount, not an image-layer path") + grep -qE '/usr/local/bin/tailscale' <<<"$entry_block" \ + && missing+=("$CATALOG: the tailscale entry writes to /usr/local/bin — that needs root and is discarded on container recreate") +fi + +# The boot path must agree with the catalog: same destination, same user. +grep -qE 'install -m 0755 [^ ]+ /usr/local/bin/tailscaled?' "$ENTRY" \ + && missing+=("$ENTRY: installs Tailscale into /usr/local/bin — the catalog installs it into the home mount, and an image-layer copy is lost on every recreate") + +# tailscaled's default control socket is /var/run/tailscale/tailscaled.sock and +# t3-code.sh calls a bare `tailscale status`. Only root can create that directory, +# so the entrypoint must, and it must not be gated behind INSTALL_TAILSCALE — +# `oh tool install tailscale` is supposed to leave the tool usable immediately. +socket_dir_line=$(grep -nE 'install -d .*-o sandbox .*/var/run/tailscale' "$ENTRY" | head -1 | cut -d: -f1) +if [ -z "$socket_dir_line" ]; then + missing+=("$ENTRY: never creates /var/run/tailscale — tailscaled's default socket path is unwritable, so a bare \`tailscale status\` cannot work") +else + guard_line=$(grep -nE '^if \[ "\$\{INSTALL_TAILSCALE:-false\}" = "true" \]' "$ENTRY" | head -1 | cut -d: -f1) + if [ -n "$guard_line" ] && [ "$socket_dir_line" -gt "$guard_line" ]; then + missing+=("$ENTRY: creates /var/run/tailscale inside the INSTALL_TAILSCALE guard — a later \`oh tool install tailscale\` would then need a reboot before the socket path exists") + fi +fi + +funnel=$(grep -rniE '\bfunnel\b' .oh/skills/t3 .devcontainer 2>/dev/null || true) +if [ -n "$funnel" ]; then + missing+=("Funnel appears in .oh/skills/t3 or .devcontainer — Funnel is public exposure and the harness ships no Funnel command") +fi + +authkeys=$(grep -rIlE 'tskey[-](auth|client|api)[-]' . --exclude-dir=.git 2>/dev/null || true) +if [ -n "$authkeys" ]; then + missing+=("a Tailscale auth key literal is committed in: $(tr '\n' ' ' <<<"$authkeys")") +fi + +if ((${#missing[@]})); then + printf 'REGRESSION: %s\n' "${missing[@]}" >&2 + exit 1 +fi + +echo "PASS: Tailscale installs pinned and checksummed into the home mount as the sandbox user, grants no capability, publishes no port, joins no tailnet on boot, and ships no Funnel or auth key" >&2 diff --git a/.oh/scripts/install.sh b/.oh/scripts/install.sh index 765835fd..654b9e80 100644 --- a/.oh/scripts/install.sh +++ b/.oh/scripts/install.sh @@ -119,7 +119,7 @@ Env vars: sandbox is never overwritten) INSTALL_HERMES=true Enable an optional agent non-interactively. Also: INSTALL_OPENCODE, INSTALL_GROK_BUILD, - INSTALL_AGENT_BROWSER + INSTALL_AGENT_BROWSER, INSTALL_TAILSCALE DOCKER_SOCKET=true Mount the host Docker socket into the sandbox non-interactively. OFF by default (socket access is effectively host root). Otherwise you're prompted (TTY), @@ -440,6 +440,7 @@ _opt_install HERMES install.hermes "Hermes — Nous self-improving a _opt_install OPENCODE install.opencode "OpenCode — OpenAI-OAuth terminal agent" _opt_install GROK_BUILD install.grokBuild "Grok Build — xAI terminal agent" _opt_install AGENT_BROWSER install.agentBrowser "agent-browser + Chromium (~1 GB)" +_opt_install TAILSCALE install.tailscale "Tailscale — private remote access for T3 Code (userspace)" banner "Host Docker socket (off by default)" if [ "$(_config_get access.dockerSocket)" = "true" ]; then @@ -493,6 +494,7 @@ printf " oh harness install hermes — Hermes agent (then 'hermes s printf " oh harness install opencode — OpenCode terminal agent\n" printf " oh harness install grok-build — xAI Grok Build\n" printf " oh tool install agent-browser — headless Chromium for screenshots / previews (~1 GB)\n" +printf " oh tool install tailscale — private tailnet access for remote / mobile T3 Code\n" printf " (each flips the matching install.* flag in oh.json)\n" printf "\n" printf " ${CYAN}Messaging gateways${NC}\n" diff --git a/.oh/scripts/migrate-harness-yaml.sh b/.oh/scripts/migrate-harness-yaml.sh index c45b073b..f2279094 100755 --- a/.oh/scripts/migrate-harness-yaml.sh +++ b/.oh/scripts/migrate-harness-yaml.sh @@ -28,6 +28,7 @@ BEGIN { envmap["install.grok_build"] = "INSTALL_GROK_BUILD" envmap["install.hermes"] = "INSTALL_HERMES" envmap["install.agent_browser"] = "INSTALL_AGENT_BROWSER" + envmap["install.tailscale"] = "INSTALL_TAILSCALE" envmap["hermes.dashboard"] = "HERMES_DASHBOARD" envmap["hermes.dashboard_port"] = "HERMES_DASHBOARD_PORT" envmap["ssh.enabled"] = "SANDBOX_SSH" @@ -141,6 +142,7 @@ _field_for() { INSTALL_GROK_BUILD) printf 'install.grokBuild boolean\n' ;; INSTALL_HERMES) printf 'install.hermes boolean\n' ;; INSTALL_AGENT_BROWSER) printf 'install.agentBrowser boolean\n' ;; + INSTALL_TAILSCALE) printf 'install.tailscale boolean\n' ;; HERMES_DASHBOARD) printf 'hermesDashboard.enabled boolean\n' ;; HERMES_DASHBOARD_PORT) printf 'hermesDashboard.port number\n' ;; SANDBOX_SSH) printf 'access.ssh boolean\n' ;; diff --git a/.oh/skills/t3/SKILL.md b/.oh/skills/t3/SKILL.md index b2a3b9f5..d3980496 100644 --- a/.oh/skills/t3/SKILL.md +++ b/.oh/skills/t3/SKILL.md @@ -1,47 +1,62 @@ --- name: t3 description: | - Start, inspect, or stop T3 Code (`npx t3`) in the Open Harness sandbox. - Use this for the browser-based T3 Code harness on port 3773, including tmux - launch, pairing URL discovery, logs, status, and shutdown. T3 Code wraps an - already-authenticated Claude Code, Codex, or OpenCode backend. - TRIGGER when: user asks to run T3 Code, start `npx t3`, open the T3 browser - UI, get the T3 pairing URL, check T3 Code status/logs, or stop T3 Code. -argument-hint: "[start|status|url|logs|stop|attach|help] [--session ] [--port ] [--log ]" + Start, inspect, pair, or stop T3 Code in the Open Harness sandbox, locally or + over a private Tailscale tailnet for phone access. Use this for the headless + `t3 serve` harness on port 3773, including tmux launch, preflight diagnosis, + pairing URL discovery, minting a pairing URL for a second device, logs, + status, and shutdown. T3 Code wraps an already-authenticated Claude Code, + Codex, or OpenCode backend. + TRIGGER when: user asks to run T3 Code, start `t3 serve`, reach T3 Code from a + phone or another machine, pair a device, get the T3 pairing URL, diagnose why + T3 Code or the tailnet will not start, check T3 Code status/logs, or stop + T3 Code. +argument-hint: "[start|status|url|pair|logs|stop|attach|doctor|help] [--session ] [--port ] [--log ] [--tailscale] [--tailscale-port

]" allowed-tools: Bash, Read disable-model-invocation: true --- # T3 Code -Run T3 Code as a sandbox-local browser harness. Treat it as a long-running -process: start it in tmux, report the pairing URL, and leave the session running -for the operator to open at `localhost:3773` through their host/VS Code port -forwarding. +Run T3 Code as a long-running sandbox process: start `t3 serve` in tmux, report +the pairing URL, and leave the session running. The operator opens it at +`localhost:3773` through host/VS Code port forwarding, or — with `--tailscale` — +from a phone on the same private tailnet. + +`npx t3` (no subcommand) is the desktop GUI launcher and is not what this skill +runs. Headless and remote access use `npx t3 serve`; a new device is added to an +already-running server with `npx t3 pair`. ## Arguments Arguments received: `$ARGUMENTS` - `ACTION`: optional first positional argument; default `start` - - `start`: start T3 Code in tmux, or report the existing session + - `start`: run the preflight, then start `t3 serve` in tmux, or report the existing session - `status`: show whether the tmux session is running and print recent output - `url`: print the latest pairing URL from the log/pane if present + - `pair`: mint a fresh one-time pairing URL against the running server, without restarting it - `logs`: print recent log lines - `stop`: kill the tmux session - `attach`: print the attach command; do not attach from an agent run + - `doctor`: run the preflight checks and print one actionable line per failure - `help`: print script usage - `--session`: tmux session name; default `agent-t3code` -- `--port`: expected UI port; default `3773` +- `--port`: expected T3 Code port; default `3773` - `--log`: log file; default `/tmp/.log` +- `--tailscale`: publish over Tailscale Serve on the tailnet (`t3 serve --tailscale-serve`, `t3 pair --tailscale`) +- `--tailscale-port`: alternate Tailscale Serve HTTPS port; default `443` If the user does not specify an action, use `start`. ## Preconditions -Before launch, remind the user that T3 Code is a UI over an existing provider. +T3 pairing is **not** provider auth. Pairing a phone does not log any provider +in, and a provider login does not pair a device. They are two separate +credentials on two separate lifecycles. + At least one backend must already be installed and authenticated inside the -sandbox: +sandbox before T3 Code is useful: ```bash claude # complete OAuth on first launch @@ -49,9 +64,19 @@ codex login opencode auth login ``` -Do not treat T3 Code itself as replacing provider login. It starts a browser UI -and prints a single-use pairing URL such as -`http://localhost:3773/pair#token=...`. +T3 Code itself prints a single-use pairing URL such as +`http://localhost:3773/pair#token=...`, or an +`https://..ts.net/...` URL in Tailscale mode. Treat that URL +and its token as a secret: never paste it into an issue, a PR, a tracked file, +or a persistent log. + +The preflight (`doctor`, and the first step of `start`) checks: + +- `tmux`, `npx`, and `node` on `PATH` +- Node satisfies `^22.16 || ^23.11 || >=24.10` (the T3 server's `engines.node`) +- with `--tailscale`: the `tailscale` binary is installed, `tailscaled` is + running and reachable, and the tailnet backend state is `Running` +- the T3 port answers on loopback when a session is already up ## Run @@ -61,27 +86,58 @@ Run the bundled script with the received arguments: bash "$CLAUDE_SKILL_DIR/scripts/t3-code.sh" $ARGUMENTS ``` -The script verifies `tmux` and `npx`, starts `npx --yes t3` in tmux for the -`start` action, waits briefly for a pairing URL, and prints follow-up commands. +Launch commands the script emits, verbatim: + +| Invocation | Command | +| --- | --- | +| `/t3 start` | `npx --yes t3 serve` | +| `/t3 start --tailscale` | `npx --yes t3 serve --tailscale-serve` | +| `/t3 start --tailscale --tailscale-port 8443` | `npx --yes t3 serve --tailscale-serve --tailscale-serve-port 8443` | +| `/t3 pair` | `npx --yes t3 pair` | +| `/t3 pair --tailscale` | `npx --yes t3 pair --tailscale` | + +The server always runs under the sandbox tmux convention +(`tmux new-session -d -s agent-t3code '... 2>&1 | tee /tmp/agent-t3code.log'`), +so it survives a terminal disconnect. It stays bound to container loopback; the +skill never binds T3 Code to a public interface. + +For the phone-side recipe and the tailnet session layout, read +[`references/tailscale-mobile.md`](references/tailscale-mobile.md). For the tmux +rules, read [`references/sandbox-processes.md`](references/sandbox-processes.md). ## Report After `start`, report: -- tmux session name -- log path +- tmux session name and log path +- the exact launch command that was used - pairing URL if found, otherwise the command to inspect logs -- local UI URL, normally `http://localhost:3773` -- reminder: if running over SSH/remote host, use VS Code port forwarding or see - `docs/connecting.md` - -For public sharing beyond the attached host, use `/cloudflared 3773` only after -confirming the operator wants a public bearer URL. +- local URL, normally `http://localhost:3773` +- in Tailscale mode, the tailnet HTTPS port and that the URL is the node's + MagicDNS name +- that `/t3 pair` adds a second device without restarting the server +- reminder: over SSH/remote host without a tailnet, use VS Code port forwarding + or see `docs/connecting.md` +- revocation paths: + - `t3 auth` — issue, inspect, and revoke T3 sessions and credentials + - `tailscale serve --https=443 off` — withdraw the Serve mapping (it persists + until you do) + - `tailscale logout`, or delete the node in the Tailscale admin console — + remove the device from the tailnet + +Never echo a pairing URL into a file the repository tracks. + +For public sharing beyond a private tailnet, use `/cloudflared 3773` only after +confirming the operator wants a public bearer URL. Tailscale is private; a +Cloudflared tunnel is not. ## Examples ```bash /t3 +/t3 doctor --tailscale +/t3 start --tailscale +/t3 pair --tailscale /t3 status /t3 logs --session agent-t3code /t3 stop diff --git a/.oh/skills/t3/references/sandbox-processes.md b/.oh/skills/t3/references/sandbox-processes.md index d175665b..d1905f2d 100644 --- a/.oh/skills/t3/references/sandbox-processes.md +++ b/.oh/skills/t3/references/sandbox-processes.md @@ -25,7 +25,7 @@ Format: `-` (kebab-case inside each segment). |----------|---------|---------| | `app-` | `app-docs`, `app-api` | User dev servers | | `cloudflared-` | `cloudflared-3000` | Cloudflare tunnels for shared previews | -| `agent-` | `agent-watcher`, `agent-batch` | Headless / long-running agent processes. Interactive CLIs (`claude`, `codex`, `opencode`) are normally foreground in a terminal or VS Code, not detached in tmux. | +| `agent-` | `agent-watcher`, `agent-batch`, `agent-t3code`, `agent-tailscaled` | Headless / long-running agent processes, including the T3 Code server (`t3 serve`) and the userspace `tailscaled` that fronts it. Interactive CLIs (`claude`, `codex`, `opencode`) are normally foreground in a terminal or VS Code, not detached in tmux. | | `client-` | `client-slack-pi`, `client-slack-hermes`, `client-discord` | External-surface clients that bridge an in-sandbox agent to a third-party UI | | `cron-` | `cron-heartbeat`, `cron-cleanup-tasks-0613-1805`, `cron-system` | Scheduled cron jobs and the cron runtime. | diff --git a/.oh/skills/t3/references/tailscale-mobile.md b/.oh/skills/t3/references/tailscale-mobile.md new file mode 100644 index 00000000..51ae9f2f --- /dev/null +++ b/.oh/skills/t3/references/tailscale-mobile.md @@ -0,0 +1,98 @@ +# T3 Code on a phone, over a private tailnet + +Reaching T3 Code from a phone without exposing it publicly. The tailnet node is +the **sandbox container**, running `tailscaled` in userspace-networking mode as +the unprivileged `sandbox` user. No container capability is added, no host port +is published, and T3 Code stays bound to container loopback. Tailscale Serve +inside the container terminates HTTPS on the tailnet and proxies to +`127.0.0.1:3773`. + +A device that is not on the tailnet cannot reach the backend at all. + +## Sessions + +Both processes follow the sandbox tmux convention (see +[`sandbox-processes.md`](sandbox-processes.md)). + +| Session | Process | Log | +| --- | --- | --- | +| `agent-tailscaled` | `tailscaled` in userspace-networking mode | `/tmp/agent-tailscaled.log` | +| `agent-t3code` | `npx --yes t3 serve --tailscale-serve` | `/tmp/agent-t3code.log` | + +Neither is started by the container entrypoint. Joining a tailnet is an explicit +human act. + +## One-time setup + +```bash +oh tool install tailscale + +tmux new-session -d -s agent-tailscaled \ + 'tailscaled --tun=userspace-networking \ + --statedir=$HOME/.tailscale 2>&1 | tee /tmp/agent-tailscaled.log' + +tailscale up +``` + +`tailscale up` prints a login URL. Open it in a browser and approve the node. +State lives in `$HOME/.tailscale`, which is a named Docker volume, so a container +recreate does not force a re-login. + +## Start the server and pair the phone + +```bash +/t3 doctor --tailscale # confirm Node, binary, daemon, and tailnet state first +/t3 start --tailscale +``` + +The server prints a connection string, a one-time pairing token, a pairing URL, +and a QR code. Install the T3 Code mobile app, then scan the QR code or paste the +`https://..ts.net/...` URL. + +The pairing token is one-time. To add a second device later, do **not** restart +the server: + +```bash +/t3 pair --tailscale +``` + +## Ports + +Tailscale Serve defaults to HTTPS on **443**. Use another port with: + +```bash +/t3 start --tailscale --tailscale-port 8443 +``` + +The hosted `https://app.t3.codes` page cannot talk to a plain-HTTP backend +(mixed content). Tailscale Serve gives you real HTTPS, so it works with both the +hosted page and the native mobile app. + +## Teardown and revocation + +```bash +/t3 stop # kill the T3 Code session +tailscale serve --https=443 off # withdraw the Serve mapping; it persists until you do +t3 auth # inspect and revoke T3 sessions and credentials +tailscale logout # remove this node's tailnet identity +``` + +Delete the node in the Tailscale admin console to revoke it from the other side. + +## Troubleshooting + +| Symptom | Fix | +| --- | --- | +| `tailscale not found in PATH` | `oh tool install tailscale`, and set `install.tailscale: true` in `oh.json` so a rebuild keeps it | +| `tailscaled is not running` | start the `agent-tailscaled` session above; check `/tmp/agent-tailscaled.log` | +| backend state is `NeedsLogin` / `Stopped` | run `tailscale up` interactively and finish the browser login | +| Node does not satisfy the range | the T3 server needs `^22.16 \|\| ^23.11 \|\| >=24.10`; raise the Node pin in `.devcontainer/Dockerfile` and rebuild | +| port 3773 not answering | read `/t3 logs`; the provider backend may have failed to start | +| phone shows the page but cannot connect | confirm the phone is on the same tailnet and the MagicDNS name resolves | + +## Secrets + +Pairing URLs and tokens are credentials. They belong in the terminal and the +`/tmp` session log only — never in a tracked file, an issue, a PR body, or a +persistent log. Tailscale auth keys are never printed or committed; the supported +path is interactive `tailscale up`. diff --git a/.oh/skills/t3/scripts/t3-code.sh b/.oh/skills/t3/scripts/t3-code.sh index d051cd5a..087fc5d8 100755 --- a/.oh/skills/t3/scripts/t3-code.sh +++ b/.oh/skills/t3/scripts/t3-code.sh @@ -3,21 +3,25 @@ set -euo pipefail usage() { cat <<'USAGE' -Usage: t3-code.sh [start|status|url|logs|stop|attach|help] [options] +Usage: t3-code.sh [start|status|url|pair|logs|stop|attach|doctor|help] [options] Actions: - start Start T3 Code in tmux, or report the existing session (default) + start Run the preflight, then start `t3 serve` in tmux, or report the existing session (default) status Show session status and recent output url Print the latest pairing URL found in the log/pane + pair Mint a fresh pairing URL against the already-running server (no restart) logs Print recent log lines stop Kill the tmux session attach Print the tmux attach command (does not attach) + doctor Run the preflight checks and report actionable errors help Show this help Options: - --session tmux session name (default: agent-t3code) - --port expected T3 Code UI port (default: 3773) - --log log path (default: /tmp/.log) + --session tmux session name (default: agent-t3code) + --port expected T3 Code port (default: 3773) + --log log path (default: /tmp/.log) + --tailscale publish over Tailscale Serve (HTTPS on the tailnet) + --tailscale-port

alternate Tailscale Serve HTTPS port (default: 443) USAGE } @@ -25,10 +29,14 @@ ACTION="start" SESSION="agent-t3code" PORT="3773" LOG="" +TAILSCALE="false" +TAILSCALE_PORT="" +TAILSCALED_SESSION="agent-tailscaled" +NODE_RANGE="^22.16 || ^23.11 || >=24.10" if [[ $# -gt 0 ]]; then case "$1" in - start|status|url|logs|stop|attach|help) + start|status|url|pair|logs|stop|attach|doctor|help) ACTION="$1" shift ;; @@ -52,6 +60,16 @@ while [[ $# -gt 0 ]]; do LOG="$2" shift 2 ;; + --tailscale) + TAILSCALE="true" + shift + ;; + --tailscale-port) + [[ $# -ge 2 ]] || { echo "ERROR: --tailscale-port requires a value" >&2; exit 2; } + TAILSCALE_PORT="$2" + TAILSCALE="true" + shift 2 + ;; -h|--help) ACTION="help" shift @@ -72,6 +90,102 @@ has_session() { tmux has-session -t "$SESSION" 2>/dev/null } +node_version_ok() { + local raw major minor + raw="$(node -v 2>/dev/null || true)" + raw="${raw#v}" + [[ "$raw" =~ ^([0-9]+)\.([0-9]+)\. ]] || return 1 + major="${BASH_REMATCH[1]}" + minor="${BASH_REMATCH[2]}" + if (( major == 22 )); then (( minor >= 16 )); return; fi + if (( major == 23 )); then (( minor >= 11 )); return; fi + if (( major == 24 )); then (( minor >= 10 )); return; fi + (( major > 24 )) +} + +tailscaled_hint() { + cat </dev/null 2>&1; then + echo "ERROR: tmux not found in PATH; every long-running sandbox process runs in tmux" >&2 + failures=$((failures + 1)) + fi + + if ! command -v npx >/dev/null 2>&1; then + echo "ERROR: npx not found in PATH; install Node.js in the sandbox image" >&2 + failures=$((failures + 1)) + fi + + if ! command -v node >/dev/null 2>&1; then + echo "ERROR: node not found in PATH; T3 Code requires Node ${NODE_RANGE}" >&2 + failures=$((failures + 1)) + elif ! node_version_ok; then + echo "ERROR: Node $(node -v 2>/dev/null) does not satisfy ${NODE_RANGE}; raise the harness Node pin in .devcontainer/Dockerfile and rebuild with 'oh rebuild'" >&2 + failures=$((failures + 1)) + fi + + if [[ "$TAILSCALE" == "true" ]]; then + if ! command -v tailscale >/dev/null 2>&1; then + echo "ERROR: tailscale not found in PATH; run 'oh tool install tailscale'" >&2 + failures=$((failures + 1)) + elif ! tailscale status --json >/dev/null 2>&1; then + echo "ERROR: tailscaled is not running or its socket is unreachable" >&2 + tailscaled_hint >&2 + failures=$((failures + 1)) + else + local state + state="$(tailscale status --json 2>/dev/null | sed -n 's/.*"BackendState"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/p' | head -n 1)" + if [[ "$state" != "Running" ]]; then + echo "ERROR: tailnet backend state is '${state:-unknown}', not 'Running'; run 'tailscale up' interactively and complete the browser login" >&2 + failures=$((failures + 1)) + fi + fi + fi + + if [[ "$ACTION" == "pair" || "$ACTION" == "doctor" ]]; then + if command -v tmux >/dev/null 2>&1 && has_session; then + if command -v curl >/dev/null 2>&1 && ! curl -fsS -o /dev/null --max-time 3 "http://127.0.0.1:${PORT}/" 2>/dev/null; then + echo "ERROR: T3 Code port ${PORT} is not answering on loopback; check '/t3 logs --session ${SESSION}' or restart with '/t3 stop' then '/t3 start'" >&2 + failures=$((failures + 1)) + fi + fi + fi + + if (( failures > 0 )); then + return 1 + fi + echo "Preflight OK (Node $(node -v 2>/dev/null), tailscale mode: ${TAILSCALE})" + return 0 +} + +serve_argv() { + local -a argv=(npx --yes t3 serve) + if [[ "$TAILSCALE" == "true" ]]; then + argv+=(--tailscale-serve) + if [[ -n "$TAILSCALE_PORT" ]]; then + argv+=(--tailscale-serve-port "$TAILSCALE_PORT") + fi + fi + printf '%s\n' "${argv[*]}" +} + +pair_argv() { + local -a argv=(npx --yes t3 pair) + if [[ "$TAILSCALE" == "true" ]]; then + argv+=(--tailscale) + if [[ -n "$TAILSCALE_PORT" ]]; then + argv+=(--tailscale-serve-port "$TAILSCALE_PORT") + fi + fi + printf '%s\n' "${argv[*]}" +} + recent_output() { if has_session; then tmux capture-pane -t "$SESSION" -p -S -160 2>/dev/null || true @@ -83,7 +197,7 @@ recent_output() { pairing_url() { recent_output \ - | grep -Eoi 'https?://[^[:space:]]*(pairingUrl|pair|token)[^[:space:]]*|pairingUrl[^[:space:]]*[[:space:]]*[:=][[:space:]]*https?://[^[:space:]]+' \ + | grep -Eoi 'https?://[^[:space:]]*\.ts\.net[^[:space:]]*|https?://[^[:space:]]*(pairingUrl|pair|token)[^[:space:]]*|pairingUrl[^[:space:]]*[[:space:]]*[:=][[:space:]]*https?://[^[:space:]]+' \ | sed 's/^[Pp]airing[Uu]rl[^:=]*[:=][[:space:]]*//' \ | tail -n 1 } @@ -93,13 +207,17 @@ print_summary() { url="$(pairing_url || true)" echo "T3 Code session: $SESSION" echo "Log: $LOG" - echo "UI: http://localhost:${PORT}" + echo "Local: http://localhost:${PORT}" + if [[ "$TAILSCALE" == "true" ]]; then + echo "Tailnet: served over HTTPS on port ${TAILSCALE_PORT:-443} at the node's MagicDNS name" + fi if [[ -n "$url" ]]; then echo "Pairing URL: $url" else echo "Pairing URL: not found yet" echo "Inspect: tmux capture-pane -t ${SESSION} -p | grep -iE 'pair|token|url'" fi + echo "Pair another device: /t3 pair$([[ "$TAILSCALE" == "true" ]] && echo ' --tailscale')" echo "Attach: tmux attach -t ${SESSION}" echo "Stop: tmux kill-session -t ${SESSION}" } @@ -109,6 +227,10 @@ case "$ACTION" in usage exit 0 ;; + doctor) + doctor || exit 1 + exit 0 + ;; attach) if has_session; then echo "Attach from an interactive terminal with: tmux attach -t ${SESSION}" @@ -144,11 +266,34 @@ case "$ACTION" in echo "$url" else echo "No pairing URL found yet for session '${SESSION}'." - echo "Try: /t3 logs --session ${SESSION}" + echo "Mint a fresh one against the running server with: /t3 pair" + echo "Or inspect: /t3 logs --session ${SESSION}" exit 1 fi exit 0 ;; + pair) + command -v tmux >/dev/null 2>&1 || { echo "ERROR: tmux not found in PATH" >&2; exit 1; } + if ! has_session; then + echo "ERROR: T3 Code session '${SESSION}' is not running; 'pair' needs a live server. Start it with: /t3 start" >&2 + exit 1 + fi + doctor >/dev/null || exit 1 + echo "Minting a one-time pairing URL against the running server (no restart)." + echo "Command: $(pair_argv)" + if [[ "$TAILSCALE" == "true" ]]; then + if [[ -n "$TAILSCALE_PORT" ]]; then + npx --yes t3 pair --tailscale --tailscale-serve-port "$TAILSCALE_PORT" + else + npx --yes t3 pair --tailscale + fi + else + npx --yes t3 pair + fi + echo + echo "Treat that URL and token as a secret. Do not paste it into an issue, PR, or tracked file." + exit 0 + ;; status) command -v tmux >/dev/null 2>&1 || { echo "ERROR: tmux not found in PATH" >&2; exit 1; } if has_session; then @@ -165,23 +310,27 @@ case "$ACTION" in exit 0 ;; start) - command -v tmux >/dev/null 2>&1 || { echo "ERROR: tmux not found in PATH" >&2; exit 1; } - command -v npx >/dev/null 2>&1 || { echo "ERROR: npx not found in PATH" >&2; exit 1; } + doctor || { echo "Preflight failed. Fix the errors above, then run '/t3 doctor' again." >&2; exit 1; } - echo "T3 Code requires at least one authenticated backend: Claude Code, Codex, or OpenCode." - echo "If none is authenticated yet, run one of: claude | codex login | opencode auth login" + echo "T3 Code is a UI over a provider backend. Provider auth (Claude Code, Codex, OpenCode) is" + echo "separate from T3 pairing: pairing a phone does NOT log you into a provider, and a provider" + echo "login does NOT pair a device. Authenticate a backend first with one of:" + echo " claude | codex login | opencode auth login" echo if has_session; then echo "T3 Code session already running: $SESSION" + echo "To add a device without restarting, use: /t3 pair$([[ "$TAILSCALE" == "true" ]] && echo ' --tailscale')" print_summary exit 0 fi mkdir -p "$(dirname "$LOG")" - : > "$LOG" - tmux new-session -d -s "$SESSION" "npx --yes t3 2>&1 | tee $(printf '%q' "$LOG")" + (umask 077; : > "$LOG") + chmod 600 "$LOG" 2>/dev/null || true + tmux new-session -d -s "$SESSION" "$(serve_argv) 2>&1 | tee $(printf '%q' "$LOG")" echo "Started T3 Code in tmux session: $SESSION" + echo "Launch command: $(serve_argv)" for _ in $(seq 1 40); do if ! has_session; then @@ -196,6 +345,12 @@ case "$ACTION" in done print_summary + echo + echo "Revoke: 't3 auth' manages T3 sessions and credentials." + if [[ "$TAILSCALE" == "true" ]]; then + echo "Withdraw the Serve mapping: tailscale serve --https=${TAILSCALE_PORT:-443} off" + echo "Remove the device: tailscale logout, or delete the node in the Tailscale admin console." + fi ;; *) echo "ERROR: unknown action: $ACTION" >&2 diff --git a/CHANGELOG.md b/CHANGELOG.md index f79e816f..a5122aa1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -29,6 +29,10 @@ Update policy and release automation live in [`/git`](.claude/skills/git/SKILL.m - Add `oh tool list --defaults` and generalize the boot provisioner over both catalogs as `provision-defaults.sh` (`OH_PROVISION_DEFAULTS`) ([#906](https://github.com/mifunedev/openharness/issues/906)). - Fix `oh harness install` hanging on a sudo password prompt inside the sandbox: every harness now installs as the sandbox user, so no install path needs root ([#908](https://github.com/mifunedev/openharness/issues/908)). - Add `skills-task-tool-coupling.sh`, a tier-A probe holding the canonical skill pack and the sandbox in agreement about the Claude-Code-only task tools ([#886](https://github.com/mifunedev/openharness/issues/886)). +- Add `install.tailscale` and `oh tool install tailscale`, an opt-in userspace Tailscale client installed into `~/.local/bin` as the sandbox user, granting no capability ([#858](https://github.com/mifunedev/openharness/issues/858)). + +### Changed +- `/t3` launches the headless `t3 serve` instead of the local-GUI `t3`, and gains `--tailscale`, a `pair` action for a second device, and a `doctor` preflight ([#858](https://github.com/mifunedev/openharness/issues/858)). ### Fixed - Give the five probes that shipped without one a `# source:` header, so every probe records the lesson it closes and the `source` column in `RESULTS.md` is fully populated ([#889](https://github.com/mifunedev/openharness/issues/889)). diff --git a/docs/configuration.md b/docs/configuration.md index a4def2e1..39468d1c 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -57,7 +57,8 @@ to; `—` means the field is consumed by the `oh` CLI itself and never rendered. All off by default. `oh harness install ` flips the matching field and installs into the running sandbox with no rebuild. The four harness fields map to `oh harness` names: `opencode`, `grok-build`, `hermes`. -`agentBrowser` is not a harness — `oh tool install agent-browser` manages it. +`agentBrowser` and `tailscale` are not harnesses — `oh tool install agent-browser` +and `oh tool install tailscale` manage them. | Field | Type | Default | Compose variable | What it does | | --- | --- | --- | --- | --- | @@ -65,6 +66,7 @@ to `oh harness` names: `opencode`, `grok-build`, `hermes`. | `install.grokBuild` | boolean | `false` | `INSTALL_GROK_BUILD` | Install the Grok Build CLI into `~/.local` at boot. `oh harness install grok-build` sets it and installs now. | | `install.hermes` | boolean | `false` | `INSTALL_HERMES` | Install the Hermes CLI into `~/.local` at boot and enable its runtime wiring (skill vendoring, `auth.json`). | | `install.agentBrowser` | boolean | `false` | `INSTALL_AGENT_BROWSER` | Install agent-browser and Chromium (about 1 GB). | +| `install.tailscale` | boolean | `false` | `INSTALL_TAILSCALE` | Install the Tailscale client for private remote access (userspace networking; no container capabilities). | ### Access diff --git a/docs/connecting.md b/docs/connecting.md index 1da84d35..eba61a79 100644 --- a/docs/connecting.md +++ b/docs/connecting.md @@ -100,12 +100,181 @@ This is NOT the default; you opt in explicitly. Be aware that binding to `0.0.0. **2. External tunnel** -The harness ships no built-in tunnel tool. For public access, bring your own: `cloudflared`, `ngrok`, `tailscale funnel`, or an nginx/Caddy reverse proxy. Start the tunnel inside the sandbox in a named tmux session (see [tmux conventions](#tmux-session-naming)). +For **public** access, use `cloudflared` (shipped in the image, see the `/cloudflared` skill), `ngrok`, or an nginx/Caddy reverse proxy. Start the tunnel inside the sandbox in a named tmux session (see [tmux conventions](#tmux-session-naming)). + +For **private** access from your own devices — including a phone — use Tailscale instead of a public tunnel. See [Mobile access over Tailscale](#mobile-access-over-tailscale). Tailscale Funnel would make a tailnet service public; it is never enabled by default and the harness ships no Funnel command. **3. Direct SSH + nginx multi-tenant routing** To SSH straight into the container — and to route several tenants' containers through one nginx reverse proxy on a single VM — enable the opt-in `sshd` overlay. See [Integrations → SSH](/docs/integrations/sshd). +## Mobile access over Tailscale + +This is the supported path for reaching T3 Code from a phone, and the supported path for reaching it from a remote sandbox at all without publishing a port. Access stays **private to your tailnet**. + +### Where Tailscale runs, and why + +`tailscaled` runs **inside the sandbox container**, in userspace-networking mode, as the unprivileged `sandbox` user. The container is the tailnet node. + +- No `NET_ADMIN`, no `/dev/net/tun`, no `privileged: true`, no host socket mount. Userspace networking needs none of them, and Tailscale Serve is fully supported in that mode. +- **No host port is published.** T3 Code stays on container loopback `127.0.0.1:3773`. Tailscale Serve inside the container proxies tailnet HTTPS to that loopback address. A device outside the tailnet has nothing to reach. +- The only compose change is one environment variable (`INSTALL_TAILSCALE`). Node identity and daemon state live in `/home/sandbox/.tailscale`, inside the single `/home/sandbox` mount, so the node does not re-authenticate on every container recreate without any per-tool volume. +- Because the container is the node, the MagicDNS name your phone saved does not change when you move the workspace to another VM. + +Installing the binary does **not** join a tailnet. The entrypoint never runs `tailscaled` and never runs `tailscale up`. Joining is an explicit human act. + +### Prerequisites + +On the remote host: + +- The sandbox is running (`oh ps`). +- Node in the sandbox satisfies T3 Code's range `^22.16 || ^23.11 || >=24.10` (`node -v`). +- A provider is authenticated in the sandbox (`claude`, `codex login`, or `opencode auth login`). +- A Tailscale account and a tailnet you control. + +On the phone: + +- The Tailscale app, signed in to the **same tailnet**. +- The T3 Code mobile app. + +The phone and the sandbox must share one tailnet. There is no other reachability path. + +### Step 1 — Install Tailscale in the sandbox + +```bash +oh tool install tailscale +``` + +This persists `install.tailscale: true` in the tracked `oh.json` so the opt-in survives container recreation, and installs the binary into a running sandbox when one is up. It is idempotent. + +If the sandbox was not running, the flag is persisted only. Run `oh sandbox` to recreate the container and let the entrypoint install the binary. No rebuild of the image is required — the install is an entrypoint step gated on `INSTALL_TAILSCALE`, not a build layer. Nothing about networking activates until you start the daemon in the next step. + +Check the state at any time: + +```bash +oh tool status tailscale +``` + +### Step 2 — Start the daemon + +Run it in a named tmux session so it survives a disconnect: + +```bash +tmux new-session -d -s agent-tailscaled \ + 'tailscaled --tun=userspace-networking \ + --statedir=$HOME/.tailscale' +``` + +### Step 3 — Join the tailnet + +```bash +tailscale up +``` + +`tailscale up` prints a login URL. Open it in a browser and approve the node. This is the supported setup: an interactive human login. **Never commit a reusable Tailscale auth key**, and never print one into a log or a tracked file. + +Confirm the node is up and note its MagicDNS name: + +```bash +tailscale status +``` + +### Step 4 — Start T3 Code in Tailscale mode + +```text +/t3 start --tailscale +``` + +This runs `npx --yes t3 serve --tailscale-serve` in the `agent-t3code` tmux session. T3 Code configures Tailscale Serve on HTTPS 443 and advertises `https://..ts.net/`. It prints a pairing URL and a QR code. + +Add `--tailscale-port 8443` if HTTPS 443 is already claimed on that node. + +Reprint the current pairing URL at any time: + +```text +/t3 url +``` + +### Step 5 — Pair the phone + +1. Open the T3 Code mobile app. +2. Scan the QR code printed in the `agent-t3code` session, or paste the `https://..ts.net/...` pairing URL. +3. The app binds to the running server. + +The pairing token is single-use. The paired session persists, so the phone reconnects later without pairing again — as long as the phone is on the tailnet and the server is running. + +### Adding another device + +Do not restart the server. Mint a fresh token against the running one: + +```text +/t3 pair --tailscale +``` + +Then scan or paste the new URL on the second device. + +### Lifecycle + +Two tmux sessions carry this setup: + +| Session | Process | +|---------|---------| +| `agent-tailscaled` | the Tailscale daemon | +| `agent-t3code` | `npx t3 serve --tailscale-serve` | + +Both survive a shell or SSH disconnect. Inspect them with `tmux ls`, attach with `tmux attach -t `, detach with `Ctrl-b d`. `/t3 status` and `/t3 logs` read the T3 session without attaching. + +After a container recreate, the tailnet identity is still in `~/.tailscale` inside the home mount, but the daemon is not running: repeat steps 2 and 4. `tailscale up` is not needed again unless you logged out. + +### Revoking access + +T3 pairing credentials and Tailscale device access are **separate**. Revoke both. + +```bash +t3 auth # inspect and revoke T3 sessions and pairing credentials +tailscale serve --https=443 off # withdraw the Serve mapping (it persists until you do) +tailscale logout # sign the sandbox node out of the tailnet +``` + +Then delete the device in the Tailscale admin console. Revoking a phone's own tailnet access is done there too. + +### Troubleshooting + +| Symptom | Cause | Fix | +|---------|-------|-----| +| `/t3 start --tailscale` reports Tailscale missing | binary not installed | `oh tool install tailscale`, then `oh sandbox` if the sandbox was down | +| `tailscale status` fails to reach the daemon | `tailscaled` not running | repeat step 2; check `tmux ls` for `agent-tailscaled` | +| Backend state is not `Running` / "logged out" | node never joined, or was logged out | `tailscale up` and complete the browser login | +| No `ts.net` URL in the T3 output | Serve was not configured | confirm `tailscale status` is `Running`, then restart with `/t3 start --tailscale` | +| Serve still answers after T3 Code stops | the Serve mapping persists | `tailscale serve --https=443 off` | +| T3 Code refuses to start with an engine error | Node outside `^22.16 \|\| ^23.11 \|\| >=24.10` | check `node -v`; the sandbox image pins Node 22.x, so upgrade past 22.16 | +| Phone cannot reach the URL at all | phone not on the tailnet | sign the phone's Tailscale app in to the same tailnet and confirm it appears in `tailscale status` | +| Phone is on the tailnet but the URL times out | Serve mapping on a different port, or the server stopped | `tailscale serve status`; `/t3 status` | +| `https://app.t3.codes` cannot connect | mixed content: the hosted page is HTTPS and a plain-HTTP tailnet endpoint is blocked | use `--tailscale-serve` (HTTPS) or the native mobile app | + +`/t3 doctor` runs the Tailscale, Node, and tooling checks in one pass and prints an actionable line per failure. + +### Operator-owned fallback: Tailscale on the host + +If your host policy forbids a daemon inside the container, you can instead run `tailscaled` on the remote host and route the sandbox port through it. This is **not** the supported path and the harness does not manage it: + +- It requires publishing `3773` from the container to the host, which widens exposure on any multi-tenant or internet-facing VM. +- `oh tool install tailscale` installs and versions the binary *inside* the sandbox, so `oh tool status tailscale` would not describe the host daemon. +- The tailnet node becomes the host, so the MagicDNS name changes when you move the workspace to another machine. + +You own the configuration and the exposure in that layout. + +### Tailscale versus Cloudflared + +| | Tailscale Serve | Cloudflared | +|---|---|---| +| Audience | your tailnet only | anyone with the URL | +| Auth | tailnet device identity | none — the URL is the bearer credential | +| Use it for | phones, remote laptops, your own devices | a public preview shared with someone off your tailnet | +| Command | `/t3 start --tailscale` | `/cloudflared 3773` | + +Cloudflared remains the right tool for public preview sharing. It is not the mobile path. Tailscale **Funnel** — which would make a tailnet service public — is never enabled by default and the harness ships no Funnel command. + ## tmux session naming All long-running processes inside the sandbox run in named tmux sessions. The naming convention is `-`: @@ -113,7 +282,7 @@ All long-running processes inside the sandbox run in named tmux sessions. The na | Category | Example | Purpose | |----------|---------|---------| | `client-` | `client-slack-pi`, `client-discord` | External-surface clients bridging an in-sandbox agent | -| `agent-` | `agent-watcher`, `agent-batch` | Headless / long-running agent processes (interactive CLIs are foreground, not tmux) | +| `agent-` | `agent-watcher`, `agent-batch`, `agent-t3code`, `agent-tailscaled` | Headless / long-running agent processes (interactive CLIs are foreground, not tmux) | | `app-` | `app-api` | Dev servers | For the full convention see [`.oh/skills/t3/references/sandbox-processes.md`](https://github.com/mifunedev/openharness/blob/development/.oh/skills/t3/references/sandbox-processes.md). @@ -148,11 +317,15 @@ T3 Code is not preinstalled; the first invocation downloads it via `npx`. If an Manual terminal fallback: ```bash -tmux new-session -d -s agent-t3code 'npx t3 2>&1 | tee /tmp/agent-t3code.log' +tmux new-session -d -s agent-t3code 'npx --yes t3 serve 2>&1 | tee /tmp/agent-t3code.log' tmux attach -t agent-t3code ``` -Watch the session output — T3 Code prints a pairing URL. Open that URL in your browser to complete the browser-based pairing step. After pairing, the UI is available at `localhost:3773` on your laptop (via VSCode auto-forwarding). +Watch the session output — T3 Code prints a pairing URL and a QR code. Open that URL in your browser to complete pairing. After pairing, the UI is available at `localhost:3773` on your laptop (via VSCode auto-forwarding). + +To pair a second device later, run `/t3 pair` — do not restart the server. + +To reach T3 Code from a phone, follow [Mobile access over Tailscale](#mobile-access-over-tailscale) instead. Detach from the tmux session without stopping it: `Ctrl-b d`. @@ -170,6 +343,6 @@ If a port is missing, confirm the tmux session is running (`tmux ls`) and that y ## Quick-reference: reach `localhost` from your laptop -| App | Container port | Laptop URL (VSCode attached) | -|-----|---------------|------------------------------| -| T3 Code UI | 3773 | `http://localhost:3773` | +| App | Container port | Laptop URL (VSCode attached) | Tailnet URL (Tailscale Serve) | +|-----|---------------|------------------------------|-------------------------------| +| T3 Code UI | 3773 (loopback only) | `http://localhost:3773` | `https://..ts.net/` | diff --git a/docs/harnesses/overview.md b/docs/harnesses/overview.md index 16b7bc15..03afb587 100644 --- a/docs/harnesses/overview.md +++ b/docs/harnesses/overview.md @@ -117,8 +117,8 @@ Web UI on `http://localhost:3773` over an already-authenticated provider. Prefer Manual terminal fallback: ```bash -tmux new-session -d -s harness-t3code 'npx t3 2>&1 | tee /tmp/harness-t3code.log' -tmux capture-pane -t harness-t3code -p | grep -i pairingUrl +tmux new-session -d -s agent-t3code 'npx --yes t3 serve 2>&1 | tee /tmp/agent-t3code.log' +tmux capture-pane -t agent-t3code -p | grep -iE 'pair|token|url' ``` Open the printed pairing URL (`http://localhost:3773/pair#token=…`) in the Simple Browser tab. Full setup: [T3 Code](./t3code.md). diff --git a/docs/harnesses/t3code.md b/docs/harnesses/t3code.md index 4f2f0a58..933deb37 100644 --- a/docs/harnesses/t3code.md +++ b/docs/harnesses/t3code.md @@ -4,24 +4,28 @@ title: "T3 Code" # T3 Code -T3 Code is a web-based coding agent harness from Theo Browne / ping.gg. Unlike the other harnesses listed here, T3 Code is **not a CLI you talk to in a terminal** — it runs a local web UI on port `3773` and orchestrates an underlying provider (Claude Code, Codex, or OpenCode) as the actual coding agent. You bring your own already-authenticated provider and T3 Code drives it from a browser. +T3 Code is a web-based coding agent harness from Theo Browne / ping.gg. Unlike the other harnesses listed here, T3 Code is **not a CLI you talk to in a terminal** — it runs a web UI backed by a server on port `3773` and orchestrates an underlying provider (Claude Code, Codex, or OpenCode) as the actual coding agent. You bring your own already-authenticated provider and T3 Code drives it from a browser or from the T3 Code mobile app. ## Purpose -Use T3 Code when you want a browser UI over the same providers the other harnesses run from the terminal — multi-thread sessions, conversational history, and a UI for review/approval flows, while reusing whatever provider auth you already have set up in the sandbox. +Use T3 Code when you want a browser or phone UI over the same providers the other harnesses run from the terminal — multi-thread sessions, conversational history, and a UI for review/approval flows, while reusing whatever provider auth you already have set up in the sandbox. -## Install +## Requirements -T3 Code is **not preinstalled** in the sandbox image. The `/t3` skill starts it on demand via `npx --yes t3` and keeps it in tmux: +T3 Code's server package requires Node `^22.16 || ^23.11 || >=24.10`. The sandbox base image is `node:22-trixie-slim`, so a 22.x older than 22.16 is the realistic failure. Check before you launch: -```text -/t3 +```bash +node -v ``` -For a direct shell launch, run: +`/t3 doctor` runs the same check and reports an actionable error when the version is out of range. -```bash -npx t3 +## Install + +T3 Code is **not preinstalled** in the sandbox image. The `/t3` skill starts it on demand via `npx --yes t3 serve` and keeps it in tmux: + +```text +/t3 ``` The first launch downloads the package and starts the server. No global install is required, but you can install it for faster subsequent starts: @@ -36,6 +40,20 @@ Verify: npx t3 --version ``` +## Which command to run + +| Command | Use it when | What it does | +|---------|-------------|--------------| +| `npx t3` | You are on the machine with the browser and want the normal local launch | Starts the server and opens the local UI flow | +| `npx t3 serve` | The server runs headless in the sandbox and you connect from elsewhere | Starts the server only, prints the connection string, a pairing token, a pairing URL, and a QR code | +| `npx t3 serve --tailscale-serve` | You want a phone or another tailnet device to reach the server privately | Same as `serve`, plus configures Tailscale Serve on HTTPS 443 and advertises `https://..ts.net/` | +| `npx t3 pair` | A server is already running and you want to add a device | Mints a fresh one-time pairing token without restarting the server | +| `npx t3 pair --tailscale` | A server is already running and the new device is on the tailnet | Publishes over Tailscale Serve HTTPS and pairs through the MagicDNS URL | + +Inside the sandbox, prefer the `/t3` skill over calling `npx` by hand — it owns the tmux session and the preflight checks. + +Use `--tailscale-serve-port ` (on `serve`) or `--tailscale-serve-port ` (on `pair --tailscale`) when HTTPS 443 is already taken on that tailnet node. `pair --tailscale` also accepts `--ttl` and `--base-dir`. + ## Authentication T3 Code currently supports Codex, Claude, and OpenCode as backends. Install and authenticate **at least one provider** in the sandbox before launching T3 Code (see the per-provider pages for details): @@ -44,42 +62,81 @@ T3 Code currently supports Codex, Claude, and OpenCode as backends. Install and - **[Claude Code](./claude-code.md)**: run `claude` and complete OAuth - **[OpenCode](./opencode.md)**: run `opencode auth login` -T3 Code itself uses a **pairing-URL** auth model: on first start it logs a one-time URL like `http://localhost:3773/pair#token=...` to stdout. Open that URL in your browser to bind the UI to the running server. The token is single-use; restart T3 Code to mint a fresh one. +Provider authentication is **separate** from T3 pairing. Pairing binds a client (browser or phone) to your running T3 server; it grants no provider credentials and does not replace `claude` / `codex login` / `opencode auth login`. + +T3 Code itself uses a **pairing-URL** auth model: on start it prints a one-time URL like `http://localhost:3773/pair#token=...` plus a QR code. Open the URL, or scan the QR from the T3 Code mobile app, to bind the client to the running server. The token is single-use. To add a second device, run `npx t3 pair` against the running server — **do not restart T3 Code**. + +Treat pairing URLs and tokens as secrets. Do not paste them into issues, pull requests, or chat. ## Run in tmux -Per [`.oh/skills/t3/references/sandbox-processes.md`](https://github.com/mifunedev/openharness/blob/development/.oh/skills/t3/references/sandbox-processes.md), long-running processes inside the sandbox go in named tmux sessions. T3 Code listens on `0.0.0.0:3773` so it can be reached from the host. Prefer the `/t3` skill when an agent is available: +Per [`.oh/skills/t3/references/sandbox-processes.md`](https://github.com/mifunedev/openharness/blob/development/.oh/skills/t3/references/sandbox-processes.md), long-running processes inside the sandbox go in named tmux sessions. T3 Code stays bound to **container loopback** (`127.0.0.1:3773`); the harness publishes no host port for it. Reach it through VSCode port forwarding, an SSH tunnel, or Tailscale Serve — see [Connecting to the Sandbox](/docs/connecting). + +Prefer the `/t3` skill when an agent is available: ```text -/t3 start # launch in tmux and print the pairing URL when available -/t3 status # inspect the tmux session and recent output -/t3 url # print the latest pairing URL found in logs -/t3 stop # stop the tmux session +/t3 doctor # preflight: tmux, npx, Node range, and Tailscale state +/t3 start # launch `npx t3 serve` in tmux and print the pairing URL +/t3 start --tailscale # launch `npx t3 serve --tailscale-serve` +/t3 status # inspect the tmux session and recent output +/t3 url # print the latest pairing URL found in logs +/t3 pair # mint a fresh pairing token for a running server +/t3 pair --tailscale # pair a new device through the MagicDNS HTTPS URL +/t3 stop # stop the tmux session ``` Manual terminal fallback: ```bash -tmux new-session -d -s agent-t3code 'npx t3 2>&1 | tee /tmp/agent-t3code.log' +tmux new-session -d -s agent-t3code 'npx --yes t3 serve 2>&1 | tee /tmp/agent-t3code.log' tmux capture-pane -t agent-t3code -p | grep -i pairingUrl ``` -Open the printed pairing URL in your host browser. Reattach to the session at any time: +Reattach to the session at any time: ```bash tmux attach -t agent-t3code ``` -If you need to share the T3 Code UI beyond your attached host session, use `/cloudflared 3773` to start a Cloudflared tunnel for the local port. +The session survives a shell or SSH disconnect. Detach with `Ctrl-b d`. + +## Mobile access over Tailscale + +`--tailscale-serve` configures Tailscale Serve on HTTPS **443** and advertises `https://..ts.net/`. The phone must be signed in to the **same tailnet** as the sandbox. Full end-to-end recipe, prerequisites, and troubleshooting: [Connecting → Mobile access over Tailscale](/docs/connecting#mobile-access-over-tailscale). + +The Serve mapping persists after T3 Code stops. Withdraw it explicitly: + +```bash +tailscale serve --https=443 off +``` + +Use `--tailscale-serve-port 8443` to publish on an alternate HTTPS port; withdraw it with `tailscale serve --https=8443 off`. + +## Revoking access + +Two independent credentials exist. Revoke both when you retire a device. + +```bash +t3 auth # issue, inspect, and revoke T3 sessions and pairing credentials +tailscale serve --https=443 off # withdraw the Serve mapping +tailscale logout # sign this node out of the tailnet +``` + +Remove the device from the tailnet in the Tailscale admin console as well — `tailscale logout` signs out the node, the admin console deletes it. + +## Sharing publicly + +Tailscale is the **private** path and the supported mobile path. If you need a genuinely public preview URL for someone who is not on your tailnet, use `/cloudflared 3773` instead. That is public bearer-URL exposure — anyone with the link reaches the port. Tailscale Funnel is **not** enabled by default and the harness ships no Funnel command. See [Security considerations](../security-considerations.md). ## Tips - T3 Code is a UI over the providers — installing T3 Code does **not** replace `claude login` / `codex login` / `opencode auth login`. Authenticate the provider first, then start T3 Code. -- The pairing token is regenerated on every server restart. Treat it as ephemeral — don't bookmark the URL. +- The hosted page at `https://app.t3.codes` is HTTPS, so it cannot talk to a plain-HTTP tailnet endpoint (mixed content). Use `--tailscale-serve`, which is HTTPS, or the native mobile app. - T3 Code uses Node's experimental SQLite at startup; the warning in the log is expected. ## Upstream documentation - [`pingdotgg/t3code` on GitHub](https://github.com/pingdotgg/t3code) +- [T3 Code remote access](https://github.com/pingdotgg/t3code/blob/main/docs/user/remote-access.md) [Connecting to the Sandbox](/docs/connecting) diff --git a/docs/installation.md b/docs/installation.md index 9de2ea32..11d4aece 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -314,6 +314,24 @@ Optional CLIs are excluded from the default image; `oh harness install ` f | Hermes | `hermes` | Nous Research's self-improving agent CLI | optional: `oh harness install hermes` | | Grok Build | `grok` | xAI's proprietary Grok Build CLI (`@xai-official/grok@0.2.39`, Node >=20) | optional: `oh harness install grok-build` | | agent-browser | `agent-browser` | Headless Chromium for web-capable agents | optional: `oh tool install agent-browser` | +| Tailscale | `tailscale` | Private tailnet access for remote/mobile T3 Code (userspace networking; no container capabilities) | optional: `oh tool install tailscale` | + +Two tools are **not** baked in and install on demand with `oh tool install `: +`agent-browser` and `tailscale`. `oh tool install` persists the opt-in in the +tracked `oh.json` (`install.agentBrowser`, `install.tailscale`) so it survives +container recreation, and installs into a running sandbox when one is up. Both +installs are idempotent. If no sandbox is running, only the flag is persisted — +run `oh sandbox` and the entrypoint installs the tool on boot. Neither needs an +image rebuild. + +Installing `tailscale` places the `tailscale` and `tailscaled` binaries in +`~/.local/bin` and nothing more. It starts no daemon and joins no tailnet. +Networking activates only when a human starts `tailscaled` in +userspace-networking mode and runs `tailscale up` interactively — see +[Connecting → Mobile access over Tailscale](connecting.md#mobile-access-over-tailscale). +Its node identity and daemon state live in `~/.tailscale`, inside the single +`/home/sandbox` mount, so the node does not re-authenticate on every container +recreate. ### Runtimes & package managers diff --git a/docs/security-considerations.md b/docs/security-considerations.md index 605772aa..ddae5963 100644 --- a/docs/security-considerations.md +++ b/docs/security-considerations.md @@ -148,6 +148,42 @@ expose to whichever trust level you choose. can't silently clobber another tenant's port. Setup + the nginx multi-tenant recipe: [Integrations → SSH](integrations/sshd.md). +- **Caveat 4 — the optional Tailscale tool (opt-in, private-by-default).** Installing + `tailscale` (`oh tool install tailscale`, persisted as `install.tailscale` in the + tracked `oh.json`) adds **no container capability**: `tailscaled` runs inside the + sandbox in **userspace-networking** mode as the unprivileged `sandbox` user, so + there is no `NET_ADMIN`, no `/dev/net/tun`, no `privileged: true`, and no host + socket mount. The only compose addition is one environment variable + (`INSTALL_TAILSCALE`); daemon state lives in `/home/sandbox/.tailscale`, + inside the single `/home/sandbox` mount. **No host port is published** — T3 Code stays on + container loopback `127.0.0.1:3773` and Tailscale Serve proxies tailnet HTTPS to + it, so a device outside the tailnet has nothing to reach. The posture: + - **Private tailnet only. Tailscale Funnel is never enabled by default and the + harness ships no Funnel command or flag.** Funnel would publish a tailnet + service to the internet; if you want that, you are configuring it yourself, + outside this tree. For a deliberately *public* preview, use `cloudflared` + instead and understand that the URL is the only credential. + - **Installation never joins a tailnet.** The entrypoint installs the binaries and + stops. It never runs `tailscaled` and never runs `tailscale up`. Joining is an + explicit interactive human act. + - **Never print or commit a reusable Tailscale auth key.** The documented and + supported setup is interactive `tailscale up` with a browser login. If an + operator insists on auth-key automation, the key is a secret and belongs in the + gitignored mode-`0600` root `.env` via the §1 secret channel — never in + `oh.json`, a script, or a log. + - **Pairing URLs and tokens are secrets.** T3 Code's pairing URL carries a + single-use token in its fragment. The `/t3` skill keeps the server log under + `/tmp/agent-t3code.log` and writes no URL into a tracked file. Do not paste a + pairing URL into an issue, a pull request, a commit message, or chat. + - **Two revocation paths, both required.** `t3 auth` issues, inspects, and revokes + T3 sessions and pairing credentials. `tailscale serve --https=443 off` withdraws + the Serve mapping, which otherwise persists after T3 Code stops. + `tailscale logout` signs the node out, and the Tailscale admin console deletes + the device. Revoking one does not revoke the other. + + Setup, lifecycle, and troubleshooting: [Connecting → Mobile access over + Tailscale](connecting.md#mobile-access-over-tailscale). + ## 5. Human merge gate / no auto-merge — **ENFORCED (process) · RECOMMENDED (hard gate)** No agent merges its own work to the trunk. diff --git a/oh.json b/oh.json index 91f571b9..c889f765 100644 --- a/oh.json +++ b/oh.json @@ -8,7 +8,8 @@ "opencode": false, "grokBuild": false, "hermes": false, - "agentBrowser": false + "agentBrowser": false, + "tailscale": false }, "access": { "ssh": false, From 159ccace6ca1aece579b484813359a9043826d1c Mon Sep 17 00:00:00 2001 From: Ryan Eggleston Date: Mon, 31 Aug 2026 15:40:42 -0600 Subject: [PATCH 11/23] skill: add a slop gate to /audit implementation (#913) The four correctness gates prove a change works. None can fail a change that works and is twice the size it needed to be. Gate 5 asks whether the diff can be smaller and still satisfy every acceptance criterion, and fails the audit while the answer is yes. Signals come from two new implementation-gates.sh modes: slop-metrics reports net lines, per-function CCN on changed TypeScript via uvx lizard, and a disclosed branch-point proxy for shell; simplicity-round reads the caller's round record. An unresolvable lizard reports unavailable rather than an empty clean result. The loop terminates by construction, not by agreement: a finding without a concrete smaller alternative is not a finding, and the loop ends at a cap of three rounds or on a round that fails to reduce netAdded, passing with SIMPLICITY-RESIDUAL for the operator. --- .oh/evals/RESULTS.md | 211 +++++++++--------- .oh/evals/probes/audit-slop-gate.sh | 75 +++++++ .oh/skills/audit/SKILL.md | 2 +- .oh/skills/audit/references/implementation.md | 70 +++++- .../audit/references/reviewer-evidence-doc.md | 6 +- .../audit/scripts/implementation-gates.sh | 57 ++++- .oh/skills/spec/references/execute.md | 31 ++- CHANGELOG.md | 2 + 8 files changed, 338 insertions(+), 116 deletions(-) create mode 100755 .oh/evals/probes/audit-slop-gate.sh diff --git a/.oh/evals/RESULTS.md b/.oh/evals/RESULTS.md index 5275d737..676304a7 100644 --- a/.oh/evals/RESULTS.md +++ b/.oh/evals/RESULTS.md @@ -6,110 +6,111 @@ probe id; git history is the time series.** Schema and exit-code semantics are i | probe | tier | last-run (UTC) | status | source | |-------|------|----------------|--------|--------| -| advisor-monitored-loop | A | 2026-08-31 21:00 | PASS | conversation 2026-06-19 (single-owner implementation workflow, issue #257) | -| agent-browser-cli | A | 2026-08-31 21:00 | PASS | retro lesson 2026-06-07 (agent-browser 0.8.5 CLI) | -| agents-identity-contract | A | 2026-08-31 21:00 | PASS | issue #854 — T3-style root identity, glossary, and skill-owned procedures | -| artifact-contract-audit | A | 2026-08-31 21:00 | PASS | issue #583/#645 — production /audit implementation Gate 1 behavior | -| audit-dispatcher-contract | A | 2026-08-31 21:00 | PASS | issue #645 — audit consolidation public taxonomy | -| audit-implementation-behavior | A | 2026-08-31 21:00 | PASS | issue #645 — implementation root/repo/browser behavior | -| audit-pr-acquire | A | 2026-08-31 21:00 | PASS | issue #645 — production PR acquisition behavior | -| audit-pr-classifier | A | 2026-08-31 21:00 | PASS | issue #645 — deterministic focused and queue PR classifier | -| audit-run-root-contract | A | 2026-08-31 21:00 | PASS | issue #645 — executable immutable audit root/run correlation | -| audit-shellcheck-coverage | A | 2026-08-31 21:00 | PASS | issue #645 — private audit scripts require release and CI lint coverage | -| audit-stale-references | A | 2026-08-31 21:00 | PASS | issue #645 — clean-breaking audit migration | -| boot-lint-glob | A | 2026-08-31 21:00 | PASS | issue #90, issue #120 | -| builder-skill-consolidation | A | 2026-08-31 21:00 | PASS | issue #643 — consolidate artifact builders behind one /builder dispatcher | -| capability-benchmark-schema | A | 2026-08-31 21:00 | PASS | issue #167 — capability benchmark instrument | -| cc-safety-net-wiring | A | 2026-08-31 21:00 | SKIPPED | .oh/tasks/cc-safety-net/prd.json US-007 2026-07-19 | -| changelog-entry-length | A | 2026-08-31 21:00 | PASS | conversation 2026-08-24 — CHANGELOG.md grew to 259KB of bullet prose because "one line" was unquantified | -| cleanup-tasks-scoped-guard | A | 2026-08-31 21:00 | PASS | issue #85 | -| cleanup-tasks-worktree-grooming | A | 2026-08-31 21:00 | PASS | issue #168; issue #327 | -| cli-publish-typecheck-scope | A | 2026-08-31 21:00 | PASS | release run 33271077312 — v0.5.0 pushed its GHCR image, then failed to publish | -| close-issues-on-development | A | 2026-08-31 21:00 | PASS | issue #841 (closing keywords never fire because the default branch is main) 2026-08-26 | -| codex-stale-response-retry | A | 2026-08-31 21:00 | PASS | issue #506 — Codex previous_response_not_found RCA | -| compose-config-path-parity | A | 2026-08-31 21:00 | PASS | PR #833 (remove harness.yaml — the wrapper and VS Code "Reopen in Container" paths must resolve the same service) 2026-08-26 | -| config-schema-parity | A | 2026-08-31 21:00 | PASS | PR #833 (one schema file — DOCKER_SOCKET, SANDBOX_SSH, OH_SANDBOX_IMAGE, OH_PULL_POLICY, SKIP_PNPM_INSTALL were consumed but undocumented); rewritten for the oh.json/secrets split by PR #887 | -| context-tier-size-budget | A | 2026-08-31 21:00 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-007) — the always-on tier was 85,256 B | -| cron-claude-codex-fallback | A | 2026-08-31 21:00 | PASS | conversation 2026-06-12 (default Codex fallback for crons) | -| cron-watchdog | A | 2026-08-31 21:00 | PASS | issues #130/#453 (cron runtime watchdog + legacy system-cron reaping) 2026-06-19 | -| crons-directory-guide | A | 2026-08-31 21:00 | PASS | issue #874 | -| curl-bash-safe-alternatives | A | 2026-08-31 21:00 | PASS | vet-run/vet integration — public curl|bash examples need review-first alternatives | -| datasets-schema | A | 2026-08-31 21:00 | PASS | issue #196 — .oh/evals/datasets verifiable trajectory corpus (Repo2RLEnv-inspired) | -| debugmcp-availability | A | 2026-08-31 21:00 | SKIPPED | issue #297 — DebugMCP MCP debug-server availability | -| default-provisioning | A | 2026-08-31 21:00 | PASS | #902 — `oh harness install` must work from inside the sandbox, where | -| delegate-model-effort-policy | A | 2026-08-31 21:00 | PASS | conversation 2026-07-11 (delegate model inheritance and thinking policy) | -| devtcp-hook | A | 2026-08-31 21:00 | PASS | retro lesson 2026-06-10 (zsh /dev/tcp) | -| docker-inspect-env-guard | A | 2026-08-31 21:00 | PASS | operator directive 2026-08-08 (agents keep the docker socket, but must | -| docs-build-fast-path | A | 2026-08-31 21:00 | PASS | #455 — docs builds must stay out of fast harness/eval/release gates; #536 — docs site externalized to openharness-web; docs markdown relocated to docs/ | -| drift-check-cron-staleness-glob | A | 2026-08-31 21:00 | PASS | issue #98; issue #225 (restart-required cron frontmatter/config drift) | -| entrypoint-pnpm-manifest-fingerprint | A | 2026-08-31 21:00 | PASS | issue #521 (manifest-aware sandbox installs) 2026-07-01 | -| eval-ci-gate | A | 2026-08-31 21:00 | PASS | #103 — eval probe suite gated in CI | -| eval-gate | A | 2026-08-31 21:00 | PASS | retro lesson 2026-06-11 (eval-gate) | -| eval-results-atomic | A | 2026-08-31 21:00 | PASS | issue #83 (eval-results-atomic-write) | -| eval-runner-exit | A | 2026-08-31 21:00 | PASS | retro lesson 2026-06-11 (eval-runner-exit) #29 | -| eval-runs-once-per-cycle | A | 2026-08-31 21:00 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-006) — /eval ran 3x per cycle on the | -| execution-target-contract | A | 2026-08-31 21:00 | PASS | issue #733 (ExecutionTarget contract + Docker Compose adapter) 2026-08-10 | -| get-oh-bootstrap | A | 2026-08-31 21:00 | PASS | get-oh.sh bootstrap — the Node-bootstrapping host-side path to the standalone `oh` CLI (also on npm as @mifune/openharness; see oh-npm-package.sh) | -| git-skill | A | 2026-08-31 21:00 | PASS | conversation 2026-06-15 — rules are not always supported; git workflow must be the /git skill | -| harness-audit-empty-output-gate | A | 2026-08-31 21:00 | PASS | issue #246 — /audit harness must fail closed on empty auditor outputs | -| harness-ci-core-paths | A | 2026-08-31 21:00 | PASS | #165 — core sandbox config files must trigger harness CI | -| harness-ci-hooks-paths | A | 2026-08-31 21:00 | PASS | issue #202 — credential/security hook changes must trigger harness CI | -| harness-yaml-migration | A | 2026-08-31 21:00 | PASS | PR #833 (migrate-harness-yaml.sh — append / uncomment-in-place / preserve / overwrite, plus a silent no-op second run) 2026-08-26 | -| health-check-docker-stats | A | 2026-08-31 21:00 | PASS | retro lesson 2026-06-10 (docker stats vs ps Size) | -| health-check-socket-degrade | A | 2026-08-31 21:00 | PASS | issue #762 (refs #756) — /health-check degrades to one statement, not nine failures | -| heartbeat-logging-contract | A | 2026-08-31 21:00 | PASS | issue #447 (heartbeat log append hardening) 2026-06-18 | -| image-seed-hygiene | A | 2026-08-31 21:00 | PASS | issue #900 (slim the sandbox image) 2026-08-30 | -| markitdown-wiki-ingest | A | 2026-08-31 21:00 | PASS | issue #649 — pinned local-document normalization contract for /wiki ingest | -| next-dev-prod | A | 2026-08-31 21:00 | SKIPPED | retro lesson 2026-06-04 | -| oh-compose-env-wiring | A | 2026-08-31 21:00 | PASS | issue #880 (oh as the only front door — oh.json is the non-secret config surface) | -| oh-config-surfaces | A | 2026-08-31 21:00 | PASS | PR #887 (config split across two authored surfaces — a tracked oh.json and a secrets-only root dotenv — with nothing left under $HOME) | -| oh-destroy-guard | A | 2026-08-31 21:00 | PASS | issue #879 — `oh` becomes the only front door, so `make destroy` must | -| oh-devcontainer-restructure | A | 2026-08-31 21:00 | PASS | consolidate devcontainer — .oh/devcontainer/ folded back into .devcontainer/ | -| oh-home-mount | A | 2026-08-31 21:00 | PASS | issue #898 (single $HOME mount) 2026-08-30 | -| oh-image-only-deploy | A | 2026-08-31 21:00 | PASS | .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only deploy) | -| oh-init-headless-config | A | 2026-08-31 21:00 | PASS | PR #827 (installer answers landed in the losing config file); retargeted to the .example.env template by PR #833, then to oh.json by PR #887 | -| oh-init-scaffold | A | 2026-08-31 21:00 | PASS | issue #531 Phase 2 | -| oh-lifecycle-surface | A | 2026-08-31 21:00 | PASS | issue #881 — the Makefile is retired and `oh` is the only front door | -| oh-npm-package | A | 2026-08-31 21:00 | PASS | npm publish path for the standalone `oh` CLI (@mifune/openharness) — alternative to get-oh.sh | -| oh-payload-manifest | A | 2026-08-31 21:00 | PASS | issue #531 follow-on (.oh payload manifest — oh update ships a declared allowlist) | -| oh-sandbox-image-mode | A | 2026-08-31 21:00 | PASS | conversation 2026-07-05 (basic Docker deployment — prebuilt-image mode) | -| oh-shipped-repo-overridable | A | 2026-08-31 21:00 | PASS | issue #531 follow-on (de-hardcode residual — shipped .oh shell scripts keep the upstream repo overridable) | -| oh-standalone-lifecycle | A | 2026-08-31 21:00 | PASS | issue #564 | -| oh-update | A | 2026-08-31 21:00 | PASS | issue #531 Phase 3 (oh update — upgrade only the .oh control plane) | -| operator-config-guard | A | 2026-08-31 21:00 | PASS | operator directives 2026-08-06 (.config/ and settings.local.json are operator-only) | -| pnpm-audit-ci-gate | A | 2026-08-31 21:00 | PASS | issue #171 — pnpm security audits must run in CI | -| post-bridge-publish-confirmation | A | 2026-08-31 21:00 | PASS | #523 — post-bridge live publishing requires an explicit final confirmation gate | -| prd-output-path-contract | A | 2026-08-31 21:00 | PASS | retro lesson 2026-06-19 | -| prompt-miner-schema-compat | A | 2026-08-31 21:00 | PASS | issue #253 — prompt-miner JSONL schema-drift guard | -| prompt-miner-symlink-entrypoint | A | 2026-08-31 21:00 | PASS | issue #663 — prompt-miner engine no-ops via the documented .claude/skills symlink | -| prompt-miner-weakness-record | A | 2026-08-31 21:00 | PASS | issue #580 — prompt-miner weakness-record (WH-xxx) cluster output | -| protected-path-deletion | A | 2026-08-31 21:00 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-001) — the critique gate was deleted, | -| protected-paths-resolve | A | 2026-08-31 21:00 | PASS | issue #753 — .claude/protected-paths.txt named 7 paths that did not exist. | -| registry-portability-gate | A | 2026-08-31 21:00 | PASS | issue #758 | -| registry-portability | A | 2026-08-31 21:00 | SKIPPED | issue #758 | -| retro-deterministic-contract | A | 2026-08-31 21:00 | PASS | issue #443 — /retro deterministic output and self-contained helper contract | -| rl-delegation-write-worker | A | 2026-08-31 21:00 | PASS | retro lesson 2026-06-10 (rl-delegation) #57 | -| rlm-context-budget | A | 2026-08-31 21:00 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-006 | -| runtime-preflight-gate | A | 2026-08-31 21:00 | PASS | issue #806 § B1 (open sandbox.substrate vs sandbox.runtime selector); | -| sandbox-boot-guard-ci | A | 2026-08-31 21:00 | PASS | issue #449 (sandbox image build CI guard) 2026-06-19; | -| sandbox-node-base | A | 2026-08-31 21:00 | PASS | openharness#878 — oh as the only front door, T0 sandbox base image | -| skill-paths | A | 2026-08-31 21:00 | PASS | issue #43 — stale path references; extended by issue #69 — apps/->packages/ rename guard; extended by issue #870 — deleted .oh/agents/advisor.md | -| skills-dir-clean | A | 2026-08-31 21:00 | PASS | conversation 2026-06-29 — Pi parses every top-level `.md` in the skills | -| skills-task-tool-coupling | A | 2026-08-31 21:00 | PASS | council review 2026-08-29 (issue #886) — /delegate instructed Claude-Code-only | -| skills-vendored | A | 2026-08-31 21:00 | PASS | absorb .mifune submodule into .oh — the skills/agents/hooks pack is vendored | -| slack-admin-command-surface | A | 2026-08-31 21:00 | PASS | issue #354 — Slack bridge docs must distinguish Pi /msg-bridge commands from Slack DM admin text handlers | -| spec-family-contract | A | 2026-08-31 21:00 | PASS | issue #265; spec-simplification issue #816; workflow authority issue #854 | -| spec-ready-finalization | A | 2026-08-31 21:00 | PASS | issue #134; spec-simplification issue #816; workflow authority issue #854 | -| ste-checker-contract | A | 2026-08-31 21:00 | PASS | issue #750 PR audit — the /ste checker had four fail-open paths (unclosed | -| submitted-by-trailers | A | 2026-08-31 21:00 | PASS | conversation 2026-06-12 (commit attribution trailers); the single-owner | -| sync-skill-contract | A | 2026-08-31 21:00 | PASS | issue #331 — /sync dispatcher skill (bidirectional origin↔upstream sync) | -| t3-headless-launch | A | 2026-08-31 21:00 | PASS | issue #858 — /t3 launched a bare `npx --yes t3`, which is the local GUI and | -| tailscale-tool-boundary | A | 2026-08-31 21:00 | PASS | issue #858 — Tailscale mobile access for T3 Code. There is no tailnet, no | -| tool-catalog-boundary | A | 2026-08-31 21:00 | PASS | agent-browser's exclusion from the harness catalog (#821) and the | -| version-parity | A | 2026-08-31 21:00 | PASS | conversation 2026-08-29 — the oh CLI became the only lifecycle door, so its | -| weigh-scorer-contract | A | 2026-08-31 21:00 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-003 (2026-06-27) | -| wiki-readme-index | A | 2026-08-31 21:00 | PASS | issue #132 — wiki README index drift guard | -| workflow-boundaries | A | 2026-08-31 21:00 | PASS | conversation 2026-06-19 (workflow consolidation, issue #259); authority moved to /spec in issue #854 | -| worktrees-layout | A | 2026-08-31 21:00 | PASS | issue #872 | +| advisor-monitored-loop | A | 2026-08-31 21:20 | PASS | conversation 2026-06-19 (single-owner implementation workflow, issue #257) | +| agent-browser-cli | A | 2026-08-31 21:20 | PASS | retro lesson 2026-06-07 (agent-browser 0.8.5 CLI) | +| agents-identity-contract | A | 2026-08-31 21:20 | PASS | issue #854 — T3-style root identity, glossary, and skill-owned procedures | +| artifact-contract-audit | A | 2026-08-31 21:20 | PASS | issue #583/#645 — production /audit implementation Gate 1 behavior | +| audit-dispatcher-contract | A | 2026-08-31 21:20 | PASS | issue #645 — audit consolidation public taxonomy | +| audit-implementation-behavior | A | 2026-08-31 21:20 | PASS | issue #645 — implementation root/repo/browser behavior | +| audit-pr-acquire | A | 2026-08-31 21:20 | PASS | issue #645 — production PR acquisition behavior | +| audit-pr-classifier | A | 2026-08-31 21:20 | PASS | issue #645 — deterministic focused and queue PR classifier | +| audit-run-root-contract | A | 2026-08-31 21:20 | PASS | issue #645 — executable immutable audit root/run correlation | +| audit-shellcheck-coverage | A | 2026-08-31 21:20 | PASS | issue #645 — private audit scripts require release and CI lint coverage | +| audit-slop-gate | A | 2026-08-31 21:20 | PASS | .claude/specs/images/reduce-slop.png — the four correctness gates all pass a change | +| audit-stale-references | A | 2026-08-31 21:20 | PASS | issue #645 — clean-breaking audit migration | +| boot-lint-glob | A | 2026-08-31 21:20 | PASS | issue #90, issue #120 | +| builder-skill-consolidation | A | 2026-08-31 21:20 | PASS | issue #643 — consolidate artifact builders behind one /builder dispatcher | +| capability-benchmark-schema | A | 2026-08-31 21:20 | PASS | issue #167 — capability benchmark instrument | +| cc-safety-net-wiring | A | 2026-08-31 21:20 | SKIPPED | .oh/tasks/cc-safety-net/prd.json US-007 2026-07-19 | +| changelog-entry-length | A | 2026-08-31 21:20 | PASS | conversation 2026-08-24 — CHANGELOG.md grew to 259KB of bullet prose because "one line" was unquantified | +| cleanup-tasks-scoped-guard | A | 2026-08-31 21:20 | PASS | issue #85 | +| cleanup-tasks-worktree-grooming | A | 2026-08-31 21:20 | PASS | issue #168; issue #327 | +| cli-publish-typecheck-scope | A | 2026-08-31 21:20 | PASS | release run 33271077312 — v0.5.0 pushed its GHCR image, then failed to publish | +| close-issues-on-development | A | 2026-08-31 21:20 | PASS | issue #841 (closing keywords never fire because the default branch is main) 2026-08-26 | +| codex-stale-response-retry | A | 2026-08-31 21:20 | PASS | issue #506 — Codex previous_response_not_found RCA | +| compose-config-path-parity | A | 2026-08-31 21:20 | PASS | PR #833 (remove harness.yaml — the wrapper and VS Code "Reopen in Container" paths must resolve the same service) 2026-08-26 | +| config-schema-parity | A | 2026-08-31 21:20 | PASS | PR #833 (one schema file — DOCKER_SOCKET, SANDBOX_SSH, OH_SANDBOX_IMAGE, OH_PULL_POLICY, SKIP_PNPM_INSTALL were consumed but undocumented); rewritten for the oh.json/secrets split by PR #887 | +| context-tier-size-budget | A | 2026-08-31 21:20 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-007) — the always-on tier was 85,256 B | +| cron-claude-codex-fallback | A | 2026-08-31 21:20 | PASS | conversation 2026-06-12 (default Codex fallback for crons) | +| cron-watchdog | A | 2026-08-31 21:20 | PASS | issues #130/#453 (cron runtime watchdog + legacy system-cron reaping) 2026-06-19 | +| crons-directory-guide | A | 2026-08-31 21:20 | PASS | issue #874 | +| curl-bash-safe-alternatives | A | 2026-08-31 21:20 | PASS | vet-run/vet integration — public curl|bash examples need review-first alternatives | +| datasets-schema | A | 2026-08-31 21:20 | PASS | issue #196 — .oh/evals/datasets verifiable trajectory corpus (Repo2RLEnv-inspired) | +| debugmcp-availability | A | 2026-08-31 21:20 | SKIPPED | issue #297 — DebugMCP MCP debug-server availability | +| default-provisioning | A | 2026-08-31 21:20 | PASS | #902 — `oh harness install` must work from inside the sandbox, where | +| delegate-model-effort-policy | A | 2026-08-31 21:20 | PASS | conversation 2026-07-11 (delegate model inheritance and thinking policy) | +| devtcp-hook | A | 2026-08-31 21:20 | PASS | retro lesson 2026-06-10 (zsh /dev/tcp) | +| docker-inspect-env-guard | A | 2026-08-31 21:20 | PASS | operator directive 2026-08-08 (agents keep the docker socket, but must | +| docs-build-fast-path | A | 2026-08-31 21:20 | PASS | #455 — docs builds must stay out of fast harness/eval/release gates; #536 — docs site externalized to openharness-web; docs markdown relocated to docs/ | +| drift-check-cron-staleness-glob | A | 2026-08-31 21:20 | PASS | issue #98; issue #225 (restart-required cron frontmatter/config drift) | +| entrypoint-pnpm-manifest-fingerprint | A | 2026-08-31 21:20 | PASS | issue #521 (manifest-aware sandbox installs) 2026-07-01 | +| eval-ci-gate | A | 2026-08-31 21:20 | PASS | #103 — eval probe suite gated in CI | +| eval-gate | A | 2026-08-31 21:20 | PASS | retro lesson 2026-06-11 (eval-gate) | +| eval-results-atomic | A | 2026-08-31 21:20 | PASS | issue #83 (eval-results-atomic-write) | +| eval-runner-exit | A | 2026-08-31 21:20 | PASS | retro lesson 2026-06-11 (eval-runner-exit) #29 | +| eval-runs-once-per-cycle | A | 2026-08-31 21:20 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-006) — /eval ran 3x per cycle on the | +| execution-target-contract | A | 2026-08-31 21:20 | PASS | issue #733 (ExecutionTarget contract + Docker Compose adapter) 2026-08-10 | +| get-oh-bootstrap | A | 2026-08-31 21:20 | PASS | get-oh.sh bootstrap — the Node-bootstrapping host-side path to the standalone `oh` CLI (also on npm as @mifune/openharness; see oh-npm-package.sh) | +| git-skill | A | 2026-08-31 21:20 | PASS | conversation 2026-06-15 — rules are not always supported; git workflow must be the /git skill | +| harness-audit-empty-output-gate | A | 2026-08-31 21:20 | PASS | issue #246 — /audit harness must fail closed on empty auditor outputs | +| harness-ci-core-paths | A | 2026-08-31 21:20 | PASS | #165 — core sandbox config files must trigger harness CI | +| harness-ci-hooks-paths | A | 2026-08-31 21:20 | PASS | issue #202 — credential/security hook changes must trigger harness CI | +| harness-yaml-migration | A | 2026-08-31 21:20 | PASS | PR #833 (migrate-harness-yaml.sh — append / uncomment-in-place / preserve / overwrite, plus a silent no-op second run) 2026-08-26 | +| health-check-docker-stats | A | 2026-08-31 21:20 | PASS | retro lesson 2026-06-10 (docker stats vs ps Size) | +| health-check-socket-degrade | A | 2026-08-31 21:20 | PASS | issue #762 (refs #756) — /health-check degrades to one statement, not nine failures | +| heartbeat-logging-contract | A | 2026-08-31 21:20 | PASS | issue #447 (heartbeat log append hardening) 2026-06-18 | +| image-seed-hygiene | A | 2026-08-31 21:20 | PASS | issue #900 (slim the sandbox image) 2026-08-30 | +| markitdown-wiki-ingest | A | 2026-08-31 21:20 | PASS | issue #649 — pinned local-document normalization contract for /wiki ingest | +| next-dev-prod | A | 2026-08-31 21:20 | SKIPPED | retro lesson 2026-06-04 | +| oh-compose-env-wiring | A | 2026-08-31 21:20 | SKIPPED | issue #880 (oh as the only front door — oh.json is the non-secret config surface) | +| oh-config-surfaces | A | 2026-08-31 21:20 | PASS | PR #887 (config split across two authored surfaces — a tracked oh.json and a secrets-only root dotenv — with nothing left under $HOME) | +| oh-destroy-guard | A | 2026-08-31 21:20 | SKIPPED | issue #879 — `oh` becomes the only front door, so `make destroy` must | +| oh-devcontainer-restructure | A | 2026-08-31 21:20 | PASS | consolidate devcontainer — .oh/devcontainer/ folded back into .devcontainer/ | +| oh-home-mount | A | 2026-08-31 21:20 | PASS | issue #898 (single $HOME mount) 2026-08-30 | +| oh-image-only-deploy | A | 2026-08-31 21:20 | PASS | .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only deploy) | +| oh-init-headless-config | A | 2026-08-31 21:20 | SKIPPED | PR #827 (installer answers landed in the losing config file); retargeted to the .example.env template by PR #833, then to oh.json by PR #887 | +| oh-init-scaffold | A | 2026-08-31 21:20 | PASS | issue #531 Phase 2 | +| oh-lifecycle-surface | A | 2026-08-31 21:20 | PASS | issue #881 — the Makefile is retired and `oh` is the only front door | +| oh-npm-package | A | 2026-08-31 21:20 | PASS | npm publish path for the standalone `oh` CLI (@mifune/openharness) — alternative to get-oh.sh | +| oh-payload-manifest | A | 2026-08-31 21:20 | PASS | issue #531 follow-on (.oh payload manifest — oh update ships a declared allowlist) | +| oh-sandbox-image-mode | A | 2026-08-31 21:20 | PASS | conversation 2026-07-05 (basic Docker deployment — prebuilt-image mode) | +| oh-shipped-repo-overridable | A | 2026-08-31 21:20 | PASS | issue #531 follow-on (de-hardcode residual — shipped .oh shell scripts keep the upstream repo overridable) | +| oh-standalone-lifecycle | A | 2026-08-31 21:20 | PASS | issue #564 | +| oh-update | A | 2026-08-31 21:20 | PASS | issue #531 Phase 3 (oh update — upgrade only the .oh control plane) | +| operator-config-guard | A | 2026-08-31 21:20 | PASS | operator directives 2026-08-06 (.config/ and settings.local.json are operator-only) | +| pnpm-audit-ci-gate | A | 2026-08-31 21:20 | PASS | issue #171 — pnpm security audits must run in CI | +| post-bridge-publish-confirmation | A | 2026-08-31 21:20 | PASS | #523 — post-bridge live publishing requires an explicit final confirmation gate | +| prd-output-path-contract | A | 2026-08-31 21:20 | PASS | retro lesson 2026-06-19 | +| prompt-miner-schema-compat | A | 2026-08-31 21:20 | PASS | issue #253 — prompt-miner JSONL schema-drift guard | +| prompt-miner-symlink-entrypoint | A | 2026-08-31 21:20 | PASS | issue #663 — prompt-miner engine no-ops via the documented .claude/skills symlink | +| prompt-miner-weakness-record | A | 2026-08-31 21:20 | PASS | issue #580 — prompt-miner weakness-record (WH-xxx) cluster output | +| protected-path-deletion | A | 2026-08-31 21:20 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-001) — the critique gate was deleted, | +| protected-paths-resolve | A | 2026-08-31 21:20 | PASS | issue #753 — .claude/protected-paths.txt named 7 paths that did not exist. | +| registry-portability-gate | A | 2026-08-31 21:20 | PASS | issue #758 | +| registry-portability | A | 2026-08-31 21:20 | SKIPPED | issue #758 | +| retro-deterministic-contract | A | 2026-08-31 21:20 | PASS | issue #443 — /retro deterministic output and self-contained helper contract | +| rl-delegation-write-worker | A | 2026-08-31 21:20 | PASS | retro lesson 2026-06-10 (rl-delegation) #57 | +| rlm-context-budget | A | 2026-08-31 21:20 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-006 | +| runtime-preflight-gate | A | 2026-08-31 21:20 | PASS | issue #806 § B1 (open sandbox.substrate vs sandbox.runtime selector); | +| sandbox-boot-guard-ci | A | 2026-08-31 21:20 | PASS | issue #449 (sandbox image build CI guard) 2026-06-19; | +| sandbox-node-base | A | 2026-08-31 21:20 | PASS | openharness#878 — oh as the only front door, T0 sandbox base image | +| skill-paths | A | 2026-08-31 21:20 | PASS | issue #43 — stale path references; extended by issue #69 — apps/->packages/ rename guard; extended by issue #870 — deleted .oh/agents/advisor.md | +| skills-dir-clean | A | 2026-08-31 21:20 | PASS | conversation 2026-06-29 — Pi parses every top-level `.md` in the skills | +| skills-task-tool-coupling | A | 2026-08-31 21:20 | PASS | council review 2026-08-29 (issue #886) — /delegate instructed Claude-Code-only | +| skills-vendored | A | 2026-08-31 21:20 | PASS | absorb .mifune submodule into .oh — the skills/agents/hooks pack is vendored | +| slack-admin-command-surface | A | 2026-08-31 21:20 | PASS | issue #354 — Slack bridge docs must distinguish Pi /msg-bridge commands from Slack DM admin text handlers | +| spec-family-contract | A | 2026-08-31 21:20 | PASS | issue #265; spec-simplification issue #816; workflow authority issue #854 | +| spec-ready-finalization | A | 2026-08-31 21:20 | PASS | issue #134; spec-simplification issue #816; workflow authority issue #854 | +| ste-checker-contract | A | 2026-08-31 21:20 | PASS | issue #750 PR audit — the /ste checker had four fail-open paths (unclosed | +| submitted-by-trailers | A | 2026-08-31 21:20 | PASS | conversation 2026-06-12 (commit attribution trailers); the single-owner | +| sync-skill-contract | A | 2026-08-31 21:20 | PASS | issue #331 — /sync dispatcher skill (bidirectional origin↔upstream sync) | +| t3-headless-launch | A | 2026-08-31 21:20 | PASS | issue #858 — /t3 launched a bare `npx --yes t3`, which is the local GUI and | +| tailscale-tool-boundary | A | 2026-08-31 21:20 | PASS | issue #858 — Tailscale mobile access for T3 Code. There is no tailnet, no | +| tool-catalog-boundary | A | 2026-08-31 21:20 | PASS | agent-browser's exclusion from the harness catalog (#821) and the | +| version-parity | A | 2026-08-31 21:20 | PASS | conversation 2026-08-29 — the oh CLI became the only lifecycle door, so its | +| weigh-scorer-contract | A | 2026-08-31 21:20 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-003 (2026-06-27) | +| wiki-readme-index | A | 2026-08-31 21:20 | PASS | issue #132 — wiki README index drift guard | +| workflow-boundaries | A | 2026-08-31 21:20 | PASS | conversation 2026-06-19 (workflow consolidation, issue #259); authority moved to /spec in issue #854 | +| worktrees-layout | A | 2026-08-31 21:20 | PASS | issue #872 | diff --git a/.oh/evals/probes/audit-slop-gate.sh b/.oh/evals/probes/audit-slop-gate.sh new file mode 100755 index 00000000..131ccd0a --- /dev/null +++ b/.oh/evals/probes/audit-slop-gate.sh @@ -0,0 +1,75 @@ +#!/usr/bin/env bash +# tier: A +# source: .claude/specs/images/reduce-slop.png — the four correctness gates all pass a change +# that works and is twice the size it needed to be. Nothing forced the diff smaller. +# desc: /audit implementation gate 5 measures slop, blocks on a concrete smaller alternative, +# and terminates by construction (round cap or a non-reducing round) rather than by taste. +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" +REF="$ROOT/.oh/skills/audit/references/implementation.md" +EXEC="$ROOT/.oh/skills/spec/references/execute.md" +GATE="$ROOT/.oh/skills/audit/scripts/implementation-gates.sh" + +fail() { echo "REGRESSION: $*" >&2; exit 1; } + +for f in "$REF" "$EXEC" "$GATE"; do [[ -f $f ]] || fail "missing: $f"; done + +grep -Fq '## The five gates (fail-fast, in order)' "$REF" || fail 'gate 5 not in the fail-fast chain' +grep -Fq '### Gate 5 — Slop (less code, low complexity)' "$REF" || fail 'gate 5 section missing' +grep -Fq 'concrete simpler alternative is not a finding' "$REF" \ + || fail 'the termination rule that keeps gate 5 from becoming taste is missing' +grep -Fq 'SIMPLICITY-RESIDUAL' "$REF" || fail 'residual disclosure missing from the verdict' +grep -Fq 'never report it as CCN' "$REF" || fail 'the bash branch-point proxy is not labelled as a proxy' +grep -Fq 'never infer green from' "$REF" || fail 'unavailable complexity tool may be read as green' +grep -Fq '.oh/tasks//simplify-rounds.json' "$REF" || fail 'gate 5 does not name the round record' +grep -Fq 'simplify-rounds.json' "$EXEC" || fail 'the caller that owns the round record does not write it' +grep -Fq 'non-reducing round' "$EXEC" || fail 'the monotone termination rule is not documented for the caller' + +root=$(mktemp -d); trap 'rm -rf "$root"' EXIT +mkdir -p "$root/.oh/tasks/demo" +git -C "$root" init -q +git -C "$root" config user.email test@example.invalid +git -C "$root" config user.name test +printf 'a\nb\nc\n' >"$root/keep.sh" +git -C "$root" add .; git -C "$root" commit -qm base +base=$(git -C "$root" rev-parse HEAD) +printf 'a\nb\nc\nif x; then y; fi\nwhile z; do w; done\n' >"$root/keep.sh" +git -C "$root" add .; git -C "$root" commit -qm change + +metrics=$(AUDIT_ROOT="$root" bash "$GATE" slop-metrics "$base") +[[ $(jq -r .netAdded <<<"$metrics") == 2 ]] || fail "netAdded wrong: $metrics" +[[ $(jq -r .netRemoved <<<"$metrics") == 0 ]] || fail "netRemoved wrong: $metrics" +[[ $(jq -r .shBranchPoints <<<"$metrics") == 2 ]] || fail "shBranchPoints wrong: $metrics" + +if AUDIT_ROOT="$root" bash "$GATE" slop-metrics no-such-ref >/dev/null 2>&1; then + fail 'unknown base ref was accepted' +fi + +stub=$(mktemp -d); printf '#!/bin/sh\nexit 127\n' >"$stub/uvx"; chmod +x "$stub/uvx" +printf 'export function f(){return 1}\n' >"$root/x.ts" +git -C "$root" add .; git -C "$root" commit -qm ts +offline=$(PATH="$stub:$PATH" AUDIT_ROOT="$root" bash "$GATE" slop-metrics "$base") +rm -rf "$stub" +[[ $(jq -r .tool <<<"$offline") == unavailable ]] \ + || fail "an unresolvable lizard must report unavailable, got: $(jq -r .tool <<<"$offline")" + +round() { AUDIT_ROOT="$root" bash "$GATE" simplicity-round demo; } +[[ $(round) == 'rounds=0 cap=3 escalate=false prevNetAdded=none' ]] || fail "missing counter: $(round)" +printf '{"rounds":2,"netAdded":410}' >"$root/.oh/tasks/demo/simplify-rounds.json" +[[ $(round) == 'rounds=2 cap=3 escalate=false prevNetAdded=410' ]] || fail "below cap: $(round)" +printf '{"rounds":3,"netAdded":380}' >"$root/.oh/tasks/demo/simplify-rounds.json" +[[ $(round) == 'rounds=3 cap=3 escalate=true prevNetAdded=380' ]] || fail "at cap must escalate: $(round)" +printf '{"rounds":"two"}' >"$root/.oh/tasks/demo/simplify-rounds.json" +if round >/dev/null 2>&1; then fail 'malformed counter was accepted'; fi +if AUDIT_ROOT="$root" bash "$GATE" simplicity-round '../etc' >/dev/null 2>&1; then + fail 'traversal slug was accepted' +fi +ln -s /tmp "$root/.oh/tasks/linked" +if AUDIT_ROOT="$root" bash "$GATE" simplicity-round linked >/dev/null 2>&1; then + fail 'symlinked task directory was accepted' +fi + +grep -Fq 'slop-metrics|simplicity-round' "$GATE" || fail 'new modes absent from the usage line' + +echo 'PASS: gate 5 measures slop and terminates on the cap or a non-reducing round' >&2 diff --git a/.oh/skills/audit/SKILL.md b/.oh/skills/audit/SKILL.md index b99b6634..0feb4f72 100644 --- a/.oh/skills/audit/SKILL.md +++ b/.oh/skills/audit/SKILL.md @@ -4,7 +4,7 @@ description: >- Explicit nine-target audit dispatcher for implementation promotability, one PR, the open PR queue, harness health, context budget, skill integrity, eval quality, drift, and correlated full campaigns. TRIGGER when: audit this task; verify this - implementation; audit PR N; classify this pull request; audit open PRs; triage + implementation; is this the simplest approach; audit PR N; classify this pull request; audit open PRs; triage the PR queue; audit the harness; find harness improvements; audit context budget; audit skills; find stale or broken skills; lint evals; find Goodharted probes; check framework drift; cron staleness; run a full audit diff --git a/.oh/skills/audit/references/implementation.md b/.oh/skills/audit/references/implementation.md index ef8b9b8b..51945ded 100644 --- a/.oh/skills/audit/references/implementation.md +++ b/.oh/skills/audit/references/implementation.md @@ -31,7 +31,7 @@ is a downstream concern and remediation belongs to the build step on --- -## The four gates (fail-fast, in order) +## The five gates (fail-fast, in order) Run in order; the **first** gate that fails decides the verdict (`AUDIT-FAIL`, naming the gate). Only when **all** applicable gates pass is the verdict @@ -158,6 +158,64 @@ repository. No clean screenshot/snapshot for an applicable story is `AUDIT-FAIL` When no story declares browser verification, this gate is **not applicable** and must not invoke `agent-browser` at all. +### Gate 5 — Slop (less code, low complexity) + +The correctness gates above prove the change *works*. None of them can fail a change +that works and is twice the size it needed to be. This gate asks the one question that +closes that hole: + +> **Can this diff be smaller and still satisfy every acceptance criterion in `prd.json`?** + +While the answer is yes, the verdict is `AUDIT-FAIL` and the build simplifies. The goal +stays one sentence on purpose — the ingenuity belongs in the execution, not in the +objective. Less code is less code to maintain and fewer places for a bug to live. + +**Signals.** Run +`"$AUDIT_ROOT/.oh/skills/audit/scripts/implementation-gates.sh" slop-metrics "$BASE"`, +which emits one JSON object. Report every number in the verdict: + +| Field | Meaning | +|---|---| +| `netAdded` / `netRemoved` | Lines the unit's diff adds and removes vs. `--base`, excluding lockfiles, `.oh/evals/RESULTS.md`, and symlinked provider mirrors. `netAdded` is the headline number the loop drives down. | +| `tsOverCcn` | Functions in the changed `.ts`/`.js`/`.mjs` files over `ccnMax` (default 10), from `uvx lizard`. **Real per-function cyclomatic complexity.** | +| `shBranchPoints` | The *net* change in branch tokens across changed `.sh` files. No complexity tool parses bash, so this is an explicit **proxy** — never report it as CCN. | +| `tool` | `lizard `, `lizard n/a (no analysable files changed)`, or `unavailable`. | + +`tool: unavailable` means `uvx lizard` could not resolve (an offline runner). The +complexity signal is then **SKIPPED and disclosed** — an empty `tsOverCcn` from an +unavailable tool is never reported as a clean complexity result. Never infer green from +silence. + +**Findings — the termination rule.** Every finding MUST cite `file:line`, name the +concrete simpler alternative, and state the lines it removes. **A finding with no +concrete simpler alternative is not a finding.** That rule is what keeps this gate an +engineering check rather than an unbounded argument about taste. Typical shapes: a +primitive the repo already has, an abstraction with exactly one call site and no +criterion requiring it, a new file where editing an existing one would do, a path no +story exercises. + +A finding is **blocking** only when its alternative satisfies every acceptance criterion +with no new work. Anything else is disclosed, non-gating. A function the diff +*introduces* above `ccnMax` is blocking; one already over the threshold on the base is +disclosed only — the same pre-existing/new distinction gate 2 makes. + +**The bounded, monotone loop.** Read the caller's round record with +`implementation-gates.sh simplicity-round "$SLUG"`, which prints +`rounds= cap=3 escalate= prevNetAdded=` from +`.oh/tasks//simplify-rounds.json`: + +- `escalate=false` and a blocking finding exists → `AUDIT-FAIL` (gate 5). The build + simplifies and re-audits. +- `escalate=true` (round cap reached), **or** `netAdded` did not strictly fall below + `prevNetAdded` on this round → stop blocking. `prevNetAdded=none` is the first round: + there is nothing to compare, so the monotone rule does not apply to it. The loop ends when the diff can no + longer be made smaller, not when taste is satisfied, so it terminates by construction. + Emit `AUDIT-PASS` with `SIMPLICITY-RESIDUAL: ` and list the residual findings for + the operator; they belong in `evidence.md`. + +This route **reads** the round record. It never writes or increments it — the +orchestrating caller owns that file, exactly as it owns `evidence.md`. + --- ## Verdict @@ -168,7 +226,9 @@ must not invoke `agent-browser` at all. State the verdict, then — on the **final line** — emit the routing token. Always name the deciding gate on `AUDIT-FAIL` and disclose any non-gating pre-existing -red from gate 2. +red from gate 2. An `AUDIT-PASS` reached at the gate-5 round cap or on a +non-reducing round carries `SIMPLICITY-RESIDUAL: ` with the residual findings; +a `PASS` that hides residual slop is the one thing this gate exists to prevent. --- @@ -180,6 +240,10 @@ red from gate 2. - **Fork PR classification.** It consumes the same private classifier JSON as `/audit pr` and `/audit prs`. - **Re-run a passing gate.** Fail-fast: stop at the first failing gate. +- **Write or increment the gate-5 round counter.** It reads + `.oh/tasks//simplify-rounds.json`; the orchestrating caller writes it. +- **Apply the simplification.** Gate 5 names the smaller alternative; removing the + code is the `implement` node's job, like every other `AUDIT-FAIL`. - **Write the reviewer evidence doc.** The per-gate observations above are what `.oh/tasks//evidence.md` is built from, but the orchestrating caller writes and commits it — see [`reviewer-evidence-doc.md`](reviewer-evidence-doc.md). @@ -195,6 +259,6 @@ Return this structured observation to the outer dispatcher; do not report a run - **Result**: OP - **Unit**: (PR # / branch ) - **Verdict**: AUDIT-PASS | AUDIT-FAIL (gate : ) -- **Gates**: graph

· eval · promotable · ui +- **Gates**: graph

· eval · promotable · ui · slop +/- () - **Observation**: ``` diff --git a/.oh/skills/audit/references/reviewer-evidence-doc.md b/.oh/skills/audit/references/reviewer-evidence-doc.md index 55647b8b..fcbda6b6 100644 --- a/.oh/skills/audit/references/reviewer-evidence-doc.md +++ b/.oh/skills/audit/references/reviewer-evidence-doc.md @@ -71,8 +71,9 @@ Every doc answers these, in this order, before the per-gate proof: plan: a criterion satisfied differently, a deliberate deviation, a mid-build scope call. Explicitly `None` when there was none. 4. **What remains unverified** — skipped gates, criteria argued rather than observed, - pre-existing reds carried forward, anything a reviewer must check by hand. - Explicitly `Nothing` when there is none. + pre-existing reds carried forward, a `SIMPLICITY-RESIDUAL` list the simplify loop + ended on, anything a reviewer must check by hand. Explicitly `Nothing` when there is + none. **Why question 0 is first and separate.** Questions 1–4 prove the change is *correct*. None of them establishes it was *worth making*. A doc can pass every gate, diverge nowhere, @@ -121,6 +122,7 @@ forward, anything needing a hand check — or "Nothing".> | Regression floor | `/eval` runner exit + delta | `rc=0`, no new green→red | PASS | | Promotable / CI | focused classifier JSON | `promotable=true`, `evidenceComplete=true` | PASS | | UI | browser criteria | n/a — no story declares browser verification | N/A | +| Slop | net lines + changed-function CCN | `+/-`, `` over CCN | PASS | ## Observed output diff --git a/.oh/skills/audit/scripts/implementation-gates.sh b/.oh/skills/audit/scripts/implementation-gates.sh index 70583894..eef4b4c6 100755 --- a/.oh/skills/audit/scripts/implementation-gates.sh +++ b/.oh/skills/audit/scripts/implementation-gates.sh @@ -2,6 +2,8 @@ set -euo pipefail : "${AUDIT_ROOT:?AUDIT_ROOT is required}" AUDIT_ROOT=$(cd "$AUDIT_ROOT" && pwd -P) +CCN_MAX=${CCN_MAX:-10} +ROUND_CAP=${ROUND_CAP:-3} mode=${1:-}; shift || true case $mode in gate1) @@ -78,5 +80,58 @@ case $mode in cmp -s "$before" "$after" || { echo 'FAIL gate4: browser preflight mutated AUDIT_ROOT content or index' >&2; exit 1; } rm -f "$before" "$after" ;; - *) echo 'usage: implementation-gates.sh ...' >&2; exit 64;; + slop-metrics) + base=${1:-development} + [[ $base =~ ^[A-Za-z0-9._/-]+$ ]] || { echo 'usage: implementation-gates.sh slop-metrics ' >&2; exit 64; } + git -C "$AUDIT_ROOT" rev-parse --verify --quiet "$base^{commit}" >/dev/null \ + || { echo "FAIL gate5: unknown base ref: $base" >&2; exit 64; } + counted(){ case $1 in *pnpm-lock.yaml|*package-lock.json|*.oh/evals/RESULTS.md) return 1;; esac; [[ ! -L $AUDIT_ROOT/$1 ]]; } + added=0; removed=0 + while read -r a r path; do + [[ $a == '-' ]] && continue + counted "$path" || continue + added=$((added + a)); removed=$((removed + r)) + done < <(git -C "$AUDIT_ROOT" diff --numstat "$base...HEAD") + sh_delta=$(git -C "$AUDIT_ROOT" diff -U0 "$base...HEAD" -- '*.sh' | awk ' + /^\+\+\+/ || /^---/ { next } + /^[+-]/ { + sign = (substr($0,1,1)=="+") ? 1 : -1; line = " " substr($0,2) " " + n = gsub(/&&|\|\|/, "", line) + n += gsub(/[^[:alnum:]_](if|elif|while|until|for|case)[^[:alnum:]_]/, " ", line) + total += sign * n + } + END { print total+0 }') + mapfile -t ts < <(git -C "$AUDIT_ROOT" diff --name-only --diff-filter=d "$base...HEAD" -- '*.ts' '*.mjs' '*.js') + tool=unavailable; over='[]' + if ((${#ts[@]})); then + if ver=$(uvx lizard --version 2>/dev/null); then + tool="lizard $ver" + warnings=$(cd "$AUDIT_ROOT" && uvx lizard -w --CCN "$CCN_MAX" "${ts[@]}" 2>/dev/null || true) + over=$(sed -nE 's/^(.+): warning: (\S+) has [0-9]+ NLOC, ([0-9]+) CCN.*/\1 \2 CCN \3/p' <<<"$warnings" | jq -R . | jq -s .) + fi + else + tool='lizard n/a (no analysable files changed)' + fi + jq -n --argjson netAdded "$added" --argjson netRemoved "$removed" \ + --argjson shBranchPoints "$sh_delta" \ + --argjson tsOverCcn "$over" --arg tool "$tool" --argjson ccnMax "$CCN_MAX" \ + '{netAdded:$netAdded,netRemoved:$netRemoved,shBranchPoints:$shBranchPoints,ccnMax:$ccnMax,tsOverCcn:$tsOverCcn,tool:$tool}' + ;; + simplicity-round) + slug=${1:-}; [[ $slug =~ ^[A-Za-z0-9][A-Za-z0-9._-]*$ ]] || { echo 'FAIL gate5: invalid slug' >&2; exit 64; } + task_dir="$AUDIT_ROOT/.oh/tasks/$slug"; counter="$task_dir/simplify-rounds.json" + resolved_task=$(realpath -e -- "$task_dir" 2>/dev/null) \ + || { echo "FAIL gate5: missing task directory: $task_dir" >&2; exit 1; } + [[ $resolved_task == "$task_dir" && ! -L $task_dir ]] \ + || { echo "FAIL gate5: task directory is symlinked: $task_dir" >&2; exit 1; } + rounds=0; prev=none + if [[ -f $counter && ! -L $counter ]]; then + jq -e '(.rounds|type)=="number"' "$counter" >/dev/null \ + || { echo "FAIL gate5: malformed counter: $counter" >&2; exit 1; } + rounds=$(jq -r '.rounds' "$counter"); prev=$(jq -r '.netAdded // "none"' "$counter") + fi + escalate=false; (( rounds >= ROUND_CAP )) && escalate=true + printf 'rounds=%s cap=%s escalate=%s prevNetAdded=%s\n' "$rounds" "$ROUND_CAP" "$escalate" "$prev" + ;; + *) echo 'usage: implementation-gates.sh ...' >&2; exit 64;; esac diff --git a/.oh/skills/spec/references/execute.md b/.oh/skills/spec/references/execute.md index 4bb1e24a..74b0a9c6 100644 --- a/.oh/skills/spec/references/execute.md +++ b/.oh/skills/spec/references/execute.md @@ -218,7 +218,7 @@ nested implementation session. **Advisor `/goal` prompt** (one line; fill the placeholders — when `$CRON_WORKTREE` is set, substitute its actual path for `` and use the "reuse" branch of step 1): -> `/goal` As the **single expert Advisor on `/worktrees`**, implement `.oh/tasks//prd.json` for PR `#` on branch `/-`. (1) **If `` is already provided** (autopilot's `$CRON_WORKTREE`, already on branch `/-`): `cd ` and do NOT create another worktree. **Otherwise** create an isolated worktree at `.worktrees//-` via `/worktrees` and `cd` into it. (2) Read `.oh/tasks//prompt.md`, implement the dependency-ready stories directly, and use `/delegate` only for bounded disjoint work. Reconcile worker results, validate every acceptance criterion, update `prd.json` and `progress.txt`, and append `STATUS: COMPLETE` only after every story passes. (3) Continue in this same Advisor session with the implementation-side audit loop, `/eval` once, required wiki revision, `/compact`, `evidence.md`, `/spec retro`, improve steps, and a fresh `/audit pr`; run `gh pr ready --repo "$SPEC_REPO"` only if that audit is promotable (CI green + mergeable + clean). Otherwise comment the blocking gate and leave the PR draft. Never `gh pr merge`. Leave this single session alive for attach. +> `/goal` As the **single expert Advisor on `/worktrees`**, implement `.oh/tasks//prd.json` for PR `#` on branch `/-`. (1) **If `` is already provided** (autopilot's `$CRON_WORKTREE`, already on branch `/-`): `cd ` and do NOT create another worktree. **Otherwise** create an isolated worktree at `.worktrees//-` via `/worktrees` and `cd` into it. (2) Read `.oh/tasks//prompt.md`, implement the dependency-ready stories directly, and use `/delegate` only for bounded disjoint work. Reconcile worker results, validate every acceptance criterion, update `prd.json` and `progress.txt`, and append `STATUS: COMPLETE` only after every story passes. (3) Continue in this same Advisor session with the implementation-side audit loop — including the gate-5 simplify sub-loop, where you delete the code each finding names and drive `netAdded` down until the round cap or a non-reducing round ends it — `/eval` once, required wiki revision, `/compact`, `evidence.md`, `/spec retro`, improve steps, and a fresh `/audit pr`; run `gh pr ready --repo "$SPEC_REPO"` only if that audit is promotable (CI green + mergeable + clean). Otherwise comment the blocking gate and leave the PR draft. Never `gh pr merge`. Leave this single session alive for attach. The Advisor owns implementation and all post-build gates inside the same session. This node's turn ends after launching it and reporting the session name; the ready-for-review PR is @@ -240,15 +240,38 @@ When implementation is complete, run the per-unit verdict gate: ``` `/audit implementation` composes `prd.json` task-graph conformance + the `/eval` regression -floor + `/audit pr` promotable classification (+ `/agent-browser` for UI stories) into one -verdict: +floor + `/audit pr` promotable classification (+ `/agent-browser` for UI stories, + the +gate-5 slop check) into one verdict: - `AUDIT-FAIL` → loop back to implementation in the same Advisor session to finish the unmet stories, then re-audit. This is the implementation-side adversary — keep looping until the Advisor satisfies the task graph. - `AUDIT-PASS` → implementation is promotable; continue to the tail. -Two gates run inside this loop and must both clear before the audit can PASS. +**The simplify sub-loop — drive `netAdded` down.** Gate 5 asks whether the diff can be +smaller and still satisfy every acceptance criterion. On an `AUDIT-FAIL (gate 5)` the +Advisor removes the code the finding names — it does not argue with it — and re-audits. +The Advisor owns the round record; the read-only audit route only reads it: + +```bash +COUNTER=".oh/tasks//simplify-rounds.json" +ROUNDS=$(jq -r '.rounds // 0' "$COUNTER" 2>/dev/null || echo 0) +NET=$(AUDIT_ROOT="$PWD" bash .oh/skills/audit/scripts/implementation-gates.sh \ + slop-metrics "$BASE" | jq -r .netAdded) +cat > "$COUNTER" < Date: Mon, 31 Aug 2026 16:18:04 -0600 Subject: [PATCH 12/23] skill: make /spec ship a PR by default when handed a plan (#917) An unrecognized first token now routes to a new `ship` node instead of printing usage, so `/spec ` runs plan then execute through to a ready-for-review PR. `ship` composes the existing nodes and owns no build mechanics; `spec-family-contract` fails if it grows a build literal, if the dispatcher stops routing the default, or if ship.md stops stating how it treats the commitment gate. The gate itself survives: handing in an approved plan file is the approval, and a bare topic with no plan file still stops after `plan`. Closes #914 Submitted-by: kre8mymedia@gmail.com --- .oh/evals/RESULTS.md | 212 +++++++++++------------ .oh/evals/probes/spec-family-contract.sh | 25 ++- .oh/skills/spec/SKILL.md | 77 +++++--- .oh/skills/spec/references/plan.md | 4 + .oh/skills/spec/references/ship.md | 121 +++++++++++++ CHANGELOG.md | 1 + 6 files changed, 304 insertions(+), 136 deletions(-) create mode 100644 .oh/skills/spec/references/ship.md diff --git a/.oh/evals/RESULTS.md b/.oh/evals/RESULTS.md index 676304a7..54af09a7 100644 --- a/.oh/evals/RESULTS.md +++ b/.oh/evals/RESULTS.md @@ -6,111 +6,111 @@ probe id; git history is the time series.** Schema and exit-code semantics are i | probe | tier | last-run (UTC) | status | source | |-------|------|----------------|--------|--------| -| advisor-monitored-loop | A | 2026-08-31 21:20 | PASS | conversation 2026-06-19 (single-owner implementation workflow, issue #257) | -| agent-browser-cli | A | 2026-08-31 21:20 | PASS | retro lesson 2026-06-07 (agent-browser 0.8.5 CLI) | -| agents-identity-contract | A | 2026-08-31 21:20 | PASS | issue #854 — T3-style root identity, glossary, and skill-owned procedures | -| artifact-contract-audit | A | 2026-08-31 21:20 | PASS | issue #583/#645 — production /audit implementation Gate 1 behavior | -| audit-dispatcher-contract | A | 2026-08-31 21:20 | PASS | issue #645 — audit consolidation public taxonomy | -| audit-implementation-behavior | A | 2026-08-31 21:20 | PASS | issue #645 — implementation root/repo/browser behavior | -| audit-pr-acquire | A | 2026-08-31 21:20 | PASS | issue #645 — production PR acquisition behavior | -| audit-pr-classifier | A | 2026-08-31 21:20 | PASS | issue #645 — deterministic focused and queue PR classifier | -| audit-run-root-contract | A | 2026-08-31 21:20 | PASS | issue #645 — executable immutable audit root/run correlation | -| audit-shellcheck-coverage | A | 2026-08-31 21:20 | PASS | issue #645 — private audit scripts require release and CI lint coverage | -| audit-slop-gate | A | 2026-08-31 21:20 | PASS | .claude/specs/images/reduce-slop.png — the four correctness gates all pass a change | -| audit-stale-references | A | 2026-08-31 21:20 | PASS | issue #645 — clean-breaking audit migration | -| boot-lint-glob | A | 2026-08-31 21:20 | PASS | issue #90, issue #120 | -| builder-skill-consolidation | A | 2026-08-31 21:20 | PASS | issue #643 — consolidate artifact builders behind one /builder dispatcher | -| capability-benchmark-schema | A | 2026-08-31 21:20 | PASS | issue #167 — capability benchmark instrument | -| cc-safety-net-wiring | A | 2026-08-31 21:20 | SKIPPED | .oh/tasks/cc-safety-net/prd.json US-007 2026-07-19 | -| changelog-entry-length | A | 2026-08-31 21:20 | PASS | conversation 2026-08-24 — CHANGELOG.md grew to 259KB of bullet prose because "one line" was unquantified | -| cleanup-tasks-scoped-guard | A | 2026-08-31 21:20 | PASS | issue #85 | -| cleanup-tasks-worktree-grooming | A | 2026-08-31 21:20 | PASS | issue #168; issue #327 | -| cli-publish-typecheck-scope | A | 2026-08-31 21:20 | PASS | release run 33271077312 — v0.5.0 pushed its GHCR image, then failed to publish | -| close-issues-on-development | A | 2026-08-31 21:20 | PASS | issue #841 (closing keywords never fire because the default branch is main) 2026-08-26 | -| codex-stale-response-retry | A | 2026-08-31 21:20 | PASS | issue #506 — Codex previous_response_not_found RCA | -| compose-config-path-parity | A | 2026-08-31 21:20 | PASS | PR #833 (remove harness.yaml — the wrapper and VS Code "Reopen in Container" paths must resolve the same service) 2026-08-26 | -| config-schema-parity | A | 2026-08-31 21:20 | PASS | PR #833 (one schema file — DOCKER_SOCKET, SANDBOX_SSH, OH_SANDBOX_IMAGE, OH_PULL_POLICY, SKIP_PNPM_INSTALL were consumed but undocumented); rewritten for the oh.json/secrets split by PR #887 | -| context-tier-size-budget | A | 2026-08-31 21:20 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-007) — the always-on tier was 85,256 B | -| cron-claude-codex-fallback | A | 2026-08-31 21:20 | PASS | conversation 2026-06-12 (default Codex fallback for crons) | -| cron-watchdog | A | 2026-08-31 21:20 | PASS | issues #130/#453 (cron runtime watchdog + legacy system-cron reaping) 2026-06-19 | -| crons-directory-guide | A | 2026-08-31 21:20 | PASS | issue #874 | -| curl-bash-safe-alternatives | A | 2026-08-31 21:20 | PASS | vet-run/vet integration — public curl|bash examples need review-first alternatives | -| datasets-schema | A | 2026-08-31 21:20 | PASS | issue #196 — .oh/evals/datasets verifiable trajectory corpus (Repo2RLEnv-inspired) | -| debugmcp-availability | A | 2026-08-31 21:20 | SKIPPED | issue #297 — DebugMCP MCP debug-server availability | -| default-provisioning | A | 2026-08-31 21:20 | PASS | #902 — `oh harness install` must work from inside the sandbox, where | -| delegate-model-effort-policy | A | 2026-08-31 21:20 | PASS | conversation 2026-07-11 (delegate model inheritance and thinking policy) | -| devtcp-hook | A | 2026-08-31 21:20 | PASS | retro lesson 2026-06-10 (zsh /dev/tcp) | -| docker-inspect-env-guard | A | 2026-08-31 21:20 | PASS | operator directive 2026-08-08 (agents keep the docker socket, but must | -| docs-build-fast-path | A | 2026-08-31 21:20 | PASS | #455 — docs builds must stay out of fast harness/eval/release gates; #536 — docs site externalized to openharness-web; docs markdown relocated to docs/ | -| drift-check-cron-staleness-glob | A | 2026-08-31 21:20 | PASS | issue #98; issue #225 (restart-required cron frontmatter/config drift) | -| entrypoint-pnpm-manifest-fingerprint | A | 2026-08-31 21:20 | PASS | issue #521 (manifest-aware sandbox installs) 2026-07-01 | -| eval-ci-gate | A | 2026-08-31 21:20 | PASS | #103 — eval probe suite gated in CI | -| eval-gate | A | 2026-08-31 21:20 | PASS | retro lesson 2026-06-11 (eval-gate) | -| eval-results-atomic | A | 2026-08-31 21:20 | PASS | issue #83 (eval-results-atomic-write) | -| eval-runner-exit | A | 2026-08-31 21:20 | PASS | retro lesson 2026-06-11 (eval-runner-exit) #29 | -| eval-runs-once-per-cycle | A | 2026-08-31 21:20 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-006) — /eval ran 3x per cycle on the | -| execution-target-contract | A | 2026-08-31 21:20 | PASS | issue #733 (ExecutionTarget contract + Docker Compose adapter) 2026-08-10 | -| get-oh-bootstrap | A | 2026-08-31 21:20 | PASS | get-oh.sh bootstrap — the Node-bootstrapping host-side path to the standalone `oh` CLI (also on npm as @mifune/openharness; see oh-npm-package.sh) | -| git-skill | A | 2026-08-31 21:20 | PASS | conversation 2026-06-15 — rules are not always supported; git workflow must be the /git skill | -| harness-audit-empty-output-gate | A | 2026-08-31 21:20 | PASS | issue #246 — /audit harness must fail closed on empty auditor outputs | -| harness-ci-core-paths | A | 2026-08-31 21:20 | PASS | #165 — core sandbox config files must trigger harness CI | -| harness-ci-hooks-paths | A | 2026-08-31 21:20 | PASS | issue #202 — credential/security hook changes must trigger harness CI | -| harness-yaml-migration | A | 2026-08-31 21:20 | PASS | PR #833 (migrate-harness-yaml.sh — append / uncomment-in-place / preserve / overwrite, plus a silent no-op second run) 2026-08-26 | -| health-check-docker-stats | A | 2026-08-31 21:20 | PASS | retro lesson 2026-06-10 (docker stats vs ps Size) | -| health-check-socket-degrade | A | 2026-08-31 21:20 | PASS | issue #762 (refs #756) — /health-check degrades to one statement, not nine failures | -| heartbeat-logging-contract | A | 2026-08-31 21:20 | PASS | issue #447 (heartbeat log append hardening) 2026-06-18 | -| image-seed-hygiene | A | 2026-08-31 21:20 | PASS | issue #900 (slim the sandbox image) 2026-08-30 | -| markitdown-wiki-ingest | A | 2026-08-31 21:20 | PASS | issue #649 — pinned local-document normalization contract for /wiki ingest | -| next-dev-prod | A | 2026-08-31 21:20 | SKIPPED | retro lesson 2026-06-04 | -| oh-compose-env-wiring | A | 2026-08-31 21:20 | SKIPPED | issue #880 (oh as the only front door — oh.json is the non-secret config surface) | -| oh-config-surfaces | A | 2026-08-31 21:20 | PASS | PR #887 (config split across two authored surfaces — a tracked oh.json and a secrets-only root dotenv — with nothing left under $HOME) | -| oh-destroy-guard | A | 2026-08-31 21:20 | SKIPPED | issue #879 — `oh` becomes the only front door, so `make destroy` must | -| oh-devcontainer-restructure | A | 2026-08-31 21:20 | PASS | consolidate devcontainer — .oh/devcontainer/ folded back into .devcontainer/ | -| oh-home-mount | A | 2026-08-31 21:20 | PASS | issue #898 (single $HOME mount) 2026-08-30 | -| oh-image-only-deploy | A | 2026-08-31 21:20 | PASS | .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only deploy) | -| oh-init-headless-config | A | 2026-08-31 21:20 | SKIPPED | PR #827 (installer answers landed in the losing config file); retargeted to the .example.env template by PR #833, then to oh.json by PR #887 | -| oh-init-scaffold | A | 2026-08-31 21:20 | PASS | issue #531 Phase 2 | -| oh-lifecycle-surface | A | 2026-08-31 21:20 | PASS | issue #881 — the Makefile is retired and `oh` is the only front door | -| oh-npm-package | A | 2026-08-31 21:20 | PASS | npm publish path for the standalone `oh` CLI (@mifune/openharness) — alternative to get-oh.sh | -| oh-payload-manifest | A | 2026-08-31 21:20 | PASS | issue #531 follow-on (.oh payload manifest — oh update ships a declared allowlist) | -| oh-sandbox-image-mode | A | 2026-08-31 21:20 | PASS | conversation 2026-07-05 (basic Docker deployment — prebuilt-image mode) | -| oh-shipped-repo-overridable | A | 2026-08-31 21:20 | PASS | issue #531 follow-on (de-hardcode residual — shipped .oh shell scripts keep the upstream repo overridable) | -| oh-standalone-lifecycle | A | 2026-08-31 21:20 | PASS | issue #564 | -| oh-update | A | 2026-08-31 21:20 | PASS | issue #531 Phase 3 (oh update — upgrade only the .oh control plane) | -| operator-config-guard | A | 2026-08-31 21:20 | PASS | operator directives 2026-08-06 (.config/ and settings.local.json are operator-only) | -| pnpm-audit-ci-gate | A | 2026-08-31 21:20 | PASS | issue #171 — pnpm security audits must run in CI | -| post-bridge-publish-confirmation | A | 2026-08-31 21:20 | PASS | #523 — post-bridge live publishing requires an explicit final confirmation gate | -| prd-output-path-contract | A | 2026-08-31 21:20 | PASS | retro lesson 2026-06-19 | -| prompt-miner-schema-compat | A | 2026-08-31 21:20 | PASS | issue #253 — prompt-miner JSONL schema-drift guard | -| prompt-miner-symlink-entrypoint | A | 2026-08-31 21:20 | PASS | issue #663 — prompt-miner engine no-ops via the documented .claude/skills symlink | -| prompt-miner-weakness-record | A | 2026-08-31 21:20 | PASS | issue #580 — prompt-miner weakness-record (WH-xxx) cluster output | -| protected-path-deletion | A | 2026-08-31 21:20 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-001) — the critique gate was deleted, | -| protected-paths-resolve | A | 2026-08-31 21:20 | PASS | issue #753 — .claude/protected-paths.txt named 7 paths that did not exist. | -| registry-portability-gate | A | 2026-08-31 21:20 | PASS | issue #758 | -| registry-portability | A | 2026-08-31 21:20 | SKIPPED | issue #758 | -| retro-deterministic-contract | A | 2026-08-31 21:20 | PASS | issue #443 — /retro deterministic output and self-contained helper contract | -| rl-delegation-write-worker | A | 2026-08-31 21:20 | PASS | retro lesson 2026-06-10 (rl-delegation) #57 | -| rlm-context-budget | A | 2026-08-31 21:20 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-006 | -| runtime-preflight-gate | A | 2026-08-31 21:20 | PASS | issue #806 § B1 (open sandbox.substrate vs sandbox.runtime selector); | -| sandbox-boot-guard-ci | A | 2026-08-31 21:20 | PASS | issue #449 (sandbox image build CI guard) 2026-06-19; | -| sandbox-node-base | A | 2026-08-31 21:20 | PASS | openharness#878 — oh as the only front door, T0 sandbox base image | -| skill-paths | A | 2026-08-31 21:20 | PASS | issue #43 — stale path references; extended by issue #69 — apps/->packages/ rename guard; extended by issue #870 — deleted .oh/agents/advisor.md | -| skills-dir-clean | A | 2026-08-31 21:20 | PASS | conversation 2026-06-29 — Pi parses every top-level `.md` in the skills | -| skills-task-tool-coupling | A | 2026-08-31 21:20 | PASS | council review 2026-08-29 (issue #886) — /delegate instructed Claude-Code-only | -| skills-vendored | A | 2026-08-31 21:20 | PASS | absorb .mifune submodule into .oh — the skills/agents/hooks pack is vendored | -| slack-admin-command-surface | A | 2026-08-31 21:20 | PASS | issue #354 — Slack bridge docs must distinguish Pi /msg-bridge commands from Slack DM admin text handlers | -| spec-family-contract | A | 2026-08-31 21:20 | PASS | issue #265; spec-simplification issue #816; workflow authority issue #854 | -| spec-ready-finalization | A | 2026-08-31 21:20 | PASS | issue #134; spec-simplification issue #816; workflow authority issue #854 | -| ste-checker-contract | A | 2026-08-31 21:20 | PASS | issue #750 PR audit — the /ste checker had four fail-open paths (unclosed | -| submitted-by-trailers | A | 2026-08-31 21:20 | PASS | conversation 2026-06-12 (commit attribution trailers); the single-owner | -| sync-skill-contract | A | 2026-08-31 21:20 | PASS | issue #331 — /sync dispatcher skill (bidirectional origin↔upstream sync) | -| t3-headless-launch | A | 2026-08-31 21:20 | PASS | issue #858 — /t3 launched a bare `npx --yes t3`, which is the local GUI and | -| tailscale-tool-boundary | A | 2026-08-31 21:20 | PASS | issue #858 — Tailscale mobile access for T3 Code. There is no tailnet, no | -| tool-catalog-boundary | A | 2026-08-31 21:20 | PASS | agent-browser's exclusion from the harness catalog (#821) and the | -| version-parity | A | 2026-08-31 21:20 | PASS | conversation 2026-08-29 — the oh CLI became the only lifecycle door, so its | -| weigh-scorer-contract | A | 2026-08-31 21:20 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-003 (2026-06-27) | -| wiki-readme-index | A | 2026-08-31 21:20 | PASS | issue #132 — wiki README index drift guard | -| workflow-boundaries | A | 2026-08-31 21:20 | PASS | conversation 2026-06-19 (workflow consolidation, issue #259); authority moved to /spec in issue #854 | -| worktrees-layout | A | 2026-08-31 21:20 | PASS | issue #872 | +| advisor-monitored-loop | A | 2026-08-31 22:13 | PASS | conversation 2026-06-19 (single-owner implementation workflow, issue #257) | +| agent-browser-cli | A | 2026-08-31 22:13 | PASS | retro lesson 2026-06-07 (agent-browser 0.8.5 CLI) | +| agents-identity-contract | A | 2026-08-31 22:13 | PASS | issue #854 — T3-style root identity, glossary, and skill-owned procedures | +| artifact-contract-audit | A | 2026-08-31 22:13 | PASS | issue #583/#645 — production /audit implementation Gate 1 behavior | +| audit-dispatcher-contract | A | 2026-08-31 22:13 | PASS | issue #645 — audit consolidation public taxonomy | +| audit-implementation-behavior | A | 2026-08-31 22:13 | PASS | issue #645 — implementation root/repo/browser behavior | +| audit-pr-acquire | A | 2026-08-31 22:13 | PASS | issue #645 — production PR acquisition behavior | +| audit-pr-classifier | A | 2026-08-31 22:13 | PASS | issue #645 — deterministic focused and queue PR classifier | +| audit-run-root-contract | A | 2026-08-31 22:13 | PASS | issue #645 — executable immutable audit root/run correlation | +| audit-shellcheck-coverage | A | 2026-08-31 22:13 | PASS | issue #645 — private audit scripts require release and CI lint coverage | +| audit-slop-gate | A | 2026-08-31 22:13 | PASS | .claude/specs/images/reduce-slop.png — the four correctness gates all pass a change | +| audit-stale-references | A | 2026-08-31 22:13 | PASS | issue #645 — clean-breaking audit migration | +| boot-lint-glob | A | 2026-08-31 22:13 | PASS | issue #90, issue #120 | +| builder-skill-consolidation | A | 2026-08-31 22:13 | PASS | issue #643 — consolidate artifact builders behind one /builder dispatcher | +| capability-benchmark-schema | A | 2026-08-31 22:13 | PASS | issue #167 — capability benchmark instrument | +| cc-safety-net-wiring | A | 2026-08-31 22:13 | SKIPPED | .oh/tasks/cc-safety-net/prd.json US-007 2026-07-19 | +| changelog-entry-length | A | 2026-08-31 22:13 | PASS | conversation 2026-08-24 — CHANGELOG.md grew to 259KB of bullet prose because "one line" was unquantified | +| cleanup-tasks-scoped-guard | A | 2026-08-31 22:13 | PASS | issue #85 | +| cleanup-tasks-worktree-grooming | A | 2026-08-31 22:13 | PASS | issue #168; issue #327 | +| cli-publish-typecheck-scope | A | 2026-08-31 22:13 | PASS | release run 33271077312 — v0.5.0 pushed its GHCR image, then failed to publish | +| close-issues-on-development | A | 2026-08-31 22:13 | PASS | issue #841 (closing keywords never fire because the default branch is main) 2026-08-26 | +| codex-stale-response-retry | A | 2026-08-31 22:13 | PASS | issue #506 — Codex previous_response_not_found RCA | +| compose-config-path-parity | A | 2026-08-31 22:13 | PASS | PR #833 (remove harness.yaml — the wrapper and VS Code "Reopen in Container" paths must resolve the same service) 2026-08-26 | +| config-schema-parity | A | 2026-08-31 22:13 | PASS | PR #833 (one schema file — DOCKER_SOCKET, SANDBOX_SSH, OH_SANDBOX_IMAGE, OH_PULL_POLICY, SKIP_PNPM_INSTALL were consumed but undocumented); rewritten for the oh.json/secrets split by PR #887 | +| context-tier-size-budget | A | 2026-08-31 22:13 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-007) — the always-on tier was 85,256 B | +| cron-claude-codex-fallback | A | 2026-08-31 22:13 | PASS | conversation 2026-06-12 (default Codex fallback for crons) | +| cron-watchdog | A | 2026-08-31 22:13 | PASS | issues #130/#453 (cron runtime watchdog + legacy system-cron reaping) 2026-06-19 | +| crons-directory-guide | A | 2026-08-31 22:13 | PASS | issue #874 | +| curl-bash-safe-alternatives | A | 2026-08-31 22:13 | PASS | vet-run/vet integration — public curl|bash examples need review-first alternatives | +| datasets-schema | A | 2026-08-31 22:13 | PASS | issue #196 — .oh/evals/datasets verifiable trajectory corpus (Repo2RLEnv-inspired) | +| debugmcp-availability | A | 2026-08-31 22:13 | SKIPPED | issue #297 — DebugMCP MCP debug-server availability | +| default-provisioning | A | 2026-08-31 22:13 | PASS | #902 — `oh harness install` must work from inside the sandbox, where | +| delegate-model-effort-policy | A | 2026-08-31 22:13 | PASS | conversation 2026-07-11 (delegate model inheritance and thinking policy) | +| devtcp-hook | A | 2026-08-31 22:13 | PASS | retro lesson 2026-06-10 (zsh /dev/tcp) | +| docker-inspect-env-guard | A | 2026-08-31 22:13 | PASS | operator directive 2026-08-08 (agents keep the docker socket, but must | +| docs-build-fast-path | A | 2026-08-31 22:13 | PASS | #455 — docs builds must stay out of fast harness/eval/release gates; #536 — docs site externalized to openharness-web; docs markdown relocated to docs/ | +| drift-check-cron-staleness-glob | A | 2026-08-31 22:13 | PASS | issue #98; issue #225 (restart-required cron frontmatter/config drift) | +| entrypoint-pnpm-manifest-fingerprint | A | 2026-08-31 22:13 | PASS | issue #521 (manifest-aware sandbox installs) 2026-07-01 | +| eval-ci-gate | A | 2026-08-31 22:13 | PASS | #103 — eval probe suite gated in CI | +| eval-gate | A | 2026-08-31 22:13 | PASS | retro lesson 2026-06-11 (eval-gate) | +| eval-results-atomic | A | 2026-08-31 22:13 | PASS | issue #83 (eval-results-atomic-write) | +| eval-runner-exit | A | 2026-08-31 22:13 | PASS | retro lesson 2026-06-11 (eval-runner-exit) #29 | +| eval-runs-once-per-cycle | A | 2026-08-31 22:13 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-006) — /eval ran 3x per cycle on the | +| execution-target-contract | A | 2026-08-31 22:13 | PASS | issue #733 (ExecutionTarget contract + Docker Compose adapter) 2026-08-10 | +| get-oh-bootstrap | A | 2026-08-31 22:13 | PASS | get-oh.sh bootstrap — the Node-bootstrapping host-side path to the standalone `oh` CLI (also on npm as @mifune/openharness; see oh-npm-package.sh) | +| git-skill | A | 2026-08-31 22:13 | PASS | conversation 2026-06-15 — rules are not always supported; git workflow must be the /git skill | +| harness-audit-empty-output-gate | A | 2026-08-31 22:13 | PASS | issue #246 — /audit harness must fail closed on empty auditor outputs | +| harness-ci-core-paths | A | 2026-08-31 22:13 | PASS | #165 — core sandbox config files must trigger harness CI | +| harness-ci-hooks-paths | A | 2026-08-31 22:13 | PASS | issue #202 — credential/security hook changes must trigger harness CI | +| harness-yaml-migration | A | 2026-08-31 22:13 | PASS | PR #833 (migrate-harness-yaml.sh — append / uncomment-in-place / preserve / overwrite, plus a silent no-op second run) 2026-08-26 | +| health-check-docker-stats | A | 2026-08-31 22:13 | PASS | retro lesson 2026-06-10 (docker stats vs ps Size) | +| health-check-socket-degrade | A | 2026-08-31 22:13 | PASS | issue #762 (refs #756) — /health-check degrades to one statement, not nine failures | +| heartbeat-logging-contract | A | 2026-08-31 22:13 | PASS | issue #447 (heartbeat log append hardening) 2026-06-18 | +| image-seed-hygiene | A | 2026-08-31 22:13 | PASS | issue #900 (slim the sandbox image) 2026-08-30 | +| markitdown-wiki-ingest | A | 2026-08-31 22:13 | PASS | issue #649 — pinned local-document normalization contract for /wiki ingest | +| next-dev-prod | A | 2026-08-31 22:13 | SKIPPED | retro lesson 2026-06-04 | +| oh-compose-env-wiring | A | 2026-08-31 22:13 | SKIPPED | issue #880 (oh as the only front door — oh.json is the non-secret config surface) | +| oh-config-surfaces | A | 2026-08-31 22:13 | PASS | PR #887 (config split across two authored surfaces — a tracked oh.json and a secrets-only root dotenv — with nothing left under $HOME) | +| oh-destroy-guard | A | 2026-08-31 22:13 | SKIPPED | issue #879 — `oh` becomes the only front door, so `make destroy` must | +| oh-devcontainer-restructure | A | 2026-08-31 22:13 | PASS | consolidate devcontainer — .oh/devcontainer/ folded back into .devcontainer/ | +| oh-home-mount | A | 2026-08-31 22:13 | PASS | issue #898 (single $HOME mount) 2026-08-30 | +| oh-image-only-deploy | A | 2026-08-31 22:13 | PASS | .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only deploy) | +| oh-init-headless-config | A | 2026-08-31 22:13 | SKIPPED | PR #827 (installer answers landed in the losing config file); retargeted to the .example.env template by PR #833, then to oh.json by PR #887 | +| oh-init-scaffold | A | 2026-08-31 22:13 | PASS | issue #531 Phase 2 | +| oh-lifecycle-surface | A | 2026-08-31 22:13 | PASS | issue #881 — the Makefile is retired and `oh` is the only front door | +| oh-npm-package | A | 2026-08-31 22:13 | PASS | npm publish path for the standalone `oh` CLI (@mifune/openharness) — alternative to get-oh.sh | +| oh-payload-manifest | A | 2026-08-31 22:13 | PASS | issue #531 follow-on (.oh payload manifest — oh update ships a declared allowlist) | +| oh-sandbox-image-mode | A | 2026-08-31 22:13 | PASS | conversation 2026-07-05 (basic Docker deployment — prebuilt-image mode) | +| oh-shipped-repo-overridable | A | 2026-08-31 22:13 | PASS | issue #531 follow-on (de-hardcode residual — shipped .oh shell scripts keep the upstream repo overridable) | +| oh-standalone-lifecycle | A | 2026-08-31 22:13 | PASS | issue #564 | +| oh-update | A | 2026-08-31 22:13 | PASS | issue #531 Phase 3 (oh update — upgrade only the .oh control plane) | +| operator-config-guard | A | 2026-08-31 22:13 | PASS | operator directives 2026-08-06 (.config/ and settings.local.json are operator-only) | +| pnpm-audit-ci-gate | A | 2026-08-31 22:13 | PASS | issue #171 — pnpm security audits must run in CI | +| post-bridge-publish-confirmation | A | 2026-08-31 22:13 | PASS | #523 — post-bridge live publishing requires an explicit final confirmation gate | +| prd-output-path-contract | A | 2026-08-31 22:13 | PASS | retro lesson 2026-06-19 | +| prompt-miner-schema-compat | A | 2026-08-31 22:13 | PASS | issue #253 — prompt-miner JSONL schema-drift guard | +| prompt-miner-symlink-entrypoint | A | 2026-08-31 22:13 | PASS | issue #663 — prompt-miner engine no-ops via the documented .claude/skills symlink | +| prompt-miner-weakness-record | A | 2026-08-31 22:13 | PASS | issue #580 — prompt-miner weakness-record (WH-xxx) cluster output | +| protected-path-deletion | A | 2026-08-31 22:13 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-001) — the critique gate was deleted, | +| protected-paths-resolve | A | 2026-08-31 22:13 | PASS | issue #753 — .claude/protected-paths.txt named 7 paths that did not exist. | +| registry-portability-gate | A | 2026-08-31 22:13 | PASS | issue #758 | +| registry-portability | A | 2026-08-31 22:13 | SKIPPED | issue #758 | +| retro-deterministic-contract | A | 2026-08-31 22:13 | PASS | issue #443 — /retro deterministic output and self-contained helper contract | +| rl-delegation-write-worker | A | 2026-08-31 22:13 | PASS | retro lesson 2026-06-10 (rl-delegation) #57 | +| rlm-context-budget | A | 2026-08-31 22:13 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-006 | +| runtime-preflight-gate | A | 2026-08-31 22:13 | PASS | issue #806 § B1 (open sandbox.substrate vs sandbox.runtime selector); | +| sandbox-boot-guard-ci | A | 2026-08-31 22:13 | PASS | issue #449 (sandbox image build CI guard) 2026-06-19; | +| sandbox-node-base | A | 2026-08-31 22:13 | PASS | openharness#878 — oh as the only front door, T0 sandbox base image | +| skill-paths | A | 2026-08-31 22:13 | PASS | issue #43 — stale path references; extended by issue #69 — apps/->packages/ rename guard; extended by issue #870 — deleted .oh/agents/advisor.md | +| skills-dir-clean | A | 2026-08-31 22:13 | PASS | conversation 2026-06-29 — Pi parses every top-level `.md` in the skills | +| skills-task-tool-coupling | A | 2026-08-31 22:13 | PASS | council review 2026-08-29 (issue #886) — /delegate instructed Claude-Code-only | +| skills-vendored | A | 2026-08-31 22:13 | PASS | absorb .mifune submodule into .oh — the skills/agents/hooks pack is vendored | +| slack-admin-command-surface | A | 2026-08-31 22:13 | PASS | issue #354 — Slack bridge docs must distinguish Pi /msg-bridge commands from Slack DM admin text handlers | +| spec-family-contract | A | 2026-08-31 22:13 | PASS | issue #265; spec-simplification issue #816; workflow authority issue #854; | +| spec-ready-finalization | A | 2026-08-31 22:13 | PASS | issue #134; spec-simplification issue #816; workflow authority issue #854 | +| ste-checker-contract | A | 2026-08-31 22:13 | PASS | issue #750 PR audit — the /ste checker had four fail-open paths (unclosed | +| submitted-by-trailers | A | 2026-08-31 22:13 | PASS | conversation 2026-06-12 (commit attribution trailers); the single-owner | +| sync-skill-contract | A | 2026-08-31 22:13 | PASS | issue #331 — /sync dispatcher skill (bidirectional origin↔upstream sync) | +| t3-headless-launch | A | 2026-08-31 22:13 | PASS | issue #858 — /t3 launched a bare `npx --yes t3`, which is the local GUI and | +| tailscale-tool-boundary | A | 2026-08-31 22:13 | PASS | issue #858 — Tailscale mobile access for T3 Code. There is no tailnet, no | +| tool-catalog-boundary | A | 2026-08-31 22:13 | PASS | agent-browser's exclusion from the harness catalog (#821) and the | +| version-parity | A | 2026-08-31 22:13 | PASS | conversation 2026-08-29 — the oh CLI became the only lifecycle door, so its | +| weigh-scorer-contract | A | 2026-08-31 22:13 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-003 (2026-06-27) | +| wiki-readme-index | A | 2026-08-31 22:13 | PASS | issue #132 — wiki README index drift guard | +| workflow-boundaries | A | 2026-08-31 22:13 | PASS | conversation 2026-06-19 (workflow consolidation, issue #259); authority moved to /spec in issue #854 | +| worktrees-layout | A | 2026-08-31 22:13 | PASS | issue #872 | diff --git a/.oh/evals/probes/spec-family-contract.sh b/.oh/evals/probes/spec-family-contract.sh index 38cf3c6c..51737899 100755 --- a/.oh/evals/probes/spec-family-contract.sh +++ b/.oh/evals/probes/spec-family-contract.sh @@ -1,7 +1,9 @@ #!/usr/bin/env bash # tier: A -# source: issue #265; spec-simplification issue #816; workflow authority issue #854 -# desc: /spec owns the three-node folder workflow and execute.md carries the complete build. +# source: issue #265; spec-simplification issue #816; workflow authority issue #854; +# ship-by-default issue #914 +# desc: /spec owns the four-node workflow — ship (the default, composing plan then +# execute), plan, execute, retro — and execute.md carries the complete build. set -u ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" @@ -9,7 +11,7 @@ SKILLS="$ROOT/.claude/skills" SPEC="$SKILLS/spec" AGENTS="$ROOT/AGENTS.md" -subs=(plan execute retro) +subs=(ship plan execute retro) retired_subs=(critique) if [ ! -f "$SPEC/SKILL.md" ]; then @@ -34,7 +36,7 @@ for s in "${retired_subs[@]}"; do done [ -e "$SKILLS/approve" ] && missing+=("approve: retired skill directory still present (.claude/skills/approve)") -for f in "$SPEC/SKILL.md" "$SPEC/references"/plan.md \ +for f in "$SPEC/SKILL.md" "$SPEC/references"/ship.md "$SPEC/references"/plan.md \ "$SPEC/references"/execute.md "$SPEC/references"/retro.md; do [ -f "$f" ] || continue rel="${f#"$ROOT"/}" @@ -46,6 +48,19 @@ for f in "$SPEC/SKILL.md" "$SPEC/references"/plan.md \ done [ -e "$SKILLS/ship-spec" ] && missing+=("ship-spec: the all-in-one composer must be absorbed and deleted, not left beside /spec") +SHIP="$SPEC/references/ship.md" +if [ -f "$SHIP" ]; then + # ship composes plan and execute; owning a build literal would fork execute.md. + for literal in 'gh pr create' 'gh pr ready' 'gh issue create' 'git push'; do + grep -qF "$literal" "$SHIP" && missing+=("references/ship.md carries the build literal '$literal' — ship composes plan and execute and must not fork execute.md") + done + grep -qF 'commitment gate' "$SHIP" || missing+=("references/ship.md does not state how it treats the commitment gate") +fi +# The dispatcher must route an unrecognized first token to ship, not to usage. +grep -qF 'ship|plan|execute|retro)' "$SPEC/SKILL.md" \ + || missing+=("SKILL.md dispatch case does not name all four nodes") +grep -qE 'DEFAULT: not a node name' "$SPEC/SKILL.md" \ + || missing+=("SKILL.md no longer routes an unrecognized first token to ship (a plan path would print usage)") EXEC="$SPEC/references/execute.md" if [ -f "$EXEC" ]; then grep -qF 'reuses those by reference' "$EXEC" && missing+=("execute.md still defers its build mechanics by reference instead of holding them") @@ -74,5 +89,5 @@ if [ "${#missing[@]}" -gt 0 ]; then exit 1 fi -echo "PASS: /spec owns the workflow, dispatches three folder-pointed procedures, carries no loop ## Handoff, keeps retired surfaces absent, and holds the build literals" >&2 +echo "PASS: /spec owns the workflow, dispatches four procedures with ship as the default composing plan and execute, carries no loop ## Handoff, keeps retired surfaces absent, and holds the build literals" >&2 exit 0 diff --git a/.oh/skills/spec/SKILL.md b/.oh/skills/spec/SKILL.md index 44b4448e..55049db7 100644 --- a/.oh/skills/spec/SKILL.md +++ b/.oh/skills/spec/SKILL.md @@ -2,17 +2,21 @@ name: spec description: >- Canonical decomposed build workflow and dispatcher. Routes the first token of - $ARGUMENTS to one of three subcommands: plan, execute, or retro. Each is pointed - at a .oh/tasks// folder (the universal interface) and is independently - runnable and fan-out-able. This skill owns the ONLY build path; there is no - all-in-one composer beside it. Full per-subcommand procedures live in - references/{plan,execute,retro}.md. - TRIGGER when: a topic/plan/issue needs to become a buildable task folder, "plan - ", "scaffold the task for " -> plan; an approved .oh/tasks// - folder needs building to a promotable PR, "execute ", "build " -> - execute; a build PASSed audit and its lessons should be captured, "retro the - build", "capture lessons for " -> retro. -argument-hint: "plan [--plan ] [--issue ] [--slug ] [--prefix ] [--repo ] [--base ] | execute [--pr ] [--repo ] [--remote ] [--base ] | retro [--dry-run]" + $ARGUMENTS to one of four subcommands: ship, plan, execute, or retro. Each of + plan/execute/retro is pointed at a .oh/tasks// folder (the universal + interface) and is independently runnable and fan-out-able; ship composes plan + then execute. This skill owns the ONLY build path; there is no all-in-one + composer beside it. Full per-subcommand procedures live in + references/{ship,plan,execute,retro}.md. + TRIGGER when: an approved plan file should become a ready PR without further + hand-holding, "/spec ", "ship this plan", "build this plan end to end" + -> ship (also the DEFAULT for an unrecognized first token); a topic/plan/issue + needs to become a buildable task folder without building it, "plan ", + "scaffold the task for " -> plan; an approved .oh/tasks// folder + needs building to a promotable PR, "execute ", "build " -> execute; a + build PASSed audit and its lessons should be captured, "retro the build", + "capture lessons for " -> retro. +argument-hint: " | ship [--issue ] [--slug ] | plan [--plan ] [--issue ] [--slug ] [--prefix ] [--repo ] [--base ] | execute [--pr ] [--repo ] [--remote ] [--base ] | retro [--dry-run]" --- # /spec — canonical workflow dispatcher @@ -23,6 +27,11 @@ selects the subcommand; everything after it is that subcommand's own argument string. Each subcommand's full procedure lives in a reference doc under `references/` — read that doc and follow it as the authoritative instructions. +**An unrecognized first token is not an error — it is `ship`.** `/spec ` +is the ordinary way in: it scaffolds the task folder and then builds it through to a +ready-for-review pull request. Naming a node explicitly (`plan`, `execute`, `retro`) +runs only that node, which is what fan-out and recovery need. + This is the **only** spec pipeline; there is no all-in-one composer beside it. `references/execute.md` holds the build mechanics in full — the issue, the branch, the draft PR, the build launch, the `/eval` and wiki gates, the promotable @@ -36,7 +45,11 @@ The canonical operative path is `spec-plan → spec-execute → merge → reset|clean`. There is no automated selection node. A human selects the work and approves -`prd.md`; that approval is the commitment gate. `/spec execute` runs +`prd.md`; that approval is the commitment gate. **Handing `/spec` an approved plan +file satisfies that gate** — writing the plan and passing it in *is* the operator's +approval, so `ship` carries it through to `execute` without a second prompt. A bare +topic with no plan file has no such approval behind it: `ship` stops after `plan` and +hands the operator the folder to approve. `/spec execute` runs `build ⇄ audit → evidence → spec-retro → improve` and stops at a ready-for-review pull request. The human alone merges. The runner performs `reset` or `clean`. @@ -49,6 +62,7 @@ evidence is absent or uncommitted. | Subcommand | Arg shape | Purpose | Procedure | |---|---|---|---| +| `ship` | ` [--issue ] [--slug ] [--prefix ] [--repo ] [--base ]` | **The default.** `plan` → `execute` in one invocation: an approved plan file becomes a ready-for-review PR. Selected by an unrecognized first token, so `/spec ` works bare | `references/ship.md` | | `plan` | ` [--plan ] [--issue ] [--slug ] [--prefix ] [--repo ] [--base ]` | Turn a topic/plan/issue into a fully-scaffolded `.oh/tasks//` four-file folder | `references/plan.md` | | `execute` | ` [--pr ] [--repo ] [--remote ] [--base ]` | `implementation ⇄ audit → evidence → spec-retro → improve` to a ready PR, stopping at the human merge gate | `references/execute.md` | | `retro` | ` [--dry-run]` | Execution-side `/retro` scoped to a built `.oh/tasks//` | `references/retro.md` | @@ -56,20 +70,30 @@ evidence is absent or uncommitted. ## Dispatch 1. Split `$ARGUMENTS`: `SUB` = the first token; `REST` = everything after it. -2. Read `references/.md` and follow it, treating `REST` as that doc's - `$ARGUMENTS` (e.g. for `/spec plan --issue 7`, the plan procedure - sees ` --issue 7`). -3. Any unrecognized or empty `SUB` → print the Subcommands table as usage and stop. +2. When `SUB` names a node (`ship`, `plan`, `execute`, `retro`), read + `references/.md` and follow it, treating `REST` as that doc's `$ARGUMENTS` + (e.g. for `/spec plan --issue 7`, the plan procedure sees + ` --issue 7`). +3. **Any other non-empty `$ARGUMENTS` is `ship`, with the whole string — `SUB` + included — as its argument.** `/spec .claude/plans/x.md` and + `/spec ship .claude/plans/x.md` are the same invocation. Do not print usage for an + argument that merely fails to name a node; a plan path is the expected input. +4. Empty `$ARGUMENTS` → print the Subcommands table as usage and stop. There is + nothing to ship. ```bash -SUB="${ARGUMENTS%% *}" # first token (subcommand) -REST="${ARGUMENTS#"$SUB"}"; REST="${REST# }" # remainder = subcommand arguments +SUB="${ARGUMENTS%% *}" # first token +REST="${ARGUMENTS#"$SUB"}"; REST="${REST# }" # remainder case "$SUB" in - plan|execute|retro) + ship|plan|execute|retro) # read references/$SUB.md and execute it with REST as its $ARGUMENTS ;; + "") + echo "usage: /spec [ship] | plan | execute | retro " + ;; *) - echo "usage: /spec [args] — see the Subcommands table" + # DEFAULT: not a node name -> ship, keeping the full argument string + # read references/ship.md and execute it with "$ARGUMENTS" as its $ARGUMENTS ;; esac ``` @@ -78,12 +102,13 @@ esac - **This skill owns the workflow** — keep the operative path, human selection, plan-approval gate, evidence gate, and human merge boundary in this skill and its - three direct references. Do not duplicate the workflow in root instructions. + four direct references. Do not duplicate the workflow in root instructions. - **The `.oh/tasks//` folder is the universal interface** — `plan` produces it; `execute` and `retro` are each pointed at it. The `` is the universal key (task directory, branch second segment, tmux session name). - **Compose, don't fork** — each node reuses existing skills rather than - re-implementing them: `plan` composes `/prd` + `/ralph`; `execute` owns the + re-implementing them: `ship` composes `plan` then `execute` and owns no build + mechanics of its own; `plan` composes `/prd` + `/ralph`; `execute` owns the implementation in one Advisor session, uses `/delegate` only for bounded fan-out, and composes `/audit implementation` + `/eval` + `/audit pr`; `retro` composes `/retro`. The build **literals** — the `gh` invocations, the branch and PR shapes, @@ -95,7 +120,8 @@ esac commitment gate**, and nothing GitHub-side exists until `execute` starts. - **Honest terminal reports** — each subcommand reports what it actually produced: `plan` the folder path and story count; `execute` `READY` or `DRAFT-BLOCKED ()` with the PR - URL; `retro` the promotion counts. There are no `STATUS: SPEC-*` tokens — all four had + URL; `ship` the same terminal report as whichever node it stopped at; `retro` the + promotion counts. There are no `STATUS: SPEC-*` tokens — all four had **zero executable consumers repo-wide**, so printing them was ceremony. The rule they encoded still holds and is what matters: never infer success from silence. A missing artifact, a crashed build, or an undecided gate is reported as blocked, never as done. @@ -103,9 +129,10 @@ esac ## When NOT to use - **selection** — choosing which issue to build is the human's job; `/spec` - builds the one folder it is handed. + builds the one plan or folder it is handed. `ship` automates the hop from plan to + execute, never the choice of what to work on. ## See Also -- `references/plan.md`, `references/execute.md`, and +- `references/ship.md`, `references/plan.md`, `references/execute.md`, and `references/retro.md` — the authoritative per-subcommand procedures. diff --git a/.oh/skills/spec/references/plan.md b/.oh/skills/spec/references/plan.md index 186c4001..edfe588c 100644 --- a/.oh/skills/spec/references/plan.md +++ b/.oh/skills/spec/references/plan.md @@ -128,6 +128,10 @@ done Within the workflow owned by `.oh/skills/spec/SKILL.md`, `plan` is the first node. The operator approves `prd.md`, then runs `/spec execute `. +Running `plan` by name is the deliberate stop-after-scaffolding path. The default entry +point, `ship`, runs this node and then continues into `execute` when the operator handed +in an approved plan file (`references/ship.md`). + The terminal artifact is the folder itself: `.oh/tasks//` carrying the four-file contract, with `prd.md` awaiting the operator's approval. Report the folder path and the story count. There is no `STATUS: SPEC-PLANNED` token — it had no executable consumer, so diff --git a/.oh/skills/spec/references/ship.md b/.oh/skills/spec/references/ship.md new file mode 100644 index 00000000..4f388210 --- /dev/null +++ b/.oh/skills/spec/references/ship.md @@ -0,0 +1,121 @@ +# `/spec ship` — an approved plan to a ready pull request + +> Detail doc for the **`ship`** subcommand of the `/spec` skill +> (`.oh/skills/spec/SKILL.md`). Argument form: +> `ship [--issue ] [--slug ] [--prefix feat|bug|task|audit|skill|agent] [--repo ] [--base ]`. +> `ship` is the **default** node: the dispatcher routes any non-empty `$ARGUMENTS` +> whose first token is not `ship`/`plan`/`execute`/`retro` here, passing the whole +> string. Authority: `.oh/skills/spec/SKILL.md`. + +`ship` runs the canonical operative path — `plan → execute` — in one invocation, so +handing `/spec` a plan file produces a ready-for-review pull request. It produces no +artifact of its own: `plan` writes the `.oh/tasks//` folder that is the universal +interface, and `execute` builds from it. + +**Core principle: `ship` composes, it does not build.** Every artifact it produces is +produced by `plan` or `execute` under their own procedures. `ship` adds no mechanics of +its own — no `gh` invocation, no branch shape, no gate. It decides one thing: whether +the commitment gate is already satisfied, and therefore whether to continue into +`execute`. + +--- + +## Inputs + +| Arg | Meaning | +|-----|---------| +| `` | A readable path to a plan file (`.claude/plans/*.md`, `/imagine` output, any markdown spec). **Its presence is the operator's approval** — see the gate below. | +| `` | Free-text description, when no plan file exists. Carries no approval; `ship` stops after `plan`. | +| `--issue ` | Issue this builds. Passed to `plan` (which embeds it in the branch name via `/ralph`). When absent, `execute`'s standalone-run path opens one. | +| `--slug ` | Override slug derivation. `[a-z0-9-]+`, ≤5 hyphen-words, not `archive`. | +| `--prefix ` | Branch/issue prefix, default `feat`, per `.claude/skills/git/SKILL.md`. | +| `--repo ` | Default `mifunedev/openharness`. Recorded by `plan`, acted on by `execute`. | +| `--base ` | Default `development`. Same. | + +The first token is a plan path when it resolves to a readable file; otherwise it is +treated as the start of a free-text topic. + +--- + +## The commitment gate + +`.oh/skills/spec/SKILL.md` makes approving `prd.md` the commitment gate — nothing +GitHub-side exists until it is crossed. `ship` does not remove that gate; it recognizes +when the operator has already crossed it. + +| Input | Gate | Behavior | +|---|---|---| +| A plan file (`` or `--plan `) | **Satisfied.** The operator wrote the plan and handed it in; requiring a second approval of a `prd.md` derived from it asks the same question twice | `plan`, then `execute` — through to a ready PR | +| A bare topic, no plan file | **Not satisfied.** Nothing has been approved; the PRD is the first artifact anyone could approve | `plan` only. Report the folder path and stop with the `/spec execute ` invocation to run after approval | + +An operator who wants the folder without the build asks for the node by name: +`/spec plan `. That is the escape hatch, and it is why `plan` stays a public +subcommand. + +--- + +## The pipeline + +1. **Resolve the input.** Determine plan-path vs topic; derive or accept ``. + Report both before doing anything, so a wrong slug is caught before files exist. + +2. **Run `plan`.** Follow `references/plan.md` with + ` [--plan ] [--issue ] [--slug ] [--prefix ...] [--repo ...] [--base ...]`. + A plan-path first token is passed to `plan` as `--plan ` with the topic derived + from the plan's own title. Verify the four-file contract + (`prd.md`, `prd.json`, `prompt.md`, `progress.txt`) before continuing — an incomplete + folder is a failure, not a clean plan, and `ship` must not build on one. + +3. **Decide at the gate** (table above). Not satisfied → report and stop. This is a + complete, honest outcome, not a blocked one. + +4. **Run `execute`.** Follow `references/execute.md` with + ` [--repo ...] [--remote ...] [--base ...]`, which owns the issue, branch, draft + PR, Advisor build, `implementation ⇄ audit` loop, evidence, `/eval` and wiki gates, + and the undraft. `ship` neither reimplements nor relaxes any of it: a + `DRAFT-BLOCKED ()` from `execute` is `ship`'s outcome verbatim. + +5. **Stop at the human merge boundary.** `ship` never merges, and never marks a PR ready + that `execute`'s own gates left draft. + +--- + +## Output + +Whatever the node it stopped at produced: + +| Stopped at | Report | +|---|---| +| `plan` (gate not satisfied) | The folder path, the story count, and `/spec execute ` as the next command | +| `execute` | `READY` or `DRAFT-BLOCKED ()` with the PR URL | + +Report which node it stopped at and why. A run that stopped after `plan` because the +input was a bare topic is a success; a run that stopped after `plan` because the +four-file contract was incomplete is a failure. Never report them the same way, and +never infer success from silence. + +--- + +## What this node does NOT do + +- **Add a build step.** Every mechanic belongs to `plan` or `execute`. If `ship` needs + new build behavior, the behavior belongs in `execute.md` — the protected single source + for build literals — not here. +- **Merge.** The human alone merges (`.oh/skills/spec/SKILL.md`). +- **Select the work.** It builds the one plan it is handed. +- **Skip a gate.** It reads the commitment gate as already satisfied when an approved + plan was passed in. Every other gate — evidence, audit, `/eval`, wiki — is `execute`'s + and is untouched. + +## Pipeline position + +Within the workflow owned by `.oh/skills/spec/SKILL.md`, `ship` is the entry point — +the node an operator reaches by typing nothing but a plan path. It occupies no position +of its own in `spec-plan → spec-execute → merge → reset|clean`; it walks the first two +and stops at the human merge boundary, exactly where `execute` stops. + +## See Also + +- `.oh/skills/spec/SKILL.md` — the dispatcher and workflow contract; the authority. +- `references/plan.md` — the folder-scaffolding node. +- `references/execute.md` — the build node and the single source for its literals. diff --git a/CHANGELOG.md b/CHANGELOG.md index c7e62bb2..2583a8fc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,7 @@ Update policy and release automation live in [`/git`](.claude/skills/git/SKILL.m ## [Unreleased] ### Changed +- Make `/spec` ship by default: an unrecognized first token routes to a new `ship` node that runs `plan` then `execute`, so `/spec ` produces a ready-for-review PR ([#914](https://github.com/mifunedev/openharness/issues/914)). - **BREAKING:** Persist the sandbox home through one `/home/sandbox` mount, not eleven per-tool volumes; set `storage.homePath` for a host path, else `_workspace` ([#898](https://github.com/mifunedev/openharness/issues/898)). - Shrink the sandbox image ~540 MB: drop build caches from the baked home seed, stage the seed once via a builder stage, and keep untracked build output out of the build context ([#900](https://github.com/mifunedev/openharness/issues/900)). - **BREAKING:** Stop baking Claude Code, Codex, and Pi into the image; boot installs them into the home mount, so a first boot needs network and runs 60-180s longer ([#904](https://github.com/mifunedev/openharness/issues/904)). From 933a59f349280b773cfd1ed10266934e6bdc5e96 Mon Sep 17 00:00:00 2001 From: Ryan Eggleston Date: Mon, 31 Aug 2026 16:24:04 -0600 Subject: [PATCH 13/23] =?UTF-8?q?skill:=20close=20the=20lesson=E2=86=92ski?= =?UTF-8?q?ll=20loop=20with=20a=20wiki=20pattern=20layer=20and=20skill-imp?= =?UTF-8?q?act=20ledger=20(#916)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * skill: ingest WikiSkill (arXiv 2608.27454) into the wiki corpus * skill: add wiki lint related-slug check and generate the index from the tracked set * skill: add kind: source|pattern to the wiki schema and a --patterns query mode * skill: add /wiki compile and the skill-impact ledger * skill: make /builder consult wiki patterns and record proposals in the ledger * skill: record verdicts in the ledger and route retro lessons through /wiki compile * skill: add wiki co-evolution probes, CB-005, and the changelog entries * skill: compile the first wiki pattern from this session's lesson * skill: record SI-0001 in the skill-impact ledger * skill: scope the ledger probe to the append region and exercise its oracles * skill: fix the ledger record parser so appends do not read as mutations * skill: score CB-005 from the first real run of the lesson-to-skill chain * skill: refresh the eval scoreboard * skill: compile four patterns from the session retro via delegated maintainer * skill: close the compile procedure ambiguities the delegated run surfaced * skill: refresh the eval scoreboard * skill: re-score CB-005 from the delegated compile run * skill: mint the two retro-nominated probes and require fault injection * skill: restore the probe contract sections and shorten two changelog entries * skill: refresh the eval scoreboard * skill: regenerate the eval scoreboard after the merge --- .oh/evals/README.md | 32 +++ .oh/evals/RESULTS.md | 221 ++++++++--------- .oh/evals/capability/RESULTS.md | 11 +- .../tasks/CB-005-compile-a-lesson.md | 71 ++++++ .oh/evals/probes/builder-wiki-proposer.sh | 39 +++ .../probes/continual-learning-20260831.sh | 45 ++++ .../probes/eval-contract-text-20260831.sh | 34 +++ .oh/evals/probes/wiki-compile-contract.sh | 57 +++++ .oh/evals/probes/wiki-kind-schema-contract.sh | 78 ++++++ .oh/evals/probes/wiki-pattern-persistence.sh | 66 ++++++ .../probes/wiki-query-pattern-isolation.sh | 83 +++++++ .oh/evals/probes/wiki-related-slugs.sh | 67 ++++++ .../probes/wiki-skill-impact-append-only.sh | 103 ++++++++ .oh/skills/benchmark/SKILL.md | 16 +- .oh/skills/builder/SKILL.md | 13 + .oh/skills/builder/references/agent.md | 6 + .oh/skills/builder/references/command.md | 6 + .oh/skills/builder/references/rule.md | 6 + .oh/skills/builder/references/skill.md | 6 + .oh/skills/spec/references/execute.md | 9 +- .oh/skills/spec/references/retro.md | 7 +- .oh/skills/wiki/SKILL.md | 36 +-- .oh/skills/wiki/corpus/README.md | 10 +- .../pattern-docs-prohibition-by-example.md | 56 +++++ .../pattern-evals-prose-literal-pinning.md | 55 +++++ .../pattern-evals-unexercised-oracle.md | 65 +++++ ...attern-wiki-external-model-over-mapping.md | 59 +++++ .../pattern-wiki-ungated-check-drift.md | 58 +++++ ...-08-31-wikiskill-experience-compilation.md | 145 ++++++++++++ .../wiki/corpus/recursive-language-models.md | 10 +- .../recursive-self-improvement-survey.md | 3 +- .oh/skills/wiki/corpus/skill-impact.md | 102 ++++++++ .../wikiskill-experience-compilation.md | 82 +++++++ .oh/skills/wiki/references/compile.md | 223 ++++++++++++++++++ .oh/skills/wiki/references/ingest.md | 2 +- .oh/skills/wiki/references/lint.md | 93 +++++++- .oh/skills/wiki/references/query.md | 130 ++++++++-- .oh/skills/wiki/references/schema.md | 153 +++++++++++- CHANGELOG.md | 8 + 39 files changed, 2092 insertions(+), 174 deletions(-) create mode 100644 .oh/evals/capability/tasks/CB-005-compile-a-lesson.md create mode 100755 .oh/evals/probes/builder-wiki-proposer.sh create mode 100755 .oh/evals/probes/continual-learning-20260831.sh create mode 100755 .oh/evals/probes/eval-contract-text-20260831.sh create mode 100755 .oh/evals/probes/wiki-compile-contract.sh create mode 100755 .oh/evals/probes/wiki-kind-schema-contract.sh create mode 100755 .oh/evals/probes/wiki-pattern-persistence.sh create mode 100755 .oh/evals/probes/wiki-query-pattern-isolation.sh create mode 100755 .oh/evals/probes/wiki-related-slugs.sh create mode 100755 .oh/evals/probes/wiki-skill-impact-append-only.sh create mode 100644 .oh/skills/wiki/corpus/pattern-docs-prohibition-by-example.md create mode 100644 .oh/skills/wiki/corpus/pattern-evals-prose-literal-pinning.md create mode 100644 .oh/skills/wiki/corpus/pattern-evals-unexercised-oracle.md create mode 100644 .oh/skills/wiki/corpus/pattern-wiki-external-model-over-mapping.md create mode 100644 .oh/skills/wiki/corpus/pattern-wiki-ungated-check-drift.md create mode 100644 .oh/skills/wiki/corpus/raw/2026-08-31-wikiskill-experience-compilation.md create mode 100644 .oh/skills/wiki/corpus/skill-impact.md create mode 100644 .oh/skills/wiki/corpus/wikiskill-experience-compilation.md create mode 100644 .oh/skills/wiki/references/compile.md diff --git a/.oh/evals/README.md b/.oh/evals/README.md index 29b33989..efab9f03 100644 --- a/.oh/evals/README.md +++ b/.oh/evals/README.md @@ -56,6 +56,38 @@ probes from an arbitrary working directory, so the canonical preamble is: ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" # .oh/evals/probes/.sh -> root ``` +### Fault injection — a probe is not green until it has been red + +A probe that has only ever been run against a passing repository proves its PASS +branch exists and nothing about whether its oracle can detect the condition it was +written for. Before landing a probe, drive its REGRESSION branch against a +deliberately broken input and confirm it fails for the stated reason. + +Commit the work under test **before** injecting faults. A sweep that restores state +with `git checkout -- .` reverts uncommitted edits in the same tree, including the +contract text the probe is meant to guard. + +Where the condition lives in git history rather than the working tree, expose the +comparison point as an environment override so the failing branch stays reachable +after the fact — `.oh/evals/probes/wiki-skill-impact-append-only.sh` takes +`WIKI_LEDGER_BASE`, and `.oh/evals/probes/wiki-pattern-persistence.sh` takes +`WIKI_PERSISTENCE_BASE`, for exactly this reason. A one-off manual check that +leaves no such handle cannot be repeated by the next author. + +Treat `SKIPPED` the same way. A probe whose skip guard can fire in the environment +that normally runs it is unexercised, not healthy; prefer a guard whose absence is +itself a REGRESSION over one that exits 2. + +### Pinning contract text + +A contract-text probe asserts a document still makes a claim. Pin the shortest +fragment that is still unique — a heading, a table cell, a code token, or four to +six distinctive words. Do not pin a whole prose sentence: `grep -qF` matches within +a single line, so a sentence stored across a hard wrap can never match, and the +assertion breaks on reflow without the contract having changed. Where a whole +sentence is genuinely required, normalize whitespace before matching rather than +pinning the stored bytes. + ### Timeout Every probe must complete within a bounded time; the `/eval` runner wraps each diff --git a/.oh/evals/RESULTS.md b/.oh/evals/RESULTS.md index 54af09a7..b08b1cf2 100644 --- a/.oh/evals/RESULTS.md +++ b/.oh/evals/RESULTS.md @@ -6,111 +6,120 @@ probe id; git history is the time series.** Schema and exit-code semantics are i | probe | tier | last-run (UTC) | status | source | |-------|------|----------------|--------|--------| -| advisor-monitored-loop | A | 2026-08-31 22:13 | PASS | conversation 2026-06-19 (single-owner implementation workflow, issue #257) | -| agent-browser-cli | A | 2026-08-31 22:13 | PASS | retro lesson 2026-06-07 (agent-browser 0.8.5 CLI) | -| agents-identity-contract | A | 2026-08-31 22:13 | PASS | issue #854 — T3-style root identity, glossary, and skill-owned procedures | -| artifact-contract-audit | A | 2026-08-31 22:13 | PASS | issue #583/#645 — production /audit implementation Gate 1 behavior | -| audit-dispatcher-contract | A | 2026-08-31 22:13 | PASS | issue #645 — audit consolidation public taxonomy | -| audit-implementation-behavior | A | 2026-08-31 22:13 | PASS | issue #645 — implementation root/repo/browser behavior | -| audit-pr-acquire | A | 2026-08-31 22:13 | PASS | issue #645 — production PR acquisition behavior | -| audit-pr-classifier | A | 2026-08-31 22:13 | PASS | issue #645 — deterministic focused and queue PR classifier | -| audit-run-root-contract | A | 2026-08-31 22:13 | PASS | issue #645 — executable immutable audit root/run correlation | -| audit-shellcheck-coverage | A | 2026-08-31 22:13 | PASS | issue #645 — private audit scripts require release and CI lint coverage | -| audit-slop-gate | A | 2026-08-31 22:13 | PASS | .claude/specs/images/reduce-slop.png — the four correctness gates all pass a change | -| audit-stale-references | A | 2026-08-31 22:13 | PASS | issue #645 — clean-breaking audit migration | -| boot-lint-glob | A | 2026-08-31 22:13 | PASS | issue #90, issue #120 | -| builder-skill-consolidation | A | 2026-08-31 22:13 | PASS | issue #643 — consolidate artifact builders behind one /builder dispatcher | -| capability-benchmark-schema | A | 2026-08-31 22:13 | PASS | issue #167 — capability benchmark instrument | -| cc-safety-net-wiring | A | 2026-08-31 22:13 | SKIPPED | .oh/tasks/cc-safety-net/prd.json US-007 2026-07-19 | -| changelog-entry-length | A | 2026-08-31 22:13 | PASS | conversation 2026-08-24 — CHANGELOG.md grew to 259KB of bullet prose because "one line" was unquantified | -| cleanup-tasks-scoped-guard | A | 2026-08-31 22:13 | PASS | issue #85 | -| cleanup-tasks-worktree-grooming | A | 2026-08-31 22:13 | PASS | issue #168; issue #327 | -| cli-publish-typecheck-scope | A | 2026-08-31 22:13 | PASS | release run 33271077312 — v0.5.0 pushed its GHCR image, then failed to publish | -| close-issues-on-development | A | 2026-08-31 22:13 | PASS | issue #841 (closing keywords never fire because the default branch is main) 2026-08-26 | -| codex-stale-response-retry | A | 2026-08-31 22:13 | PASS | issue #506 — Codex previous_response_not_found RCA | -| compose-config-path-parity | A | 2026-08-31 22:13 | PASS | PR #833 (remove harness.yaml — the wrapper and VS Code "Reopen in Container" paths must resolve the same service) 2026-08-26 | -| config-schema-parity | A | 2026-08-31 22:13 | PASS | PR #833 (one schema file — DOCKER_SOCKET, SANDBOX_SSH, OH_SANDBOX_IMAGE, OH_PULL_POLICY, SKIP_PNPM_INSTALL were consumed but undocumented); rewritten for the oh.json/secrets split by PR #887 | -| context-tier-size-budget | A | 2026-08-31 22:13 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-007) — the always-on tier was 85,256 B | -| cron-claude-codex-fallback | A | 2026-08-31 22:13 | PASS | conversation 2026-06-12 (default Codex fallback for crons) | -| cron-watchdog | A | 2026-08-31 22:13 | PASS | issues #130/#453 (cron runtime watchdog + legacy system-cron reaping) 2026-06-19 | -| crons-directory-guide | A | 2026-08-31 22:13 | PASS | issue #874 | -| curl-bash-safe-alternatives | A | 2026-08-31 22:13 | PASS | vet-run/vet integration — public curl|bash examples need review-first alternatives | -| datasets-schema | A | 2026-08-31 22:13 | PASS | issue #196 — .oh/evals/datasets verifiable trajectory corpus (Repo2RLEnv-inspired) | -| debugmcp-availability | A | 2026-08-31 22:13 | SKIPPED | issue #297 — DebugMCP MCP debug-server availability | -| default-provisioning | A | 2026-08-31 22:13 | PASS | #902 — `oh harness install` must work from inside the sandbox, where | -| delegate-model-effort-policy | A | 2026-08-31 22:13 | PASS | conversation 2026-07-11 (delegate model inheritance and thinking policy) | -| devtcp-hook | A | 2026-08-31 22:13 | PASS | retro lesson 2026-06-10 (zsh /dev/tcp) | -| docker-inspect-env-guard | A | 2026-08-31 22:13 | PASS | operator directive 2026-08-08 (agents keep the docker socket, but must | -| docs-build-fast-path | A | 2026-08-31 22:13 | PASS | #455 — docs builds must stay out of fast harness/eval/release gates; #536 — docs site externalized to openharness-web; docs markdown relocated to docs/ | -| drift-check-cron-staleness-glob | A | 2026-08-31 22:13 | PASS | issue #98; issue #225 (restart-required cron frontmatter/config drift) | -| entrypoint-pnpm-manifest-fingerprint | A | 2026-08-31 22:13 | PASS | issue #521 (manifest-aware sandbox installs) 2026-07-01 | -| eval-ci-gate | A | 2026-08-31 22:13 | PASS | #103 — eval probe suite gated in CI | -| eval-gate | A | 2026-08-31 22:13 | PASS | retro lesson 2026-06-11 (eval-gate) | -| eval-results-atomic | A | 2026-08-31 22:13 | PASS | issue #83 (eval-results-atomic-write) | -| eval-runner-exit | A | 2026-08-31 22:13 | PASS | retro lesson 2026-06-11 (eval-runner-exit) #29 | -| eval-runs-once-per-cycle | A | 2026-08-31 22:13 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-006) — /eval ran 3x per cycle on the | -| execution-target-contract | A | 2026-08-31 22:13 | PASS | issue #733 (ExecutionTarget contract + Docker Compose adapter) 2026-08-10 | -| get-oh-bootstrap | A | 2026-08-31 22:13 | PASS | get-oh.sh bootstrap — the Node-bootstrapping host-side path to the standalone `oh` CLI (also on npm as @mifune/openharness; see oh-npm-package.sh) | -| git-skill | A | 2026-08-31 22:13 | PASS | conversation 2026-06-15 — rules are not always supported; git workflow must be the /git skill | -| harness-audit-empty-output-gate | A | 2026-08-31 22:13 | PASS | issue #246 — /audit harness must fail closed on empty auditor outputs | -| harness-ci-core-paths | A | 2026-08-31 22:13 | PASS | #165 — core sandbox config files must trigger harness CI | -| harness-ci-hooks-paths | A | 2026-08-31 22:13 | PASS | issue #202 — credential/security hook changes must trigger harness CI | -| harness-yaml-migration | A | 2026-08-31 22:13 | PASS | PR #833 (migrate-harness-yaml.sh — append / uncomment-in-place / preserve / overwrite, plus a silent no-op second run) 2026-08-26 | -| health-check-docker-stats | A | 2026-08-31 22:13 | PASS | retro lesson 2026-06-10 (docker stats vs ps Size) | -| health-check-socket-degrade | A | 2026-08-31 22:13 | PASS | issue #762 (refs #756) — /health-check degrades to one statement, not nine failures | -| heartbeat-logging-contract | A | 2026-08-31 22:13 | PASS | issue #447 (heartbeat log append hardening) 2026-06-18 | -| image-seed-hygiene | A | 2026-08-31 22:13 | PASS | issue #900 (slim the sandbox image) 2026-08-30 | -| markitdown-wiki-ingest | A | 2026-08-31 22:13 | PASS | issue #649 — pinned local-document normalization contract for /wiki ingest | -| next-dev-prod | A | 2026-08-31 22:13 | SKIPPED | retro lesson 2026-06-04 | -| oh-compose-env-wiring | A | 2026-08-31 22:13 | SKIPPED | issue #880 (oh as the only front door — oh.json is the non-secret config surface) | -| oh-config-surfaces | A | 2026-08-31 22:13 | PASS | PR #887 (config split across two authored surfaces — a tracked oh.json and a secrets-only root dotenv — with nothing left under $HOME) | -| oh-destroy-guard | A | 2026-08-31 22:13 | SKIPPED | issue #879 — `oh` becomes the only front door, so `make destroy` must | -| oh-devcontainer-restructure | A | 2026-08-31 22:13 | PASS | consolidate devcontainer — .oh/devcontainer/ folded back into .devcontainer/ | -| oh-home-mount | A | 2026-08-31 22:13 | PASS | issue #898 (single $HOME mount) 2026-08-30 | -| oh-image-only-deploy | A | 2026-08-31 22:13 | PASS | .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only deploy) | -| oh-init-headless-config | A | 2026-08-31 22:13 | SKIPPED | PR #827 (installer answers landed in the losing config file); retargeted to the .example.env template by PR #833, then to oh.json by PR #887 | -| oh-init-scaffold | A | 2026-08-31 22:13 | PASS | issue #531 Phase 2 | -| oh-lifecycle-surface | A | 2026-08-31 22:13 | PASS | issue #881 — the Makefile is retired and `oh` is the only front door | -| oh-npm-package | A | 2026-08-31 22:13 | PASS | npm publish path for the standalone `oh` CLI (@mifune/openharness) — alternative to get-oh.sh | -| oh-payload-manifest | A | 2026-08-31 22:13 | PASS | issue #531 follow-on (.oh payload manifest — oh update ships a declared allowlist) | -| oh-sandbox-image-mode | A | 2026-08-31 22:13 | PASS | conversation 2026-07-05 (basic Docker deployment — prebuilt-image mode) | -| oh-shipped-repo-overridable | A | 2026-08-31 22:13 | PASS | issue #531 follow-on (de-hardcode residual — shipped .oh shell scripts keep the upstream repo overridable) | -| oh-standalone-lifecycle | A | 2026-08-31 22:13 | PASS | issue #564 | -| oh-update | A | 2026-08-31 22:13 | PASS | issue #531 Phase 3 (oh update — upgrade only the .oh control plane) | -| operator-config-guard | A | 2026-08-31 22:13 | PASS | operator directives 2026-08-06 (.config/ and settings.local.json are operator-only) | -| pnpm-audit-ci-gate | A | 2026-08-31 22:13 | PASS | issue #171 — pnpm security audits must run in CI | -| post-bridge-publish-confirmation | A | 2026-08-31 22:13 | PASS | #523 — post-bridge live publishing requires an explicit final confirmation gate | -| prd-output-path-contract | A | 2026-08-31 22:13 | PASS | retro lesson 2026-06-19 | -| prompt-miner-schema-compat | A | 2026-08-31 22:13 | PASS | issue #253 — prompt-miner JSONL schema-drift guard | -| prompt-miner-symlink-entrypoint | A | 2026-08-31 22:13 | PASS | issue #663 — prompt-miner engine no-ops via the documented .claude/skills symlink | -| prompt-miner-weakness-record | A | 2026-08-31 22:13 | PASS | issue #580 — prompt-miner weakness-record (WH-xxx) cluster output | -| protected-path-deletion | A | 2026-08-31 22:13 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-001) — the critique gate was deleted, | -| protected-paths-resolve | A | 2026-08-31 22:13 | PASS | issue #753 — .claude/protected-paths.txt named 7 paths that did not exist. | -| registry-portability-gate | A | 2026-08-31 22:13 | PASS | issue #758 | -| registry-portability | A | 2026-08-31 22:13 | SKIPPED | issue #758 | -| retro-deterministic-contract | A | 2026-08-31 22:13 | PASS | issue #443 — /retro deterministic output and self-contained helper contract | -| rl-delegation-write-worker | A | 2026-08-31 22:13 | PASS | retro lesson 2026-06-10 (rl-delegation) #57 | -| rlm-context-budget | A | 2026-08-31 22:13 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-006 | -| runtime-preflight-gate | A | 2026-08-31 22:13 | PASS | issue #806 § B1 (open sandbox.substrate vs sandbox.runtime selector); | -| sandbox-boot-guard-ci | A | 2026-08-31 22:13 | PASS | issue #449 (sandbox image build CI guard) 2026-06-19; | -| sandbox-node-base | A | 2026-08-31 22:13 | PASS | openharness#878 — oh as the only front door, T0 sandbox base image | -| skill-paths | A | 2026-08-31 22:13 | PASS | issue #43 — stale path references; extended by issue #69 — apps/->packages/ rename guard; extended by issue #870 — deleted .oh/agents/advisor.md | -| skills-dir-clean | A | 2026-08-31 22:13 | PASS | conversation 2026-06-29 — Pi parses every top-level `.md` in the skills | -| skills-task-tool-coupling | A | 2026-08-31 22:13 | PASS | council review 2026-08-29 (issue #886) — /delegate instructed Claude-Code-only | -| skills-vendored | A | 2026-08-31 22:13 | PASS | absorb .mifune submodule into .oh — the skills/agents/hooks pack is vendored | -| slack-admin-command-surface | A | 2026-08-31 22:13 | PASS | issue #354 — Slack bridge docs must distinguish Pi /msg-bridge commands from Slack DM admin text handlers | -| spec-family-contract | A | 2026-08-31 22:13 | PASS | issue #265; spec-simplification issue #816; workflow authority issue #854; | -| spec-ready-finalization | A | 2026-08-31 22:13 | PASS | issue #134; spec-simplification issue #816; workflow authority issue #854 | -| ste-checker-contract | A | 2026-08-31 22:13 | PASS | issue #750 PR audit — the /ste checker had four fail-open paths (unclosed | -| submitted-by-trailers | A | 2026-08-31 22:13 | PASS | conversation 2026-06-12 (commit attribution trailers); the single-owner | -| sync-skill-contract | A | 2026-08-31 22:13 | PASS | issue #331 — /sync dispatcher skill (bidirectional origin↔upstream sync) | -| t3-headless-launch | A | 2026-08-31 22:13 | PASS | issue #858 — /t3 launched a bare `npx --yes t3`, which is the local GUI and | -| tailscale-tool-boundary | A | 2026-08-31 22:13 | PASS | issue #858 — Tailscale mobile access for T3 Code. There is no tailnet, no | -| tool-catalog-boundary | A | 2026-08-31 22:13 | PASS | agent-browser's exclusion from the harness catalog (#821) and the | -| version-parity | A | 2026-08-31 22:13 | PASS | conversation 2026-08-29 — the oh CLI became the only lifecycle door, so its | -| weigh-scorer-contract | A | 2026-08-31 22:13 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-003 (2026-06-27) | -| wiki-readme-index | A | 2026-08-31 22:13 | PASS | issue #132 — wiki README index drift guard | -| workflow-boundaries | A | 2026-08-31 22:13 | PASS | conversation 2026-06-19 (workflow consolidation, issue #259); authority moved to /spec in issue #854 | -| worktrees-layout | A | 2026-08-31 22:13 | PASS | issue #872 | +| advisor-monitored-loop | A | 2026-08-31 22:19 | PASS | conversation 2026-06-19 (single-owner implementation workflow, issue #257) | +| agent-browser-cli | A | 2026-08-31 22:19 | PASS | retro lesson 2026-06-07 (agent-browser 0.8.5 CLI) | +| agents-identity-contract | A | 2026-08-31 22:19 | PASS | issue #854 — T3-style root identity, glossary, and skill-owned procedures | +| artifact-contract-audit | A | 2026-08-31 22:19 | PASS | issue #583/#645 — production /audit implementation Gate 1 behavior | +| audit-dispatcher-contract | A | 2026-08-31 22:19 | PASS | issue #645 — audit consolidation public taxonomy | +| audit-implementation-behavior | A | 2026-08-31 22:19 | PASS | issue #645 — implementation root/repo/browser behavior | +| audit-pr-acquire | A | 2026-08-31 22:19 | PASS | issue #645 — production PR acquisition behavior | +| audit-pr-classifier | A | 2026-08-31 22:19 | PASS | issue #645 — deterministic focused and queue PR classifier | +| audit-run-root-contract | A | 2026-08-31 22:19 | PASS | issue #645 — executable immutable audit root/run correlation | +| audit-shellcheck-coverage | A | 2026-08-31 22:19 | PASS | issue #645 — private audit scripts require release and CI lint coverage | +| audit-slop-gate | A | 2026-08-31 22:19 | PASS | .claude/specs/images/reduce-slop.png — the four correctness gates all pass a change | +| audit-stale-references | A | 2026-08-31 22:19 | PASS | issue #645 — clean-breaking audit migration | +| boot-lint-glob | A | 2026-08-31 22:19 | PASS | issue #90, issue #120 | +| builder-skill-consolidation | A | 2026-08-31 22:19 | PASS | issue #643 — consolidate artifact builders behind one /builder dispatcher | +| builder-wiki-proposer | A | 2026-08-31 22:19 | PASS | wikiskill arXiv:2608.27454 — the skill proposer reads accumulated knowledge first | +| capability-benchmark-schema | A | 2026-08-31 22:19 | PASS | issue #167 — capability benchmark instrument | +| cc-safety-net-wiring | A | 2026-08-31 22:19 | SKIPPED | .oh/tasks/cc-safety-net/prd.json US-007 2026-07-19 | +| changelog-entry-length | A | 2026-08-31 22:19 | PASS | conversation 2026-08-24 — CHANGELOG.md grew to 259KB of bullet prose because "one line" was unquantified | +| cleanup-tasks-scoped-guard | A | 2026-08-31 22:19 | PASS | issue #85 | +| cleanup-tasks-worktree-grooming | A | 2026-08-31 22:19 | PASS | issue #168; issue #327 | +| cli-publish-typecheck-scope | A | 2026-08-31 22:19 | PASS | release run 33271077312 — v0.5.0 pushed its GHCR image, then failed to publish | +| close-issues-on-development | A | 2026-08-31 22:19 | PASS | issue #841 (closing keywords never fire because the default branch is main) 2026-08-26 | +| codex-stale-response-retry | A | 2026-08-31 22:19 | PASS | issue #506 — Codex previous_response_not_found RCA | +| compose-config-path-parity | A | 2026-08-31 22:19 | PASS | PR #833 (remove harness.yaml — the wrapper and VS Code "Reopen in Container" paths must resolve the same service) 2026-08-26 | +| config-schema-parity | A | 2026-08-31 22:19 | PASS | PR #833 (one schema file — DOCKER_SOCKET, SANDBOX_SSH, OH_SANDBOX_IMAGE, OH_PULL_POLICY, SKIP_PNPM_INSTALL were consumed but undocumented); rewritten for the oh.json/secrets split by PR #887 | +| context-tier-size-budget | A | 2026-08-31 22:19 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-007) — the always-on tier was 85,256 B | +| continual-learning-20260831 | A | 2026-08-31 22:19 | PASS | retro lesson 2026-08-31 (unexercised oracle) — a probe green in a 112-probe run carried three parser defects | +| cron-claude-codex-fallback | A | 2026-08-31 22:19 | PASS | conversation 2026-06-12 (default Codex fallback for crons) | +| cron-watchdog | A | 2026-08-31 22:19 | PASS | issues #130/#453 (cron runtime watchdog + legacy system-cron reaping) 2026-06-19 | +| crons-directory-guide | A | 2026-08-31 22:19 | PASS | issue #874 | +| curl-bash-safe-alternatives | A | 2026-08-31 22:19 | PASS | vet-run/vet integration — public curl|bash examples need review-first alternatives | +| datasets-schema | A | 2026-08-31 22:19 | PASS | issue #196 — .oh/evals/datasets verifiable trajectory corpus (Repo2RLEnv-inspired) | +| debugmcp-availability | A | 2026-08-31 22:19 | SKIPPED | issue #297 — DebugMCP MCP debug-server availability | +| default-provisioning | A | 2026-08-31 22:19 | PASS | #902 — `oh harness install` must work from inside the sandbox, where | +| delegate-model-effort-policy | A | 2026-08-31 22:19 | PASS | conversation 2026-07-11 (delegate model inheritance and thinking policy) | +| devtcp-hook | A | 2026-08-31 22:19 | PASS | retro lesson 2026-06-10 (zsh /dev/tcp) | +| docker-inspect-env-guard | A | 2026-08-31 22:19 | PASS | operator directive 2026-08-08 (agents keep the docker socket, but must | +| docs-build-fast-path | A | 2026-08-31 22:19 | PASS | #455 — docs builds must stay out of fast harness/eval/release gates; #536 — docs site externalized to openharness-web; docs markdown relocated to docs/ | +| drift-check-cron-staleness-glob | A | 2026-08-31 22:19 | PASS | issue #98; issue #225 (restart-required cron frontmatter/config drift) | +| entrypoint-pnpm-manifest-fingerprint | A | 2026-08-31 22:19 | PASS | issue #521 (manifest-aware sandbox installs) 2026-07-01 | +| eval-ci-gate | A | 2026-08-31 22:19 | PASS | #103 — eval probe suite gated in CI | +| eval-contract-text-20260831 | A | 2026-08-31 22:19 | PASS | retro lesson 2026-08-31 (prose-literal pinning) — two assertions failed on first run purely from source line wrapping | +| eval-gate | A | 2026-08-31 22:19 | PASS | retro lesson 2026-06-11 (eval-gate) | +| eval-results-atomic | A | 2026-08-31 22:19 | PASS | issue #83 (eval-results-atomic-write) | +| eval-runner-exit | A | 2026-08-31 22:19 | PASS | retro lesson 2026-06-11 (eval-runner-exit) #29 | +| eval-runs-once-per-cycle | A | 2026-08-31 22:19 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-006) — /eval ran 3x per cycle on the | +| execution-target-contract | A | 2026-08-31 22:19 | PASS | issue #733 (ExecutionTarget contract + Docker Compose adapter) 2026-08-10 | +| get-oh-bootstrap | A | 2026-08-31 22:19 | PASS | get-oh.sh bootstrap — the Node-bootstrapping host-side path to the standalone `oh` CLI (also on npm as @mifune/openharness; see oh-npm-package.sh) | +| git-skill | A | 2026-08-31 22:19 | PASS | conversation 2026-06-15 — rules are not always supported; git workflow must be the /git skill | +| harness-audit-empty-output-gate | A | 2026-08-31 22:19 | PASS | issue #246 — /audit harness must fail closed on empty auditor outputs | +| harness-ci-core-paths | A | 2026-08-31 22:19 | PASS | #165 — core sandbox config files must trigger harness CI | +| harness-ci-hooks-paths | A | 2026-08-31 22:19 | PASS | issue #202 — credential/security hook changes must trigger harness CI | +| harness-yaml-migration | A | 2026-08-31 22:19 | PASS | PR #833 (migrate-harness-yaml.sh — append / uncomment-in-place / preserve / overwrite, plus a silent no-op second run) 2026-08-26 | +| health-check-docker-stats | A | 2026-08-31 22:19 | PASS | retro lesson 2026-06-10 (docker stats vs ps Size) | +| health-check-socket-degrade | A | 2026-08-31 22:19 | PASS | issue #762 (refs #756) — /health-check degrades to one statement, not nine failures | +| heartbeat-logging-contract | A | 2026-08-31 22:19 | PASS | issue #447 (heartbeat log append hardening) 2026-06-18 | +| image-seed-hygiene | A | 2026-08-31 22:19 | PASS | issue #900 (slim the sandbox image) 2026-08-30 | +| markitdown-wiki-ingest | A | 2026-08-31 22:19 | PASS | issue #649 — pinned local-document normalization contract for /wiki ingest | +| next-dev-prod | A | 2026-08-31 22:19 | SKIPPED | retro lesson 2026-06-04 | +| oh-compose-env-wiring | A | 2026-08-31 22:19 | SKIPPED | issue #880 (oh as the only front door — oh.json is the non-secret config surface) | +| oh-config-surfaces | A | 2026-08-31 22:19 | PASS | PR #887 (config split across two authored surfaces — a tracked oh.json and a secrets-only root dotenv — with nothing left under $HOME) | +| oh-destroy-guard | A | 2026-08-31 22:19 | SKIPPED | issue #879 — `oh` becomes the only front door, so `make destroy` must | +| oh-devcontainer-restructure | A | 2026-08-31 22:19 | PASS | consolidate devcontainer — .oh/devcontainer/ folded back into .devcontainer/ | +| oh-home-mount | A | 2026-08-31 22:19 | PASS | issue #898 (single $HOME mount) 2026-08-30 | +| oh-image-only-deploy | A | 2026-08-31 22:19 | PASS | .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only deploy) | +| oh-init-headless-config | A | 2026-08-31 22:19 | SKIPPED | PR #827 (installer answers landed in the losing config file); retargeted to the .example.env template by PR #833, then to oh.json by PR #887 | +| oh-init-scaffold | A | 2026-08-31 22:19 | PASS | issue #531 Phase 2 | +| oh-lifecycle-surface | A | 2026-08-31 22:19 | PASS | issue #881 — the Makefile is retired and `oh` is the only front door | +| oh-npm-package | A | 2026-08-31 22:19 | PASS | npm publish path for the standalone `oh` CLI (@mifune/openharness) — alternative to get-oh.sh | +| oh-payload-manifest | A | 2026-08-31 22:19 | PASS | issue #531 follow-on (.oh payload manifest — oh update ships a declared allowlist) | +| oh-sandbox-image-mode | A | 2026-08-31 22:19 | PASS | conversation 2026-07-05 (basic Docker deployment — prebuilt-image mode) | +| oh-shipped-repo-overridable | A | 2026-08-31 22:19 | PASS | issue #531 follow-on (de-hardcode residual — shipped .oh shell scripts keep the upstream repo overridable) | +| oh-standalone-lifecycle | A | 2026-08-31 22:19 | PASS | issue #564 | +| oh-update | A | 2026-08-31 22:19 | PASS | issue #531 Phase 3 (oh update — upgrade only the .oh control plane) | +| operator-config-guard | A | 2026-08-31 22:19 | PASS | operator directives 2026-08-06 (.config/ and settings.local.json are operator-only) | +| pnpm-audit-ci-gate | A | 2026-08-31 22:19 | PASS | issue #171 — pnpm security audits must run in CI | +| post-bridge-publish-confirmation | A | 2026-08-31 22:19 | PASS | #523 — post-bridge live publishing requires an explicit final confirmation gate | +| prd-output-path-contract | A | 2026-08-31 22:19 | PASS | retro lesson 2026-06-19 | +| prompt-miner-schema-compat | A | 2026-08-31 22:19 | PASS | issue #253 — prompt-miner JSONL schema-drift guard | +| prompt-miner-symlink-entrypoint | A | 2026-08-31 22:19 | PASS | issue #663 — prompt-miner engine no-ops via the documented .claude/skills symlink | +| prompt-miner-weakness-record | A | 2026-08-31 22:19 | PASS | issue #580 — prompt-miner weakness-record (WH-xxx) cluster output | +| protected-path-deletion | A | 2026-08-31 22:19 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-001) — the critique gate was deleted, | +| protected-paths-resolve | A | 2026-08-31 22:19 | PASS | issue #753 — .claude/protected-paths.txt named 7 paths that did not exist. | +| registry-portability-gate | A | 2026-08-31 22:19 | PASS | issue #758 | +| registry-portability | A | 2026-08-31 22:19 | SKIPPED | issue #758 | +| retro-deterministic-contract | A | 2026-08-31 22:19 | PASS | issue #443 — /retro deterministic output and self-contained helper contract | +| rl-delegation-write-worker | A | 2026-08-31 22:19 | PASS | retro lesson 2026-06-10 (rl-delegation) #57 | +| rlm-context-budget | A | 2026-08-31 22:19 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-006 | +| runtime-preflight-gate | A | 2026-08-31 22:19 | PASS | issue #806 § B1 (open sandbox.substrate vs sandbox.runtime selector); | +| sandbox-boot-guard-ci | A | 2026-08-31 22:19 | PASS | issue #449 (sandbox image build CI guard) 2026-06-19; | +| sandbox-node-base | A | 2026-08-31 22:19 | PASS | openharness#878 — oh as the only front door, T0 sandbox base image | +| skill-paths | A | 2026-08-31 22:19 | PASS | issue #43 — stale path references; extended by issue #69 — apps/->packages/ rename guard; extended by issue #870 — deleted .oh/agents/advisor.md | +| skills-dir-clean | A | 2026-08-31 22:19 | PASS | conversation 2026-06-29 — Pi parses every top-level `.md` in the skills | +| skills-task-tool-coupling | A | 2026-08-31 22:19 | PASS | council review 2026-08-29 (issue #886) — /delegate instructed Claude-Code-only | +| skills-vendored | A | 2026-08-31 22:19 | PASS | absorb .mifune submodule into .oh — the skills/agents/hooks pack is vendored | +| slack-admin-command-surface | A | 2026-08-31 22:19 | PASS | issue #354 — Slack bridge docs must distinguish Pi /msg-bridge commands from Slack DM admin text handlers | +| spec-family-contract | A | 2026-08-31 22:19 | PASS | issue #265; spec-simplification issue #816; workflow authority issue #854; | +| spec-ready-finalization | A | 2026-08-31 22:19 | PASS | issue #134; spec-simplification issue #816; workflow authority issue #854 | +| ste-checker-contract | A | 2026-08-31 22:19 | PASS | issue #750 PR audit — the /ste checker had four fail-open paths (unclosed | +| submitted-by-trailers | A | 2026-08-31 22:19 | PASS | conversation 2026-06-12 (commit attribution trailers); the single-owner | +| sync-skill-contract | A | 2026-08-31 22:19 | PASS | issue #331 — /sync dispatcher skill (bidirectional origin↔upstream sync) | +| t3-headless-launch | A | 2026-08-31 22:19 | PASS | issue #858 — /t3 launched a bare `npx --yes t3`, which is the local GUI and | +| tailscale-tool-boundary | A | 2026-08-31 22:19 | PASS | issue #858 — Tailscale mobile access for T3 Code. There is no tailnet, no | +| tool-catalog-boundary | A | 2026-08-31 22:19 | PASS | agent-browser's exclusion from the harness catalog (#821) and the | +| version-parity | A | 2026-08-31 22:19 | PASS | conversation 2026-08-29 — the oh CLI became the only lifecycle door, so its | +| weigh-scorer-contract | A | 2026-08-31 22:19 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-003 (2026-06-27) | +| wiki-compile-contract | A | 2026-08-31 22:19 | PASS | wikiskill arXiv:2608.27454 — Wiki Maintainer role added as /wiki compile | +| wiki-kind-schema-contract | A | 2026-08-31 22:19 | PASS | wikiskill arXiv:2608.27454 — pattern layer added to the wiki corpus | +| wiki-pattern-persistence | A | 2026-08-31 22:19 | SKIPPED | wikiskill arXiv:2608.27454 — pattern pages are never rolled back | +| wiki-query-pattern-isolation | A | 2026-08-31 22:19 | PASS | wikiskill arXiv:2608.27454 — proposer-only pattern access | +| wiki-readme-index | A | 2026-08-31 22:19 | PASS | issue #132 — wiki README index drift guard | +| wiki-related-slugs | A | 2026-08-31 22:19 | PASS | wikiskill arXiv:2608.27454 — wiki lint related-slug check | +| wiki-skill-impact-append-only | A | 2026-08-31 22:19 | PASS | wikiskill arXiv:2608.27454 — skill-change ledger, never rolled back | +| workflow-boundaries | A | 2026-08-31 22:19 | PASS | conversation 2026-06-19 (workflow consolidation, issue #259); authority moved to /spec in issue #854 | +| worktrees-layout | A | 2026-08-31 22:19 | PASS | issue #872 | diff --git a/.oh/evals/capability/RESULTS.md b/.oh/evals/capability/RESULTS.md index b8d373a5..e8d604ee 100644 --- a/.oh/evals/capability/RESULTS.md +++ b/.oh/evals/capability/RESULTS.md @@ -14,8 +14,9 @@ inspection; rows are re-scored and overwritten per id by the `run.sh` runner. | CB-002 | 2026-06-19 | PASS | PARTIAL | PASS | 1.67 | spec-* workflow (select→plan→execute→merge) ships via /autopilot→/spec to a ready PR with an honest audit gate and no auto-merge; retargeted in #497 from the removed loop-walk; task re-authored 2026-08-24 (US-003) when /ship-spec was absorbed into /spec execute — score predates that re-authoring | | CB-003 | 2026-06-15 | PASS | PASS | PARTIAL | 1.67 | **RETIRED** — historical row, no longer scored. Original basis: /retro compounds durable lessons (loop-node-name-pipe-trap, eval-results-new-probe-row); promotion orchestrator-gated. `/retro` is now report-only, so the identity-promotion cycle this task scored no longer exists; the task spec and its `DS-020-lens-diversity` dataset are removed. | | CB-004 | 2026-07-03 | PARTIAL | PARTIAL | PARTIAL | 1.00 | **RETIRED — unproven, not disproven.** Original basis: repo-map contract + A/B manifest/scorer exist (#462), but no completed workload-mix token/tool/time benchmark yet · Δ +0.00 machinery-added vs 1.00 baseline · check=PASS. The task spec, its held-out A/B workload manifest, and its scorer all landed 2026-07-03 and the paired A/B workload was **never run once** in the ~2 months they stood — the row held at `Δ +0.00 machinery-added` for that entire span because no measurement was ever taken. The benchmark did not show `.oh/context/REPO_MAP.md` to be useless; it showed nothing at all. `REPO_MAP.md` and this benchmark are removed as **unproven** — for want of evidence in either direction, not against evidence. | +| CB-005 | 2026-08-31 | PASS | PARTIAL | PARTIAL | 1.33 | Second run, delegated. `/retro` -> a sub-agent maintainer drafted four pattern pages under the write gate with no operator prose -> promoted after review -> `pattern-evals-unexercised-oracle` was then read and acted on, driving the REGRESSION branch of four probes that had never been exercised (all four caught their injected fault) -> `SI-0002` proposes the `/wiki compile` fixes that run surfaced, citing `[[pattern-wiki-external-model-over-mapping]]`. Persistence oracles exercised in run 1 remain green. `cost-time` PARTIAL: the delegated draft needed operator review and three of its judgment calls became procedure fixes. `unattended` PARTIAL not PASS: the harness wrote the pattern prose, but the operator still chose which drafts to promote and wrote `SI-0002`. Prior 0.67. - + > **Baseline reset — 0.3.0 (autopilot removal).** Every score above was taken > while an unattended `autopilot` runner existed. That runner and its `select` @@ -30,3 +31,11 @@ inspection; rows are re-scored and overwritten per id by the `run.sh` runner. > `CB-004` retired as **unproven**: its manifest, scorer, and ablation harness all > existed from 2026-07-03, the paired workload was never run, and the harness > therefore holds no measurement of `REPO_MAP.md`'s value in either direction. + +> **CB-005 lowers the suite score on purpose.** Adding an axis the harness scores +> `1.33` on drops the mean from 1.50 to 1.44. That is the honest reading, not a +> regression: the harness was previously blind to whether a lesson ever became a +> durable artifact, and measuring it for the first time shows it partly does — 0.00 -> 0.67 -> 1.33 across two runs. +> A ceiling that only ever rises is measuring the wrong things. **Do not read +> 1.50 -> 1.44 as a ceiling delta** — the two means are taken over different task +> sets. The comparable number is CB-005's own 0.00 -> 0.67 -> 1.33. diff --git a/.oh/evals/capability/tasks/CB-005-compile-a-lesson.md b/.oh/evals/capability/tasks/CB-005-compile-a-lesson.md new file mode 100644 index 00000000..ba0de56e --- /dev/null +++ b/.oh/evals/capability/tasks/CB-005-compile-a-lesson.md @@ -0,0 +1,71 @@ +--- +id: CB-005 +slug: compile-a-lesson +title: "Compile one session's lesson into a validated skill change" +axes: [success, cost-time, unattended] +skills: [/retro, /wiki, /builder, /benchmark] +created: 2026-08-31 +--- + +# CB-005 · Compile one session's lesson into a validated skill change + +## Task +Carry one supported lesson end-to-end, from observation to a recorded verdict: +`/retro` produces a `supported` hypothesis at `medium`+ confidence → `/wiki compile` +creates or patches exactly one `kind: pattern` page → `/builder` proposes one atomic +edit to one artifact, **citing the motivating pattern slug it read** → `/benchmark` +emits a verdict → `.oh/skills/wiki/corpus/skill-impact.md` holds a `PROPOSED` record +and its matching `-V` verdict record. + +The capability under test is whether the harness can turn an observation into a +durable, cited, validated artifact change without inventing a session journal — and +whether a **rejected** proposal still leaves knowledge behind. + +This is the axis the harness could not previously see. `/retro` nominates probe ids +and writes nothing; before this task existed, the newest probe carrying `retro lesson` +provenance was dated 2026-06-19, roughly ten weeks stale, and no instrument measured +that gap. `CB-003` scored the nearest capability and was retired when `/retro` became +report-only, leaving the ceiling with no view of knowledge persistence at all. + +## Success signal +- A tracked `.oh/skills/wiki/corpus/pattern-*.md` whose `## Detail` names an + observable symptom and a `path:line` root cause, and whose `sources:` uses the + pinned `@` evidence form. +- `/builder`'s report names the `[[pattern-...]]` slug it read, or records + `none (direct request)` honestly. +- `skill-impact.md` gained exactly one `SI-nnnn` record and one `SI-nnnn-V` record; + no existing record was edited. +- On a `REJECTED` verdict, the pattern page and both ledger records survive the + revert, with `wiki-pattern-persistence.sh` and `wiki-skill-impact-append-only.sh` + green. +- `/eval` green; `wiki-readme-index.sh` green after the index regeneration. + +## Rubric +| Axis | PASS | PARTIAL | FAIL | +|------|------|---------|------| +| success | The full chain runs — retro → compile → builder → benchmark → two ledger records — and the rejected-path persistence invariant is exercised at least once against a real revert | The chain runs but the pattern prose was hand-written by the operator, or the persistence invariant was asserted by probe text rather than exercised against a revert | The lesson lands only in a commit message, or `/wiki compile` produced a dated per-run page instead of one page per failure mode | +| cost-time | One pass carries the lesson through, no rework | One re-entry (a pattern page rewritten, or a proposal re-scoped) before the chain completes | Repeated rework, or the chain abandoned partway | +| unattended | No operator-authored pattern prose anywhere in the chain | The operator wrote or substantially rewrote the pattern body | The operator drove each step by hand | + +## Scoring method +v1 rubric inspection **after at least one real run**. + +It is legitimate to score `PARTIAL` on day one. It is **not** legitimate to score +`PASS` by rubric inspection with zero runs — that is precisely the `CB-004` failure +this task exists partly to avoid: a manifest, a scorer, and an ablation harness all +landed and stood for two months while the measurement was never taken once, so the +row held at "machinery-added" for that entire span and the benchmark showed nothing +in either direction. + +## Evidence basis +The three-layer separation and the never-rollback invariant this task scores come +from `[[wikiskill-experience-compilation]]` (arXiv 2608.27454), whose ablation +measured persistent knowledge for the proposer at +15.0 against no persistence. The +harness-side contract lives in `.oh/skills/wiki/references/schema.md` §§ 2, 7a, 8 and +`.oh/skills/wiki/references/compile.md`. + +## Baseline reset +Scored for the first time in the change that introduced `/wiki compile`, the +`kind: pattern` layer, and the `skill-impact.md` ledger. There is no pre-change +baseline: before that change the chain had no owner past `/retro`'s report, so the +honest prior value on every axis is `FAIL` (score 0.00). diff --git a/.oh/evals/probes/builder-wiki-proposer.sh b/.oh/evals/probes/builder-wiki-proposer.sh new file mode 100755 index 00000000..2b4e0baf --- /dev/null +++ b/.oh/evals/probes/builder-wiki-proposer.sh @@ -0,0 +1,39 @@ +#!/usr/bin/env bash +# tier: A +# source: wikiskill arXiv:2608.27454 — the skill proposer reads accumulated knowledge first +# desc: /builder consults wiki patterns and the skill-impact ledger before proposing, and records the proposal after landing; frontmatter and dispatch remain owned by builder-skill-consolidation.sh +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" +SKILL="$ROOT/.oh/skills/builder/SKILL.md" + +if [[ ! -f "$SKILL" ]]; then + echo "SKIPPED: /builder dispatcher absent" >&2 + exit 2 +fi + +failures=() +need() { grep -qF -- "$1" "$SKILL" || failures+=("builder/SKILL.md missing contract text: $1"); } + +need '/wiki query --patterns' +need '.oh/skills/wiki/corpus/skill-impact.md' +need 'Do not re-propose a change recorded there as `REJECTED`' +need 'Append a `PROPOSED` record' +need 'none (direct request)' +need 'Never edit an existing record' + +# The ledger this skill is told to write must actually exist. +[[ -f "$ROOT/.oh/skills/wiki/corpus/skill-impact.md" ]] \ + || failures+=("builder cites .oh/skills/wiki/corpus/skill-impact.md but the ledger does not exist") + +# The reads builder is told to perform must be within its declared tool allowlist. +grep -q '^allowed-tools: Read, Write, Edit, Glob, Grep, Bash$' "$SKILL" \ + || failures+=("builder allowed-tools changed — the wiki and ledger steps need no new tool, so a change here is a design smell") + +if ((${#failures[@]})); then + printf 'REGRESSION: %s\n' "${failures[@]}" >&2 + exit 1 +fi + +echo "PASS: /builder reads wiki patterns and the skill-impact ledger before proposing, within its existing tool allowlist" >&2 +exit 0 diff --git a/.oh/evals/probes/continual-learning-20260831.sh b/.oh/evals/probes/continual-learning-20260831.sh new file mode 100755 index 00000000..132bdf84 --- /dev/null +++ b/.oh/evals/probes/continual-learning-20260831.sh @@ -0,0 +1,45 @@ +#!/usr/bin/env bash +# tier: A +# source: retro lesson 2026-08-31 (unexercised oracle) — a probe green in a 112-probe run carried three parser defects +# desc: the probe contract requires driving a new probe's REGRESSION branch against a broken input, and the two probes whose oracle lives in git history keep their comparison-point override reachable +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" +README="$ROOT/.oh/evals/README.md" + +if [[ ! -f "$README" ]]; then + echo "SKIPPED: evals README absent: .oh/evals/README.md" >&2 + exit 2 +fi + +failures=() +need() { grep -qF -- "$1" "$README" || failures+=("evals/README.md missing contract text: $1"); } + +need '### Fault injection' +need 'drive its REGRESSION branch' +need 'expose the' +need 'is unexercised, not healthy' + +# The documented overrides must still exist, or the guidance names a dead handle. +declare -A OVERRIDES=( + ["wiki-skill-impact-append-only"]="WIKI_LEDGER_BASE" + ["wiki-pattern-persistence"]="WIKI_PERSISTENCE_BASE" +) +for id in "${!OVERRIDES[@]}"; do + probe="$ROOT/.oh/evals/probes/$id.sh" + var="${OVERRIDES[$id]}" + if [[ ! -f "$probe" ]]; then + failures+=("$id.sh is named in the fault-injection guidance but does not exist") + continue + fi + grep -qF "\${$var:-}" "$probe" \ + || failures+=("$id.sh no longer reads $var — its failing branch is unreachable after the fact") +done + +if ((${#failures[@]})); then + printf 'REGRESSION: %s\n' "${failures[@]}" >&2 + exit 1 +fi + +echo "PASS: the probe contract requires fault injection and both documented comparison-point overrides are live" >&2 +exit 0 diff --git a/.oh/evals/probes/eval-contract-text-20260831.sh b/.oh/evals/probes/eval-contract-text-20260831.sh new file mode 100755 index 00000000..b6f14d03 --- /dev/null +++ b/.oh/evals/probes/eval-contract-text-20260831.sh @@ -0,0 +1,34 @@ +#!/usr/bin/env bash +# tier: A +# source: retro lesson 2026-08-31 (prose-literal pinning) — two assertions failed on first run purely from source line wrapping +# desc: no probe pins a fixed-string literal long enough to straddle a hard wrap in the document it guards; the contract for pinning is documented +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" +PROBES="$ROOT/.oh/evals/probes" +README="$ROOT/.oh/evals/README.md" +MAX=72 # a pinned literal longer than this is likely to span a hard wrap + +[[ -d "$PROBES" ]] || { echo "SKIPPED: probe dir absent" >&2; exit 2; } + +failures=() +grep -qF -- '### Pinning contract text' "$README" \ + || failures+=("evals/README.md missing the '### Pinning contract text' contract") + +# Flag long fixed-string literals passed to grep -F / -qF across the corpus. +while IFS= read -r hit; do + [[ -n "$hit" ]] || continue + file="${hit%%:*}" + [[ "$(basename "$file")" == "$(basename "${BASH_SOURCE[0]}")" ]] && continue + lit="$(sed -E "s/.*grep -[A-Za-z]*F[A-Za-z]* -- '([^']*)'.*/\1/" <<<"$hit")" + [[ "$lit" == "$hit" ]] && continue + (( ${#lit} > MAX )) && failures+=("$(basename "$file"): pinned literal is ${#lit} chars (>$MAX), likely to straddle a wrap: ${lit:0:48}...") +done < <(grep -rn "grep -[A-Za-z]*F[A-Za-z]* -- '" "$PROBES" 2>/dev/null || true) + +if ((${#failures[@]})); then + printf 'REGRESSION: %s\n' "${failures[@]}" >&2 + exit 1 +fi + +echo "PASS: no probe pins an over-long fixed-string literal, and the pinning contract is documented" >&2 +exit 0 diff --git a/.oh/evals/probes/wiki-compile-contract.sh b/.oh/evals/probes/wiki-compile-contract.sh new file mode 100755 index 00000000..1ec72120 --- /dev/null +++ b/.oh/evals/probes/wiki-compile-contract.sh @@ -0,0 +1,57 @@ +#!/usr/bin/env bash +# tier: A +# source: wikiskill arXiv:2608.27454 — Wiki Maintainer role added as /wiki compile +# desc: the /wiki dispatcher routes a fourth compile subcommand, compile.md defers to the schema merge rules, forbids snapshotting retro reports into raw/, and the deleted memory tier is not reintroduced +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" +SKILL="$ROOT/.oh/skills/wiki/SKILL.md" +COMPILE="$ROOT/.oh/skills/wiki/references/compile.md" + +failures=() + +if [[ ! -f "$SKILL" ]]; then + echo "SKIPPED: wiki dispatcher absent" >&2 + exit 2 +fi +if [[ ! -f "$COMPILE" ]]; then + echo "REGRESSION: /wiki compile reference missing: .oh/skills/wiki/references/compile.md" >&2 + exit 1 +fi + +need() { grep -qF -- "$2" "$1" || failures+=("$(basename "$1") missing contract text: $2"); } + +need "$SKILL" 'four subcommands: ingest, query, lint, or compile' +need "$SKILL" 'references/{ingest,query,lint,compile}.md' +need "$SKILL" 'compile [--from ] [--task ] [--dry-run]' +need "$SKILL" '| `compile` | Read `references/compile.md`' +need "$SKILL" "\`compile\`'s pattern-page writes" + +need "$COMPILE" '/wiki compile [--from ] [--task ] [--dry-run]' +need "$COMPILE" 'MUST NOT write a `raw/` snapshot of a `/retro` report' +need "$COMPILE" '§ 7 as amended by § 7a' +need "$COMPILE" 'orchestrator-only' +need "$COMPILE" 'One page per failure mode' +need "$COMPILE" '## Contents' + +# The deleted memory tier must not return through this door. compile.md is +# expected to NAME the tier while explaining why it is not one, so only the +# operative machinery tokens are forbidden. +for f in "$SKILL" "$COMPILE"; do + grep -qE 'MEMORY_DIR|MEMORY\.md|Memory Improvement Protocol' "$f" \ + && failures+=("$(basename "$f") reintroduces the removed memory tier") +done +need "$COMPILE" '## Why this is not a session journal' +need "$COMPILE" 'One page per **failure mode**' + +# compile.md must stay a single-level reference under the 500-line house cap. +lines="$(wc -l < "$COMPILE")" +(( lines < 500 )) || failures+=("compile.md is $lines lines, over the 500-line cap") + +if ((${#failures[@]})); then + printf 'REGRESSION: %s\n' "${failures[@]}" >&2 + exit 1 +fi + +echo "PASS: /wiki routes compile, and compile.md defers to the schema merge rules without reviving the memory tier" >&2 +exit 0 diff --git a/.oh/evals/probes/wiki-kind-schema-contract.sh b/.oh/evals/probes/wiki-kind-schema-contract.sh new file mode 100755 index 00000000..d185a3f0 --- /dev/null +++ b/.oh/evals/probes/wiki-kind-schema-contract.sh @@ -0,0 +1,78 @@ +#!/usr/bin/env bash +# tier: A +# source: wikiskill arXiv:2608.27454 — pattern layer added to the wiki corpus +# desc: the wiki schema declares kind: source|pattern with an absent-means-source default, the pattern merge amendment, and the persistence invariant; every tracked entry obeys the kind/filename/placement rules +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" +SCHEMA="$ROOT/.oh/skills/wiki/references/schema.md" +CORPUS_REL=".oh/skills/wiki/corpus" + +if [[ ! -f "$SCHEMA" ]]; then + echo "SKIPPED: wiki schema absent: $SCHEMA" >&2 + exit 2 +fi + +failures=() + +need() { + grep -qF -- "$1" "$SCHEMA" || failures+=("schema.md missing contract text: $1") +} + +need '| `kind` | enum | no |' +need 'An absent `kind:` field means `kind: source`.' +need 'Consumers that filter on `kind` MUST apply the' +need 'Pattern placement.' +need 'a pattern in a subdirectory would be invisible to both' +need '## 7a. Pattern amendment to the body-merge strategy' +need 'is **append-only**' +need '## 8. Pattern persistence invariant' +need 'is never rolled back' +need 'Reverting a `corpus/` path as collateral of a skill revert is forbidden.' + +# Structural checks over tracked corpus entries. +while IFS= read -r rel; do + base="$(basename "$rel")" + [[ "$base" == "README.md" ]] && continue + [[ "$base" == "skill-impact.md" ]] && continue + abs="$ROOT/$rel" + [[ -f "$abs" ]] || continue + + fm="$(awk '/^---$/{f=!f; next} f{print}' "$abs")" + slug="$(grep '^slug:' <<<"$fm" | awk '{print $2}' | head -1 || true)" + [[ -z "$slug" ]] && continue + kind="$(grep '^kind:' <<<"$fm" | awk '{print $2}' | head -1 || true)" + kind="${kind:-source}" + + case "$kind" in + source|pattern) ;; + *) failures+=("$rel: kind must be source or pattern, got '$kind'");; + esac + + if [[ "$base" == pattern-* && "$kind" != "pattern" ]]; then + failures+=("$rel: filename says pattern but kind is '$kind'") + fi + if [[ "$kind" == "pattern" && "$base" != pattern-* ]]; then + failures+=("$rel: kind is pattern but filename lacks the pattern- prefix") + fi + if [[ "$kind" == "pattern" ]]; then + grep -q '^## Relevant Source Files$' "$abs" \ + || failures+=("$rel: kind: pattern requires a '## Relevant Source Files' section") + grep -q '^sources:' <<<"$fm" \ + || failures+=("$rel: kind: pattern requires at least one sources: entry") + fi +done < <(git -C "$ROOT" ls-files -- "$CORPUS_REL/*.md" ":!:$CORPUS_REL/raw/*") + +# Placement: no tracked entry may live in a corpus subdirectory other than raw/. +while IFS= read -r rel; do + [[ -n "$rel" ]] || continue + failures+=("$rel: corpus entries are flat; only raw/ may be a subdirectory") +done < <(git -C "$ROOT" ls-files -- "$CORPUS_REL/*/*" ":!:$CORPUS_REL/raw/*") + +if ((${#failures[@]})); then + printf 'REGRESSION: %s\n' "${failures[@]}" >&2 + exit 1 +fi + +echo "PASS: wiki kind schema declared and every tracked corpus entry obeys the kind, filename, and placement rules" >&2 +exit 0 diff --git a/.oh/evals/probes/wiki-pattern-persistence.sh b/.oh/evals/probes/wiki-pattern-persistence.sh new file mode 100755 index 00000000..8d3ba307 --- /dev/null +++ b/.oh/evals/probes/wiki-pattern-persistence.sh @@ -0,0 +1,66 @@ +#!/usr/bin/env bash +# tier: A +# source: wikiskill arXiv:2608.27454 — pattern pages are never rolled back +# desc: every kind: pattern entry tracked at the merge-base is still tracked at HEAD, and no pattern's sources: list has shrunk +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" +CORPUS_REL=".oh/skills/wiki/corpus" + +# WIKI_PERSISTENCE_BASE overrides the comparison point. It exists so the invariant +# can actually be exercised against a real revert instead of only asserted: cut a +# scratch branch, revert the skill change, and run this probe with the pre-revert +# commit as the base. +base="${WIKI_PERSISTENCE_BASE:-}" +[[ -n "$base" ]] && base="$(git -C "$ROOT" rev-parse "$base" 2>/dev/null || true)" +for cand in development main master; do + [[ -n "$base" ]] && break + if git -C "$ROOT" show-ref --verify --quiet "refs/heads/$cand"; then + base="$(git -C "$ROOT" merge-base HEAD "$cand" 2>/dev/null || true)" + [[ -n "$base" ]] && break + fi +done +if [[ -z "$base" ]]; then + echo "SKIPPED: no merge-base against development/main/master (shallow or detached checkout)" >&2 + exit 2 +fi + +base_patterns=() +while IFS= read -r rel; do + [[ -n "$rel" ]] || continue + [[ "$(basename "$rel")" == pattern-* ]] || continue + base_patterns+=("$rel") +done < <(git -C "$ROOT" ls-tree -r --name-only "$base" -- "$CORPUS_REL" | grep '\.md$' || true) + +if ((${#base_patterns[@]} == 0)); then + echo "SKIPPED: no kind: pattern entries tracked at the merge-base — nothing to protect yet" >&2 + exit 2 +fi + + +deleted=(); shrunk=() +for rel in "${base_patterns[@]}"; do + if ! git -C "$ROOT" ls-files --error-unmatch "$rel" >/dev/null 2>&1; then + deleted+=("$rel") + continue + fi + before="$(git -C "$ROOT" show "$base:$rel" | awk '/^---$/{f=!f; next} f{print}' \ + | awk '/^sources:/{s=1; next} s && /^[a-z_-]+:/{s=0} s && /^[[:space:]]*- /{n++} END{print n+0}')" + after="$(awk '/^---$/{f=!f; next} f{print}' "$ROOT/$rel" \ + | awk '/^sources:/{s=1; next} s && /^[a-z_-]+:/{s=0} s && /^[[:space:]]*- /{n++} END{print n+0}')" + if (( after < before )); then + shrunk+=("$rel ($before -> $after)") + fi +done + +if ((${#deleted[@]})); then + printf 'REGRESSION: pattern page removed since the merge-base — see schema.md § 8, patterns are never rolled back: %s\n' "${deleted[@]}" >&2 + exit 1 +fi +if ((${#shrunk[@]})); then + printf 'REGRESSION: pattern sources: list shrank since the merge-base (provenance is append-only): %s\n' "${shrunk[@]}" >&2 + exit 1 +fi + +echo "PASS: every pattern page tracked at the merge-base survives at HEAD with its provenance intact" >&2 +exit 0 diff --git a/.oh/evals/probes/wiki-query-pattern-isolation.sh b/.oh/evals/probes/wiki-query-pattern-isolation.sh new file mode 100755 index 00000000..446f6c0a --- /dev/null +++ b/.oh/evals/probes/wiki-query-pattern-isolation.sh @@ -0,0 +1,83 @@ +#!/usr/bin/env bash +# tier: A +# source: wikiskill arXiv:2608.27454 — proposer-only pattern access +# desc: /wiki query declares two disjoint kind modes with per-mode caps and term-hit ranking on the locked awk, and no kind: pattern entry appears in a default-mode result set +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" +QUERY="$ROOT/.oh/skills/wiki/references/query.md" +CORPUS_REL=".oh/skills/wiki/corpus" + +if [[ ! -f "$QUERY" ]]; then + echo "SKIPPED: wiki query reference absent: $QUERY" >&2 + exit 2 +fi + +failures=() +need() { grep -qF -- "$1" "$QUERY" || failures+=("query.md missing contract text: $1"); } + +need '/wiki query [--patterns]' +need 'an absent `kind:` counts as `source`' +need 'There is deliberately **no `--all` mode**' +need 'This is a **default, not a boundary**' +need 'WANT_KIND=source' +need 'WANT_KIND=pattern' +need 'CAP=3' +need 'CAP=5' +need 'kind="${kind:-source}"' +need "awk '/^---\$/{f=!f; next} f{print}'" + +if ((${#failures[@]})); then + printf 'REGRESSION: %s\n' "${failures[@]}" >&2 + exit 1 +fi + +# Behavioral: default mode must never surface a kind: pattern entry. +patterns=() +while IFS= read -r rel; do + [[ -n "$rel" ]] || continue + abs="$ROOT/$rel" + [[ -f "$abs" ]] || continue + kind="$(awk '/^---$/{f=!f; next} f{print}' "$abs" | grep '^kind:' | awk '{print $2}' | head -1 || true)" + [[ "${kind:-source}" == "pattern" ]] && patterns+=("$rel") +done < <(git -C "$ROOT" ls-files -- "$CORPUS_REL/*.md" ":!:$CORPUS_REL/raw/*") + +if ((${#patterns[@]} == 0)); then + echo "SKIPPED: no tracked kind: pattern entries yet — contract text verified, isolation untestable" >&2 + exit 2 +fi + +# Run the documented default-mode filter (query.md § 3) over the whole tracked +# corpus, using a term drawn from a pattern page's own frontmatter. No pattern +# entry may survive it. +probe_term="$(awk '/^---$/{f=!f; next} f{print}' "$ROOT/${patterns[0]}" \ + | grep '^slug:' | awk '{print $2}' | head -1)" + +WANT_KIND=source +matched=() +while IFS= read -r rel; do + abs="$ROOT/$rel" + [[ -f "$abs" ]] || continue + fm="$(awk '/^---$/{f=!f; next} f{print}' "$abs")" + kind="$(grep '^kind:' <<<"$fm" | awk '{print $2}' | head -1 || true)" + kind="${kind:-source}" + [[ "$kind" == "$WANT_KIND" ]] || continue + grep -qi -- "$probe_term" <<<"$fm" || continue + matched+=("$rel") +done < <(git -C "$ROOT" ls-files -- "$CORPUS_REL/*.md" ":!:$CORPUS_REL/raw/*") + +leaked=() +for m in "${matched[@]:-}"; do + [[ -n "$m" ]] || continue + for p in "${patterns[@]}"; do + [[ "$m" == "$p" ]] && leaked+=("$m") + done +done + +if ((${#leaked[@]})); then + printf 'REGRESSION: pattern entry survived the default-mode kind filter: %s\n' "${leaked[@]}" >&2 + exit 1 +fi + +echo "PASS: /wiki query declares two disjoint kind modes and no pattern entry passes the default filter" >&2 +exit 0 diff --git a/.oh/evals/probes/wiki-related-slugs.sh b/.oh/evals/probes/wiki-related-slugs.sh new file mode 100755 index 00000000..94f35d9d --- /dev/null +++ b/.oh/evals/probes/wiki-related-slugs.sh @@ -0,0 +1,67 @@ +#!/usr/bin/env bash +# tier: A +# source: wikiskill arXiv:2608.27454 — wiki lint related-slug check +# desc: every related: frontmatter slug in a tracked wiki entry resolves to an existing tracked entry, and /wiki lint declares the check +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" +LINT="$ROOT/.oh/skills/wiki/references/lint.md" +CORPUS_REL=".oh/skills/wiki/corpus" + +if [[ ! -f "$LINT" ]]; then + echo "SKIPPED: wiki lint reference absent" >&2 + exit 2 +fi + +failures=() +for lit in '### 7a. Broken related-slug check' 'RELATED_BROKEN' 'Broken related-slug findings'; do + grep -qF -- "$lit" "$LINT" || failures+=("lint.md missing contract text: $lit") +done +if ((${#failures[@]})); then + printf 'REGRESSION: %s\n' "${failures[@]}" >&2 + exit 1 +fi + +declare -A KNOWN +entries=() +while IFS= read -r rel; do + base="$(basename "$rel")" + [[ "$base" == "README.md" ]] && continue + [[ "$base" == "skill-impact.md" ]] && continue + abs="$ROOT/$rel" + [[ -f "$abs" ]] || continue + slug="$(awk '/^---$/{f=!f; next} f{print}' "$abs" | grep '^slug:' | awk '{print $2}' | head -1 || true)" + [[ -z "$slug" ]] && continue + KNOWN["$slug"]=1 + entries+=("$rel") +done < <(git -C "$ROOT" ls-files -- "$CORPUS_REL/*.md" ":!:$CORPUS_REL/raw/*") + +if ((${#entries[@]} == 0)); then + echo "SKIPPED: no tracked corpus entries" >&2 + exit 2 +fi + +broken=(); seen_related=0 +for rel in "${entries[@]}"; do + fm="$(awk '/^---$/{f=!f; next} f{print}' "$ROOT/$rel")" + line="$(grep '^related:' <<<"$fm" | head -1 || true)" + [[ -z "$line" ]] && continue + seen_related=1 + rel_slugs="$(sed 's/^related: *//; s/[][]//g; s/,/ /g' <<<"$line")" + for r in $rel_slugs; do + [[ -z "$r" ]] && continue + [[ -n "${KNOWN[$r]:-}" ]] || broken+=("$rel -> related: $r (no such entry)") + done +done + +if (( seen_related == 0 )); then + echo "SKIPPED: no tracked entry declares a related: field" >&2 + exit 2 +fi +if ((${#broken[@]})); then + printf 'REGRESSION: %s\n' "${broken[@]}" >&2 + exit 1 +fi + +echo "PASS: every related: slug in the tracked corpus resolves to an existing entry" >&2 +exit 0 diff --git a/.oh/evals/probes/wiki-skill-impact-append-only.sh b/.oh/evals/probes/wiki-skill-impact-append-only.sh new file mode 100755 index 00000000..4107dfd6 --- /dev/null +++ b/.oh/evals/probes/wiki-skill-impact-append-only.sh @@ -0,0 +1,103 @@ +#!/usr/bin/env bash +# tier: A +# source: wikiskill arXiv:2608.27454 — skill-change ledger, never rolled back +# desc: skill-impact.md is tracked, carries no slug (so it never enters the corpus index), and every SI record present at the merge-base is present and byte-identical at HEAD +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" +LEDGER_REL=".oh/skills/wiki/corpus/skill-impact.md" +LEDGER="$ROOT/$LEDGER_REL" + +if [[ ! -f "$LEDGER" ]]; then + echo "SKIPPED: ledger absent: $LEDGER_REL" >&2 + exit 2 +fi +if ! git -C "$ROOT" ls-files --error-unmatch "$LEDGER_REL" >/dev/null 2>&1; then + echo "REGRESSION: $LEDGER_REL exists but is untracked — the ledger must be reviewable in a pull request" >&2 + exit 1 +fi +if grep -q '^slug:' "$LEDGER"; then + echo "REGRESSION: $LEDGER_REL carries a slug: field — it would become a corpus index row" >&2 + exit 1 +fi + +# Required record keys must be documented, so a writer cannot omit them silently. +failures=() +for lit in '**target**' '**motivating patterns**' '**for**' '**verdict**' 'Never edit an existing record'; do + grep -qF -- "$lit" "$LEDGER" || failures+=("ledger missing documented key: $lit") +done +if ((${#failures[@]})); then + printf 'REGRESSION: %s\n' "${failures[@]}" >&2 + exit 1 +fi + +# WIKI_LEDGER_BASE overrides the comparison point so the append-only invariant can +# be exercised against a real mutation rather than only asserted. +base="${WIKI_LEDGER_BASE:-}" +[[ -n "$base" ]] && base="$(git -C "$ROOT" rev-parse "$base" 2>/dev/null || true)" +for cand in development main master; do + [[ -n "$base" ]] && break + if git -C "$ROOT" show-ref --verify --quiet "refs/heads/$cand"; then + base="$(git -C "$ROOT" merge-base HEAD "$cand" 2>/dev/null || true)" + [[ -n "$base" ]] && break + fi +done +if [[ -z "$base" ]]; then + echo "SKIPPED: no merge-base against development/main/master (shallow or detached checkout)" >&2 + exit 2 +fi + +if ! git -C "$ROOT" cat-file -e "$base:$LEDGER_REL" 2>/dev/null; then + echo "PASS: $LEDGER_REL is new on this branch; nothing to compare against the merge-base" >&2 + exit 0 +fi + +MARKER=' + +## SI-0001 · 2026-08-31 · builder · PROPOSED + +- **proposal**: add a `related:`-slug resolution check to `/wiki lint` and a deterministic probe that fails on the findings, so an unrun report-only check cannot hide broken links +- **target**: `.oh/skills/wiki/references/lint.md` +- **motivating patterns**: [[pattern-wiki-ungated-check-drift]] +- **proposer**: /builder skill, wiki co-evolution change (branch `skill/wiki-coevolution`) +- **diff**: `8fab04ab` — `/wiki lint` § 7a plus `.oh/evals/probes/wiki-related-slugs.sh` + +## SI-0001-V · 2026-08-31 · benchmark · ACCEPTED + +- **for**: SI-0001 +- **floor**: /eval rc=0, 0 regressions over 112 probes (`.oh/evals/RESULTS.md`@af1c14ec) +- **ceiling**: suite score 1.50 -> 1.22 — **not a comparable delta.** The suite gained CB-005 in the same change, so the mean is taken over a different task set than the 1.50 it is being compared to. The meaningful number is CB-005's own first score, 0.67, against an honest prior of 0.00. +- **verdict**: BENEFICIAL — ACCEPTED. The floor held, and the change moved the one axis it targeted from an unmeasured 0.00 to a measured 0.67. Recorded with the caveat above rather than as a clean ceiling rise, because a rise produced by adding a task the harness scores badly on is not the same evidence as a rise on a fixed task set. + +## SI-0002 · 2026-08-31 · builder · PROPOSED + +- **proposal**: close three ambiguities in `/wiki compile` § 3-4 that a delegated maintainer run hit — the slug subsystem vocabulary, per-retro fan-out, and dual shas for a defect observed and fixed in one session +- **target**: `.oh/skills/wiki/references/compile.md` +- **motivating patterns**: [[pattern-wiki-external-model-over-mapping]] — its workaround is that a mapping is complete only when its exclusions are written down in the local vocabulary; the slug-token mismatch is the same defect one level down, a foreign taxonomy left un-translated in the local procedure +- **proposer**: /builder skill, prompted by the delegated `/wiki compile` run's flagged judgment calls +- **diff**: `.oh/skills/wiki/references/compile.md` § 3 fan-out and subsystem-token rules, § 4 dual-sha rule + +## SI-0003 · 2026-08-31 · builder · PROPOSED + +- **proposal**: document fault injection and short-fragment pinning in the probe contract, and mint the two probes that guard them, closing the retro nominations in the same session that nominated them +- **target**: `.oh/evals/README.md` +- **motivating patterns**: [[pattern-evals-unexercised-oracle]], [[pattern-evals-prose-literal-pinning]] +- **proposer**: /builder skill, closing the `/retro` nominations rather than leaving them to decay +- **diff**: `.oh/evals/README.md` §§ "Fault injection" and "Pinning contract text"; `.oh/evals/probes/continual-learning-20260831.sh`; `.oh/evals/probes/eval-contract-text-20260831.sh`. Both probes had every REGRESSION branch driven against injected faults before landing (5 injections, 5 caught). diff --git a/.oh/skills/wiki/corpus/wikiskill-experience-compilation.md b/.oh/skills/wiki/corpus/wikiskill-experience-compilation.md new file mode 100644 index 00000000..5aa0fdb9 --- /dev/null +++ b/.oh/skills/wiki/corpus/wikiskill-experience-compilation.md @@ -0,0 +1,82 @@ +--- +title: "WikiSkill: Compiling Agent Experience into Persistent Knowledge (arXiv 2608.27454)" +slug: wikiskill-experience-compilation +tags: [skill-evolution, persistent-knowledge, continual-learning, harness-evolution, self-improvement, wiki, ablation, skill-transfer] +created: 2026-08-31 +updated: 2026-08-31 +sources: + - raw/2026-08-31-wikiskill-experience-compilation.md +related: [recursive-self-improvement-survey, molt-agentic-reinforcement-learning, audit-architecture] +confidence: provisional +--- + +# WikiSkill: Compiling Agent Experience into Persistent Knowledge (arXiv 2608.27454) + +Tang, Rashtchian, Ferng, Tomkins, Juan, Vu — arXiv 2608.27454 (2026-08-27), +cs.AI/cs.CL. Source: . + +## Relevant Source Files +- `.oh/skills/wiki/references/schema.md` — the local analogue of the paper's wiki layer schema. +- `.oh/skills/wiki/corpus/raw/` — the local analogue of the paper's raw layer, holding source snapshots rather than agent traces. +- `.oh/skills/builder/SKILL.md` — the local analogue of the paper's Skill Proposer. +- `.oh/skills/retro/SKILL.md` — report-only lesson producer; the paper's maintainer role has no local owner. +- `.oh/evals/capability/RESULTS.md` — the ceiling instrument the paper's ablation argues should move. + +## Summary +WikiSkill co-evolves an agent's executable skills with a persistent knowledge +base. Its claim is that insights guiding skill development "remain scattered +across optimization histories," so it separates three layers — immutable raw +traces, an accumulated wiki, and executable skills — and makes the agent that +*proposes* skill edits read the wiki first. Its ablation isolates persistence as +the load-bearing component, not the skill format. + +## Detail +**Three layers.** `raw/` holds full rollout traces. `wiki/` holds `patterns/` +(one page per failure mode or successful strategy, each with an actionable +workaround), an `index.md` catalog, a per-iteration `logs.md`, and +`skill-impact.md` recording each proposal's diff, target, validation score, and +Accepted/Rejected outcome. `skills/` holds `SKILL.md` plus a `PURPOSE.md` mapping +each skill back to the patterns that motivated it. + +**The loop.** Per iteration: roll out with the current skills; a *Wiki Maintainer* +consolidates a sampled subset of traces into pattern pages by incremental, +patch-based edits; a *Wiki-Informed Skill Proposer* — a ReAct agent given the wiki +index, the impact ledger, and an outcome summary — emits **one atomic proposal +targeting one skill**; the proposal is validated and accepted only if the metric +improves, else the skill reverts. + +**The invariant that matters.** "The wiki is never rolled back regardless of the +acceptance decision." A rejected proposal still leaves durable knowledge, which +is what stops the same edit being proposed again. + +**Ablation (Gemini-3.5-Flash, mean).** No skill 40.4 · wiki visible to the +inference agent only 43.8 · **wiki visible to the skill proposer only 63.7** · +both 60.9. Persistence for the proposer is worth **+15.0**; leaking the wiki into +the inference path *costs* **2.8**. Skills also transfer across model families, +sometimes beating self-evolved skills — and sometimes transferring negatively +when a skill encodes model-specific low-level workarounds. + +**Stated limitations.** Skill retrieval is deliberately unsolved (skills are +injected wholesale); strict gating rejects neutral proposals the authors admit +could pay off later; there is no wiki pruning; long-horizon tasks are uncovered. + +**Local reading.** The harness already has raw snapshots, a generated index, a +proposer, and a two-signal validation metric. It lacks the pattern layer, the +impact ledger, and any wiki read on the proposer path — the three parts the +ablation credits. + +## System Relationships +```mermaid +flowchart LR + R["raw/ — traces"] -->|maintainer| W["wiki/ — patterns, index, impact"] + W -->|proposer reads| P["one atomic proposal"] + P --> S["skills/"] + S -->|rollout| R + V["validation metric"] --> S + V -->|verdict appended, never rolled back| W +``` + +## See Also +- [[recursive-self-improvement-survey]] +- [[molt-agentic-reinforcement-learning]] +- [[audit-architecture]] diff --git a/.oh/skills/wiki/references/compile.md b/.oh/skills/wiki/references/compile.md new file mode 100644 index 00000000..dac45c25 --- /dev/null +++ b/.oh/skills/wiki/references/compile.md @@ -0,0 +1,223 @@ +# /wiki compile — reference + +> Full procedure for the `compile` subcommand of the `/wiki` dispatcher. The +> dispatcher (`.oh/skills/wiki/SKILL.md`) routes here when the first `$ARGUMENTS` +> token is `compile`. Canonical schema: `.oh/skills/wiki/references/schema.md`. + +## Contents + +- [Wiki Compile](#wiki-compile) — what this subcommand is for +- [When to Use / When NOT to Use](#when-to-use) +- [Argument Interface (locked)](#argument-interface-locked) +- [Instructions](#instructions) — §§ 1-6 +- [Why this is not a session journal](#why-this-is-not-a-session-journal) +- [Anti-Patterns](#anti-patterns) + +# Wiki Compile + +Consolidate a `/retro` or `/spec retro` report into `kind: pattern` entries — the +harness's durable record of its own failure modes and working strategies. + +This is the Wiki Maintainer role. It exists because the harness produces lessons and +discards them: `/retro` nominates probe ids and writes nothing, and no skill owns the +edit its `proceduralize` triage prescribes. `compile` is the write step that closes +that gap without touching `/retro`, whose report-only contract is guarded by +`.oh/evals/probes/retro-deterministic-contract.sh` and must stay intact. + +## When to Use + +- After `/retro` or `/spec retro` emits a report with at least one `supported` + hypothesis at `medium` or `high` confidence. +- To record counter-evidence against a pattern a later run refuted. +- Before `/builder` proposes a skill change, so the proposal has a pattern to cite. + +## When NOT to Use + +- **`/wiki ingest`** — for an external source. `compile` never fetches a URL and + never writes to `corpus/raw/`. +- **A per-run note.** One page per failure mode, never one per run. See the + anti-patterns. +- **An `inconclusive` hypothesis.** It is not knowledge yet. + +## Argument Interface (locked) + +``` +/wiki compile [--from ] [--task ] [--dry-run] +``` + +| Argument | Meaning | +|----------|---------| +| *(none)* | Consume the `/retro` report already present in the current session's context. This is the normal path — `/retro` writes no file, so its report exists only as terminal output. | +| `--from ` | Read the report from a file: an operator-saved copy, or a sub-agent draft at `$TMPDIR/oh-wiki-drafts/.md`. | +| `--task ` | Scope to `.oh/tasks//`. Used to derive pinned-evidence `sources:` paths and to read `prd.md`, `progress.txt`, and `critique.md` as corroborating evidence. | +| `--dry-run` | Print the proposed create-or-patch for each target page. Write nothing. | + +The interface is locked; adding a flag requires editing this reference and +`.oh/evals/probes/wiki-compile-contract.sh`. + +## Instructions + +### 1. Read the report + +Locate the report's `## Hypotheses` table and its promotion-candidate lines, which +`/retro` emits in this exact form: + +``` +- [ · · harden|proceduralize|eval] — probe: | basis: +``` + +With `--from `, read that file instead. With neither a `--from` path nor a +report in context, print the usage line and exit 0. Do not invent a report. + +### 2. Select what is eligible + +Reuse `/retro`'s own promotion bar rather than inventing a second one. + +| Verdict | Confidence | Action | +|---------|-----------|--------| +| `supported` | `high` or `medium` | create or patch a pattern page | +| `refuted` | `high` | patch an **existing** pattern that asserts the refuted claim, adding counter-evidence; never create a new page | +| `refuted` | `low` | no write | +| `inconclusive` | any | **never** written | + +### 3. Derive the target slug — one page per failure mode + +The slug is `pattern--`, derived from the hypothesis's +subsystem and the mode it describes — never from the date or the run. + +``` +GOOD pattern-eval-probe-provenance-decay +BAD pattern-2026-08-31-retro-findings +``` + +The `` token is the **corpus's** subsystem vocabulary — the prefix a +reader would grep for (`evals`, `wiki`, `docs`, `spec`) — not `/retro`'s +five-lens taxonomy, which names where a signal was *noticed* rather than what the +page is about. A lesson noticed through the continual-learning lens about probe +behavior is `pattern-evals-...`, never `pattern-continual-learning-...`. + +**Fan-out.** One retro may legitimately yield several pages when it surfaced +several distinct modes, but each additional page must carry its own root cause and +its own workaround. If two candidate pages would share a workaround, they are one +mode: merge them. If they share a mechanism but their fixes point in opposite +directions, they are two. + +Enumerate existing patterns before writing: + +```bash +ls .oh/skills/wiki/corpus/pattern-*.md 2>/dev/null +``` + +If a page for that failure mode exists, **patch it**. A run that surfaces three +lessons about one failure mode produces one patch, not three pages. + +### 4. Create or patch + +**Create** follows the pattern body layout in +`.oh/skills/wiki/references/schema.md` § 2: `kind: pattern`, `confidence: +provisional`, a required `## Relevant Source Files`, and `## Detail` carrying +`**Symptom.**`, `**Root cause.**`, and `**Workaround.**` as bold leads. + +`sources:` uses the pinned-evidence form `@` — for +example `.oh/tasks//progress.txt@a1b2c3d` or `.oh/evals/RESULTS.md@a1b2c3d`. +Resolve the sha with `git rev-parse --short HEAD` at the time of the observation. +When a defect was observed and fixed in the same session, pin **both** shas — the +before-state is the evidence for the symptom and the after-state is the evidence for +the workaround. Multiple `sources:` entries are expected, not exceptional. + +**`/wiki compile` MUST NOT write a `raw/` snapshot of a `/retro` report.** `raw/` +holds snapshots of external sources. Persisting retro reports there would recreate +the per-session journal tier the harness deliberately removed, wearing a new name, +and would launder around `/retro`'s report-only contract. + +**Patch** applies the body-merge strategy in +`.oh/skills/wiki/references/schema.md` § 7 as amended by § 7a. This reference does +not restate those steps and must not diverge from them. The load-bearing part of +§ 7a: `**Workaround.**` is append-only, and a workaround shown not to work is +annotated `(superseded YYYY-MM-DD, SI-nnnn)` rather than deleted. + +### 5. Promote and reindex + +Pattern pages are always force-added. An untracked pattern page is invisible +provenance, and it is the corpus's only durable record of what a rejected cycle +taught. + +```bash +git add -f .oh/skills/wiki/corpus/pattern-.md +``` + +Then regenerate the index by running `/wiki lint` (non-dry-run) and verify: + +```bash +bash .oh/evals/probes/wiki-readme-index.sh +``` + +`compile` never hand-edits `.oh/skills/wiki/corpus/README.md` — `lint` owns it. + +### 6. Report + +Print to the terminal and write no report file: + +``` +Slugs-Created: ... +Slugs-Patched: ... +Hypotheses-Compiled: of +Skipped: — / +Result: OP | DRY-RUN | FAIL +``` + +## Write gate + +Pattern-page writes are **orchestrator-only**, the same rule that governs `ingest` +writes and `lint`'s index regeneration. A sub-agent proposes a draft at +`$TMPDIR/oh-wiki-drafts/.md`; the orchestrator promotes it with +`/wiki compile --from $TMPDIR/oh-wiki-drafts/.md`. A sub-agent that writes +directly to the corpus is out of scope and may be reverted. + +## Why this is not a session journal + +The `.oh/memory` tier was removed as a concept (`CHANGELOG.md`) because it held one +entry per session, keyed by date, with no consumer. Every structural property here is +the opposite: + +| `.oh/memory` (deleted) | `corpus/pattern-*.md` | +|---|---| +| One entry per session, keyed by date | One page per **failure mode**, keyed by subsystem and mode | +| Grew with every run | Grows only when a run teaches something not already recorded | +| No consumer; nothing read it | Read by the proposer role through `/wiki query --patterns` | +| Any skill could write | Orchestrator-only, through this one subcommand | +| Gitignored, unreviewable | Force-added and reviewed in the PR that lands it | + +The sharp test is `/retro`'s own anti-pattern, "inventing a file to save a lesson +in". A dated per-run page fails that test. A page named for a failure mode, patched +rather than appended to, and cited by a skill proposal, passes it. + +## Anti-Patterns + +- **One page per run** — the single failure mode of this subcommand. A page named + for a date is a journal entry. Name it for the failure mode and patch on repeat. +- **Snapshotting the retro report into `corpus/raw/`** — see § 4. `raw/` is for + external sources; `/retro` output is ephemeral by contract. +- **Compiling an `inconclusive` hypothesis** — the report already judged it not to be + knowledge. Compiling it launders a guess into the corpus. +- **Deleting or blanking a pattern page because the change it motivated was + rejected** — forbidden by `.oh/skills/wiki/references/schema.md` § 8. That + knowledge is the rejected cycle's entire output. +- **Replacing `## Detail` wholesale** — that is § 7 behavior for source pages. For a + pattern it erases accumulated failure knowledge; § 7a governs instead. +- **Restating the merge steps here** — § 7 and § 7a own them. A second copy will + drift. +- **Retired audit vocabulary in pattern prose** — the token list enforced by + `.oh/evals/probes/audit-stale-references.sh` covers every tracked file, this corpus + included. Read that probe's pattern before writing about an audit subsystem, and + use the current route names. +- **Skipping the reindex** — a new tracked pattern page without a regenerated + `README.md` is an immediate `wiki-readme-index.sh` regression. + +## See Also + +- `.oh/skills/wiki/references/schema.md` — § 2 pattern layout and placement, § 5 confidence, § 7a merge amendment, § 8 persistence invariant +- `.oh/skills/wiki/references/query.md` — the `--patterns` read path +- `.oh/skills/wiki/references/lint.md` — index regeneration and the health checks +- `.oh/skills/retro/SKILL.md` — the report this subcommand consumes; report-only by contract +- `.oh/skills/wiki/corpus/skill-impact.md` — where the proposal a pattern motivates is recorded diff --git a/.oh/skills/wiki/references/ingest.md b/.oh/skills/wiki/references/ingest.md index 10a2cf6b..bd078bd9 100644 --- a/.oh/skills/wiki/references/ingest.md +++ b/.oh/skills/wiki/references/ingest.md @@ -520,7 +520,7 @@ awk '/^---$/{f=!f; next} f{print}' .oh/skills/wiki/corpus/.md `.oh/skills/wiki/corpus/README.md` is the human/LLM index and is owned by `/wiki lint`, not by hand edits. After creating or updating a tracked `.oh/skills/wiki/corpus/.md` entry for a user-facing deliverable (especially when the user asked to "add to the wiki", or when you will commit/push the wiki change), run `/wiki lint` or follow its atomic regeneration protocol so the index includes the new entry before finalizing. Remember that `.oh/skills/wiki/corpus/raw/*` snapshots are gitignored by design; the tracked deliverable is usually `.oh/skills/wiki/corpus/.md` plus the regenerated `.oh/skills/wiki/corpus/README.md`, while the raw snapshot remains local provenance unless policy changes. -If you cannot run the full `/wiki lint` skill, do not hand-maintain the table casually: enumerate `.oh/skills/wiki/corpus/*.md`, extract frontmatter with the canonical `awk '/^---$/{f=!f; next} f{print}'` command, sort by `updated:` descending, write `.oh/skills/wiki/corpus/README.md.tmp`, validate it is non-empty and contains `| Slug | Title | Tags | Updated |`, then atomically rename it to `.oh/skills/wiki/corpus/README.md`. Log the lint/index refresh separately per `/wiki lint`'s Memory Improvement Protocol. +If you cannot run the full `/wiki lint` skill, do not hand-maintain the table casually: enumerate the **git-tracked** entry set with the pathspec in `/wiki lint` § 9a, extract frontmatter with the canonical `awk '/^---$/{f=!f; next} f{print}'` command, sort by `updated:` descending, write `.oh/skills/wiki/corpus/README.md.tmp`, validate it is non-empty and contains `| Slug | Title | Tags | Updated |`, then atomically rename it to `.oh/skills/wiki/corpus/README.md`. ### 8. Orchestrator-only write gate diff --git a/.oh/skills/wiki/references/lint.md b/.oh/skills/wiki/references/lint.md index aacafbf0..8bb2235e 100644 --- a/.oh/skills/wiki/references/lint.md +++ b/.oh/skills/wiki/references/lint.md @@ -80,10 +80,19 @@ ENTRIES_COUNT=${#WIKI_ENTRIES[@]} This enumerates `.oh/skills/wiki/corpus/*.md` directly — NOT via `.oh/skills/wiki/corpus/README.md` (the README is the output of this skill, not its input). Sub-article files at `.oh/skills/wiki/corpus//.md` are not matched by this glob; they are scoped for -a future iteration. +a future iteration. `kind: pattern` entries are flat `corpus/.md` files by the +`.oh/skills/wiki/references/schema.md` § 2 placement rule, precisely because this glob does +not descend. + +**Two entry sets, deliberately.** The health checks (§§ 4-7a) run over this +**working-tree** set, so a local scratch entry is still lintable. The README +regeneration (§ 9) runs over the **git-tracked** set instead — see § 9a. Building +the index from the working tree makes an untracked scratch entry a CI regression, +because `.oh/evals/probes/wiki-readme-index.sh` reconstructs the expected table +from git-tracked files only. If `$ENTRIES_COUNT = 0`, skip all check steps and proceed directly to -§ 7 (README regeneration with empty corpus) and § 8 (Memory Protocol). +§ 9 (README regeneration with an empty corpus). ### 3. Extract frontmatter for each entry @@ -304,6 +313,41 @@ If `${#BROKEN_LINKS[@]} = 0`, print ` (none)`. See `.oh/skills/wiki/references/schema.md` § 4 for the cross-link convention and grep patterns that govern outbound link syntax (`\[\[[a-z0-9-]+\]\]`). +### 7a. Broken related-slug check + +A broken related-slug is an entry whose `related:` frontmatter list names a slug +with no matching entry. This is distinct from § 7: a `[[slug]]` body link is a +navigational claim, a `related:` slug is a frontmatter adjacency claim. They fail +for different reasons and are remediated differently, so they are counted +separately. + +```bash +RELATED_BROKEN=() + +for slug in "${!ENTRY_SLUGS[@]}"; do + frontmatter=$(awk '/^---$/{f=!f; next} f{print}' "${ENTRY_PATH[$slug]}") + rel=$(echo "$frontmatter" | grep '^related:' | sed 's/^related: *//; s/[][]//g; s/,/ /g') + for r in $rel; do + [ -z "$r" ] && continue + if [ -z "${ENTRY_SLUGS[$r]+_}" ]; then + RELATED_BROKEN+=("$slug → related: $r (no such entry)") + fi + done +done +``` + +Print findings: + +``` +=== Broken related-slug findings (${#RELATED_BROKEN[@]}) === +``` + +For each finding, print one line: ` - → related: (no such entry)`. +If `${#RELATED_BROKEN[@]} = 0`, print ` (none)`. + +Report-only. This check never edits frontmatter and never removes a slug — the +orchestrator decides whether to repoint the slug or author the missing entry. + ### 8. Contradiction detection (stub) Contradiction detection is explicitly **descoped** for v1. The function prints @@ -322,12 +366,35 @@ the stub text above is the complete implementation for v1. Build the `.oh/skills/wiki/corpus/README.md` entries table. Sort all entries by `updated:` descending (most recently updated first). -#### 9a. Sort entries by updated date descending +#### 9a. Select the git-tracked entry set, then sort by updated date descending + +The index domain is the **git-tracked** entry set, not the working tree. The +pathspec below is byte-identical to the one in +`.oh/evals/probes/wiki-readme-index.sh`; the two must never diverge, or a local +scratch entry becomes a CI regression. The `corpus/*/*` exclusion keeps a future +subdirectory from silently splitting the two sets — git's pathspec `*` is not +path-aware and descends, while § 2's shell glob does not. + +```bash +TRACKED_SLUGS=() +while IFS= read -r rel; do + [ "$(basename "$rel")" = "README.md" ] && continue + [ "$(basename "$rel")" = "skill-impact.md" ] && continue + abs="$HARNESS/$rel" + [ -f "$abs" ] || continue + slug=$(awk '/^---$/{f=!f; next} f{print}' "$abs" | grep '^slug:' | awk '{print $2}' | head -1) + [ -z "$slug" ] && continue + TRACKED_SLUGS+=("$slug") +done < <(git -C "$HARNESS" ls-files -- \ + '.oh/skills/wiki/corpus/*.md' \ + ':!:.oh/skills/wiki/corpus/raw/*' \ + ':!:.oh/skills/wiki/corpus/*/*') +``` ```bash SORTED_SLUGS=() RANK_LINES=() -for slug in "${!ENTRY_UPDATED[@]}"; do +for slug in "${TRACKED_SLUGS[@]}"; do updated="${ENTRY_UPDATED[$slug]:-0000-00-00}" RANK_LINES+=("$updated $slug") done @@ -426,7 +493,7 @@ extraction to prevent silent divergence (a match that works in one skill must work in the other). Any future change to this extraction method requires updating both skills atomically. -## Five Check Types — Summary +## Six Check Types — Summary | # | Type | Finding trigger | Recommendation | Autonomously sets flag? | |---|------|-----------------|---------------|------------------------| @@ -434,9 +501,10 @@ updating both skills atomically. | 2 | Deprecated | `confidence: deprecated` | consider archive or delete | No — report only | | 3 | Orphan | zero inbound `[[slug]]` references | (informational; true positive even for single-entry corpus) | No | | 4 | Broken outbound | `[[slug]]` in body where `slug` has no matching entry | (informational; fix by adding the entry or correcting the link) | No | +| 4a | Broken related-slug | `related:` frontmatter slug with no matching entry | repoint the slug or author the missing entry | No | | 5 | Contradiction | descoped | n/a — stub only | n/a | -These five types are always reported separately. Types 1 and 2 (both related to +These six types are always reported separately. Types 1 and 2 (both related to "staleness" in a loose sense) MUST NOT be conflated — they have distinct triggers and distinct recommendations. @@ -455,14 +523,21 @@ and distinct recommendations. - **Conflating orphans with broken outbound links** — orphans have no INBOUND links (other entries don't reference them); broken outbound links reference slugs that DO NOT EXIST. They are separate checks with different remediation paths. +- **Conflating broken `related:` slugs with broken `[[slug]]` body links** — the + first is a frontmatter adjacency claim, the second is a navigational link. They are + distinct checks with distinct remediation, reported under separate headings. +- **Regenerating the index from the working tree** — § 9a builds the Index from the + git-tracked entry set, matching `.oh/evals/probes/wiki-readme-index.sh` byte for + byte. Using the § 2 working-tree glob makes an untracked local scratch entry a CI + regression. - **Non-atomic README write** — writing directly to `.oh/skills/wiki/corpus/README.md` without the tmp → validate → rename protocol risks corruption. Always use the three-step atomic write in § 9c. - **Grepping `.oh/skills/wiki/corpus/README.md` for entries** — the README is the output of this skill, not its input. Always enumerate `.oh/skills/wiki/corpus/*.md` directly. -- **Skipping a direct log** — every direct invocation (OP, DRY-RUN, FAIL) appends - a log entry. Audit-child mode is the sole exception and returns its observation - to the outer dispatcher instead. +- **Writing a run log** — there is no log tier. Report OP / DRY-RUN / FAIL and the + findings to the terminal. Audit-child mode returns its observation to the outer + dispatcher instead. - **Hardcoding today's date** — always compute UTC date at runtime with `date -u +%Y-%m-%d`. diff --git a/.oh/skills/wiki/references/query.md b/.oh/skills/wiki/references/query.md index 32a565e1..c52598c8 100644 --- a/.oh/skills/wiki/references/query.md +++ b/.oh/skills/wiki/references/query.md @@ -34,11 +34,32 @@ slow as the corpus grows. ## Argument Interface (locked) ``` -/wiki query +/wiki query [--patterns] ``` -`` is one or more whitespace-separated words. The interface is locked; -do not add flags or positional arguments without editing this SKILL.md. +`` is one or more whitespace-separated words. The interface is locked to +`` plus the `--patterns` flag; adding further flags or positional arguments +requires editing this reference and +`.oh/evals/probes/wiki-query-pattern-isolation.sh`. + +## Two disjoint modes + +`kind:` (`.oh/skills/wiki/references/schema.md` § 2) splits the corpus into source +entries and pattern entries. The two modes never mix. + +| Mode | Returns | Read cap | Ranking | +|------|---------|----------|---------| +| default | `kind: source` only — **an absent `kind:` counts as `source`** | 3 | `updated:` descending | +| `--patterns` | `kind: pattern` only | **5** | term-hit count descending, `updated:` descending as tiebreak | + +There is deliberately **no `--all` mode**. The evidence this split comes from +(`[[wikiskill-experience-compilation]]`) measured the mixed configuration as +worst-of-both: the proposer-only setting scored 63.7 while giving both roles the same +view scored 60.9. Two disjoint modes make the asymmetry the path of least resistance. + +This is a **default, not a boundary**. Any session can read a pattern file directly; +nothing prevents it. The flag keeps patterns out of ordinary results, which is what +the measurement supports — do not describe it as isolation. ## Multi-Word OR Semantics @@ -63,14 +84,22 @@ cap constrains how much context is loaded regardless of match count. ## Instructions -### 1. Parse the topic argument +### 1. Parse the topic argument and mode ```bash -TOPIC="$ARGUMENTS" +ARGUMENTS="${ARGUMENTS:-}" +WANT_KIND=source +CAP=3 +if echo "$ARGUMENTS" | grep -q -- '--patterns'; then + WANT_KIND=pattern + CAP=5 +fi +TOPIC=$(echo "$ARGUMENTS" | sed 's/--patterns//g' | xargs) ``` Split `$TOPIC` on whitespace to produce an array of search terms. Each term -is used independently in the grep step below. +is used independently in the grep step below. An empty `$TOPIC` after stripping the +flag is a usage error: print the `argument-hint` line and exit 0. ### 2. Collect all wiki entry paths @@ -85,7 +114,9 @@ done This enumerates `.oh/skills/wiki/corpus/*.md` directly — NOT via `.oh/skills/wiki/corpus/README.md` (the README is a human-orientation index regenerated by `/wiki lint`, not a query backend). Sub-article files at `.oh/skills/wiki/corpus//.md` are not matched by this glob; -they are scoped for a future iteration. +they are scoped for a future iteration. `kind: pattern` entries are flat +`corpus/.md` files by the `.oh/skills/wiki/references/schema.md` § 2 placement +rule, precisely because this glob does not descend. If no `.oh/skills/wiki/corpus/*.md` files exist, jump to step 5 (empty result). @@ -101,26 +132,42 @@ awk '/^---$/{f=!f; next} f{print}' .oh/skills/wiki/corpus/.md Then grep the extracted frontmatter for any of the topic terms. An entry is a match if the grep succeeds for **at least one** term. -Full loop: +Full loop. Two things happen per entry beyond the v1 behavior: the `kind:` filter +rejects entries belonging to the other mode, and the number of matching terms is +counted for the ranking step. ```bash MATCHES=() +declare -A HITS for entry in "${WIKI_ENTRIES[@]}"; do slug=$(basename "$entry" .md) frontmatter=$(awk '/^---$/{f=!f; next} f{print}' "$entry") - matched=false + + # kind filter — absent means source (schema.md § 2) + kind=$(echo "$frontmatter" | grep '^kind:' | awk '{print $2}' | head -1) + kind="${kind:-source}" + [ "$kind" = "$WANT_KIND" ] || continue + + hits=0 for term in $TOPIC; do - if echo "$frontmatter" | grep -qi "$term"; then - matched=true - break + if echo "$frontmatter" | grep -qi -- "$term"; then + hits=$((hits + 1)) fi done - if [ "$matched" = true ]; then + if [ "$hits" -gt 0 ]; then MATCHES+=("$entry") + HITS["$entry"]=$hits fi done ``` +The `kind` value is read from the output of the § 6 canonical extraction command, +exactly like `updated:` in step 6 — the extraction itself is untouched. + +Note the loop now counts *distinct matching terms* rather than breaking on the first +match. OR semantics are unchanged: one hit still qualifies. The count is only used +for ranking. + The grep targets the full extracted frontmatter block — which contains the `title:`, `slug:`, and `tags:` fields — so all three fields are searched in a single pass. The grep is case-insensitive (`-i`). @@ -153,32 +200,55 @@ Before reading entries into context, sort the match list so the most recently updated entry is first. Extract the `updated:` field from each match's frontmatter using the canonical command: +In `--patterns` mode, `updated:` is the **tiebreak**, not the primary key: entries are +ranked by how many topic terms they matched, descending, and only then by recency. + ```bash -# Build a sortable list: " " +# Build a sortable list: " " (hits zero-padded to sort as text) RANKED=() for m in "${MATCHES[@]}"; do updated=$(awk '/^---$/{f=!f; next} f{print}' "$m" | grep '^updated:' | awk '{print $2}') - RANKED+=("$updated $m") + if [ "$WANT_KIND" = pattern ]; then + RANKED+=("$(printf '%03d' "${HITS[$m]}") $updated $m") + else + RANKED+=("000 $updated $m") + fi done -# Sort descending (most recent first), extract paths +# Sort descending, extract paths SORTED_PATHS=() while IFS= read -r line; do - SORTED_PATHS+=("${line#* }") + rest="${line#* }" # drop hit count + SORTED_PATHS+=("${rest#* }") # drop updated date done < <(printf '%s\n' "${RANKED[@]}" | sort -r) ``` +Default mode pins every hit count to `000`, so the sort reduces to `updated:` +descending — v1 behavior, byte for byte. + +**Why ranking exists in `--patterns` mode only.** Pattern pages accumulate +monotonically and are never pruned, so a recency-only order degenerates into "most +recently compiled" — the wrong bias for a proposer that needs the *relevant* failure +mode, not the newest one. Source pages have a natural refresh cycle through +re-ingest, so their recency ordering still carries signal; leave it alone. + +**Honest caveat.** At roughly one pattern per merged skill change, the pattern layer +will hold single digits of pages for months. The cap-5-plus-hit-ranking is cheap +insurance against a flood that is not yet happening, not a fix for a live problem. Do +not add scoring beyond this — no term frequency, no TF-IDF, no embeddings. + If an entry has no `updated:` field (malformed frontmatter), it sorts to the bottom. `/wiki lint` should surface such entries as a finding. ### 7. Apply the read cap -The read cap is **hardcoded to 3** in v1. Changing the cap requires editing -this SKILL.md — it is NOT configurable via a flag. +The read cap is set by the mode in step 1: **3** for the default source mode +(unchanged from v1) and **5** for `--patterns`. Neither is configurable by a further +flag; changing either requires editing this reference and its probe. ```bash MATCH_COUNT=${#SORTED_PATHS[@]} -CAP=3 +# CAP was set in step 1: 3 (source) or 5 (--patterns) if [ "$MATCH_COUNT" -lt "$CAP" ]; then # Sub-cap: read ALL matches @@ -194,8 +264,8 @@ Two cases: | `Match-Count` | Behavior | |---------------|----------| | 0 | Print empty-result message; exit 0; read nothing | -| 1 or 2 | Read ALL matches into context (sub-cap: `0 < N < 3`) | -| 3 or more | Read top 3 by `updated:` descending; skip the rest | +| below `$CAP` | Read ALL matches into context | +| at or above `$CAP` | Read the top `$CAP` by the mode's ranking; skip the rest | ### 8. Read matched entries into context @@ -241,11 +311,21 @@ work in the other). on a young or sparse corpus. Print the message, exit 0, log it. - **Hard-coding today's date** in the glob or path — always compute UTC date at runtime. -- **Skipping the log** — every invocation appends a log entry, including - empty-result runs and error runs. No exceptions. +- **Writing a run log** — there is no log tier. Summarize the slugs read, the match + count, and how many were skipped to the terminal, and stop. +- **Mixing kinds in one result set** — there is no `--all`. Pattern entries and + source entries answer different questions for different roles, and the measurement + behind the split scored the mixed configuration worst-of-both. +- **Forgetting the `kind:` default** — an entry with no `kind:` field is a `source` + entry. A filter that tests `kind = "source"` against a raw, unset value drops all + fourteen entries authored before the field existed. +- **Ranking patterns by recency alone** — patterns are never pruned, so recency + ordering degenerates into "most recently compiled". Rank by term-hit count first. ## See Also - `.oh/skills/wiki/references/schema.md` — the locked schema, § 6 (frontmatter extraction canonical command), § 2 (entry schema), § 4 (cross-link convention) -- `/wiki ingest` — add or update an entry +- `/wiki ingest` — add or update a `kind: source` entry +- `/wiki compile` — create or patch a `kind: pattern` entry from a `/retro` report - `/wiki lint` — health-check the corpus and regenerate `.oh/skills/wiki/corpus/README.md` +- `.oh/evals/probes/wiki-query-pattern-isolation.sh` — the guard on this contract diff --git a/.oh/skills/wiki/references/schema.md b/.oh/skills/wiki/references/schema.md index 400da662..875fc56e 100644 --- a/.oh/skills/wiki/references/schema.md +++ b/.oh/skills/wiki/references/schema.md @@ -12,7 +12,7 @@ The sharp test: *Is this a fact or synthesis about a topic, intended to be read | Surface | Holds | Written by | When wiki wins instead | | --- | --- | --- | --- | -| `.oh/skills/*/SKILL.md` | Behavioral norms (prescriptive) | Deliberate orchestrator revision | Wiki holds **facts**, skills hold **how to behave** | +| `.oh/skills/*/SKILL.md` | Behavioral norms (prescriptive) | Deliberate orchestrator revision | Wiki holds **facts**, skills hold **how to behave**. A `kind: pattern` entry sits closest to this line: it records that a workaround *worked*, which is evidence; the skill records that the workaround *must be applied*, which is a norm. When a pattern's workaround becomes a rule, it is promoted into a skill and the pattern stays as the evidence for it | | `docs/` | Human-facing prose | Orchestrator / contributors | Wiki is LLM-readable; docs are human-readable | | `.claude/skills/*/SKILL.md` | Executable procedures | Orchestrator | Skills are *how to do*; wiki is *what is true* | | `.oh/skills/wiki/corpus/raw/` | Immutable source captures (snapshots of fetched pages, papers) | Skills writing snapshots only | Same surface; raw is upstream, wiki entries are synthesis | @@ -29,6 +29,7 @@ Every wiki entry is a single markdown file at `.oh/skills/wiki/corpus/.md` --- title: "GitHub Token Workflow Scope" slug: gh-token-workflow-scope +kind: source # source | pattern; absent means source tags: [git, github, auth, ci] created: 2026-05-23 updated: 2026-05-23 @@ -45,6 +46,7 @@ Field definitions: | --- | --- | --- | --- | | `title` | string | yes | Human-readable entry title | | `slug` | string | yes | Matches filename without `.md`; charset `[a-z0-9-]+` | +| `kind` | enum | no | `source` \| `pattern`. **An absent `kind:` field means `kind: source`.** Set to `pattern` only by `/wiki compile` | | `tags` | list of strings | yes | Used by `/wiki query` for frontmatter-only grep | | `created` | date (YYYY-MM-DD) | yes | UTC date of initial creation; never updated | | `updated` | date (YYYY-MM-DD) | yes | UTC date of most recent ingest/edit; always updated on write | @@ -52,6 +54,39 @@ Field definitions: | `related` | list of slugs | no | Slugs of conceptually adjacent entries | | `confidence` | enum | yes | `provisional` \| `confirmed` \| `deprecated` | +### Entry kinds + +The corpus holds two kinds of entry, distinguished by `kind:`. + +| `kind` | Holds | Written by | Read by | +| --- | --- | --- | --- | +| `source` | Facts and synthesis about an external topic, backed by a `raw/` snapshot | `/wiki ingest` | any session, via `/wiki query ` | +| `pattern` | A failure mode or successful strategy observed in this harness's own runs, with an actionable workaround | `/wiki compile` | the proposer role, via `/wiki query --patterns` | + +**Back-compatibility.** `kind` is the only optional-with-default field in this +schema. Every entry authored before the field existed is a `source` entry, and none +require editing. `/wiki ingest` MAY emit `kind: source` explicitly on new source +pages but is not required to. **Consumers that filter on `kind` MUST apply the +default: read the field, and treat empty as `source`.** + +**Pattern placement.** `kind: pattern` entries are flat files at +`.oh/skills/wiki/corpus/pattern--.md` — never in a +subdirectory. The `corpus/*.md` glob used by `/wiki query` and `/wiki lint` does not +descend, so a pattern in a subdirectory would be invisible to both while still +visible to `.oh/evals/probes/wiki-readme-index.sh`'s git pathspec, which does. The +`pattern-` filename prefix is a redundant, greppable encoding of the `kind:` field: +`ls corpus/pattern-*.md` answers "what has this harness learned" without parsing +YAML, and the two must always agree. When a pattern page overflows the word cap, +split it into a second flat pattern page and cross-link; do not create a +sub-article. + +**Why the proposer, and not every session, reads patterns.** The source this rule +comes from measured it: giving the skill proposer access to accumulated knowledge was +worth +15.0 points, while additionally giving the inference agent that same access +*cost* 2.8 (`[[wikiskill-experience-compilation]]`). `--patterns` is a default, not a +boundary — any session can read a pattern file directly. Say "default", never +"isolation". + ### Body layout ```markdown @@ -86,6 +121,61 @@ New or substantially revised architecture pages follow one shape: source files f - **Synthesis stays separate from evidence**: use prose to explain what the cited files imply, but do not let unsupported interpretation look like a source fact. - **Navigation closes the loop**: `## See Also` points to adjacent wiki entries using `[[slug]]` links, so a reader can walk between related pages. +### Pattern body layout (`kind: pattern`) + +A pattern page uses the **same sections in the same order** as a source page. The +paper's Symptom / Root cause / Workaround / Evidence all fit inside them as bold +leads, so patterns need no structural exception and no special case in `/wiki lint`. +The only rule change is that `## Relevant Source Files` — where the evidence lands — +is **required** for `kind: pattern`, where it is optional for `kind: source`. + +```markdown +# + +## Relevant Source Files +- `` — the artifact the pattern is about +- `@` — the run that produced the observation + +## Summary +<2-3 sentences: what goes wrong (or what reliably works), and in which subsystem.> + +## Detail +**Symptom.** + +**Root cause.** + +**Workaround.** + +## See Also +- [[]] +``` + +Title a pattern for the failure mode, not the incident that revealed it: +`pattern-eval-probe-provenance-decay`, not `pattern-2026-08-31-retro-findings`. One +page per failure mode, never one per run — a dated per-run page is a session journal, +which this corpus is not. + +**`sources:` for a pattern.** A pattern entry MUST carry at least one `sources:` +entry. Each is either a `raw/-.md` snapshot path (when the pattern +is grounded in an ingested source) **or** a pinned repository-evidence path of the +form `@` — for example +`.oh/tasks//progress.txt@a1b2c3d`, `.oh/evals/RESULTS.md@a1b2c3d`. The +`@` suffix is required: it buys for a mutable tracked file the same +reproducibility that immutability buys for a `raw/` snapshot. + +**`/wiki compile` MUST NOT write a `raw/` snapshot of a `/retro` report.** `raw/` +holds snapshots of external sources. A `/retro` report is this harness's own +ephemeral output, and `/retro` is report-only by contract; persisting its reports +under `raw/` would recreate the per-session journal tier the harness deliberately +removed, wearing a new name. + +**Authoring constraint.** Pattern prose discusses harness subsystems, so it is the +most likely place for retired vocabulary to reappear. `.oh/evals/probes/audit-stale-references.sh` +greps every tracked file, this corpus included, for retired route and skill names. +Read that probe's pattern before writing about an audit subsystem, and use the +current route names. + ### Word cap and sub-articles Every entry should stay concise enough to read whole into context. Default cap is ≤ 600 words (title excluded, frontmatter excluded). Architecture/harness entries may reach ≤ 900 words when needed for source-file evidence and diagrams. When a topic overflows, split into sub-articles named `.oh/skills/wiki/corpus//.md`. The parent entry becomes an index: its `## Detail` section lists child slugs as `[[parent/child]]` cross-links; each child carries its own frontmatter with its own `slug` (e.g., `gh-auth/sandbox`), `sources`, and `confidence`. @@ -158,6 +248,11 @@ Lifecycle flow: [entry removed or moved to .oh/skills/wiki/corpus/archive/.md] ``` +**Patterns.** A `kind: pattern` entry is created `provisional` by `/wiki compile`. +The orchestrator promotes it to `confirmed` when a skill proposal it motivated is +recorded `ACCEPTED` in `.oh/skills/wiki/corpus/skill-impact.md`. **A `REJECTED` +proposal never demotes or deprecates its motivating pattern** — see § 8. + The archive vs. delete decision for `deprecated` entries is not yet defined — defer to `.oh/skills/wiki/references/schema.md` update after the first deprecation in practice. --- @@ -223,3 +318,59 @@ When `/wiki ingest` is invoked with a source whose derived slug matches an exist 7. **Do NOT concatenate bodies**: the prior `## Summary` and `## Detail` content is replaced, not concatenated. The entry stays ≤ 600 words. **Rationale**: bodies grow unbounded if concatenated across multiple ingests, eventually exceeding the 600-word cap and diluting the entry's utility. The replace-in-place strategy keeps entries fresh and bounded while the `sources:` list preserves the full provenance trail. + +--- + +## 7a. Pattern amendment to the body-merge strategy + +Applies only when the target entry has `kind: pattern`. All of § 7 holds except +steps 1, 2, and 7, which are amended as follows. + +**1'. `## Summary` is replaced** — unchanged from § 7 step 1. The summary is a +rolling 2-3 sentence statement of the current understanding. + +**2'. `## Detail` is merged, not replaced.** + +- `**Symptom.**` and `**Root cause.**` are rewritten in place ONLY when the new + evidence contradicts them. New corroborating evidence adds a citation, not a + rewrite. +- `**Workaround.**` is **append-only**. A new workaround is appended. A workaround + shown not to work is annotated `(superseded YYYY-MM-DD, SI-nnnn)` and left in + place. It is never deleted. + +**7'. The word cap is met by compressing older evidence into one clause, never by +dropping a distinct root cause.** When a pattern page holds two or more distinct root +causes and exceeds the cap, split it into two flat pattern pages (§ 2, pattern +placement) and cross-link them. + +**Rationale**: § 7's replace-in-place strategy keeps a source page fresh against a +moving upstream. A pattern page has no upstream — it is this harness's own +accumulated experience, and replacing it discards exactly the knowledge the page +exists to hold. + +--- + +## 8. Pattern persistence invariant + +**A `kind: pattern` entry is never rolled back.** + +When a skill proposal is rejected and the skill edit is reverted, the revert covers +the skill artifact **only**. The pattern page that motivated the proposal stays, its +`confidence` is unchanged, its `sources:` list is unchanged, and its accumulated +`**Workaround.**` text is unchanged. `/wiki compile` records the rejection as +evidence — annotating the workaround that failed with `(superseded YYYY-MM-DD, +SI-nnnn)` — rather than deleting it. The `skill-impact.md` record of the rejected +proposal is likewise never removed. + +**Reverting a `corpus/` path as collateral of a skill revert is forbidden.** + +Rationale: the knowledge that an approach was tried and did not work is the most +valuable output of a rejected cycle, and it is the only thing preventing the same +proposal being made again. Rolling it back with the code destroys exactly the +persistence this layer exists to provide. + +Prose is not enforcement. The oracles are +`.oh/evals/probes/wiki-pattern-persistence.sh` (pattern pages present at the +merge-base are present at HEAD, and no pattern's `sources:` list has shrunk) and +`.oh/evals/probes/wiki-skill-impact-append-only.sh` (ledger records are added, never +removed or edited in place). diff --git a/CHANGELOG.md b/CHANGELOG.md index 2583a8fc..2852bbb6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -26,6 +26,14 @@ Update policy and release automation live in [`/git`](.claude/skills/git/SKILL.m ### Added - Add gate 5 to `/audit implementation`: fail a promotable change while its diff can still be smaller. Measures net lines and per-function CCN on changed TypeScript via `uvx lizard`. ([#912](https://github.com/mifunedev/openharness/issues/912)) - Add `audit-slop-gate.sh`, a tier-A probe holding gate 5's termination contract: a finding needs a concrete smaller alternative, and the loop ends on the cap or a non-reducing round. ([#912](https://github.com/mifunedev/openharness/issues/912)) +- Add `/wiki compile`, a `kind: pattern` corpus layer, and an append-only `skill-impact.md` ledger, so a `/retro` lesson becomes a page `/builder` reads before proposing. ([#916](https://github.com/mifunedev/openharness/pull/916)) +- Add a `--patterns` mode to `/wiki query` that filters on `kind:`, reads up to five pattern entries, and ranks them by term-hit count before recency. ([#916](https://github.com/mifunedev/openharness/pull/916)) +- Add a fault-injection requirement to the probe contract: a probe is not green until its REGRESSION branch has been driven against a broken input. ([#916](https://github.com/mifunedev/openharness/pull/916)) +- Add capability task `CB-005` scoring whether a lesson reaches a validated skill change; two runs score 0.67 then 1.33, moving the suite mean to 1.44 over a changed task set. ([#916](https://github.com/mifunedev/openharness/pull/916)) + +### Fixed +- Fix `/wiki lint` generating the corpus index from the working tree instead of the git-tracked set, which made any untracked scratch entry a `wiki-readme-index.sh` regression. ([#916](https://github.com/mifunedev/openharness/pull/916)) +- Fix three unresolved `related:` and `[[slug]]` links in the `recursive-language-models` wiki entry, and add the `/wiki lint` check that would have caught them. ([#916](https://github.com/mifunedev/openharness/pull/916)) - Provision the default harnesses into `/home/sandbox/.local` at boot, gated by `OH_PROVISION_HARNESSES`, so `oh harness install` also works from inside the sandbox ([#902](https://github.com/mifunedev/openharness/issues/902)). - Add `oh-home-mount.sh`, a tier-A probe holding the single-`$HOME`-mount contract: one mount per compose file, the baked `/opt/home-seed`, and the checkout prune that replaces `-xdev` ([#898](https://github.com/mifunedev/openharness/issues/898)). - Assert boot-provisioned harnesses in the boot smoke and reject a baked default harness in `verify-sandbox-image.sh`, so CI exercises the install path ([#904](https://github.com/mifunedev/openharness/issues/904)). From 1fdeee538c541537ea45ddb572d04d3dbf6781e0 Mon Sep 17 00:00:00 2001 From: Ryan Eggleston Date: Mon, 31 Aug 2026 16:29:12 -0600 Subject: [PATCH 14/23] skill: add /escalate so an unattended session can reach a human (#921) A cron fire, a detached Advisor, or a background agent that is blocked on an operator decision has nowhere to put it. The GitHub path is proven broken: on #799 the automation's gh identity is the operator's own account, so seven comments produced zero notifications and a one-line kill-switch took ten days. /escalate is one deterministic script with no live agent and no attached terminal in the loop. --needs is required, because an escalation that names no decision is a log line. --key suppresses a repeat for 12h so a looping session cannot train the operator to ignore the channel. Channel health is checked before every send. An unavailable channel is a no-op, not a raised error -- a dead channel must not take down the session reporting through it. But a no-op is still a non-delivery, so the reason is printed, the JSON carries ok:false, and every attempt appends to .oh/logs/escalations.jsonl. Silence is the failure this closes. .oh/logs/ is new: gitignored by default with a tracked README, matching the .oh/tasks/ idiom, for records that must outlive the session that wrote them. --- .github/workflows/ci-harness.yml | 2 +- .gitignore | 2 + .oh/evals/RESULTS.md | 231 ++++++++++++------------ .oh/evals/probes/escalate-contract.sh | 55 ++++++ .oh/logs/README.md | 29 +++ .oh/skills/escalate/SKILL.md | 132 ++++++++++++++ .oh/skills/escalate/scripts/escalate.sh | 111 ++++++++++++ CHANGELOG.md | 3 + 8 files changed, 449 insertions(+), 116 deletions(-) create mode 100755 .oh/evals/probes/escalate-contract.sh create mode 100644 .oh/logs/README.md create mode 100644 .oh/skills/escalate/SKILL.md create mode 100755 .oh/skills/escalate/scripts/escalate.sh diff --git a/.github/workflows/ci-harness.yml b/.github/workflows/ci-harness.yml index 645536ca..0f818e97 100644 --- a/.github/workflows/ci-harness.yml +++ b/.github/workflows/ci-harness.yml @@ -125,7 +125,7 @@ jobs: run: | command -v shellcheck >/dev/null 2>&1 || { sudo apt-get update && sudo apt-get install -y shellcheck; } shellcheck --version - shellcheck -S warning .devcontainer/*.sh .oh/install/*.sh .oh/scripts/*.sh .oh/skills/audit/scripts/*.sh .claude/hooks/*.sh .claude/skills/retro/scripts/*.sh + shellcheck -S warning .devcontainer/*.sh .oh/install/*.sh .oh/scripts/*.sh .oh/skills/audit/scripts/*.sh .oh/skills/escalate/scripts/*.sh .claude/hooks/*.sh .claude/skills/retro/scripts/*.sh - name: Hadolint Dockerfile uses: hadolint/hadolint-action@54c9adbab1582c2ef04b2016b760714a4bfde3cf # v3.1.0 diff --git a/.gitignore b/.gitignore index e4a9924b..7d50295b 100644 --- a/.gitignore +++ b/.gitignore @@ -23,6 +23,8 @@ projects/* # Task artifacts are local runtime state; keep only the directory guide tracked. .oh/tasks/* !.oh/tasks/README.md +.oh/logs/* +!.oh/logs/README.md # Build artifacts **/.venv/ diff --git a/.oh/evals/RESULTS.md b/.oh/evals/RESULTS.md index b08b1cf2..ce74e5d2 100644 --- a/.oh/evals/RESULTS.md +++ b/.oh/evals/RESULTS.md @@ -6,120 +6,121 @@ probe id; git history is the time series.** Schema and exit-code semantics are i | probe | tier | last-run (UTC) | status | source | |-------|------|----------------|--------|--------| -| advisor-monitored-loop | A | 2026-08-31 22:19 | PASS | conversation 2026-06-19 (single-owner implementation workflow, issue #257) | -| agent-browser-cli | A | 2026-08-31 22:19 | PASS | retro lesson 2026-06-07 (agent-browser 0.8.5 CLI) | -| agents-identity-contract | A | 2026-08-31 22:19 | PASS | issue #854 — T3-style root identity, glossary, and skill-owned procedures | -| artifact-contract-audit | A | 2026-08-31 22:19 | PASS | issue #583/#645 — production /audit implementation Gate 1 behavior | -| audit-dispatcher-contract | A | 2026-08-31 22:19 | PASS | issue #645 — audit consolidation public taxonomy | -| audit-implementation-behavior | A | 2026-08-31 22:19 | PASS | issue #645 — implementation root/repo/browser behavior | -| audit-pr-acquire | A | 2026-08-31 22:19 | PASS | issue #645 — production PR acquisition behavior | -| audit-pr-classifier | A | 2026-08-31 22:19 | PASS | issue #645 — deterministic focused and queue PR classifier | -| audit-run-root-contract | A | 2026-08-31 22:19 | PASS | issue #645 — executable immutable audit root/run correlation | -| audit-shellcheck-coverage | A | 2026-08-31 22:19 | PASS | issue #645 — private audit scripts require release and CI lint coverage | -| audit-slop-gate | A | 2026-08-31 22:19 | PASS | .claude/specs/images/reduce-slop.png — the four correctness gates all pass a change | -| audit-stale-references | A | 2026-08-31 22:19 | PASS | issue #645 — clean-breaking audit migration | -| boot-lint-glob | A | 2026-08-31 22:19 | PASS | issue #90, issue #120 | -| builder-skill-consolidation | A | 2026-08-31 22:19 | PASS | issue #643 — consolidate artifact builders behind one /builder dispatcher | -| builder-wiki-proposer | A | 2026-08-31 22:19 | PASS | wikiskill arXiv:2608.27454 — the skill proposer reads accumulated knowledge first | -| capability-benchmark-schema | A | 2026-08-31 22:19 | PASS | issue #167 — capability benchmark instrument | -| cc-safety-net-wiring | A | 2026-08-31 22:19 | SKIPPED | .oh/tasks/cc-safety-net/prd.json US-007 2026-07-19 | -| changelog-entry-length | A | 2026-08-31 22:19 | PASS | conversation 2026-08-24 — CHANGELOG.md grew to 259KB of bullet prose because "one line" was unquantified | -| cleanup-tasks-scoped-guard | A | 2026-08-31 22:19 | PASS | issue #85 | -| cleanup-tasks-worktree-grooming | A | 2026-08-31 22:19 | PASS | issue #168; issue #327 | -| cli-publish-typecheck-scope | A | 2026-08-31 22:19 | PASS | release run 33271077312 — v0.5.0 pushed its GHCR image, then failed to publish | -| close-issues-on-development | A | 2026-08-31 22:19 | PASS | issue #841 (closing keywords never fire because the default branch is main) 2026-08-26 | -| codex-stale-response-retry | A | 2026-08-31 22:19 | PASS | issue #506 — Codex previous_response_not_found RCA | -| compose-config-path-parity | A | 2026-08-31 22:19 | PASS | PR #833 (remove harness.yaml — the wrapper and VS Code "Reopen in Container" paths must resolve the same service) 2026-08-26 | -| config-schema-parity | A | 2026-08-31 22:19 | PASS | PR #833 (one schema file — DOCKER_SOCKET, SANDBOX_SSH, OH_SANDBOX_IMAGE, OH_PULL_POLICY, SKIP_PNPM_INSTALL were consumed but undocumented); rewritten for the oh.json/secrets split by PR #887 | -| context-tier-size-budget | A | 2026-08-31 22:19 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-007) — the always-on tier was 85,256 B | -| continual-learning-20260831 | A | 2026-08-31 22:19 | PASS | retro lesson 2026-08-31 (unexercised oracle) — a probe green in a 112-probe run carried three parser defects | -| cron-claude-codex-fallback | A | 2026-08-31 22:19 | PASS | conversation 2026-06-12 (default Codex fallback for crons) | -| cron-watchdog | A | 2026-08-31 22:19 | PASS | issues #130/#453 (cron runtime watchdog + legacy system-cron reaping) 2026-06-19 | -| crons-directory-guide | A | 2026-08-31 22:19 | PASS | issue #874 | -| curl-bash-safe-alternatives | A | 2026-08-31 22:19 | PASS | vet-run/vet integration — public curl|bash examples need review-first alternatives | -| datasets-schema | A | 2026-08-31 22:19 | PASS | issue #196 — .oh/evals/datasets verifiable trajectory corpus (Repo2RLEnv-inspired) | -| debugmcp-availability | A | 2026-08-31 22:19 | SKIPPED | issue #297 — DebugMCP MCP debug-server availability | -| default-provisioning | A | 2026-08-31 22:19 | PASS | #902 — `oh harness install` must work from inside the sandbox, where | -| delegate-model-effort-policy | A | 2026-08-31 22:19 | PASS | conversation 2026-07-11 (delegate model inheritance and thinking policy) | -| devtcp-hook | A | 2026-08-31 22:19 | PASS | retro lesson 2026-06-10 (zsh /dev/tcp) | -| docker-inspect-env-guard | A | 2026-08-31 22:19 | PASS | operator directive 2026-08-08 (agents keep the docker socket, but must | -| docs-build-fast-path | A | 2026-08-31 22:19 | PASS | #455 — docs builds must stay out of fast harness/eval/release gates; #536 — docs site externalized to openharness-web; docs markdown relocated to docs/ | -| drift-check-cron-staleness-glob | A | 2026-08-31 22:19 | PASS | issue #98; issue #225 (restart-required cron frontmatter/config drift) | -| entrypoint-pnpm-manifest-fingerprint | A | 2026-08-31 22:19 | PASS | issue #521 (manifest-aware sandbox installs) 2026-07-01 | -| eval-ci-gate | A | 2026-08-31 22:19 | PASS | #103 — eval probe suite gated in CI | -| eval-contract-text-20260831 | A | 2026-08-31 22:19 | PASS | retro lesson 2026-08-31 (prose-literal pinning) — two assertions failed on first run purely from source line wrapping | -| eval-gate | A | 2026-08-31 22:19 | PASS | retro lesson 2026-06-11 (eval-gate) | -| eval-results-atomic | A | 2026-08-31 22:19 | PASS | issue #83 (eval-results-atomic-write) | -| eval-runner-exit | A | 2026-08-31 22:19 | PASS | retro lesson 2026-06-11 (eval-runner-exit) #29 | -| eval-runs-once-per-cycle | A | 2026-08-31 22:19 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-006) — /eval ran 3x per cycle on the | -| execution-target-contract | A | 2026-08-31 22:19 | PASS | issue #733 (ExecutionTarget contract + Docker Compose adapter) 2026-08-10 | -| get-oh-bootstrap | A | 2026-08-31 22:19 | PASS | get-oh.sh bootstrap — the Node-bootstrapping host-side path to the standalone `oh` CLI (also on npm as @mifune/openharness; see oh-npm-package.sh) | -| git-skill | A | 2026-08-31 22:19 | PASS | conversation 2026-06-15 — rules are not always supported; git workflow must be the /git skill | -| harness-audit-empty-output-gate | A | 2026-08-31 22:19 | PASS | issue #246 — /audit harness must fail closed on empty auditor outputs | -| harness-ci-core-paths | A | 2026-08-31 22:19 | PASS | #165 — core sandbox config files must trigger harness CI | -| harness-ci-hooks-paths | A | 2026-08-31 22:19 | PASS | issue #202 — credential/security hook changes must trigger harness CI | -| harness-yaml-migration | A | 2026-08-31 22:19 | PASS | PR #833 (migrate-harness-yaml.sh — append / uncomment-in-place / preserve / overwrite, plus a silent no-op second run) 2026-08-26 | -| health-check-docker-stats | A | 2026-08-31 22:19 | PASS | retro lesson 2026-06-10 (docker stats vs ps Size) | -| health-check-socket-degrade | A | 2026-08-31 22:19 | PASS | issue #762 (refs #756) — /health-check degrades to one statement, not nine failures | -| heartbeat-logging-contract | A | 2026-08-31 22:19 | PASS | issue #447 (heartbeat log append hardening) 2026-06-18 | -| image-seed-hygiene | A | 2026-08-31 22:19 | PASS | issue #900 (slim the sandbox image) 2026-08-30 | -| markitdown-wiki-ingest | A | 2026-08-31 22:19 | PASS | issue #649 — pinned local-document normalization contract for /wiki ingest | -| next-dev-prod | A | 2026-08-31 22:19 | SKIPPED | retro lesson 2026-06-04 | -| oh-compose-env-wiring | A | 2026-08-31 22:19 | SKIPPED | issue #880 (oh as the only front door — oh.json is the non-secret config surface) | -| oh-config-surfaces | A | 2026-08-31 22:19 | PASS | PR #887 (config split across two authored surfaces — a tracked oh.json and a secrets-only root dotenv — with nothing left under $HOME) | -| oh-destroy-guard | A | 2026-08-31 22:19 | SKIPPED | issue #879 — `oh` becomes the only front door, so `make destroy` must | -| oh-devcontainer-restructure | A | 2026-08-31 22:19 | PASS | consolidate devcontainer — .oh/devcontainer/ folded back into .devcontainer/ | -| oh-home-mount | A | 2026-08-31 22:19 | PASS | issue #898 (single $HOME mount) 2026-08-30 | -| oh-image-only-deploy | A | 2026-08-31 22:19 | PASS | .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only deploy) | -| oh-init-headless-config | A | 2026-08-31 22:19 | SKIPPED | PR #827 (installer answers landed in the losing config file); retargeted to the .example.env template by PR #833, then to oh.json by PR #887 | -| oh-init-scaffold | A | 2026-08-31 22:19 | PASS | issue #531 Phase 2 | -| oh-lifecycle-surface | A | 2026-08-31 22:19 | PASS | issue #881 — the Makefile is retired and `oh` is the only front door | -| oh-npm-package | A | 2026-08-31 22:19 | PASS | npm publish path for the standalone `oh` CLI (@mifune/openharness) — alternative to get-oh.sh | -| oh-payload-manifest | A | 2026-08-31 22:19 | PASS | issue #531 follow-on (.oh payload manifest — oh update ships a declared allowlist) | -| oh-sandbox-image-mode | A | 2026-08-31 22:19 | PASS | conversation 2026-07-05 (basic Docker deployment — prebuilt-image mode) | -| oh-shipped-repo-overridable | A | 2026-08-31 22:19 | PASS | issue #531 follow-on (de-hardcode residual — shipped .oh shell scripts keep the upstream repo overridable) | -| oh-standalone-lifecycle | A | 2026-08-31 22:19 | PASS | issue #564 | -| oh-update | A | 2026-08-31 22:19 | PASS | issue #531 Phase 3 (oh update — upgrade only the .oh control plane) | -| operator-config-guard | A | 2026-08-31 22:19 | PASS | operator directives 2026-08-06 (.config/ and settings.local.json are operator-only) | -| pnpm-audit-ci-gate | A | 2026-08-31 22:19 | PASS | issue #171 — pnpm security audits must run in CI | -| post-bridge-publish-confirmation | A | 2026-08-31 22:19 | PASS | #523 — post-bridge live publishing requires an explicit final confirmation gate | -| prd-output-path-contract | A | 2026-08-31 22:19 | PASS | retro lesson 2026-06-19 | -| prompt-miner-schema-compat | A | 2026-08-31 22:19 | PASS | issue #253 — prompt-miner JSONL schema-drift guard | -| prompt-miner-symlink-entrypoint | A | 2026-08-31 22:19 | PASS | issue #663 — prompt-miner engine no-ops via the documented .claude/skills symlink | -| prompt-miner-weakness-record | A | 2026-08-31 22:19 | PASS | issue #580 — prompt-miner weakness-record (WH-xxx) cluster output | -| protected-path-deletion | A | 2026-08-31 22:19 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-001) — the critique gate was deleted, | -| protected-paths-resolve | A | 2026-08-31 22:19 | PASS | issue #753 — .claude/protected-paths.txt named 7 paths that did not exist. | -| registry-portability-gate | A | 2026-08-31 22:19 | PASS | issue #758 | -| registry-portability | A | 2026-08-31 22:19 | SKIPPED | issue #758 | -| retro-deterministic-contract | A | 2026-08-31 22:19 | PASS | issue #443 — /retro deterministic output and self-contained helper contract | -| rl-delegation-write-worker | A | 2026-08-31 22:19 | PASS | retro lesson 2026-06-10 (rl-delegation) #57 | -| rlm-context-budget | A | 2026-08-31 22:19 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-006 | -| runtime-preflight-gate | A | 2026-08-31 22:19 | PASS | issue #806 § B1 (open sandbox.substrate vs sandbox.runtime selector); | -| sandbox-boot-guard-ci | A | 2026-08-31 22:19 | PASS | issue #449 (sandbox image build CI guard) 2026-06-19; | -| sandbox-node-base | A | 2026-08-31 22:19 | PASS | openharness#878 — oh as the only front door, T0 sandbox base image | -| skill-paths | A | 2026-08-31 22:19 | PASS | issue #43 — stale path references; extended by issue #69 — apps/->packages/ rename guard; extended by issue #870 — deleted .oh/agents/advisor.md | -| skills-dir-clean | A | 2026-08-31 22:19 | PASS | conversation 2026-06-29 — Pi parses every top-level `.md` in the skills | -| skills-task-tool-coupling | A | 2026-08-31 22:19 | PASS | council review 2026-08-29 (issue #886) — /delegate instructed Claude-Code-only | -| skills-vendored | A | 2026-08-31 22:19 | PASS | absorb .mifune submodule into .oh — the skills/agents/hooks pack is vendored | -| slack-admin-command-surface | A | 2026-08-31 22:19 | PASS | issue #354 — Slack bridge docs must distinguish Pi /msg-bridge commands from Slack DM admin text handlers | -| spec-family-contract | A | 2026-08-31 22:19 | PASS | issue #265; spec-simplification issue #816; workflow authority issue #854; | -| spec-ready-finalization | A | 2026-08-31 22:19 | PASS | issue #134; spec-simplification issue #816; workflow authority issue #854 | -| ste-checker-contract | A | 2026-08-31 22:19 | PASS | issue #750 PR audit — the /ste checker had four fail-open paths (unclosed | -| submitted-by-trailers | A | 2026-08-31 22:19 | PASS | conversation 2026-06-12 (commit attribution trailers); the single-owner | -| sync-skill-contract | A | 2026-08-31 22:19 | PASS | issue #331 — /sync dispatcher skill (bidirectional origin↔upstream sync) | -| t3-headless-launch | A | 2026-08-31 22:19 | PASS | issue #858 — /t3 launched a bare `npx --yes t3`, which is the local GUI and | -| tailscale-tool-boundary | A | 2026-08-31 22:19 | PASS | issue #858 — Tailscale mobile access for T3 Code. There is no tailnet, no | -| tool-catalog-boundary | A | 2026-08-31 22:19 | PASS | agent-browser's exclusion from the harness catalog (#821) and the | -| version-parity | A | 2026-08-31 22:19 | PASS | conversation 2026-08-29 — the oh CLI became the only lifecycle door, so its | -| weigh-scorer-contract | A | 2026-08-31 22:19 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-003 (2026-06-27) | -| wiki-compile-contract | A | 2026-08-31 22:19 | PASS | wikiskill arXiv:2608.27454 — Wiki Maintainer role added as /wiki compile | -| wiki-kind-schema-contract | A | 2026-08-31 22:19 | PASS | wikiskill arXiv:2608.27454 — pattern layer added to the wiki corpus | -| wiki-pattern-persistence | A | 2026-08-31 22:19 | SKIPPED | wikiskill arXiv:2608.27454 — pattern pages are never rolled back | -| wiki-query-pattern-isolation | A | 2026-08-31 22:19 | PASS | wikiskill arXiv:2608.27454 — proposer-only pattern access | -| wiki-readme-index | A | 2026-08-31 22:19 | PASS | issue #132 — wiki README index drift guard | -| wiki-related-slugs | A | 2026-08-31 22:19 | PASS | wikiskill arXiv:2608.27454 — wiki lint related-slug check | -| wiki-skill-impact-append-only | A | 2026-08-31 22:19 | PASS | wikiskill arXiv:2608.27454 — skill-change ledger, never rolled back | -| workflow-boundaries | A | 2026-08-31 22:19 | PASS | conversation 2026-06-19 (workflow consolidation, issue #259); authority moved to /spec in issue #854 | -| worktrees-layout | A | 2026-08-31 22:19 | PASS | issue #872 | +| advisor-monitored-loop | A | 2026-08-31 22:25 | PASS | conversation 2026-06-19 (single-owner implementation workflow, issue #257) | +| agent-browser-cli | A | 2026-08-31 22:25 | PASS | retro lesson 2026-06-07 (agent-browser 0.8.5 CLI) | +| agents-identity-contract | A | 2026-08-31 22:25 | PASS | issue #854 — T3-style root identity, glossary, and skill-owned procedures | +| artifact-contract-audit | A | 2026-08-31 22:25 | PASS | issue #583/#645 — production /audit implementation Gate 1 behavior | +| audit-dispatcher-contract | A | 2026-08-31 22:25 | PASS | issue #645 — audit consolidation public taxonomy | +| audit-implementation-behavior | A | 2026-08-31 22:25 | PASS | issue #645 — implementation root/repo/browser behavior | +| audit-pr-acquire | A | 2026-08-31 22:25 | PASS | issue #645 — production PR acquisition behavior | +| audit-pr-classifier | A | 2026-08-31 22:25 | PASS | issue #645 — deterministic focused and queue PR classifier | +| audit-run-root-contract | A | 2026-08-31 22:25 | PASS | issue #645 — executable immutable audit root/run correlation | +| audit-shellcheck-coverage | A | 2026-08-31 22:25 | PASS | issue #645 — private audit scripts require release and CI lint coverage | +| audit-slop-gate | A | 2026-08-31 22:25 | PASS | .claude/specs/images/reduce-slop.png — the four correctness gates all pass a change | +| audit-stale-references | A | 2026-08-31 22:25 | PASS | issue #645 — clean-breaking audit migration | +| boot-lint-glob | A | 2026-08-31 22:25 | PASS | issue #90, issue #120 | +| builder-skill-consolidation | A | 2026-08-31 22:25 | PASS | issue #643 — consolidate artifact builders behind one /builder dispatcher | +| builder-wiki-proposer | A | 2026-08-31 22:25 | PASS | wikiskill arXiv:2608.27454 — the skill proposer reads accumulated knowledge first | +| capability-benchmark-schema | A | 2026-08-31 22:25 | PASS | issue #167 — capability benchmark instrument | +| cc-safety-net-wiring | A | 2026-08-31 22:25 | SKIPPED | .oh/tasks/cc-safety-net/prd.json US-007 2026-07-19 | +| changelog-entry-length | A | 2026-08-31 22:25 | PASS | conversation 2026-08-24 — CHANGELOG.md grew to 259KB of bullet prose because "one line" was unquantified | +| cleanup-tasks-scoped-guard | A | 2026-08-31 22:25 | PASS | issue #85 | +| cleanup-tasks-worktree-grooming | A | 2026-08-31 22:25 | PASS | issue #168; issue #327 | +| cli-publish-typecheck-scope | A | 2026-08-31 22:25 | PASS | release run 33271077312 — v0.5.0 pushed its GHCR image, then failed to publish | +| close-issues-on-development | A | 2026-08-31 22:25 | PASS | issue #841 (closing keywords never fire because the default branch is main) 2026-08-26 | +| codex-stale-response-retry | A | 2026-08-31 22:25 | PASS | issue #506 — Codex previous_response_not_found RCA | +| compose-config-path-parity | A | 2026-08-31 22:25 | PASS | PR #833 (remove harness.yaml — the wrapper and VS Code "Reopen in Container" paths must resolve the same service) 2026-08-26 | +| config-schema-parity | A | 2026-08-31 22:25 | PASS | PR #833 (one schema file — DOCKER_SOCKET, SANDBOX_SSH, OH_SANDBOX_IMAGE, OH_PULL_POLICY, SKIP_PNPM_INSTALL were consumed but undocumented); rewritten for the oh.json/secrets split by PR #887 | +| context-tier-size-budget | A | 2026-08-31 22:25 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-007) — the always-on tier was 85,256 B | +| continual-learning-20260831 | A | 2026-08-31 22:25 | PASS | retro lesson 2026-08-31 (unexercised oracle) — a probe green in a 112-probe run carried three parser defects | +| cron-claude-codex-fallback | A | 2026-08-31 22:25 | PASS | conversation 2026-06-12 (default Codex fallback for crons) | +| cron-watchdog | A | 2026-08-31 22:25 | PASS | issues #130/#453 (cron runtime watchdog + legacy system-cron reaping) 2026-06-19 | +| crons-directory-guide | A | 2026-08-31 22:25 | PASS | issue #874 | +| curl-bash-safe-alternatives | A | 2026-08-31 22:25 | PASS | vet-run/vet integration — public curl|bash examples need review-first alternatives | +| datasets-schema | A | 2026-08-31 22:25 | PASS | issue #196 — .oh/evals/datasets verifiable trajectory corpus (Repo2RLEnv-inspired) | +| debugmcp-availability | A | 2026-08-31 22:25 | SKIPPED | issue #297 — DebugMCP MCP debug-server availability | +| default-provisioning | A | 2026-08-31 22:25 | PASS | #902 — `oh harness install` must work from inside the sandbox, where | +| delegate-model-effort-policy | A | 2026-08-31 22:25 | PASS | conversation 2026-07-11 (delegate model inheritance and thinking policy) | +| devtcp-hook | A | 2026-08-31 22:25 | PASS | retro lesson 2026-06-10 (zsh /dev/tcp) | +| docker-inspect-env-guard | A | 2026-08-31 22:25 | PASS | operator directive 2026-08-08 (agents keep the docker socket, but must | +| docs-build-fast-path | A | 2026-08-31 22:25 | PASS | #455 — docs builds must stay out of fast harness/eval/release gates; #536 — docs site externalized to openharness-web; docs markdown relocated to docs/ | +| drift-check-cron-staleness-glob | A | 2026-08-31 22:25 | PASS | issue #98; issue #225 (restart-required cron frontmatter/config drift) | +| entrypoint-pnpm-manifest-fingerprint | A | 2026-08-31 22:25 | PASS | issue #521 (manifest-aware sandbox installs) 2026-07-01 | +| escalate-contract | A | 2026-08-31 22:25 | PASS | issue #799 — seven comments on a GitHub thread produced zero notifications and nobody | +| eval-ci-gate | A | 2026-08-31 22:25 | PASS | #103 — eval probe suite gated in CI | +| eval-contract-text-20260831 | A | 2026-08-31 22:25 | PASS | retro lesson 2026-08-31 (prose-literal pinning) — two assertions failed on first run purely from source line wrapping | +| eval-gate | A | 2026-08-31 22:25 | PASS | retro lesson 2026-06-11 (eval-gate) | +| eval-results-atomic | A | 2026-08-31 22:25 | PASS | issue #83 (eval-results-atomic-write) | +| eval-runner-exit | A | 2026-08-31 22:25 | PASS | retro lesson 2026-06-11 (eval-runner-exit) #29 | +| eval-runs-once-per-cycle | A | 2026-08-31 22:25 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-006) — /eval ran 3x per cycle on the | +| execution-target-contract | A | 2026-08-31 22:25 | PASS | issue #733 (ExecutionTarget contract + Docker Compose adapter) 2026-08-10 | +| get-oh-bootstrap | A | 2026-08-31 22:25 | PASS | get-oh.sh bootstrap — the Node-bootstrapping host-side path to the standalone `oh` CLI (also on npm as @mifune/openharness; see oh-npm-package.sh) | +| git-skill | A | 2026-08-31 22:25 | PASS | conversation 2026-06-15 — rules are not always supported; git workflow must be the /git skill | +| harness-audit-empty-output-gate | A | 2026-08-31 22:25 | PASS | issue #246 — /audit harness must fail closed on empty auditor outputs | +| harness-ci-core-paths | A | 2026-08-31 22:25 | PASS | #165 — core sandbox config files must trigger harness CI | +| harness-ci-hooks-paths | A | 2026-08-31 22:25 | PASS | issue #202 — credential/security hook changes must trigger harness CI | +| harness-yaml-migration | A | 2026-08-31 22:25 | PASS | PR #833 (migrate-harness-yaml.sh — append / uncomment-in-place / preserve / overwrite, plus a silent no-op second run) 2026-08-26 | +| health-check-docker-stats | A | 2026-08-31 22:25 | PASS | retro lesson 2026-06-10 (docker stats vs ps Size) | +| health-check-socket-degrade | A | 2026-08-31 22:25 | PASS | issue #762 (refs #756) — /health-check degrades to one statement, not nine failures | +| heartbeat-logging-contract | A | 2026-08-31 22:25 | PASS | issue #447 (heartbeat log append hardening) 2026-06-18 | +| image-seed-hygiene | A | 2026-08-31 22:25 | PASS | issue #900 (slim the sandbox image) 2026-08-30 | +| markitdown-wiki-ingest | A | 2026-08-31 22:25 | PASS | issue #649 — pinned local-document normalization contract for /wiki ingest | +| next-dev-prod | A | 2026-08-31 22:25 | SKIPPED | retro lesson 2026-06-04 | +| oh-compose-env-wiring | A | 2026-08-31 22:25 | SKIPPED | issue #880 (oh as the only front door — oh.json is the non-secret config surface) | +| oh-config-surfaces | A | 2026-08-31 22:25 | PASS | PR #887 (config split across two authored surfaces — a tracked oh.json and a secrets-only root dotenv — with nothing left under $HOME) | +| oh-destroy-guard | A | 2026-08-31 22:25 | SKIPPED | issue #879 — `oh` becomes the only front door, so `make destroy` must | +| oh-devcontainer-restructure | A | 2026-08-31 22:25 | PASS | consolidate devcontainer — .oh/devcontainer/ folded back into .devcontainer/ | +| oh-home-mount | A | 2026-08-31 22:25 | PASS | issue #898 (single $HOME mount) 2026-08-30 | +| oh-image-only-deploy | A | 2026-08-31 22:25 | PASS | .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only deploy) | +| oh-init-headless-config | A | 2026-08-31 22:25 | SKIPPED | PR #827 (installer answers landed in the losing config file); retargeted to the .example.env template by PR #833, then to oh.json by PR #887 | +| oh-init-scaffold | A | 2026-08-31 22:25 | PASS | issue #531 Phase 2 | +| oh-lifecycle-surface | A | 2026-08-31 22:25 | PASS | issue #881 — the Makefile is retired and `oh` is the only front door | +| oh-npm-package | A | 2026-08-31 22:25 | PASS | npm publish path for the standalone `oh` CLI (@mifune/openharness) — alternative to get-oh.sh | +| oh-payload-manifest | A | 2026-08-31 22:25 | PASS | issue #531 follow-on (.oh payload manifest — oh update ships a declared allowlist) | +| oh-sandbox-image-mode | A | 2026-08-31 22:25 | PASS | conversation 2026-07-05 (basic Docker deployment — prebuilt-image mode) | +| oh-shipped-repo-overridable | A | 2026-08-31 22:25 | PASS | issue #531 follow-on (de-hardcode residual — shipped .oh shell scripts keep the upstream repo overridable) | +| oh-standalone-lifecycle | A | 2026-08-31 22:25 | PASS | issue #564 | +| oh-update | A | 2026-08-31 22:25 | PASS | issue #531 Phase 3 (oh update — upgrade only the .oh control plane) | +| operator-config-guard | A | 2026-08-31 22:25 | PASS | operator directives 2026-08-06 (.config/ and settings.local.json are operator-only) | +| pnpm-audit-ci-gate | A | 2026-08-31 22:25 | PASS | issue #171 — pnpm security audits must run in CI | +| post-bridge-publish-confirmation | A | 2026-08-31 22:25 | PASS | #523 — post-bridge live publishing requires an explicit final confirmation gate | +| prd-output-path-contract | A | 2026-08-31 22:25 | PASS | retro lesson 2026-06-19 | +| prompt-miner-schema-compat | A | 2026-08-31 22:25 | PASS | issue #253 — prompt-miner JSONL schema-drift guard | +| prompt-miner-symlink-entrypoint | A | 2026-08-31 22:25 | PASS | issue #663 — prompt-miner engine no-ops via the documented .claude/skills symlink | +| prompt-miner-weakness-record | A | 2026-08-31 22:25 | PASS | issue #580 — prompt-miner weakness-record (WH-xxx) cluster output | +| protected-path-deletion | A | 2026-08-31 22:25 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-001) — the critique gate was deleted, | +| protected-paths-resolve | A | 2026-08-31 22:25 | PASS | issue #753 — .claude/protected-paths.txt named 7 paths that did not exist. | +| registry-portability-gate | A | 2026-08-31 22:25 | PASS | issue #758 | +| registry-portability | A | 2026-08-31 22:25 | SKIPPED | issue #758 | +| retro-deterministic-contract | A | 2026-08-31 22:25 | PASS | issue #443 — /retro deterministic output and self-contained helper contract | +| rl-delegation-write-worker | A | 2026-08-31 22:25 | PASS | retro lesson 2026-06-10 (rl-delegation) #57 | +| rlm-context-budget | A | 2026-08-31 22:25 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-006 | +| runtime-preflight-gate | A | 2026-08-31 22:25 | PASS | issue #806 § B1 (open sandbox.substrate vs sandbox.runtime selector); | +| sandbox-boot-guard-ci | A | 2026-08-31 22:25 | PASS | issue #449 (sandbox image build CI guard) 2026-06-19; | +| sandbox-node-base | A | 2026-08-31 22:25 | PASS | openharness#878 — oh as the only front door, T0 sandbox base image | +| skill-paths | A | 2026-08-31 22:25 | PASS | issue #43 — stale path references; extended by issue #69 — apps/->packages/ rename guard; extended by issue #870 — deleted .oh/agents/advisor.md | +| skills-dir-clean | A | 2026-08-31 22:25 | PASS | conversation 2026-06-29 — Pi parses every top-level `.md` in the skills | +| skills-task-tool-coupling | A | 2026-08-31 22:25 | PASS | council review 2026-08-29 (issue #886) — /delegate instructed Claude-Code-only | +| skills-vendored | A | 2026-08-31 22:25 | PASS | absorb .mifune submodule into .oh — the skills/agents/hooks pack is vendored | +| slack-admin-command-surface | A | 2026-08-31 22:25 | PASS | issue #354 — Slack bridge docs must distinguish Pi /msg-bridge commands from Slack DM admin text handlers | +| spec-family-contract | A | 2026-08-31 22:25 | PASS | issue #265; spec-simplification issue #816; workflow authority issue #854; | +| spec-ready-finalization | A | 2026-08-31 22:25 | PASS | issue #134; spec-simplification issue #816; workflow authority issue #854 | +| ste-checker-contract | A | 2026-08-31 22:25 | PASS | issue #750 PR audit — the /ste checker had four fail-open paths (unclosed | +| submitted-by-trailers | A | 2026-08-31 22:25 | PASS | conversation 2026-06-12 (commit attribution trailers); the single-owner | +| sync-skill-contract | A | 2026-08-31 22:25 | PASS | issue #331 — /sync dispatcher skill (bidirectional origin↔upstream sync) | +| t3-headless-launch | A | 2026-08-31 22:25 | PASS | issue #858 — /t3 launched a bare `npx --yes t3`, which is the local GUI and | +| tailscale-tool-boundary | A | 2026-08-31 22:25 | PASS | issue #858 — Tailscale mobile access for T3 Code. There is no tailnet, no | +| tool-catalog-boundary | A | 2026-08-31 22:25 | PASS | agent-browser's exclusion from the harness catalog (#821) and the | +| version-parity | A | 2026-08-31 22:25 | PASS | conversation 2026-08-29 — the oh CLI became the only lifecycle door, so its | +| weigh-scorer-contract | A | 2026-08-31 22:25 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-003 (2026-06-27) | +| wiki-compile-contract | A | 2026-08-31 22:25 | PASS | wikiskill arXiv:2608.27454 — Wiki Maintainer role added as /wiki compile | +| wiki-kind-schema-contract | A | 2026-08-31 22:25 | PASS | wikiskill arXiv:2608.27454 — pattern layer added to the wiki corpus | +| wiki-pattern-persistence | A | 2026-08-31 22:25 | SKIPPED | wikiskill arXiv:2608.27454 — pattern pages are never rolled back | +| wiki-query-pattern-isolation | A | 2026-08-31 22:25 | PASS | wikiskill arXiv:2608.27454 — proposer-only pattern access | +| wiki-readme-index | A | 2026-08-31 22:25 | PASS | issue #132 — wiki README index drift guard | +| wiki-related-slugs | A | 2026-08-31 22:25 | PASS | wikiskill arXiv:2608.27454 — wiki lint related-slug check | +| wiki-skill-impact-append-only | A | 2026-08-31 22:25 | PASS | wikiskill arXiv:2608.27454 — skill-change ledger, never rolled back | +| workflow-boundaries | A | 2026-08-31 22:25 | PASS | conversation 2026-06-19 (workflow consolidation, issue #259); authority moved to /spec in issue #854 | +| worktrees-layout | A | 2026-08-31 22:25 | PASS | issue #872 | diff --git a/.oh/evals/probes/escalate-contract.sh b/.oh/evals/probes/escalate-contract.sh new file mode 100755 index 00000000..b3b4ac56 --- /dev/null +++ b/.oh/evals/probes/escalate-contract.sh @@ -0,0 +1,55 @@ +#!/usr/bin/env bash +# tier: A +# source: issue #799 — seven comments on a GitHub thread produced zero notifications and nobody +# knew delivery had failed; the kill-switch took ten days instead of one. +# desc: /escalate reaches a human or says so. An unavailable channel is a no-op, never a raised +# error and never silence: the reason is printed and the attempt is recorded under .oh/logs/. +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" +S="$ROOT/.oh/skills/escalate/scripts/escalate.sh" +SKILL="$ROOT/.oh/skills/escalate/SKILL.md" + +fail() { echo "REGRESSION: $*" >&2; exit 1; } + +[[ -f $S && -x $S ]] || fail 'escalate script missing or not executable' +[[ -f $SKILL ]] || fail 'escalate SKILL.md missing' +[[ -f $ROOT/.oh/logs/README.md ]] || fail '.oh/logs/README.md missing — the log directory has no contract' +grep -Fq '.oh/logs/*' "$ROOT/.gitignore" || fail '.oh/logs contents are not gitignored' +grep -Fq '!.oh/logs/README.md' "$ROOT/.gitignore" || fail '.oh/logs/README.md is not exempted from the ignore' + +grep -Fq 'Exit 0 is not proof' "$SKILL" || fail 'SKILL.md does not warn that exit 0 is not delivery' +grep -Fq 'conversations.info' "$SKILL" || fail 'SKILL.md does not document the channel health check' + +tmp=$(mktemp -d); trap 'rm -rf "$tmp"' EXIT +log="$tmp/escalations.jsonl" +run() { ESCALATE_LOG="$log" ESCALATE_STATE_DIR="$tmp/state" ESCALATE_BRIDGE_CONFIG="$tmp/bridge.json" bash "$S" "$@"; } + +run --summary s >/dev/null 2>&1 && fail 'missing --needs was accepted' +[[ $(run --summary s >/dev/null 2>&1; echo $?) == 64 ]] || fail 'usage error must exit 64' +run --needs n >/dev/null 2>&1 && fail 'missing --summary was accepted' + +printf '{"auth":{"channels":{}}}' >"$tmp/bridge.json" +out=$(run --summary s --needs n 2>/dev/null) || fail 'no enabled channel must no-op, not raise' +[[ $(jq -r .ok <<<"$out") == false ]] || fail 'no-op must report ok=false' +[[ $(jq -r .skipped <<<"$out") == true ]] || fail 'no-op must report skipped=true' +[[ -n $(jq -r '.reason // empty' <<<"$out") ]] || fail 'no-op must name a reason' + +err=$(run --summary s --needs n 2>&1 >/dev/null) +grep -Fq 'operator was NOT reached' <<<"$err" || fail 'a no-op must say the operator was not reached' + +[[ -f $log ]] || fail 'a no-op was not recorded under the log path' +[[ $(jq -sr 'length' "$log") -ge 1 ]] || fail 'log holds no record' +[[ $(jq -sr 'last | .ok' "$log") == false ]] || fail 'logged record does not mark non-delivery' +[[ $(jq -sr 'last | .summary' "$log") == s ]] || fail 'logged record does not carry the escalation content' + +printf '{"auth":{"channels":{"C1":{"enabled":true}}}}' >"$tmp/bridge.json" +dry=$(ESCALATE_LOG="$log" ESCALATE_BRIDGE_CONFIG="$tmp/bridge.json" PI_SLACK_BOT_TOKEN=x \ + bash "$S" --dry-run --summary s --needs n) || fail 'dry-run failed' +[[ $(jq -r .channel <<<"$dry") == C1 ]] || fail 'channel not resolved from the bridge config' +[[ $(jq -r .dryRun <<<"$dry") == true ]] || fail 'dry-run must not claim delivery' + +grep -Fq 'Authorization: Bearer %s' "$S" || fail 'token must be passed via a header file, not argv' +grep -Eq '\-H "Authorization: Bearer \$' "$S" && fail 'token interpolated into argv where /proc exposes it' + +echo 'PASS: escalate no-ops loudly on an unavailable channel and records every attempt' >&2 diff --git a/.oh/logs/README.md b/.oh/logs/README.md new file mode 100644 index 00000000..8fb4ea14 --- /dev/null +++ b/.oh/logs/README.md @@ -0,0 +1,29 @@ +# `.oh/logs/` + +Durable operational records written by unattended sessions. Everything here is +gitignored except this file: a log is evidence for the next session and for the +operator, never repository content. + +| File | Written by | Holds | +| ---- | ---------- | ----- | +| `escalations.jsonl` | `.oh/skills/escalate/scripts/escalate.sh` | One JSON object per escalation attempt — delivered and not delivered alike | + +Use this directory when a session must leave a record that outlives it: an +escalation the operator has not answered, a decision made without consent because +none could be obtained, a finding that would otherwise exist only in a tmux +scrollback. + +Do not use it for process logs a service already owns (`/tmp/cron-*.log`, +`/tmp/client-slack-*.log`), for anything a probe or `RESULTS.md` already asserts, +or as a substitute for `evidence.md` in a task folder. + +Each record is one line of JSON so the file stays append-only and greppable +without a parser: + +```bash +jq -c 'select(.ok == false)' .oh/logs/escalations.jsonl # what never reached a human +``` + +Rotation is manual. These files are small and are read by humans after something +went wrong; truncate them when they stop being useful, and never rewrite a record +in place. diff --git a/.oh/skills/escalate/SKILL.md b/.oh/skills/escalate/SKILL.md new file mode 100644 index 00000000..7f144747 --- /dev/null +++ b/.oh/skills/escalate/SKILL.md @@ -0,0 +1,132 @@ +--- +name: escalate +description: | + Deliver a human-addressed escalation from an unattended session to the + operator's Slack channel, so a finding that needs a person does not die in a + log nobody tails or a GitHub thread that notifies no one. + TRIGGER when: an unattended, detached, cron, or background session is blocked + and needs an operator decision; a finding requires consent the session cannot + give (restart infrastructure, spend budget, change identity, publish + externally); a guard or cap refuses and only a human can lift it; asked to + "escalate this", "tell the operator", "notify me", or "ping me in Slack". + Do NOT trigger for routine progress, completion notices, or anything the + session can resolve itself. No-ops when the channel is unavailable. +argument-hint: "--summary --needs [--tried ] [--key ] [--link ]" +allowed-tools: Bash +--- + +# Escalate + +Send one escalation to the operator through the Slack gateway's channel. Use the +bundled script; it is the only supported path and it works with no live agent and +no attached terminal. + +```bash +bash .oh/skills/escalate/scripts/escalate.sh \ + --summary "" \ + --needs "" \ + --tried "" \ + --key "" \ + --link "" +``` + +## When to escalate + +Escalate only when the session is **blocked on a human**. The bar is a decision, +not a status. + +| Escalate | Do not escalate | +|---|---| +| A guard, cap, or permission refuses and only the operator can lift it | A task finished, or is progressing normally | +| An action needs consent the session cannot give — restart infrastructure, spend budget, publish externally, change identity | Something the session can fix, retry, or route around itself | +| A finding will silently rot if nobody sees it before the next run | A finding already captured where the operator will see it | + +An escalation that names no decision is a log line. `--needs` is required for +exactly that reason: if you cannot state what the human must decide, you are +reporting, not escalating. + +## Rules + +1. **Say what you already tried.** An operator who cannot tell what was attempted + has to redo the diagnosis before they can act. Use `--tried`. +2. **Pass a `--key` for anything recurring.** The same key is suppressed for 12 + hours (`ESCALATE_QUIET_HOURS`). A repeating session that escalates the same + finding every run trains the operator to ignore the channel — the + `prompt-miner` cron sent the same escalation nine times before anyone acted. + Use `--force` only when the situation genuinely changed. +3. **Never assume delivery. Exit 0 is not proof.** An unavailable channel is a + **no-op**, not an error: the script exits `0` so a blocked session is not + itself broken by a dead channel, and prints `{"ok":false,"skipped":true,...}`. + Read `.ok`. When it is false the operator was **not** reached, and the session + must surface it where a human will look — a PR comment, `evidence.md`. The + script has already written the record for you (see below); it has not made + anyone read it. +4. **One escalation per blocker.** Do not narrate a session in Slack. +5. **Read the reply channel honestly.** This is one-way. The script delivers a + message; it does not wait for or receive an answer. A session that needs an + answer must stop and leave durable state, not poll. + +## Exit codes + +Branch on `.ok` in the JSON on stdout, not on the exit code alone. + +| Exit | stdout | Meaning | What the session must do | +|---|---|---|---| +| `0` | `{"ok":true,...,"ts":...}` | Delivered | Continue | +| `0` | `{"ok":false,"skipped":true,"reason":...}` | **No-op** — the channel is unavailable | **Record the escalation elsewhere; the operator did not see it** | +| `64` | — | Bad usage: missing `--summary` or `--needs` | Fix the call | +| `75` | — | Suppressed by the quiet window for this `--key` | Continue; the operator was already told | + +## Channel health + +Every send is preceded by a `conversations.info` check on the resolved channel. +The escalation **no-ops** — never raises — when any of these hold: + +- no `PI_SLACK_BOT_TOKEN` is resolvable (checked before any network call); +- no `--channel` and no enabled channel in the bridge config; +- Slack is unreachable, or answers `not_in_channel`, `channel_not_found`, + `invalid_auth`, or any other error; +- the channel is archived; +- `chat.postMessage` itself is rejected. + +A dead channel must not take down the session that was trying to report through +it. That is the whole reason this is a no-op instead of a failure — but a no-op +is still a **non-delivery**, and rule 3 governs what the session owes the +operator afterwards. Silence here is the exact failure this skill exists to +remove, so the reason is always printed to stderr and returned in the JSON. + +## Resolution + +- **Token**: `PI_SLACK_BOT_TOKEN` from the environment, else from + `.devcontainer/.env` — the same order `.oh/scripts/gateway.sh` uses. The token + is passed to `curl` through a header file, never on the command line where + `/proc` would expose it. +- **Channel**: `--channel`, else the first `enabled` entry in + `~/.pi/msg-bridge.json` under `auth.channels`. +- **State**: quiet-window markers in `~/.oh/escalate/` (`ESCALATE_STATE_DIR`). +- **Log**: every attempt, delivered or not, appends one JSON line to + `$OH_PROJECT_ROOT/.oh/logs/escalations.jsonl` (`ESCALATE_LOG`). The path + resolves to the **harness root**, so sessions in worktrees and cron checkouts + all write one canonical trail. Logging never fails the send. + + ```bash + jq -c 'select(.ok == false)' .oh/logs/escalations.jsonl # what never reached a human + ``` + + This is what makes a no-op recoverable: the next session, or the operator after + the fact, can see an escalation was attempted and died in a dead channel. See + [`.oh/logs/README.md`](../../logs/README.md). +- **Timeout**: `ESCALATE_TIMEOUT` seconds per Slack call, default 10 — an + unreachable gateway must not hang an unattended session. + +Verify wiring without sending anything by adding `--dry-run`, which prints the +resolved channel and the exact rendered message. + +## Not this skill + +- **Sending a routine message to Slack.** This skill is for escalations; its + format and dedupe assume a human must act. +- **Talking to the gateway agent.** The `client-slack-pi` tmux session is an + interactive agent. Do not type into it to send a message — that requires a + human at a keyboard, which is the failure this skill exists to remove. +- **Receiving replies.** One-way by design. diff --git a/.oh/skills/escalate/scripts/escalate.sh b/.oh/skills/escalate/scripts/escalate.sh new file mode 100755 index 00000000..b1e3d9a9 --- /dev/null +++ b/.oh/skills/escalate/scripts/escalate.sh @@ -0,0 +1,111 @@ +#!/usr/bin/env bash +set -euo pipefail + +HARNESS="${OH_PROJECT_ROOT:-$(cd "$(dirname "${BASH_SOURCE[0]}")/../../../.." && pwd)}" +SLACK_ENV="$HARNESS/.devcontainer/.env" +BRIDGE_CONFIG="${ESCALATE_BRIDGE_CONFIG:-$HOME/.pi/msg-bridge.json}" +STATE_DIR="${ESCALATE_STATE_DIR:-$HOME/.oh/escalate}" +LOG_FILE="${ESCALATE_LOG:-$HARNESS/.oh/logs/escalations.jsonl}" +QUIET_HOURS="${ESCALATE_QUIET_HOURS:-12}" + +summary='' needs='' tried='' link='' key='' channel='' dry_run=0 force=0 +while [ $# -gt 0 ]; do + case $1 in + --summary) summary=${2:-}; shift 2 ;; + --needs) needs=${2:-}; shift 2 ;; + --tried) tried=${2:-}; shift 2 ;; + --link) link=${2:-}; shift 2 ;; + --key) key=${2:-}; shift 2 ;; + --channel) channel=${2:-}; shift 2 ;; + --dry-run) dry_run=1; shift ;; + --force) force=1; shift ;; + *) echo "escalate: unknown argument: $1" >&2; exit 64 ;; + esac +done + +[ -n "$summary" ] || { echo 'escalate: --summary is required' >&2; exit 64; } +[ -n "$needs" ] || { echo 'escalate: --needs is required — an escalation names the decision only a human can make' >&2; exit 64; } + +record() { + mkdir -p "$(dirname "$LOG_FILE")" 2>/dev/null || return 0 + printf '%s\n' "$1" >>"$LOG_FILE" 2>/dev/null || true +} + +noop() { + printf 'escalate: no-op — %s; the operator was NOT reached\n' "$1" >&2 + entry=$(jq -c -n --arg at "$(date -u +%FT%TZ)" --arg reason "$1" --arg channel "${channel:-}" \ + --arg summary "$summary" --arg needs "$needs" --arg tried "$tried" --arg link "$link" --arg key "$key" \ + '{at:$at,ok:false,skipped:true,reason:$reason,channel:$channel,summary:$summary,needs:$needs,tried:$tried,link:$link,key:$key}') + record "$entry" + jq -n --arg reason "$1" --arg channel "${channel:-}" '{ok:false,skipped:true,reason:$reason,channel:$channel}' + exit 0 +} + +if [ -z "${PI_SLACK_BOT_TOKEN:-}" ] && [ -f "$SLACK_ENV" ]; then + t=$(grep -E '^PI_SLACK_BOT_TOKEN=' "$SLACK_ENV" | tail -1 | cut -d= -f2-) + [ -n "$t" ] && export PI_SLACK_BOT_TOKEN="$t" + unset t +fi +[ -n "${PI_SLACK_BOT_TOKEN:-}" ] || noop 'no PI_SLACK_BOT_TOKEN in the environment or .devcontainer/.env' + +if [ -z "$channel" ]; then + [ -f "$BRIDGE_CONFIG" ] || noop "no --channel and no bridge config at $BRIDGE_CONFIG" + channel=$(jq -r 'first((.auth.channels // {}) | to_entries[] | select(.value.enabled == true) | .key) // empty' "$BRIDGE_CONFIG") + [ -n "$channel" ] || noop "no enabled channel in $BRIDGE_CONFIG" +fi + +host=$(hostname 2>/dev/null || echo unknown) +branch=$(git -C "$HARNESS" rev-parse --abbrev-ref HEAD 2>/dev/null || echo unknown) +text=$(printf '*Escalation from an unattended session*\n\n%s\n\n*Needs a human to:* %s' "$summary" "$needs") +[ -n "$tried" ] && text=$(printf '%s\n\n*Already tried:* %s' "$text" "$tried") +[ -n "$link" ] && text=$(printf '%s\n\n%s' "$text" "$link") +text=$(printf '%s\n\n_%s · %s · %s_' "$text" "$host" "$branch" "$(date -u +%FT%TZ)") + +if [ "$dry_run" -eq 1 ]; then + jq -n --arg channel "$channel" --arg text "$text" '{dryRun:true,channel:$channel,text:$text}' + exit 0 +fi + +slack_api() { + curl -sS --max-time "${ESCALATE_TIMEOUT:-10}" "https://slack.com/api/$1" \ + -H @<(printf 'Authorization: Bearer %s\n' "$PI_SLACK_BOT_TOKEN") "${@:2}" +} + +health=$(slack_api conversations.info -G --data-urlencode "channel=$channel") \ + || noop "Slack unreachable while checking channel $channel" +if [ "$(jq -r '.ok' <<<"$health")" != true ]; then + noop "channel $channel unavailable: $(jq -r '.error // "unknown"' <<<"$health")" +fi +if [ "$(jq -r '.channel.is_archived // false' <<<"$health")" = true ]; then + noop "channel $channel is archived" +fi + +if [ -n "$key" ] && [ "$force" -eq 0 ]; then + mkdir -p "$STATE_DIR" + marker="$STATE_DIR/$(printf '%s' "$key" | tr -c 'A-Za-z0-9._-' '_')" + if [ -f "$marker" ]; then + last=$(cat "$marker" 2>/dev/null || echo 0) + age=$(( $(date -u +%s) - last )) + if [ "$age" -lt $(( QUIET_HOURS * 3600 )) ]; then + printf 'escalate: suppressed — key %s already escalated %sh ago (quiet window %sh); use --force to override\n' \ + "$key" "$(( age / 3600 ))" "$QUIET_HOURS" >&2 + exit 75 + fi + fi +fi + +payload=$(jq -n --arg channel "$channel" --arg text "$text" '{channel:$channel,text:$text}') +response=$(printf '%s' "$payload" | curl -sS -X POST https://slack.com/api/chat.postMessage \ + -H 'Content-Type: application/json; charset=utf-8' \ + -H @<(printf 'Authorization: Bearer %s\n' "$PI_SLACK_BOT_TOKEN") \ + --data @- ) || noop 'transport failure calling chat.postMessage' + +if [ "$(jq -r '.ok' <<<"$response")" != true ]; then + noop "Slack rejected the message: $(jq -r '.error // "unknown"' <<<"$response")" +fi + +[ -n "$key" ] && { mkdir -p "$STATE_DIR"; date -u +%s >"$STATE_DIR/$(printf '%s' "$key" | tr -c 'A-Za-z0-9._-' '_')"; } +record "$(jq -c -n --arg at "$(date -u +%FT%TZ)" --arg channel "$channel" --arg ts "$(jq -r .ts <<<"$response")" \ + --arg summary "$summary" --arg needs "$needs" --arg tried "$tried" --arg link "$link" --arg key "$key" \ + '{at:$at,ok:true,channel:$channel,ts:$ts,summary:$summary,needs:$needs,tried:$tried,link:$link,key:$key}')" +jq -c '{ok,channel,ts}' <<<"$response" diff --git a/CHANGELOG.md b/CHANGELOG.md index 2852bbb6..3cb4d04d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -24,6 +24,9 @@ Update policy and release automation live in [`/git`](.claude/skills/git/SKILL.m - **BREAKING:** Retire the `projectRoot` / `OH_PROJECT_ROOT` config knob — the checkout is fixed at `/home/sandbox/harness`, nested inside the home mount ([#898](https://github.com/mifunedev/openharness/issues/898)). ### Added +- Add `/escalate`: an unattended session delivers a human-addressed escalation to the operator's Slack channel. An unavailable channel no-ops loudly rather than failing the session. ([#919](https://github.com/mifunedev/openharness/issues/919)) +- Add `.oh/logs/`, gitignored by default with a tracked README, for records that outlive the session that wrote them; `/escalate` appends every attempt to `escalations.jsonl`. ([#919](https://github.com/mifunedev/openharness/issues/919)) +- Add `escalate-contract.sh`, a tier-A probe: a no-op names its reason, is recorded, and `--dry-run` makes no network call. ([#919](https://github.com/mifunedev/openharness/issues/919)) - Add gate 5 to `/audit implementation`: fail a promotable change while its diff can still be smaller. Measures net lines and per-function CCN on changed TypeScript via `uvx lizard`. ([#912](https://github.com/mifunedev/openharness/issues/912)) - Add `audit-slop-gate.sh`, a tier-A probe holding gate 5's termination contract: a finding needs a concrete smaller alternative, and the loop ends on the cap or a non-reducing round. ([#912](https://github.com/mifunedev/openharness/issues/912)) - Add `/wiki compile`, a `kind: pattern` corpus layer, and an append-only `skill-impact.md` ledger, so a `/retro` lesson becomes a page `/builder` reads before proposing. ([#916](https://github.com/mifunedev/openharness/pull/916)) From db6fe74f895186329cc3a106869c5ddffc37c527 Mon Sep 17 00:00:00 2001 From: Ryan Eggleston Date: Mon, 31 Aug 2026 16:41:01 -0600 Subject: [PATCH 15/23] task: give .oh/logs an AGENTS.md guide instead of a README (#925) Root AGENTS.md draws the line already: a nested AGENTS.md belongs to directories whose contents are produced apart from the root context, and everything else uses a README. .oh/logs/ is that shape -- unattended sessions write to it without the root context loaded, and the guide's reader is the next agent deciding whether a record belongs there. Renames the guide, adds the CLAUDE.md provider symlink, un-ignores both, and names the fourth directory in the root sentence. The escalate-contract probe now asserts the symlink target rather than only the guide's presence. Adding the directory to that sentence pushed AGENTS.md to 9507 B against a 9500 B budget and the context-tier ratchet caught it, so the sentence is compressed rather than the budget raised. --- .gitignore | 3 +- .oh/evals/RESULTS.md | 232 +++++++++++++------------- .oh/evals/probes/escalate-contract.sh | 8 +- .oh/logs/{README.md => AGENTS.md} | 6 + .oh/logs/CLAUDE.md | 1 + .oh/skills/escalate/SKILL.md | 2 +- AGENTS.md | 4 +- CHANGELOG.md | 3 + 8 files changed, 137 insertions(+), 122 deletions(-) rename .oh/logs/{README.md => AGENTS.md} (82%) create mode 120000 .oh/logs/CLAUDE.md diff --git a/.gitignore b/.gitignore index 7d50295b..2a993679 100644 --- a/.gitignore +++ b/.gitignore @@ -24,7 +24,8 @@ projects/* .oh/tasks/* !.oh/tasks/README.md .oh/logs/* -!.oh/logs/README.md +!.oh/logs/AGENTS.md +!.oh/logs/CLAUDE.md # Build artifacts **/.venv/ diff --git a/.oh/evals/RESULTS.md b/.oh/evals/RESULTS.md index ce74e5d2..a5223f82 100644 --- a/.oh/evals/RESULTS.md +++ b/.oh/evals/RESULTS.md @@ -6,121 +6,121 @@ probe id; git history is the time series.** Schema and exit-code semantics are i | probe | tier | last-run (UTC) | status | source | |-------|------|----------------|--------|--------| -| advisor-monitored-loop | A | 2026-08-31 22:25 | PASS | conversation 2026-06-19 (single-owner implementation workflow, issue #257) | -| agent-browser-cli | A | 2026-08-31 22:25 | PASS | retro lesson 2026-06-07 (agent-browser 0.8.5 CLI) | -| agents-identity-contract | A | 2026-08-31 22:25 | PASS | issue #854 — T3-style root identity, glossary, and skill-owned procedures | -| artifact-contract-audit | A | 2026-08-31 22:25 | PASS | issue #583/#645 — production /audit implementation Gate 1 behavior | -| audit-dispatcher-contract | A | 2026-08-31 22:25 | PASS | issue #645 — audit consolidation public taxonomy | -| audit-implementation-behavior | A | 2026-08-31 22:25 | PASS | issue #645 — implementation root/repo/browser behavior | -| audit-pr-acquire | A | 2026-08-31 22:25 | PASS | issue #645 — production PR acquisition behavior | -| audit-pr-classifier | A | 2026-08-31 22:25 | PASS | issue #645 — deterministic focused and queue PR classifier | -| audit-run-root-contract | A | 2026-08-31 22:25 | PASS | issue #645 — executable immutable audit root/run correlation | -| audit-shellcheck-coverage | A | 2026-08-31 22:25 | PASS | issue #645 — private audit scripts require release and CI lint coverage | -| audit-slop-gate | A | 2026-08-31 22:25 | PASS | .claude/specs/images/reduce-slop.png — the four correctness gates all pass a change | -| audit-stale-references | A | 2026-08-31 22:25 | PASS | issue #645 — clean-breaking audit migration | -| boot-lint-glob | A | 2026-08-31 22:25 | PASS | issue #90, issue #120 | -| builder-skill-consolidation | A | 2026-08-31 22:25 | PASS | issue #643 — consolidate artifact builders behind one /builder dispatcher | -| builder-wiki-proposer | A | 2026-08-31 22:25 | PASS | wikiskill arXiv:2608.27454 — the skill proposer reads accumulated knowledge first | -| capability-benchmark-schema | A | 2026-08-31 22:25 | PASS | issue #167 — capability benchmark instrument | -| cc-safety-net-wiring | A | 2026-08-31 22:25 | SKIPPED | .oh/tasks/cc-safety-net/prd.json US-007 2026-07-19 | -| changelog-entry-length | A | 2026-08-31 22:25 | PASS | conversation 2026-08-24 — CHANGELOG.md grew to 259KB of bullet prose because "one line" was unquantified | -| cleanup-tasks-scoped-guard | A | 2026-08-31 22:25 | PASS | issue #85 | -| cleanup-tasks-worktree-grooming | A | 2026-08-31 22:25 | PASS | issue #168; issue #327 | -| cli-publish-typecheck-scope | A | 2026-08-31 22:25 | PASS | release run 33271077312 — v0.5.0 pushed its GHCR image, then failed to publish | -| close-issues-on-development | A | 2026-08-31 22:25 | PASS | issue #841 (closing keywords never fire because the default branch is main) 2026-08-26 | -| codex-stale-response-retry | A | 2026-08-31 22:25 | PASS | issue #506 — Codex previous_response_not_found RCA | -| compose-config-path-parity | A | 2026-08-31 22:25 | PASS | PR #833 (remove harness.yaml — the wrapper and VS Code "Reopen in Container" paths must resolve the same service) 2026-08-26 | -| config-schema-parity | A | 2026-08-31 22:25 | PASS | PR #833 (one schema file — DOCKER_SOCKET, SANDBOX_SSH, OH_SANDBOX_IMAGE, OH_PULL_POLICY, SKIP_PNPM_INSTALL were consumed but undocumented); rewritten for the oh.json/secrets split by PR #887 | -| context-tier-size-budget | A | 2026-08-31 22:25 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-007) — the always-on tier was 85,256 B | -| continual-learning-20260831 | A | 2026-08-31 22:25 | PASS | retro lesson 2026-08-31 (unexercised oracle) — a probe green in a 112-probe run carried three parser defects | -| cron-claude-codex-fallback | A | 2026-08-31 22:25 | PASS | conversation 2026-06-12 (default Codex fallback for crons) | -| cron-watchdog | A | 2026-08-31 22:25 | PASS | issues #130/#453 (cron runtime watchdog + legacy system-cron reaping) 2026-06-19 | -| crons-directory-guide | A | 2026-08-31 22:25 | PASS | issue #874 | -| curl-bash-safe-alternatives | A | 2026-08-31 22:25 | PASS | vet-run/vet integration — public curl|bash examples need review-first alternatives | -| datasets-schema | A | 2026-08-31 22:25 | PASS | issue #196 — .oh/evals/datasets verifiable trajectory corpus (Repo2RLEnv-inspired) | -| debugmcp-availability | A | 2026-08-31 22:25 | SKIPPED | issue #297 — DebugMCP MCP debug-server availability | -| default-provisioning | A | 2026-08-31 22:25 | PASS | #902 — `oh harness install` must work from inside the sandbox, where | -| delegate-model-effort-policy | A | 2026-08-31 22:25 | PASS | conversation 2026-07-11 (delegate model inheritance and thinking policy) | -| devtcp-hook | A | 2026-08-31 22:25 | PASS | retro lesson 2026-06-10 (zsh /dev/tcp) | -| docker-inspect-env-guard | A | 2026-08-31 22:25 | PASS | operator directive 2026-08-08 (agents keep the docker socket, but must | -| docs-build-fast-path | A | 2026-08-31 22:25 | PASS | #455 — docs builds must stay out of fast harness/eval/release gates; #536 — docs site externalized to openharness-web; docs markdown relocated to docs/ | -| drift-check-cron-staleness-glob | A | 2026-08-31 22:25 | PASS | issue #98; issue #225 (restart-required cron frontmatter/config drift) | -| entrypoint-pnpm-manifest-fingerprint | A | 2026-08-31 22:25 | PASS | issue #521 (manifest-aware sandbox installs) 2026-07-01 | -| escalate-contract | A | 2026-08-31 22:25 | PASS | issue #799 — seven comments on a GitHub thread produced zero notifications and nobody | -| eval-ci-gate | A | 2026-08-31 22:25 | PASS | #103 — eval probe suite gated in CI | -| eval-contract-text-20260831 | A | 2026-08-31 22:25 | PASS | retro lesson 2026-08-31 (prose-literal pinning) — two assertions failed on first run purely from source line wrapping | -| eval-gate | A | 2026-08-31 22:25 | PASS | retro lesson 2026-06-11 (eval-gate) | -| eval-results-atomic | A | 2026-08-31 22:25 | PASS | issue #83 (eval-results-atomic-write) | -| eval-runner-exit | A | 2026-08-31 22:25 | PASS | retro lesson 2026-06-11 (eval-runner-exit) #29 | -| eval-runs-once-per-cycle | A | 2026-08-31 22:25 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-006) — /eval ran 3x per cycle on the | -| execution-target-contract | A | 2026-08-31 22:25 | PASS | issue #733 (ExecutionTarget contract + Docker Compose adapter) 2026-08-10 | -| get-oh-bootstrap | A | 2026-08-31 22:25 | PASS | get-oh.sh bootstrap — the Node-bootstrapping host-side path to the standalone `oh` CLI (also on npm as @mifune/openharness; see oh-npm-package.sh) | -| git-skill | A | 2026-08-31 22:25 | PASS | conversation 2026-06-15 — rules are not always supported; git workflow must be the /git skill | -| harness-audit-empty-output-gate | A | 2026-08-31 22:25 | PASS | issue #246 — /audit harness must fail closed on empty auditor outputs | -| harness-ci-core-paths | A | 2026-08-31 22:25 | PASS | #165 — core sandbox config files must trigger harness CI | -| harness-ci-hooks-paths | A | 2026-08-31 22:25 | PASS | issue #202 — credential/security hook changes must trigger harness CI | -| harness-yaml-migration | A | 2026-08-31 22:25 | PASS | PR #833 (migrate-harness-yaml.sh — append / uncomment-in-place / preserve / overwrite, plus a silent no-op second run) 2026-08-26 | -| health-check-docker-stats | A | 2026-08-31 22:25 | PASS | retro lesson 2026-06-10 (docker stats vs ps Size) | -| health-check-socket-degrade | A | 2026-08-31 22:25 | PASS | issue #762 (refs #756) — /health-check degrades to one statement, not nine failures | -| heartbeat-logging-contract | A | 2026-08-31 22:25 | PASS | issue #447 (heartbeat log append hardening) 2026-06-18 | -| image-seed-hygiene | A | 2026-08-31 22:25 | PASS | issue #900 (slim the sandbox image) 2026-08-30 | -| markitdown-wiki-ingest | A | 2026-08-31 22:25 | PASS | issue #649 — pinned local-document normalization contract for /wiki ingest | -| next-dev-prod | A | 2026-08-31 22:25 | SKIPPED | retro lesson 2026-06-04 | -| oh-compose-env-wiring | A | 2026-08-31 22:25 | SKIPPED | issue #880 (oh as the only front door — oh.json is the non-secret config surface) | -| oh-config-surfaces | A | 2026-08-31 22:25 | PASS | PR #887 (config split across two authored surfaces — a tracked oh.json and a secrets-only root dotenv — with nothing left under $HOME) | -| oh-destroy-guard | A | 2026-08-31 22:25 | SKIPPED | issue #879 — `oh` becomes the only front door, so `make destroy` must | -| oh-devcontainer-restructure | A | 2026-08-31 22:25 | PASS | consolidate devcontainer — .oh/devcontainer/ folded back into .devcontainer/ | -| oh-home-mount | A | 2026-08-31 22:25 | PASS | issue #898 (single $HOME mount) 2026-08-30 | -| oh-image-only-deploy | A | 2026-08-31 22:25 | PASS | .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only deploy) | -| oh-init-headless-config | A | 2026-08-31 22:25 | SKIPPED | PR #827 (installer answers landed in the losing config file); retargeted to the .example.env template by PR #833, then to oh.json by PR #887 | -| oh-init-scaffold | A | 2026-08-31 22:25 | PASS | issue #531 Phase 2 | -| oh-lifecycle-surface | A | 2026-08-31 22:25 | PASS | issue #881 — the Makefile is retired and `oh` is the only front door | -| oh-npm-package | A | 2026-08-31 22:25 | PASS | npm publish path for the standalone `oh` CLI (@mifune/openharness) — alternative to get-oh.sh | -| oh-payload-manifest | A | 2026-08-31 22:25 | PASS | issue #531 follow-on (.oh payload manifest — oh update ships a declared allowlist) | -| oh-sandbox-image-mode | A | 2026-08-31 22:25 | PASS | conversation 2026-07-05 (basic Docker deployment — prebuilt-image mode) | -| oh-shipped-repo-overridable | A | 2026-08-31 22:25 | PASS | issue #531 follow-on (de-hardcode residual — shipped .oh shell scripts keep the upstream repo overridable) | -| oh-standalone-lifecycle | A | 2026-08-31 22:25 | PASS | issue #564 | -| oh-update | A | 2026-08-31 22:25 | PASS | issue #531 Phase 3 (oh update — upgrade only the .oh control plane) | -| operator-config-guard | A | 2026-08-31 22:25 | PASS | operator directives 2026-08-06 (.config/ and settings.local.json are operator-only) | -| pnpm-audit-ci-gate | A | 2026-08-31 22:25 | PASS | issue #171 — pnpm security audits must run in CI | -| post-bridge-publish-confirmation | A | 2026-08-31 22:25 | PASS | #523 — post-bridge live publishing requires an explicit final confirmation gate | -| prd-output-path-contract | A | 2026-08-31 22:25 | PASS | retro lesson 2026-06-19 | -| prompt-miner-schema-compat | A | 2026-08-31 22:25 | PASS | issue #253 — prompt-miner JSONL schema-drift guard | -| prompt-miner-symlink-entrypoint | A | 2026-08-31 22:25 | PASS | issue #663 — prompt-miner engine no-ops via the documented .claude/skills symlink | -| prompt-miner-weakness-record | A | 2026-08-31 22:25 | PASS | issue #580 — prompt-miner weakness-record (WH-xxx) cluster output | -| protected-path-deletion | A | 2026-08-31 22:25 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-001) — the critique gate was deleted, | -| protected-paths-resolve | A | 2026-08-31 22:25 | PASS | issue #753 — .claude/protected-paths.txt named 7 paths that did not exist. | -| registry-portability-gate | A | 2026-08-31 22:25 | PASS | issue #758 | -| registry-portability | A | 2026-08-31 22:25 | SKIPPED | issue #758 | -| retro-deterministic-contract | A | 2026-08-31 22:25 | PASS | issue #443 — /retro deterministic output and self-contained helper contract | -| rl-delegation-write-worker | A | 2026-08-31 22:25 | PASS | retro lesson 2026-06-10 (rl-delegation) #57 | -| rlm-context-budget | A | 2026-08-31 22:25 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-006 | -| runtime-preflight-gate | A | 2026-08-31 22:25 | PASS | issue #806 § B1 (open sandbox.substrate vs sandbox.runtime selector); | -| sandbox-boot-guard-ci | A | 2026-08-31 22:25 | PASS | issue #449 (sandbox image build CI guard) 2026-06-19; | -| sandbox-node-base | A | 2026-08-31 22:25 | PASS | openharness#878 — oh as the only front door, T0 sandbox base image | -| skill-paths | A | 2026-08-31 22:25 | PASS | issue #43 — stale path references; extended by issue #69 — apps/->packages/ rename guard; extended by issue #870 — deleted .oh/agents/advisor.md | -| skills-dir-clean | A | 2026-08-31 22:25 | PASS | conversation 2026-06-29 — Pi parses every top-level `.md` in the skills | -| skills-task-tool-coupling | A | 2026-08-31 22:25 | PASS | council review 2026-08-29 (issue #886) — /delegate instructed Claude-Code-only | -| skills-vendored | A | 2026-08-31 22:25 | PASS | absorb .mifune submodule into .oh — the skills/agents/hooks pack is vendored | -| slack-admin-command-surface | A | 2026-08-31 22:25 | PASS | issue #354 — Slack bridge docs must distinguish Pi /msg-bridge commands from Slack DM admin text handlers | -| spec-family-contract | A | 2026-08-31 22:25 | PASS | issue #265; spec-simplification issue #816; workflow authority issue #854; | -| spec-ready-finalization | A | 2026-08-31 22:25 | PASS | issue #134; spec-simplification issue #816; workflow authority issue #854 | -| ste-checker-contract | A | 2026-08-31 22:25 | PASS | issue #750 PR audit — the /ste checker had four fail-open paths (unclosed | -| submitted-by-trailers | A | 2026-08-31 22:25 | PASS | conversation 2026-06-12 (commit attribution trailers); the single-owner | -| sync-skill-contract | A | 2026-08-31 22:25 | PASS | issue #331 — /sync dispatcher skill (bidirectional origin↔upstream sync) | -| t3-headless-launch | A | 2026-08-31 22:25 | PASS | issue #858 — /t3 launched a bare `npx --yes t3`, which is the local GUI and | -| tailscale-tool-boundary | A | 2026-08-31 22:25 | PASS | issue #858 — Tailscale mobile access for T3 Code. There is no tailnet, no | -| tool-catalog-boundary | A | 2026-08-31 22:25 | PASS | agent-browser's exclusion from the harness catalog (#821) and the | -| version-parity | A | 2026-08-31 22:25 | PASS | conversation 2026-08-29 — the oh CLI became the only lifecycle door, so its | -| weigh-scorer-contract | A | 2026-08-31 22:25 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-003 (2026-06-27) | -| wiki-compile-contract | A | 2026-08-31 22:25 | PASS | wikiskill arXiv:2608.27454 — Wiki Maintainer role added as /wiki compile | -| wiki-kind-schema-contract | A | 2026-08-31 22:25 | PASS | wikiskill arXiv:2608.27454 — pattern layer added to the wiki corpus | -| wiki-pattern-persistence | A | 2026-08-31 22:25 | SKIPPED | wikiskill arXiv:2608.27454 — pattern pages are never rolled back | -| wiki-query-pattern-isolation | A | 2026-08-31 22:25 | PASS | wikiskill arXiv:2608.27454 — proposer-only pattern access | -| wiki-readme-index | A | 2026-08-31 22:25 | PASS | issue #132 — wiki README index drift guard | -| wiki-related-slugs | A | 2026-08-31 22:25 | PASS | wikiskill arXiv:2608.27454 — wiki lint related-slug check | -| wiki-skill-impact-append-only | A | 2026-08-31 22:25 | PASS | wikiskill arXiv:2608.27454 — skill-change ledger, never rolled back | -| workflow-boundaries | A | 2026-08-31 22:25 | PASS | conversation 2026-06-19 (workflow consolidation, issue #259); authority moved to /spec in issue #854 | -| worktrees-layout | A | 2026-08-31 22:25 | PASS | issue #872 | +| advisor-monitored-loop | A | 2026-08-31 22:35 | PASS | conversation 2026-06-19 (single-owner implementation workflow, issue #257) | +| agent-browser-cli | A | 2026-08-31 22:35 | PASS | retro lesson 2026-06-07 (agent-browser 0.8.5 CLI) | +| agents-identity-contract | A | 2026-08-31 22:35 | PASS | issue #854 — T3-style root identity, glossary, and skill-owned procedures | +| artifact-contract-audit | A | 2026-08-31 22:35 | PASS | issue #583/#645 — production /audit implementation Gate 1 behavior | +| audit-dispatcher-contract | A | 2026-08-31 22:35 | PASS | issue #645 — audit consolidation public taxonomy | +| audit-implementation-behavior | A | 2026-08-31 22:35 | PASS | issue #645 — implementation root/repo/browser behavior | +| audit-pr-acquire | A | 2026-08-31 22:35 | PASS | issue #645 — production PR acquisition behavior | +| audit-pr-classifier | A | 2026-08-31 22:35 | PASS | issue #645 — deterministic focused and queue PR classifier | +| audit-run-root-contract | A | 2026-08-31 22:35 | PASS | issue #645 — executable immutable audit root/run correlation | +| audit-shellcheck-coverage | A | 2026-08-31 22:35 | PASS | issue #645 — private audit scripts require release and CI lint coverage | +| audit-slop-gate | A | 2026-08-31 22:35 | PASS | .claude/specs/images/reduce-slop.png — the four correctness gates all pass a change | +| audit-stale-references | A | 2026-08-31 22:35 | PASS | issue #645 — clean-breaking audit migration | +| boot-lint-glob | A | 2026-08-31 22:35 | PASS | issue #90, issue #120 | +| builder-skill-consolidation | A | 2026-08-31 22:35 | PASS | issue #643 — consolidate artifact builders behind one /builder dispatcher | +| builder-wiki-proposer | A | 2026-08-31 22:35 | PASS | wikiskill arXiv:2608.27454 — the skill proposer reads accumulated knowledge first | +| capability-benchmark-schema | A | 2026-08-31 22:35 | PASS | issue #167 — capability benchmark instrument | +| cc-safety-net-wiring | A | 2026-08-31 22:35 | SKIPPED | .oh/tasks/cc-safety-net/prd.json US-007 2026-07-19 | +| changelog-entry-length | A | 2026-08-31 22:35 | PASS | conversation 2026-08-24 — CHANGELOG.md grew to 259KB of bullet prose because "one line" was unquantified | +| cleanup-tasks-scoped-guard | A | 2026-08-31 22:35 | PASS | issue #85 | +| cleanup-tasks-worktree-grooming | A | 2026-08-31 22:35 | PASS | issue #168; issue #327 | +| cli-publish-typecheck-scope | A | 2026-08-31 22:35 | PASS | release run 33271077312 — v0.5.0 pushed its GHCR image, then failed to publish | +| close-issues-on-development | A | 2026-08-31 22:35 | PASS | issue #841 (closing keywords never fire because the default branch is main) 2026-08-26 | +| codex-stale-response-retry | A | 2026-08-31 22:35 | PASS | issue #506 — Codex previous_response_not_found RCA | +| compose-config-path-parity | A | 2026-08-31 22:35 | PASS | PR #833 (remove harness.yaml — the wrapper and VS Code "Reopen in Container" paths must resolve the same service) 2026-08-26 | +| config-schema-parity | A | 2026-08-31 22:35 | PASS | PR #833 (one schema file — DOCKER_SOCKET, SANDBOX_SSH, OH_SANDBOX_IMAGE, OH_PULL_POLICY, SKIP_PNPM_INSTALL were consumed but undocumented); rewritten for the oh.json/secrets split by PR #887 | +| context-tier-size-budget | A | 2026-08-31 22:35 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-007) — the always-on tier was 85,256 B | +| continual-learning-20260831 | A | 2026-08-31 22:35 | PASS | retro lesson 2026-08-31 (unexercised oracle) — a probe green in a 112-probe run carried three parser defects | +| cron-claude-codex-fallback | A | 2026-08-31 22:35 | PASS | conversation 2026-06-12 (default Codex fallback for crons) | +| cron-watchdog | A | 2026-08-31 22:35 | PASS | issues #130/#453 (cron runtime watchdog + legacy system-cron reaping) 2026-06-19 | +| crons-directory-guide | A | 2026-08-31 22:35 | PASS | issue #874 | +| curl-bash-safe-alternatives | A | 2026-08-31 22:35 | PASS | vet-run/vet integration — public curl|bash examples need review-first alternatives | +| datasets-schema | A | 2026-08-31 22:35 | PASS | issue #196 — .oh/evals/datasets verifiable trajectory corpus (Repo2RLEnv-inspired) | +| debugmcp-availability | A | 2026-08-31 22:35 | SKIPPED | issue #297 — DebugMCP MCP debug-server availability | +| default-provisioning | A | 2026-08-31 22:35 | PASS | #902 — `oh harness install` must work from inside the sandbox, where | +| delegate-model-effort-policy | A | 2026-08-31 22:35 | PASS | conversation 2026-07-11 (delegate model inheritance and thinking policy) | +| devtcp-hook | A | 2026-08-31 22:35 | PASS | retro lesson 2026-06-10 (zsh /dev/tcp) | +| docker-inspect-env-guard | A | 2026-08-31 22:35 | PASS | operator directive 2026-08-08 (agents keep the docker socket, but must | +| docs-build-fast-path | A | 2026-08-31 22:35 | PASS | #455 — docs builds must stay out of fast harness/eval/release gates; #536 — docs site externalized to openharness-web; docs markdown relocated to docs/ | +| drift-check-cron-staleness-glob | A | 2026-08-31 22:35 | PASS | issue #98; issue #225 (restart-required cron frontmatter/config drift) | +| entrypoint-pnpm-manifest-fingerprint | A | 2026-08-31 22:35 | PASS | issue #521 (manifest-aware sandbox installs) 2026-07-01 | +| escalate-contract | A | 2026-08-31 22:35 | PASS | issue #799 — seven comments on a GitHub thread produced zero notifications and nobody | +| eval-ci-gate | A | 2026-08-31 22:35 | PASS | #103 — eval probe suite gated in CI | +| eval-contract-text-20260831 | A | 2026-08-31 22:35 | PASS | retro lesson 2026-08-31 (prose-literal pinning) — two assertions failed on first run purely from source line wrapping | +| eval-gate | A | 2026-08-31 22:35 | PASS | retro lesson 2026-06-11 (eval-gate) | +| eval-results-atomic | A | 2026-08-31 22:35 | PASS | issue #83 (eval-results-atomic-write) | +| eval-runner-exit | A | 2026-08-31 22:35 | PASS | retro lesson 2026-06-11 (eval-runner-exit) #29 | +| eval-runs-once-per-cycle | A | 2026-08-31 22:35 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-006) — /eval ran 3x per cycle on the | +| execution-target-contract | A | 2026-08-31 22:35 | PASS | issue #733 (ExecutionTarget contract + Docker Compose adapter) 2026-08-10 | +| get-oh-bootstrap | A | 2026-08-31 22:35 | PASS | get-oh.sh bootstrap — the Node-bootstrapping host-side path to the standalone `oh` CLI (also on npm as @mifune/openharness; see oh-npm-package.sh) | +| git-skill | A | 2026-08-31 22:35 | PASS | conversation 2026-06-15 — rules are not always supported; git workflow must be the /git skill | +| harness-audit-empty-output-gate | A | 2026-08-31 22:35 | PASS | issue #246 — /audit harness must fail closed on empty auditor outputs | +| harness-ci-core-paths | A | 2026-08-31 22:35 | PASS | #165 — core sandbox config files must trigger harness CI | +| harness-ci-hooks-paths | A | 2026-08-31 22:35 | PASS | issue #202 — credential/security hook changes must trigger harness CI | +| harness-yaml-migration | A | 2026-08-31 22:35 | PASS | PR #833 (migrate-harness-yaml.sh — append / uncomment-in-place / preserve / overwrite, plus a silent no-op second run) 2026-08-26 | +| health-check-docker-stats | A | 2026-08-31 22:35 | PASS | retro lesson 2026-06-10 (docker stats vs ps Size) | +| health-check-socket-degrade | A | 2026-08-31 22:35 | PASS | issue #762 (refs #756) — /health-check degrades to one statement, not nine failures | +| heartbeat-logging-contract | A | 2026-08-31 22:35 | PASS | issue #447 (heartbeat log append hardening) 2026-06-18 | +| image-seed-hygiene | A | 2026-08-31 22:35 | PASS | issue #900 (slim the sandbox image) 2026-08-30 | +| markitdown-wiki-ingest | A | 2026-08-31 22:35 | PASS | issue #649 — pinned local-document normalization contract for /wiki ingest | +| next-dev-prod | A | 2026-08-31 22:35 | SKIPPED | retro lesson 2026-06-04 | +| oh-compose-env-wiring | A | 2026-08-31 22:35 | SKIPPED | issue #880 (oh as the only front door — oh.json is the non-secret config surface) | +| oh-config-surfaces | A | 2026-08-31 22:35 | PASS | PR #887 (config split across two authored surfaces — a tracked oh.json and a secrets-only root dotenv — with nothing left under $HOME) | +| oh-destroy-guard | A | 2026-08-31 22:35 | SKIPPED | issue #879 — `oh` becomes the only front door, so `make destroy` must | +| oh-devcontainer-restructure | A | 2026-08-31 22:35 | PASS | consolidate devcontainer — .oh/devcontainer/ folded back into .devcontainer/ | +| oh-home-mount | A | 2026-08-31 22:35 | PASS | issue #898 (single $HOME mount) 2026-08-30 | +| oh-image-only-deploy | A | 2026-08-31 22:35 | PASS | .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only deploy) | +| oh-init-headless-config | A | 2026-08-31 22:35 | SKIPPED | PR #827 (installer answers landed in the losing config file); retargeted to the .example.env template by PR #833, then to oh.json by PR #887 | +| oh-init-scaffold | A | 2026-08-31 22:35 | PASS | issue #531 Phase 2 | +| oh-lifecycle-surface | A | 2026-08-31 22:35 | PASS | issue #881 — the Makefile is retired and `oh` is the only front door | +| oh-npm-package | A | 2026-08-31 22:35 | PASS | npm publish path for the standalone `oh` CLI (@mifune/openharness) — alternative to get-oh.sh | +| oh-payload-manifest | A | 2026-08-31 22:35 | PASS | issue #531 follow-on (.oh payload manifest — oh update ships a declared allowlist) | +| oh-sandbox-image-mode | A | 2026-08-31 22:35 | PASS | conversation 2026-07-05 (basic Docker deployment — prebuilt-image mode) | +| oh-shipped-repo-overridable | A | 2026-08-31 22:35 | PASS | issue #531 follow-on (de-hardcode residual — shipped .oh shell scripts keep the upstream repo overridable) | +| oh-standalone-lifecycle | A | 2026-08-31 22:35 | PASS | issue #564 | +| oh-update | A | 2026-08-31 22:35 | PASS | issue #531 Phase 3 (oh update — upgrade only the .oh control plane) | +| operator-config-guard | A | 2026-08-31 22:35 | PASS | operator directives 2026-08-06 (.config/ and settings.local.json are operator-only) | +| pnpm-audit-ci-gate | A | 2026-08-31 22:35 | PASS | issue #171 — pnpm security audits must run in CI | +| post-bridge-publish-confirmation | A | 2026-08-31 22:35 | PASS | #523 — post-bridge live publishing requires an explicit final confirmation gate | +| prd-output-path-contract | A | 2026-08-31 22:35 | PASS | retro lesson 2026-06-19 | +| prompt-miner-schema-compat | A | 2026-08-31 22:35 | PASS | issue #253 — prompt-miner JSONL schema-drift guard | +| prompt-miner-symlink-entrypoint | A | 2026-08-31 22:35 | PASS | issue #663 — prompt-miner engine no-ops via the documented .claude/skills symlink | +| prompt-miner-weakness-record | A | 2026-08-31 22:35 | PASS | issue #580 — prompt-miner weakness-record (WH-xxx) cluster output | +| protected-path-deletion | A | 2026-08-31 22:35 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-001) — the critique gate was deleted, | +| protected-paths-resolve | A | 2026-08-31 22:35 | PASS | issue #753 — .claude/protected-paths.txt named 7 paths that did not exist. | +| registry-portability-gate | A | 2026-08-31 22:35 | PASS | issue #758 | +| registry-portability | A | 2026-08-31 22:35 | SKIPPED | issue #758 | +| retro-deterministic-contract | A | 2026-08-31 22:35 | PASS | issue #443 — /retro deterministic output and self-contained helper contract | +| rl-delegation-write-worker | A | 2026-08-31 22:35 | PASS | retro lesson 2026-06-10 (rl-delegation) #57 | +| rlm-context-budget | A | 2026-08-31 22:35 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-006 | +| runtime-preflight-gate | A | 2026-08-31 22:35 | PASS | issue #806 § B1 (open sandbox.substrate vs sandbox.runtime selector); | +| sandbox-boot-guard-ci | A | 2026-08-31 22:35 | PASS | issue #449 (sandbox image build CI guard) 2026-06-19; | +| sandbox-node-base | A | 2026-08-31 22:35 | PASS | openharness#878 — oh as the only front door, T0 sandbox base image | +| skill-paths | A | 2026-08-31 22:35 | PASS | issue #43 — stale path references; extended by issue #69 — apps/->packages/ rename guard; extended by issue #870 — deleted .oh/agents/advisor.md | +| skills-dir-clean | A | 2026-08-31 22:35 | PASS | conversation 2026-06-29 — Pi parses every top-level `.md` in the skills | +| skills-task-tool-coupling | A | 2026-08-31 22:35 | PASS | council review 2026-08-29 (issue #886) — /delegate instructed Claude-Code-only | +| skills-vendored | A | 2026-08-31 22:35 | PASS | absorb .mifune submodule into .oh — the skills/agents/hooks pack is vendored | +| slack-admin-command-surface | A | 2026-08-31 22:35 | PASS | issue #354 — Slack bridge docs must distinguish Pi /msg-bridge commands from Slack DM admin text handlers | +| spec-family-contract | A | 2026-08-31 22:35 | PASS | issue #265; spec-simplification issue #816; workflow authority issue #854; | +| spec-ready-finalization | A | 2026-08-31 22:35 | PASS | issue #134; spec-simplification issue #816; workflow authority issue #854 | +| ste-checker-contract | A | 2026-08-31 22:35 | PASS | issue #750 PR audit — the /ste checker had four fail-open paths (unclosed | +| submitted-by-trailers | A | 2026-08-31 22:35 | PASS | conversation 2026-06-12 (commit attribution trailers); the single-owner | +| sync-skill-contract | A | 2026-08-31 22:35 | PASS | issue #331 — /sync dispatcher skill (bidirectional origin↔upstream sync) | +| t3-headless-launch | A | 2026-08-31 22:35 | PASS | issue #858 — /t3 launched a bare `npx --yes t3`, which is the local GUI and | +| tailscale-tool-boundary | A | 2026-08-31 22:35 | PASS | issue #858 — Tailscale mobile access for T3 Code. There is no tailnet, no | +| tool-catalog-boundary | A | 2026-08-31 22:35 | PASS | agent-browser's exclusion from the harness catalog (#821) and the | +| version-parity | A | 2026-08-31 22:35 | PASS | conversation 2026-08-29 — the oh CLI became the only lifecycle door, so its | +| weigh-scorer-contract | A | 2026-08-31 22:35 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-003 (2026-06-27) | +| wiki-compile-contract | A | 2026-08-31 22:35 | PASS | wikiskill arXiv:2608.27454 — Wiki Maintainer role added as /wiki compile | +| wiki-kind-schema-contract | A | 2026-08-31 22:35 | PASS | wikiskill arXiv:2608.27454 — pattern layer added to the wiki corpus | +| wiki-pattern-persistence | A | 2026-08-31 22:35 | PASS | wikiskill arXiv:2608.27454 — pattern pages are never rolled back | +| wiki-query-pattern-isolation | A | 2026-08-31 22:35 | PASS | wikiskill arXiv:2608.27454 — proposer-only pattern access | +| wiki-readme-index | A | 2026-08-31 22:35 | PASS | issue #132 — wiki README index drift guard | +| wiki-related-slugs | A | 2026-08-31 22:35 | PASS | wikiskill arXiv:2608.27454 — wiki lint related-slug check | +| wiki-skill-impact-append-only | A | 2026-08-31 22:35 | PASS | wikiskill arXiv:2608.27454 — skill-change ledger, never rolled back | +| workflow-boundaries | A | 2026-08-31 22:35 | PASS | conversation 2026-06-19 (workflow consolidation, issue #259); authority moved to /spec in issue #854 | +| worktrees-layout | A | 2026-08-31 22:35 | PASS | issue #872 | diff --git a/.oh/evals/probes/escalate-contract.sh b/.oh/evals/probes/escalate-contract.sh index b3b4ac56..85c21e39 100755 --- a/.oh/evals/probes/escalate-contract.sh +++ b/.oh/evals/probes/escalate-contract.sh @@ -14,9 +14,13 @@ fail() { echo "REGRESSION: $*" >&2; exit 1; } [[ -f $S && -x $S ]] || fail 'escalate script missing or not executable' [[ -f $SKILL ]] || fail 'escalate SKILL.md missing' -[[ -f $ROOT/.oh/logs/README.md ]] || fail '.oh/logs/README.md missing — the log directory has no contract' +[[ -f $ROOT/.oh/logs/AGENTS.md ]] || fail '.oh/logs/AGENTS.md missing — the log directory has no contract' +[[ -L $ROOT/.oh/logs/CLAUDE.md && $(readlink "$ROOT/.oh/logs/CLAUDE.md") == AGENTS.md ]] \ + || fail '.oh/logs/CLAUDE.md must be a symlink to the sibling AGENTS.md' grep -Fq '.oh/logs/*' "$ROOT/.gitignore" || fail '.oh/logs contents are not gitignored' -grep -Fq '!.oh/logs/README.md' "$ROOT/.gitignore" || fail '.oh/logs/README.md is not exempted from the ignore' +for keep in '!.oh/logs/AGENTS.md' '!.oh/logs/CLAUDE.md'; do + grep -Fq "$keep" "$ROOT/.gitignore" || fail "$keep is not exempted from the ignore" +done grep -Fq 'Exit 0 is not proof' "$SKILL" || fail 'SKILL.md does not warn that exit 0 is not delivery' grep -Fq 'conversations.info' "$SKILL" || fail 'SKILL.md does not document the channel health check' diff --git a/.oh/logs/README.md b/.oh/logs/AGENTS.md similarity index 82% rename from .oh/logs/README.md rename to .oh/logs/AGENTS.md index 8fb4ea14..19074455 100644 --- a/.oh/logs/README.md +++ b/.oh/logs/AGENTS.md @@ -1,5 +1,7 @@ # `.oh/logs/` +`CLAUDE.md` is a provider-compatibility symlink to this file. Edit `AGENTS.md`. + Durable operational records written by unattended sessions. Everything here is gitignored except this file: a log is evidence for the next session and for the operator, never repository content. @@ -24,6 +26,10 @@ without a parser: jq -c 'select(.ok == false)' .oh/logs/escalations.jsonl # what never reached a human ``` +Write here only when the record must survive the session. A session that can +still act on a finding acts on it; a session that cannot leaves the record and +says so where a human looks. + Rotation is manual. These files are small and are read by humans after something went wrong; truncate them when they stop being useful, and never rewrite a record in place. diff --git a/.oh/logs/CLAUDE.md b/.oh/logs/CLAUDE.md new file mode 120000 index 00000000..47dc3e3d --- /dev/null +++ b/.oh/logs/CLAUDE.md @@ -0,0 +1 @@ +AGENTS.md \ No newline at end of file diff --git a/.oh/skills/escalate/SKILL.md b/.oh/skills/escalate/SKILL.md index 7f144747..e72f1216 100644 --- a/.oh/skills/escalate/SKILL.md +++ b/.oh/skills/escalate/SKILL.md @@ -115,7 +115,7 @@ remove, so the reason is always printed to stderr and returned in the JSON. This is what makes a no-op recoverable: the next session, or the operator after the fact, can see an escalation was attempted and died in a dead channel. See - [`.oh/logs/README.md`](../../logs/README.md). + [`.oh/logs/AGENTS.md`](../../logs/AGENTS.md). - **Timeout**: `ESCALATE_TIMEOUT` seconds per Slack call, default 10 — an unreachable gateway must not hang an unattended session. diff --git a/AGENTS.md b/AGENTS.md index ed86bd7d..99c6f363 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -141,8 +141,8 @@ silently skip a surface. ## How to work in this repository This file is the only always-on context. A nested `AGENTS.md` exists only in -`.worktrees/`, `projects/`, and `crons/`, whose contents run or are checked -out apart from it. Every other directory uses a `README.md`. +`.worktrees/`, `projects/`, `crons/`, and `.oh/logs/`, whose contents are +produced apart from it. Every other directory uses a `README.md`. Use the lifecycle in this order: diff --git a/CHANGELOG.md b/CHANGELOG.md index 3cb4d04d..c32d4897 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -23,6 +23,9 @@ Update policy and release automation live in [`/git`](.claude/skills/git/SKILL.m - **BREAKING:** Retire the DeepAgents harness — `deepagents-cli` is deprecated upstream. `install.deepagents` is no longer a settable oh.json field ([#910](https://github.com/mifunedev/openharness/issues/910)). - **BREAKING:** Retire the `projectRoot` / `OH_PROJECT_ROOT` config knob — the checkout is fixed at `/home/sandbox/harness`, nested inside the home mount ([#898](https://github.com/mifunedev/openharness/issues/898)). +### Changed +- `.oh/logs/` carries an `AGENTS.md` with a `CLAUDE.md` symlink instead of a `README.md`, matching the directories whose contents are produced apart from the root context. ([#924](https://github.com/mifunedev/openharness/issues/924)) + ### Added - Add `/escalate`: an unattended session delivers a human-addressed escalation to the operator's Slack channel. An unavailable channel no-ops loudly rather than failing the session. ([#919](https://github.com/mifunedev/openharness/issues/919)) - Add `.oh/logs/`, gitignored by default with a tracked README, for records that outlive the session that wrote them; `/escalate` appends every attempt to `escalations.jsonl`. ([#919](https://github.com/mifunedev/openharness/issues/919)) From 1c5f37230822ec2bbc5ed316be92ad295722b693 Mon Sep 17 00:00:00 2001 From: Ryan Eggleston Date: Mon, 31 Aug 2026 18:07:29 -0600 Subject: [PATCH 16/23] FROM task/920-compose-env-boundary TO development (#922) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * task: scaffold compose-env-boundary task Four-file contract: - prd.md: 11 user stories - prd.json: schemaVersion 1, branchName task/920-compose-env-boundary - prompt.md: the rendered single-owner task prompt - progress.txt: empty header Tracks #920. PRD generated by /prd; converted by /ralph. Submitted-by: Claude * task: make the CLI the only install/config surface Move eleven settings out of the compose environment: block into oh.json, read inside the container through the oh CLI. A value belongs in compose only if a process outside the sandbox — or the entrypoint before the control plane is readable — must act on it. - entrypoint.sh gains oh_config/oh_config_truthy over `oh config show`, and loses the duplicate agent-browser and Tailscale installers; the tool catalog is now the sole owner of both pins and Tailscale's two checksums. - The Hermes wiring keys off `command -v hermes` instead of INSTALL_HERMES, so it runs identically in both sandbox flavors. Flavor B previously carried none of the flavor-A-only keys, leaving all of it dead there. - The sshd overlay keeps its ports: and loses its environment:; sshd's mode comes from access.ssh* in oh.json. - Retire docker-compose.hermes-dashboard.yml and its published 127.0.0.1:9119. - config-render.ts stops rendering ten keys and adds each to RETIRED_KEYS, so a re-added put() throws; tools/catalog.ts drops entrypointGuard. - cron-runtime.ts resolves the agent binary from CRON_AGENT_BIN, then oh.json, then "claude". - tool-catalog-boundary and tailscale-tool-boundary are inverted to assert the post-epic boundary; every zero-exposure assertion is kept. Every oh.json field survives — only its .env projection is gone. Tracks #920. Submitted-by: Claude * task: detect the sandbox flavor instead of declaring it Replace the OH_IMAGE_ONLY compose flag with a runtime check. Both conditions are load-bearing: mountpoint alone misreads an empty bind mounted straight at the project root as a checkout, and the .oh/ test alone would send a seeded no-bind volume through the host-UID sync on its second boot. The detected mode is logged on both paths, so a wrong detection shows up in `oh logs` rather than silently. `.oh/.image-seeded` is now gitignored, so a misdetection can never leave an untracked marker in a real checkout. With the flag gone the two compose files' environment: blocks are byte-identical; a new tier-A probe, compose-env-boundary.sh, holds that across every compose file and overlay: no INSTALL_*, no OH_IMAGE_ONLY, and every key either rendered by config-render.ts or one of four documented literals. oh-image-only-deploy.sh drops the three assertions that only required documentation prose to contain or omit a string; every behavioral assertion is kept and the gate check now targets detection. Tracks #920. Submitted-by: Claude * task: record compose-env-boundary evidence and eval result Story graph complete (11/11), /eval clean at this HEAD, and evidence.md answers back to the approved plan: what improved, what was built, five divergences, and what stays unverified without a Docker daemon. Tracks #920. Submitted-by: Claude * task: refresh the eval scoreboard on the merged tree Regenerated after merging development rather than resolving the generated RESULTS.md by hand. 117 probes, runner exit 0, no REGRESSION/TIMEOUT/ERROR rows. Tracks #920. Submitted-by: Claude * task: key skills-vendored on the hermes binary, not INSTALL_HERMES The probe drove link-providers.sh with INSTALL_HERMES=true and asserted the Hermes skill symlink appeared. #920 replaced that flag with a `command -v hermes` check, so on a CI runner with no hermes the link is correctly never created and the assertion failed. It now stubs hermes on a bare PATH inside the clean clone and asserts both directions: absent with no binary, present with one. Verified in an isolated clone that each direction flips when the gate is mutated to `true` / `false` — the probe clones committed state, so an in-place working-tree mutation cannot reach it. Tracks #920. Submitted-by: Claude * task: strip explanatory comments from .devcontainer Comments are a second, unverified description of behavior. Remove every explanatory comment from .devcontainer/ (Dockerfile, all compose files, entrypoint.sh, the two client scripts) and the stale .dockerignore block that still named the retired OH_IMAGE_ONLY flag. Kept only machine-read comment-shaped data: the shebangs, the '# >>> name >>>' fences that oh-home-mount.sh, oh-image-only-deploy.sh and entrypoint.test.ts extract functions from, and the header written into the generated sshd config. Re-anchor four assertions in three tests that sliced entrypoint.sh on comment prose so they slice on code instead; one of them (entrypoint.test.ts reconBranch) had been anchored on the pre-#920 'elif [ -d "$HARNESS_DIR" ]' line and was passing vacuously on a whole-block slice. --- .devcontainer/Dockerfile | 2 - .devcontainer/client-slack-supervise.sh | 3 - .devcontainer/docker-compose.docker-sock.yml | 12 - .../docker-compose.hermes-dashboard.yml | 16 - .devcontainer/docker-compose.image-only.yml | 33 -- .devcontainer/docker-compose.ssh.yml | 19 - .devcontainer/docker-compose.yml | 51 --- .devcontainer/entrypoint.sh | 153 +++----- .devcontainer/seed-msg-bridge.sh | 1 - .dockerignore | 4 - .github/workflows/sandbox-boot-guard.yml | 26 +- .gitignore | 3 + .oh/README.md | 8 +- .oh/cli/src/__tests__/harness-catalog.test.ts | 22 +- .oh/cli/src/__tests__/tool-catalog.test.ts | 76 ++-- .../src/lib/__tests__/config-render.test.ts | 81 ++-- .oh/cli/src/lib/config-render.ts | 32 +- .oh/cli/src/lib/tools/catalog.ts | 3 - .oh/evals/RESULTS.md | 233 +++++------ .../probes/compose-config-path-parity.sh | 8 +- .oh/evals/probes/compose-env-boundary.sh | 91 +++++ .../entrypoint-pnpm-manifest-fingerprint.sh | 13 +- .../probes/oh-devcontainer-restructure.sh | 5 +- .oh/evals/probes/oh-image-only-deploy.sh | 62 +-- .oh/evals/probes/sandbox-boot-guard-ci.sh | 6 +- .oh/evals/probes/skills-vendored.sh | 24 +- .oh/evals/probes/tailscale-tool-boundary.sh | 48 +-- .oh/evals/probes/tool-catalog-boundary.sh | 32 +- .oh/install/banner.sh | 12 +- .oh/scripts/__tests__/boot-banner.test.ts | 2 +- .oh/scripts/__tests__/compose-args.test.ts | 10 +- .oh/scripts/__tests__/cron-runtime.test.ts | 39 ++ .../__tests__/docs-compose-overlays.test.ts | 5 + .../__tests__/entrypoint-pnpm-install.test.ts | 10 +- .oh/scripts/__tests__/entrypoint.test.ts | 17 +- .../__tests__/sandbox-healthcheck.test.ts | 7 +- .oh/scripts/cron-runtime.ts | 35 +- .oh/scripts/docker-compose.sh | 5 - .oh/scripts/link-providers.sh | 4 +- .oh/scripts/sandbox-healthcheck.sh | 12 +- .oh/skills/wiki/corpus/README.md | 3 +- .../wiki/corpus/compose-env-boundary.md | 61 +++ .../corpus/sandbox-dependency-installs.md | 11 +- .../compose-env-boundary/eval-result.json | 7 + .oh/tasks/compose-env-boundary/evidence.md | 209 ++++++++++ .oh/tasks/compose-env-boundary/prd.json | 210 ++++++++++ .oh/tasks/compose-env-boundary/prd.md | 371 ++++++++++++++++++ .oh/tasks/compose-env-boundary/progress.txt | 112 ++++++ .oh/tasks/compose-env-boundary/prompt.md | 31 ++ .oh/templates/full/.hermes/README.md | 2 +- CHANGELOG.md | 4 + docs/configuration.md | 68 ++-- docs/connecting.md | 6 +- docs/deployment-prebuilt-image.md | 33 +- docs/harnesses/grok-build.md | 4 +- docs/harnesses/hermes.md | 73 ++-- docs/harnesses/opencode.md | 6 +- docs/harnesses/overview.md | 7 +- docs/installation.md | 6 +- docs/integrations/langfuse.md | 10 +- docs/integrations/sshd.md | 32 +- docs/lifecycle-commands.md | 1 - docs/rfcs/rfc-brain-hands-boundary.md | 2 +- docs/runtimes/microsandbox.md | 1 - docs/security-considerations.md | 6 +- 65 files changed, 1783 insertions(+), 718 deletions(-) delete mode 100644 .devcontainer/docker-compose.hermes-dashboard.yml create mode 100755 .oh/evals/probes/compose-env-boundary.sh create mode 100644 .oh/skills/wiki/corpus/compose-env-boundary.md create mode 100644 .oh/tasks/compose-env-boundary/eval-result.json create mode 100644 .oh/tasks/compose-env-boundary/evidence.md create mode 100644 .oh/tasks/compose-env-boundary/prd.json create mode 100644 .oh/tasks/compose-env-boundary/prd.md create mode 100644 .oh/tasks/compose-env-boundary/progress.txt create mode 100644 .oh/tasks/compose-env-boundary/prompt.md diff --git a/.devcontainer/Dockerfile b/.devcontainer/Dockerfile index 1f0f7162..964785b2 100644 --- a/.devcontainer/Dockerfile +++ b/.devcontainer/Dockerfile @@ -88,7 +88,6 @@ RUN printf '%s\n' \ 'export NPM_USER_PREFIX="/home/sandbox/.local"' \ 'export PNPM_HOME="/usr/local/share/pnpm"' \ 'export PATH="$NPM_USER_PREFIX/bin:$PNPM_HOME:$PATH"' \ - '# uv/Python env written by .oh/scripts/provision-python.sh (PRIME_AGENT_KERNEL_PYTHON).' \ '[ -r "$HOME/.local/share/oh/python-env.sh" ] && . "$HOME/.local/share/oh/python-env.sh"' \ | tee -a /home/sandbox/.profile /home/sandbox/.zprofile \ && chown sandbox:sandbox /home/sandbox/.profile /home/sandbox/.zprofile @@ -120,7 +119,6 @@ RUN mkdir -p /.devcontainer && echo '{"workspaceFolder":"/home/sandbox/harness", LABEL devcontainer.metadata='[{"remoteUser":"sandbox","workspaceFolder":"/home/sandbox/harness"}]' LABEL org.opencontainers.image.licenses="Apache-2.0" -# boot (entrypoint OH_IMAGE_ONLY / seed_workspace_volume, OH_IMAGE_SEED_SRC). COPY --chown=sandbox:sandbox . /opt/oh-seed/ COPY .devcontainer/entrypoint.sh /usr/local/bin/entrypoint.sh diff --git a/.devcontainer/client-slack-supervise.sh b/.devcontainer/client-slack-supervise.sh index 2ce66abd..eebd0aec 100755 --- a/.devcontainer/client-slack-supervise.sh +++ b/.devcontainer/client-slack-supervise.sh @@ -1,6 +1,4 @@ #!/usr/bin/env bash -# the pane pty), with NO `| tee` pipe and NO `--mode rpc`. On a TTY pi resolves to -# session no longer needs `--mode rpc` to avoid the idle exit. set -u BACKEND="${GATEWAY_BACKEND:-pi}" @@ -76,7 +74,6 @@ while true; do pkill -f 'pi-messenger-bridge/dist/index.js'; } ) /dev/null 2>&1 & WD=$! - # flood, stays alive at idle), stderr -> $LOG. No pipe, no --mode rpc. pi --extension "$BRIDGE_ENTRY" --extension "$RECOVERY_ENTRY" --approve 2>>"$LOG" rc=$? else diff --git a/.devcontainer/docker-compose.docker-sock.yml b/.devcontainer/docker-compose.docker-sock.yml index 07d7d983..171b0f23 100644 --- a/.devcontainer/docker-compose.docker-sock.yml +++ b/.devcontainer/docker-compose.docker-sock.yml @@ -1,15 +1,3 @@ -# Docker-socket overlay (opt-in — see docs/security-considerations.md). -# -# Applied by .oh/scripts/docker-compose.sh only when DOCKER_SOCKET is truthy -# (the DOCKER_SOCKET key in .devcontainer/.env). -# Mounting the host Docker socket gives the sandboxed agent effectively host -# root (it can start a privileged container that mounts the host filesystem), -# so enable this ONLY when the agent must drive Docker. entrypoint.sh guards on -# the socket's presence, so the sandbox boots fine whether or not this is applied. -# -# The VS Code "Reopen in Container" path reads docker-compose.yml directly and -# bypasses the wrapper, so it never applies this overlay; add this file to -# devcontainer.json's dockerComposeFile list to enable the socket there. services: sandbox: volumes: diff --git a/.devcontainer/docker-compose.hermes-dashboard.yml b/.devcontainer/docker-compose.hermes-dashboard.yml deleted file mode 100644 index 5e06c536..00000000 --- a/.devcontainer/docker-compose.hermes-dashboard.yml +++ /dev/null @@ -1,16 +0,0 @@ -# Hermes dashboard overlay. -# -# Included automatically by .oh/scripts/docker-compose.sh only when HERMES_DASHBOARD is -# truthy via `HERMES_DASHBOARD` in .devcontainer/.env. The host bind stays -# loopback-only; the in-container process binds all interfaces so Docker's port -# publishing can reach it. - -services: - sandbox: - ports: - - "127.0.0.1:${HERMES_DASHBOARD_PORT:-9119}:${HERMES_DASHBOARD_PORT:-9119}" - environment: - - HERMES_DASHBOARD=true - - HERMES_DASHBOARD_HOST=0.0.0.0 - - HERMES_DASHBOARD_PORT=${HERMES_DASHBOARD_PORT:-9119} - - HERMES_DASHBOARD_INSECURE=true diff --git a/.devcontainer/docker-compose.image-only.yml b/.devcontainer/docker-compose.image-only.yml index 6328e8e4..2404b4eb 100644 --- a/.devcontainer/docker-compose.image-only.yml +++ b/.devcontainer/docker-compose.image-only.yml @@ -1,26 +1,7 @@ -# Image-Only Sandbox Docker Compose Configuration (Flavor B, #609) -# =================================================================== -# Standalone recipe for running the PUBLISHED image with no repo checkout -# and no local image build — there is no parent-directory bind mount and -# no build stanza in this file. Everything the sandbox persists, workspace -# and control plane included, lives in the single /home/sandbox mount: the -# named volume ${SANDBOX_NAME}_workspace by default, or an absolute HOST -# path when OH_HOME_MOUNT is set. The workspace at /home/sandbox/harness is -# a directory inside that mount, seeded from the image's baked /opt/oh-seed -# on first boot by entrypoint.sh (see OH_IMAGE_ONLY handling); the rest of -# the home comes from /opt/home-seed via seed_home. Edits made inside the -# running sandbox persist across image pulls and container recreation. -# -# Usage: -# docker compose -f .devcontainer/docker-compose.image-only.yml up -d -# -# See docs/deployment-prebuilt-image.md for the full Flavor B writeup. - name: ${SANDBOX_NAME:-openharness} services: sandbox: - # Always runs the published image — never builds locally. image: ${OH_SANDBOX_IMAGE:-ghcr.io/mifunedev/openharness:latest} pull_policy: ${OH_PULL_POLICY:-always} container_name: ${SANDBOX_NAME:-openharness} @@ -33,20 +14,11 @@ services: - SANDBOX_PASSWORD=${SANDBOX_PASSWORD:-test1234} - TZ=${TZ:-America/Los_Angeles} - CLAUDE_DANGEROUSLY_SKIP_PERMISSIONS=true - # cc-safety-net destructive-command guard modes (install-decision.md): - # STRICT closes the fail-open hole on unparseable shell; WORKTREE unblocks - # bare reset/clean/checkout inside linked worktrees. No PARANOID/DEBUG; - # CC_SAFETY_NET_OFF left unset (kill-switch, opt-in per session). - CC_SAFETY_NET_STRICT=1 - CC_SAFETY_NET_WORKTREE=1 - GIT_USER_NAME=${GIT_USER_NAME:-} - GIT_USER_EMAIL=${GIT_USER_EMAIL:-} - GH_TOKEN=${GH_TOKEN:-} - - XAI_API_KEY=${XAI_API_KEY:-} - - OH_IMAGE_ONLY=1 - - CRON_AGENT_BIN=${CRON_AGENT_BIN:-claude} - - LANGFUSE_BASE_URL=${LANGFUSE_BASE_URL:-} - - LANGFUSE_PRIVACY_PRESET=${LANGFUSE_PRIVACY_PRESET:-metadata-only} stdin_open: true tty: true init: true @@ -57,11 +29,6 @@ services: interval: 30s timeout: 10s retries: 3 - # Boot installs the default harnesses AND the default tools (herdr, - # cloudflared) into the home mount instead of unpacking them from the - # image (entrypoint.sh, bounded by OH_PROVISION_DEFAULTS_TIMEOUT). A cold - # first boot on a fresh home mount therefore spends up to 240s downloading - # before the control plane is reachable; 600s leaves room for the rest. start_period: 600s restart: unless-stopped diff --git a/.devcontainer/docker-compose.ssh.yml b/.devcontainer/docker-compose.ssh.yml index 5bba9a83..251d32cc 100644 --- a/.devcontainer/docker-compose.ssh.yml +++ b/.devcontainer/docker-compose.ssh.yml @@ -1,23 +1,4 @@ -# sshd overlay — direct SSH into the sandbox container. -# -# Included automatically by .oh/scripts/docker-compose.sh only when SANDBOX_SSH is truthy -# via `SANDBOX_SSH` in .devcontainer/.env. The host bind -# stays loopback-only (127.0.0.1) so the daemon is reachable from the host and -# through a host-side reverse proxy, but not from the public interface by -# default — switch the bind to 0.0.0.0 for direct remote SSH, or front it with -# nginx for multi-tenant routing. See docs/integrations/sshd.md. -# -# Auth is public-key by default: put your public key(s) in -# SANDBOX_SSH_AUTHORIZED_KEYS (in .devcontainer/.env). Password auth (using -# SANDBOX_PASSWORD) stays off unless SANDBOX_SSH_PASSWORD_AUTH is truthy. -# The entrypoint starts sshd as a background daemon alongside `sleep infinity`. - services: sandbox: ports: - "127.0.0.1:${SANDBOX_SSH_PORT:-2222}:22" - environment: - - SANDBOX_SSH=true - - SANDBOX_SSH_PORT=${SANDBOX_SSH_PORT:-2222} - - SANDBOX_SSH_PASSWORD_AUTH=${SANDBOX_SSH_PASSWORD_AUTH:-false} - - SANDBOX_SSH_AUTHORIZED_KEYS=${SANDBOX_SSH_AUTHORIZED_KEYS:-} diff --git a/.devcontainer/docker-compose.yml b/.devcontainer/docker-compose.yml index 72ddf66a..5d7b6d36 100644 --- a/.devcontainer/docker-compose.yml +++ b/.devcontainer/docker-compose.yml @@ -1,38 +1,7 @@ -# Main Sandbox Docker Compose Configuration -# =========================================== -# The sandbox persists everything under the agent's home directory through a -# single mount at /home/sandbox. Leave OH_HOME_MOUNT unset and Docker manages -# it as the named volume ${SANDBOX_NAME}_workspace; set it (via -# storage.homePath in oh.json) to an absolute HOST path and the same mount -# becomes a bind. The image ships its home baked at /opt/home-seed, which -# entrypoint.sh's seed_home restores into whichever mount lands there, so both -# kinds behave identically. -# -# The repo checkout is bind-mounted at /home/sandbox/harness, nested inside -# that mount; the daemon orders mounts by destination depth, so the checkout -# always lands on top. Its location is fixed, not configurable. -# -# Downstream harness packs and Pi extensions can register additional -# overlays by appending paths to composeOverrides[] in oh.json; -# `oh sandbox` and `.oh/scripts/install.sh` pick those up via jq. -# -# The host Docker socket is NOT mounted here by default (socket access is -# effectively host root). It is an opt-in overlay — docker-compose.docker-sock.yml -# — applied by .oh/scripts/docker-compose.sh only when access.dockerSocket is -# true in oh.json. The VS Code "Reopen in Container" path reads THIS file -# directly and bypasses that wrapper, so it never gets the socket; add the -# overlay to devcontainer.json to enable it there. See -# docs/security-considerations.md. - name: ${SANDBOX_NAME:-openharness} services: sandbox: - # Default is a LOCAL build tagged sandbox-. Set image.ref - # in oh.json (e.g. ghcr.io/mifunedev/openharness:latest) - # to run the prebuilt image instead — the CLI path pairs this with - # `up -d --no-build` (`oh sandbox --image`); the VS Code "Reopen in - # Container" path relies on pull_policy since it can't pass --no-build. image: ${OH_SANDBOX_IMAGE:-sandbox-${SANDBOX_NAME:-openharness}} pull_policy: ${OH_PULL_POLICY:-missing} container_name: ${SANDBOX_NAME:-openharness} @@ -49,26 +18,11 @@ services: - SANDBOX_PASSWORD=${SANDBOX_PASSWORD:-test1234} - TZ=${TZ:-America/Los_Angeles} - CLAUDE_DANGEROUSLY_SKIP_PERMISSIONS=true - # cc-safety-net destructive-command guard modes (install-decision.md): - # STRICT closes the fail-open hole on unparseable shell; WORKTREE unblocks - # bare reset/clean/checkout inside linked worktrees. No PARANOID/DEBUG; - # CC_SAFETY_NET_OFF left unset (kill-switch, opt-in per session). - CC_SAFETY_NET_STRICT=1 - CC_SAFETY_NET_WORKTREE=1 - GIT_USER_NAME=${GIT_USER_NAME:-} - GIT_USER_EMAIL=${GIT_USER_EMAIL:-} - GH_TOKEN=${GH_TOKEN:-} - - XAI_API_KEY=${XAI_API_KEY:-} - - INSTALL_AGENT_BROWSER=${INSTALL_AGENT_BROWSER:-false} - - INSTALL_TAILSCALE=${INSTALL_TAILSCALE:-false} - - SKIP_PNPM_INSTALL=${SKIP_PNPM_INSTALL:-0} - - INSTALL_HERMES=${INSTALL_HERMES:-false} - - HERMES_HOME=/home/sandbox/harness/.hermes - - HERMES_DASHBOARD=${HERMES_DASHBOARD:-false} - - HERMES_DASHBOARD_PORT=${HERMES_DASHBOARD_PORT:-9119} - - CRON_AGENT_BIN=${CRON_AGENT_BIN:-claude} - - LANGFUSE_BASE_URL=${LANGFUSE_BASE_URL:-} - - LANGFUSE_PRIVACY_PRESET=${LANGFUSE_PRIVACY_PRESET:-metadata-only} stdin_open: true tty: true init: true @@ -79,11 +33,6 @@ services: interval: 30s timeout: 10s retries: 3 - # Boot installs the default harnesses AND the default tools (herdr, - # cloudflared) into the home mount instead of unpacking them from the - # image (entrypoint.sh, bounded by OH_PROVISION_DEFAULTS_TIMEOUT). A cold - # first boot on a fresh home mount therefore spends up to 240s downloading - # before the control plane is reachable; 600s leaves room for the rest. start_period: 600s restart: unless-stopped diff --git a/.devcontainer/entrypoint.sh b/.devcontainer/entrypoint.sh index 5b0346fa..b5d3e3aa 100644 --- a/.devcontainer/entrypoint.sh +++ b/.devcontainer/entrypoint.sh @@ -79,13 +79,32 @@ seed_workspace_volume() { } # <<< seed_workspace_volume <<< +# >>> oh_config >>> +OH_CONFIG_JSON="" +oh_config() { + local filter="$1" fallback="${2-}" out + command -v jq >/dev/null 2>&1 || { printf '%s' "$fallback"; return 0; } + if [ -z "$OH_CONFIG_JSON" ]; then + OH_CONFIG_JSON="$(cd "$HARNESS" 2>/dev/null && gosu sandbox "${OH_BIN:-oh}" config show 2>/dev/null)" || OH_CONFIG_JSON="" + [ -n "$OH_CONFIG_JSON" ] || OH_CONFIG_JSON="{}" + fi + out="$(printf '%s' "$OH_CONFIG_JSON" | jq -r "$filter" 2>/dev/null)" || out="" + if [ -z "$out" ] || [ "$out" = "null" ]; then printf '%s' "$fallback"; else printf '%s' "$out"; fi +} + +oh_config_truthy() { + case "$(printf '%s' "$(oh_config "$1" "${2:-false}")" | tr '[:upper:]' '[:lower:]')" in + 1|true|yes|on) return 0 ;; + *) return 1 ;; + esac +} +# <<< oh_config <<< + OH_PROJECT_ROOT="${OH_PROJECT_ROOT:-/home/sandbox/harness}" HARNESS="${HARNESS:-$OH_PROJECT_ROOT}" seed_home /home/sandbox || echo "[entrypoint] WARNING: home seed incomplete; some baked dotfiles may be missing" >&2 -# ─── Host UID reconciliation ──────────────────────────────────────── - uid_reconcile_step() { local description="$1" shift @@ -99,16 +118,8 @@ uid_reconcile_step() { } HARNESS_DIR="$OH_PROJECT_ROOT" -if [ "${OH_IMAGE_ONLY:-}" = "1" ]; then - echo "[entrypoint] OH_IMAGE_ONLY=1 — no-bind mode; skipping host UID/GID sync" - seed_workspace_volume "$OH_PROJECT_ROOT" - if [ "${OH_IMAGE_SEEDED_THIS_BOOT:-0}" = "1" ]; then - echo "[entrypoint] seeded control plane into $OH_PROJECT_ROOT from ${OH_IMAGE_SEED_SRC:-/opt/oh-seed}" - chown -R "$(id -u sandbox):$(id -g sandbox)" "$OH_PROJECT_ROOT" 2>/dev/null || true - else - chown "$(id -u sandbox):$(id -g sandbox)" "$OH_PROJECT_ROOT" 2>/dev/null || true - fi -elif [ -d "$HARNESS_DIR" ]; then +if mountpoint -q "$HARNESS_DIR" 2>/dev/null && [ -d "$HARNESS_DIR/.oh" ]; then + echo "[entrypoint] checkout bind detected at $HARNESS_DIR — syncing host UID/GID" HOST_UID=$(stat -c '%u' "$HARNESS_DIR") HOST_GID=$(stat -c '%g' "$HARNESS_DIR") SANDBOX_UID=$(id -u sandbox) @@ -132,13 +143,21 @@ elif [ -d "$HARNESS_DIR" ]; then echo "[entrypoint] WARNING: sandbox UID/GID reconciliation incomplete; continuing with current ownership" >&2 fi fi +else + echo "[entrypoint] no checkout bind at $HARNESS_DIR — seeding from ${OH_IMAGE_SEED_SRC:-/opt/oh-seed}" + seed_workspace_volume "$OH_PROJECT_ROOT" + if [ "${OH_IMAGE_SEEDED_THIS_BOOT:-0}" = "1" ]; then + echo "[entrypoint] seeded control plane into $OH_PROJECT_ROOT from ${OH_IMAGE_SEED_SRC:-/opt/oh-seed}" + chown -R "$(id -u sandbox):$(id -g sandbox)" "$OH_PROJECT_ROOT" 2>/dev/null || true + else + chown "$(id -u sandbox):$(id -g sandbox)" "$OH_PROJECT_ROOT" 2>/dev/null || true + fi fi PW="${SANDBOX_PASSWORD:-test1234}" echo "sandbox:${PW}" | chpasswd || echo "[entrypoint] WARNING: failed to set sandbox password" >&2 unset PW -# UID/GID reconciliation can change the numeric identity behind the sandbox repair_home_mount_ownership HARNESS="${HARNESS:-$OH_PROJECT_ROOT}" @@ -165,9 +184,8 @@ if [ "${OH_PROVISION_PYTHON:-true}" = "true" ] \ fi fi -# Hermes keeps all runtime state — including auth.json — inside the -if [ "${INSTALL_HERMES:-false}" = "true" ]; then - HERMES_RUNTIME="${HERMES_HOME:-$HARNESS/.hermes}" +if command -v hermes >/dev/null 2>&1; then + HERMES_RUNTIME="$HARNESS/.hermes" HERMES_LEGACY_AUTH="/home/sandbox/.hermes/auth.json" mkdir -p "$HERMES_RUNTIME" @@ -202,23 +220,17 @@ if [ "${INSTALL_HERMES:-false}" = "true" ]; then [ -d "$d" ] && chown -hR "$(sandbox_ownership)" "$d" 2>/dev/null || true done - if [ "${HERMES_DASHBOARD:-false}" = "true" ] && command -v hermes &>/dev/null \ - && command -v tmux &>/dev/null; then - _dash_port="${HERMES_DASHBOARD_PORT:-9119}" + if oh_config_truthy '.hermesDashboard.enabled' && command -v tmux &>/dev/null; then + _dash_port="$(oh_config '.hermesDashboard.port' 9119)" case "$_dash_port" in *[!0-9]|"") - echo "[entrypoint] HERMES_DASHBOARD_PORT='${_dash_port}' is not numeric — skipping dashboard launch" + echo "[entrypoint] hermesDashboard.port='${_dash_port}' is not numeric — skipping dashboard launch" ;; *) if ! gosu sandbox tmux has-session -t app-hermes-dashboard 2>/dev/null; then - _dash_host="${HERMES_DASHBOARD_HOST:-127.0.0.1}" - _dash_insecure="" - case "${HERMES_DASHBOARD_INSECURE:-}" in - [Tt][Rr][Uu][Ee]|1|[Yy][Ee][Ss]|[Oo][Nn]) _dash_insecure=" --insecure" ;; - esac gosu sandbox tmux new-session -d -s app-hermes-dashboard \ - "hermes dashboard --no-open --host \"${_dash_host}\" --port \"${_dash_port}\"${_dash_insecure} 2>&1 | tee /tmp/app-hermes-dashboard.log" - echo "[entrypoint] starting Hermes dashboard on ${_dash_host}:${_dash_port}" + "hermes dashboard --no-open --host 127.0.0.1 --port \"${_dash_port}\" 2>&1 | tee /tmp/app-hermes-dashboard.log" + echo "[entrypoint] starting Hermes dashboard on 127.0.0.1:${_dash_port}" else echo "[entrypoint] app-hermes-dashboard tmux session already running — skipping" fi @@ -227,7 +239,7 @@ if [ "${INSTALL_HERMES:-false}" = "true" ]; then fi fi -if [ "${SANDBOX_SSH:-false}" = "true" ] && [ -x /usr/sbin/sshd ]; then +if oh_config_truthy '.access.ssh' && [ -x /usr/sbin/sshd ]; then if pgrep -x sshd >/dev/null 2>&1; then echo "[entrypoint] sshd already running — skipping" else @@ -236,9 +248,10 @@ if [ "${SANDBOX_SSH:-false}" = "true" ] && [ -x /usr/sbin/sshd ]; then _ssh_dir=/home/sandbox/.ssh _have_keys=0 - if [ -n "${SANDBOX_SSH_AUTHORIZED_KEYS:-}" ]; then + _ssh_authorized_keys="$(oh_config '.access.sshAuthorizedKeys' '')" + if [ -n "$_ssh_authorized_keys" ]; then mkdir -p "$_ssh_dir" - _ssh_keys="${SANDBOX_SSH_AUTHORIZED_KEYS//\\n/$'\n'}" + _ssh_keys="${_ssh_authorized_keys//\\n/$'\n'}" printf '%s\n' "$_ssh_keys" > "$_ssh_dir/authorized_keys" unset _ssh_keys chmod 700 "$_ssh_dir" @@ -250,9 +263,9 @@ if [ "${SANDBOX_SSH:-false}" = "true" ] && [ -x /usr/sbin/sshd ]; then fi _pw_auth=no - case "$(printf '%s' "${SANDBOX_SSH_PASSWORD_AUTH:-false}" | tr '[:upper:]' '[:lower:]')" in - 1|true|yes|on) _pw_auth=yes ;; - esac + if oh_config_truthy '.access.sshPasswordAuth'; then + _pw_auth=yes + fi mkdir -p /etc/ssh/sshd_config.d cat > /etc/ssh/sshd_config.d/openharness.conf <&2 - echo "[entrypoint] no one can log in. Set SANDBOX_SSH_AUTHORIZED_KEYS in .devcontainer/.env" >&2 - echo "[entrypoint] or SANDBOX_SSH_PASSWORD_AUTH=true in .devcontainer/.env. See docs/integrations/sshd.md" >&2 + echo "[entrypoint] no one can log in. Run: oh config set access.sshAuthorizedKeys ''" >&2 + echo "[entrypoint] or: oh config set access.sshPasswordAuth true. See docs/integrations/sshd.md" >&2 fi if /usr/sbin/sshd; then @@ -345,7 +358,6 @@ if [ -n "${GH_TOKEN:-}" ] && gosu sandbox env -u GH_TOKEN -u GITHUB_TOKEN gh aut fi fi -# from the marker stored alongside node_modules. Set SKIP_PNPM_INSTALL=1 to opt pnpm_workspace_package_patterns() { local workspace="$1/pnpm-workspace.yaml" [ -f "$workspace" ] || return 0 @@ -448,7 +460,7 @@ pnpm_manifest_fingerprint() { done | sha256sum | awk '{print $1}' } -if [ -f "$HARNESS/package.json" ] && [ "${SKIP_PNPM_INSTALL:-0}" != "1" ]; then +if [ -f "$HARNESS/package.json" ] && ! oh_config_truthy '.build.skipPnpmInstall'; then PNPM_INSTALL_MARKER_FILENAME=".openharness-root-pnpm-manifest.sha256" PNPM_INSTALL_MARKER="$HARNESS/node_modules/$PNPM_INSTALL_MARKER_FILENAME" PNPM_MANIFEST_FINGERPRINT="$(pnpm_manifest_fingerprint "$HARNESS")" @@ -541,79 +553,12 @@ else echo "[entrypoint] Slack not configured (or pi missing) — skipping client-slack-pi" fi -if [ "${INSTALL_AGENT_BROWSER:-false}" = "true" ] && ! command -v agent-browser &>/dev/null; then - echo "[entrypoint] Installing agent-browser (INSTALL_AGENT_BROWSER=true)..." - pnpm add -g agent-browser@0.8.5 \ - && find "$PNPM_HOME" -name "agent-browser-linux-*" -exec chmod +x {} \; \ - && agent-browser install --with-deps 2>&1 | tail -5 \ - && echo "[entrypoint] agent-browser installed" \ - || echo "[entrypoint] agent-browser install failed — skipping" -fi - -# tailscaled defaults its control socket to /var/run/tailscale/tailscaled.sock, -# and t3-code.sh calls a bare `tailscale status` that expects exactly that path. -# Only root can create it, so the entrypoint must — unconditionally, not behind -# the guard below: `oh tool install tailscale` promises the tool is usable in the -# already-running container, and gating this on INSTALL_TAILSCALE would make an -# install-now/use-now flow wait for a reboot. An empty directory costs nothing. install -d -o sandbox -g sandbox -m 0755 /var/run/tailscale 2>/dev/null || true -if [ "${INSTALL_TAILSCALE:-false}" = "true" ]; then - install -d -o sandbox -g sandbox -m 0700 /home/sandbox/.tailscale 2>/dev/null || true - - if ! gosu sandbox bash -lc 'command -v tailscale' >/dev/null 2>&1; then - case "$(dpkg --print-architecture)" in - amd64) - ts_tarball=tailscale_1.102.3_amd64.tgz - ts_sha=36ddd9b51be57ffc2990cf76323cfa13643bfbb1b8a969f6183fa164741cdef5 - ;; - arm64) - ts_tarball=tailscale_1.102.3_arm64.tgz - ts_sha=a0fa1b154af8c61f862a2259f559f7396d96c0225f4a863eae2333e1546bbe25 - ;; - *) - ts_tarball="" - ts_sha="" - ;; - esac - - if [ -z "$ts_tarball" ]; then - echo "[entrypoint] WARNING: no pinned Tailscale build for $(dpkg --print-architecture) — skipping" >&2 - else - echo "[entrypoint] Installing ${ts_tarball%.tgz} (INSTALL_TAILSCALE=true)..." - # Install into the home mount as the sandbox user, matching the tool - # catalog. /usr/local/bin is an image-layer path: it is lost on every - # container recreate, so the old location re-downloaded Tailscale on every - # fresh container, and left a root-owned binary no running sandbox could - # upgrade in place. - ts_tmp="$(mktemp -d)" - chown sandbox:sandbox "$ts_tmp" - if gosu sandbox bash -lc " - set -e - prefix=\"\${NPM_USER_PREFIX:-\$HOME/.local}\" - curl -fsSL 'https://pkgs.tailscale.com/stable/${ts_tarball}' -o '$ts_tmp/$ts_tarball' - echo '${ts_sha} $ts_tmp/$ts_tarball' | sha256sum -c - - tar -xzf '$ts_tmp/$ts_tarball' -C '$ts_tmp' - install -d \"\$prefix/bin\" - install -m 0755 '$ts_tmp/${ts_tarball%.tgz}/tailscale' \"\$prefix/bin/tailscale\" - install -m 0755 '$ts_tmp/${ts_tarball%.tgz}/tailscaled' \"\$prefix/bin/tailscaled\" - "; then - echo "[entrypoint] ${ts_tarball%.tgz} installed into the home mount" - else - echo "[entrypoint] WARNING: Tailscale install failed — skipping" >&2 - fi - rm -rf "$ts_tmp" - unset ts_tmp - fi - unset ts_tarball ts_sha - fi -fi - for hook in /usr/local/bin/*-entrypoint-hook.sh; do [ -x "$hook" ] && "$hook" done -# First-boot message if onboarding not complete if [ ! -f "/home/sandbox/.claude/.onboarded" ]; then echo "" echo " ┌─────────────────────────────────────────────────┐" diff --git a/.devcontainer/seed-msg-bridge.sh b/.devcontainer/seed-msg-bridge.sh index e6c86600..a3e7d426 100755 --- a/.devcontainer/seed-msg-bridge.sh +++ b/.devcontainer/seed-msg-bridge.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# (autoConnect, showWidget, …) but PRESERVE the operator's runtime set -u seed="${1:-}" diff --git a/.dockerignore b/.dockerignore index ff1bdfca..e3df1c5f 100644 --- a/.dockerignore +++ b/.dockerignore @@ -1,8 +1,4 @@ **/.env* -# Exclude .claude/ CONTENTS (local plans, specs, screenshots, auth state) but -# re-include the tracked control-plane config the OH_IMAGE_ONLY seed needs. -# Without these in /opt/oh-seed, a no-bind volume boots without -# .claude/protected-paths.txt and link-providers.sh --init crash-loops. .claude/* !.claude/protected-paths.txt !.claude/settings.json diff --git a/.github/workflows/sandbox-boot-guard.yml b/.github/workflows/sandbox-boot-guard.yml index 1591267e..bb6ac3b0 100644 --- a/.github/workflows/sandbox-boot-guard.yml +++ b/.github/workflows/sandbox-boot-guard.yml @@ -67,9 +67,10 @@ jobs: - name: Validate base compose configuration run: bash .oh/scripts/docker-compose.sh config --quiet - - name: Validate Hermes dashboard compose overlay + - name: Validate sshd compose overlay env: - HERMES_DASHBOARD: "true" + SANDBOX_SSH: "true" + SANDBOX_SSH_PORT_CHECK: "off" run: bash .oh/scripts/docker-compose.sh config --quiet - name: Build sandbox image locally @@ -95,8 +96,8 @@ jobs: # overrun the healthcheck's ~690s unhealthy deadline and fail the boot with # "missing required tmux session: cron-system". Seeding node_modules here # moves that slow/variable install OUT of the healthcheck-timed window; the - # sandbox smoke step below also sets SKIP_PNPM_INSTALL=1 so the pre-seeded - # CI path remains an explicit boot-window escape hatch even before a marker + # step below also sets build.skipPnpmInstall in oh.json so the pre-seeded CI + # path remains an explicit boot-window escape hatch even before a marker # exists. The runner workspace is a named volume shared at the same path # with the dind daemon, so the node_modules written here is exactly what the # container's bind mount sees. The container's UID is synced to the @@ -132,6 +133,17 @@ jobs: sh -c 'ls -la /mnt/node_modules/croner/package.json' echo "OK: pre-seeded node_modules is visible to the daemon that boots the sandbox" + # Deps are pre-seeded above and proven visible; build.skipPnpmInstall keeps + # the in-container install out of the healthcheck-timed window. This is the + # Option C contract: pnpm slowness can no longer trip the ~690s unhealthy + # deadline. entrypoint.sh reads this through `oh config show`, so it is set + # in oh.json rather than passed as an environment knob. + - name: Skip the in-container pnpm install + run: | + tmp=$(mktemp) + jq '.build.skipPnpmInstall = true' oh.json > "$tmp" + mv "$tmp" oh.json + # This step is the ONLY place the boot-time harness install runs end to end. # It boots on a fresh home volume (the smoke tears down with `down -v`), so # provision-defaults.sh does real npm work every run and the smoke asserts @@ -146,10 +158,4 @@ jobs: # 3x30s retry window (~690s), not just the old 300s one. BOOT_SMOKE_TIMEOUT_SECONDS: "1200" BOOT_SMOKE_INTERVAL_SECONDS: "10" - # Deps are pre-seeded above and proven visible; SKIP_PNPM_INSTALL=1 - # keeps the in-container install out of the healthcheck-timed window. - # This is the Option C contract: pnpm slowness can no longer trip the - # ~690s unhealthy deadline. What remains inside that window is the - # cron-system spawn and the boot-time harness install. - SKIP_PNPM_INSTALL: "1" run: bash .oh/scripts/sandbox-boot-smoke.sh diff --git a/.gitignore b/.gitignore index 2a993679..d8b73969 100644 --- a/.gitignore +++ b/.gitignore @@ -85,5 +85,8 @@ packages/*/node_modules/ # Per-session agent memory and cron runtime state — local scratch, like the # wiki corpus above. Nothing here is tracked on any branch; the operator # whitelists an entry with `git add -f` if one is ever worth keeping. +# Written by entrypoint.sh's seed_workspace_volume when there is no checkout bind. +# It must never land in a real checkout, so it is ignored on every branch. +.oh/.image-seeded .oh/memory/ .oh/crons/ diff --git a/.oh/README.md b/.oh/README.md index 510707fd..02121556 100644 --- a/.oh/README.md +++ b/.oh/README.md @@ -154,8 +154,8 @@ source instead of the bundled `.oh/templates/`. - `.devcontainer/` — the **full devcontainer**, pinned to root by the devcontainer spec / `.dockerignore` / hadolint (which don't honor a symlinked directory). It holds the VS Code `devcontainer.json`, the user-owned `.env`, and every build - asset: `Dockerfile`, `docker-compose.yml` + the hermes-dashboard overlay, - `entrypoint.sh`, and the two client scripts (`client-slack-supervise.sh` / + asset: `Dockerfile`, `docker-compose.yml` + the docker-socket and sshd + overlays, `entrypoint.sh`, and the two client scripts (`client-slack-supervise.sh` / `seed-msg-bridge.sh`). Everything the sandbox boots from lives here, in the one conventional location — no split, no compat shim. - `oh.json` and `.env.example` — the two authored configuration surfaces, and @@ -185,8 +185,8 @@ The harness's own devcontainer lives in the one conventional location — top-le **`.devcontainer/`** — rather than split across `.oh/`. It holds: - the build/bootstrap assets: `Dockerfile`, `docker-compose.yml` + the - `docker-compose.hermes-dashboard.yml` overlay, `entrypoint.sh`, - `client-slack-supervise.sh`, `seed-msg-bridge.sh`; + `docker-compose.docker-sock.yml` and `docker-compose.ssh.yml` overlays, + `entrypoint.sh`, `client-slack-supervise.sh`, `seed-msg-bridge.sh`; - the VS Code `devcontainer.json` (hand-maintained; its `dockerComposeFile` points at the same-dir `docker-compose.yml`) plus the user-owned `.env`. diff --git a/.oh/cli/src/__tests__/harness-catalog.test.ts b/.oh/cli/src/__tests__/harness-catalog.test.ts index b35a49ce..238e695c 100644 --- a/.oh/cli/src/__tests__/harness-catalog.test.ts +++ b/.oh/cli/src/__tests__/harness-catalog.test.ts @@ -49,9 +49,9 @@ describe("harness catalog", () => { } }); - it("excludes agent_browser — it shares the INSTALL_* namespace but is not a harness", () => { + it("excludes agent_browser — it shares the install.* namespace but is not a harness", () => { expect(HARNESS_CATALOG.some((h) => h.harnessKey === "agent_browser")).toBe(false); - expect(CONFIG_DOC).toMatch(/^\| `install\.agentBrowser` \|.*`INSTALL_AGENT_BROWSER`/m); + expect(CONFIG_DOC).toMatch(/^\| `install\.agentBrowser` \|/m); }); it("documents every harness under docs/harnesses/.md", () => { @@ -96,11 +96,13 @@ describe("harness catalog", () => { ); it.each(optional.map((h) => [h.id, h] as const))( - "%s: its oh.json key stays documented in docs/configuration.md", + "%s: its oh.json key stays documented in docs/configuration.md, with no compose projection", (_id, h) => { - const arg = `INSTALL_${(h.harnessKey as string).toUpperCase()}`; + const field = (h.harnessKey as string).replace(/_(.)/g, (_m, c: string) => + c.toUpperCase(), + ); expect(CONFIG_DOC).toMatch( - new RegExp(`^\\| \`install\\.[A-Za-z]+\` \\|.*\`${arg}\``, "m"), + new RegExp(`^\\| \`install\\.${field}\` \\|[^|]*\\|[^|]*\\| — \\|`, "m"), ); }, ); @@ -111,12 +113,12 @@ describe("harness catalog", () => { expect(DOCKERFILE).not.toContain("bash -s 0.2.39"); }); - // INSTALL_HERMES survives as a RUNTIME flag: link-providers.sh vendors the - // Hermes skill pack from it and entrypoint.sh wires auth.json. Only its - // build-arg role is gone. - it("keeps INSTALL_HERMES as a container environment variable", () => { - expect(COMPOSE_YML).toContain("- INSTALL_HERMES=${INSTALL_HERMES:-false}"); + it("gates the Hermes wiring on the binary, never on INSTALL_HERMES", () => { + expect(COMPOSE_YML).not.toContain("INSTALL_HERMES"); expect(DOCKERFILE).not.toContain("INSTALL_HERMES"); + expect(ENTRYPOINT).not.toContain("INSTALL_HERMES"); + expect(ENTRYPOINT).toContain("if command -v hermes >/dev/null 2>&1; then"); + expect(read(".oh/scripts/link-providers.sh")).not.toContain("INSTALL_HERMES"); }); it("installs every harness as the sandbox user, never root", () => { diff --git a/.oh/cli/src/__tests__/tool-catalog.test.ts b/.oh/cli/src/__tests__/tool-catalog.test.ts index 66d0adbd..ec878e52 100644 --- a/.oh/cli/src/__tests__/tool-catalog.test.ts +++ b/.oh/cli/src/__tests__/tool-catalog.test.ts @@ -130,27 +130,27 @@ describe("the three catalogs are disjoint", () => { }); }); -describe("agent-browser matches the entrypoint that really installs it", () => { +describe("agent-browser is installed from the catalog, not the boot path", () => { const ab = findTool("agent-browser")!; const ENTRYPOINT = read(".devcontainer/entrypoint.sh"); - it("carries the entrypoint guard, not a build arg", () => { - expect(ab.entrypointGuard).toBe("INSTALL_AGENT_BROWSER"); + it("declares the oh.json opt-in and neither a build arg nor an entrypoint guard", () => { expect(Object.keys(ab)).not.toContain("buildArg"); + expect(Object.keys(ab)).not.toContain("entrypointGuard"); expect(ab.toolKey).toBe("agent_browser"); }); - it("is installed by the entrypoint and is ABSENT from the Dockerfile", () => { - expect(ENTRYPOINT).toContain("INSTALL_AGENT_BROWSER"); + it("is absent from the boot path and the Dockerfile", () => { + expect(ENTRYPOINT).not.toContain("INSTALL_AGENT_BROWSER"); + expect(ENTRYPOINT).not.toContain("agent-browser@"); expect(read(".devcontainer/Dockerfile")).not.toContain("INSTALL_AGENT_BROWSER"); }); - it("pins the same version the entrypoint pins", () => { + it("is the sole owner of the pinned version", () => { expect(ab.installArgv!.join(" ")).toContain("agent-browser@0.8.5"); - expect(ENTRYPOINT).toContain("agent-browser@0.8.5"); }); - it("reproduces each of the entrypoint's three install steps", () => { + it("carries every install step itself", () => { const argv = ab.installArgv!.join(" "); for (const step of [ "pnpm add -g agent-browser@0.8.5", @@ -158,11 +158,10 @@ describe("agent-browser matches the entrypoint that really installs it", () => { "agent-browser install --with-deps", ]) { expect(argv, step).toContain(step); - expect(ENTRYPOINT, step).toContain(step); } }); - it("drops the entrypoint's log cosmetics, which would eat the exit code", () => { + it("drops log cosmetics, which would eat the exit code", () => { const argv = ab.installArgv!.join(" "); expect(argv).not.toContain("tail -5"); expect(argv).not.toContain("[entrypoint]"); @@ -173,59 +172,47 @@ describe("agent-browser matches the entrypoint that really installs it", () => { expect(read(".oh/cli/src/commands/init.ts")).toContain("~1 GB"); }); - it("keeps the env plumbing wired end to end", () => { - expect(read(".devcontainer/docker-compose.yml")).toContain("INSTALL_AGENT_BROWSER"); - expect(read("docs/configuration.md")).toMatch( - /^\| `install\.agentBrowser` \|.*`INSTALL_AGENT_BROWSER`/m, - ); + it("is reachable only through oh.json — never through compose", () => { + expect(read(".devcontainer/docker-compose.yml")).not.toContain("INSTALL_AGENT_BROWSER"); + expect(read(".oh/cli/src/lib/config-render.ts")).toContain('"INSTALL_AGENT_BROWSER"'); + expect(read("docs/configuration.md")).toMatch(/^\| `install\.agentBrowser` \|/m); }); }); -describe("tailscale matches the entrypoint that really installs it", () => { +describe("tailscale is installed from the catalog, not the boot path", () => { const ts = findTool("tailscale")!; const ENTRYPOINT = read(".devcontainer/entrypoint.sh"); const VERSION = "1.102.3"; const SHA_AMD64 = "36ddd9b51be57ffc2990cf76323cfa13643bfbb1b8a969f6183fa164741cdef5"; const SHA_ARM64 = "a0fa1b154af8c61f862a2259f559f7396d96c0225f4a863eae2333e1546bbe25"; - it("carries the entrypoint guard, not a build arg", () => { - expect(ts.entrypointGuard).toBe("INSTALL_TAILSCALE"); + it("declares the oh.json opt-in and neither a build arg nor an entrypoint guard", () => { expect(Object.keys(ts)).not.toContain("buildArg"); + expect(Object.keys(ts)).not.toContain("entrypointGuard"); expect(ts.toolKey).toBe("tailscale"); expect(ts.kind).toBe("opt-in"); }); - it("is installed by the entrypoint and is ABSENT from the Dockerfile", () => { - expect(ENTRYPOINT).toContain("INSTALL_TAILSCALE"); + it("is absent from the boot path and the Dockerfile", () => { + expect(ENTRYPOINT).not.toContain("INSTALL_TAILSCALE"); + expect(ENTRYPOINT).not.toContain(`tailscale_${VERSION}_`); expect(read(".devcontainer/Dockerfile")).not.toContain("INSTALL_TAILSCALE"); }); - it("pins the same version the entrypoint pins", () => { - expect(ts.installArgv!.join(" ")).toContain(`tailscale_${VERSION}_`); - expect(ENTRYPOINT).toContain(`tailscale_${VERSION}_`); - }); - - it("verifies the same per-arch sha256 the entrypoint verifies", () => { + it("is the sole owner of the pinned version and both checksums", () => { const argv = ts.installArgv!.join(" "); + expect(argv).toContain(`tailscale_${VERSION}_`); for (const sha of [SHA_AMD64, SHA_ARM64]) { expect(argv, sha).toContain(sha); - expect(ENTRYPOINT, sha).toContain(sha); + expect(ENTRYPOINT, sha).not.toContain(sha); } expect(argv).toContain("sha256sum -c -"); - expect(ENTRYPOINT).toContain("sha256sum -c -"); }); - it("downloads from the pinned stable base the entrypoint uses", () => { - const base = "https://pkgs.tailscale.com/stable/"; - expect(ts.installArgv!.join(" ")).toContain(base); - expect(ENTRYPOINT).toContain(base); + it("downloads from the pinned stable base", () => { + expect(ts.installArgv!.join(" ")).toContain("https://pkgs.tailscale.com/stable/"); }); - // tailscaled runs unprivileged under --tun=userspace-networking, so nothing - // here needs root. A root install would hang `oh tool install tailscale` on a - // sudo password prompt (commands/tool.ts uses stdio:"inherit", and - // /etc/sudoers.d/sandbox has no NOPASSWD), and would put the binaries in an - // image-layer path discarded on every container recreate. it("installs as the sandbox user into the home mount", () => { expect(ts.installUser).toBe("sandbox"); const argv = ts.installArgv!.join(" "); @@ -234,9 +221,6 @@ describe("tailscale matches the entrypoint that really installs it", () => { expect(argv).not.toContain("/usr/local/bin/tailscaled"); }); - // /var/run/tailscale is tailscaled's default socket directory and only root - // can create it, so it belongs to the entrypoint, not to an install that runs - // as the sandbox user. it("leaves the root-owned socket directory to the entrypoint", () => { expect(ts.installArgv!.join(" ")).not.toContain("/var/run/tailscale"); expect(ENTRYPOINT).toContain("/var/run/tailscale"); @@ -248,7 +232,7 @@ describe("tailscale matches the entrypoint that really installs it", () => { expect(argv).not.toMatch(/(^|[^d])tailscaled\s+--tun/); }); - it("drops the entrypoint's log cosmetics, which would eat the exit code", () => { + it("drops log cosmetics, which would eat the exit code", () => { const argv = ts.installArgv!.join(" "); expect(argv).not.toContain("[entrypoint]"); expect(argv).not.toContain("tail -"); @@ -258,11 +242,10 @@ describe("tailscale matches the entrypoint that really installs it", () => { expect(ts.downloadSize).toBeUndefined(); }); - it("keeps the env plumbing wired end to end", () => { - expect(read(".devcontainer/docker-compose.yml")).toContain("INSTALL_TAILSCALE"); - expect(read("docs/configuration.md")).toMatch( - /^\| `install\.tailscale` \|.*`INSTALL_TAILSCALE`/m, - ); + it("is reachable only through oh.json — never through compose", () => { + expect(read(".devcontainer/docker-compose.yml")).not.toContain("INSTALL_TAILSCALE"); + expect(read(".oh/cli/src/lib/config-render.ts")).toContain('"INSTALL_TAILSCALE"'); + expect(read("docs/configuration.md")).toMatch(/^\| `install\.tailscale` \|/m); }); }); @@ -271,7 +254,6 @@ describe("baked-in tools", () => { for (const t of TOOL_CATALOG) { if (t.kind !== "baked-in") continue; expect(t.toolKey, t.id).toBeUndefined(); - expect(t.entrypointGuard, t.id).toBeUndefined(); expect(t.installArgv, t.id).toBeUndefined(); } }); diff --git a/.oh/cli/src/lib/__tests__/config-render.test.ts b/.oh/cli/src/lib/__tests__/config-render.test.ts index 7bd4b70e..78019e07 100644 --- a/.oh/cli/src/lib/__tests__/config-render.test.ts +++ b/.oh/cli/src/lib/__tests__/config-render.test.ts @@ -9,7 +9,39 @@ import { defaultOhConfig, type OhConfig } from "../oh-config.js"; const REPO_ROOT = join(dirname(fileURLToPath(import.meta.url)), "..", "..", "..", "..", ".."); const DEVCONTAINER = join(REPO_ROOT, ".devcontainer"); -const RETIRED = ["WORKTREES_DIR", "PROJECTS_DIR", "CRONS_DIR", "OH_PROJECT_ROOT"]; +const RETIRED = [ + "WORKTREES_DIR", + "PROJECTS_DIR", + "CRONS_DIR", + "OH_PROJECT_ROOT", + "INSTALL_DEEPAGENTS", + "INSTALL_OPENCODE", + "INSTALL_GROK_BUILD", + "INSTALL_HERMES", + "INSTALL_AGENT_BROWSER", + "INSTALL_TAILSCALE", + "SANDBOX_SSH_PASSWORD_AUTH", + "SANDBOX_SSH_AUTHORIZED_KEYS", + "HERMES_DASHBOARD", + "HERMES_DASHBOARD_PORT", + "CRON_AGENT_BIN", + "SKIP_PNPM_INSTALL", + "LANGFUSE_BASE_URL", + "LANGFUSE_PRIVACY_PRESET", +]; + +const HOST_SIDE_KEYS = [ + "SANDBOX_NAME", + "TZ", + "OH_HOME_MOUNT", + "GIT_USER_NAME", + "GIT_USER_EMAIL", + "DOCKER_SOCKET", + "SANDBOX_SSH", + "SANDBOX_SSH_PORT", + "OH_SANDBOX_IMAGE", + "OH_PULL_POLICY", +]; function composeInterpolatedVars(): string[] { const found = new Set(); @@ -48,33 +80,22 @@ describe("renderComposeEnv", () => { } }); - it("carries every non-secret setting through from oh.json", () => { + it("carries every host-side setting through from oh.json", () => { const text = renderComposeEnv(fullConfig()); expect(text).toContain("SANDBOX_NAME=demo"); expect(text).toContain("TZ=America/Los_Angeles"); expect(text).toContain("OH_HOME_MOUNT=/srv/oh-home"); expect(text).toContain("GIT_USER_NAME=Ada"); expect(text).toContain("GIT_USER_EMAIL=ada@example.com"); - expect(text).toContain("INSTALL_OPENCODE=false"); - expect(text).toContain("INSTALL_GROK_BUILD=false"); - // #910: deepagents is retired; the key must no longer be rendered. - expect(text).not.toContain("INSTALL_DEEPAGENTS"); - expect(text).toContain("INSTALL_HERMES=false"); - expect(text).toContain("INSTALL_AGENT_BROWSER=false"); - expect(text).toContain("INSTALL_TAILSCALE=false"); expect(text).toContain("DOCKER_SOCKET=true"); expect(text).toContain("SANDBOX_SSH=true"); expect(text).toContain("SANDBOX_SSH_PORT=2022"); - expect(text).toContain("SANDBOX_SSH_PASSWORD_AUTH=true"); - expect(text).toContain("SANDBOX_SSH_AUTHORIZED_KEYS=ssh-ed25519 AAAA you@laptop"); - expect(text).toContain("HERMES_DASHBOARD=false"); - expect(text).toContain("HERMES_DASHBOARD_PORT=9119"); - expect(text).toContain("CRON_AGENT_BIN=claude"); - expect(text).toContain("SKIP_PNPM_INSTALL=0"); expect(text).toContain("OH_SANDBOX_IMAGE=ghcr.io/mifunedev/openharness:latest"); expect(text).toContain("OH_PULL_POLICY=always"); - expect(text).toContain("LANGFUSE_BASE_URL=http://langfuse-web:3000"); - expect(text).toContain("LANGFUSE_PRIVACY_PRESET=metadata-only"); + }); + + it("renders the host-side set and nothing else", () => { + expect(keysOf(fullConfig()).sort()).toEqual([...HOST_SIDE_KEYS].sort()); }); it("covers every variable the real compose files interpolate", () => { @@ -94,9 +115,18 @@ describe("renderComposeEnv", () => { for (const key of SECRET_KEYS) expect(rendered).not.toContain(key); }); - it("emits no retired *_DIR variable", () => { + it("emits no retired variable", () => { const text = renderComposeEnv(fullConfig()); - for (const key of RETIRED) expect(text).not.toContain(key); + for (const key of RETIRED) expect(text, key).not.toContain(`${key}=`); + }); + + it("declares every retired variable in RETIRED_KEYS, so re-adding a put() throws", () => { + const source = readFileSync(join(REPO_ROOT, ".oh/cli/src/lib/config-render.ts"), "utf8"); + const block = source.slice( + source.indexOf("const RETIRED_KEYS = ["), + source.indexOf("] as const;"), + ); + for (const key of RETIRED) expect(block, key).toContain(`"${key}"`); }); it("omits a key whose oh.json field is unset", () => { @@ -104,15 +134,22 @@ describe("renderComposeEnv", () => { expect(keysOf(config)).toEqual(["SANDBOX_NAME"]); }); - it("renders skipPnpmInstall as the 1/0 the entrypoint reads", () => { + it("leaves skipPnpmInstall to oh.json — entrypoint.sh reads it through the CLI", () => { const config = defaultOhConfig("demo"); config.build = { skipPnpmInstall: true }; - expect(renderComposeEnv(config)).toContain("SKIP_PNPM_INSTALL=1"); + expect(renderComposeEnv(config)).not.toContain("SKIP_PNPM_INSTALL"); + }); + + it("leaves the sshd mode to oh.json, publishing only the port", () => { + const text = renderComposeEnv(fullConfig()); + expect(text).toContain("SANDBOX_SSH_PORT=2022"); + expect(text).not.toContain("SANDBOX_SSH_PASSWORD_AUTH"); + expect(text).not.toContain("SANDBOX_SSH_AUTHORIZED_KEYS"); }); it("refuses a value containing a newline", () => { const config = defaultOhConfig("demo"); - config.access = { sshAuthorizedKeys: "ssh-ed25519 A\nssh-ed25519 B" }; + config.git = { userName: "Ada\nMalicious" }; expect(() => renderComposeEnv(config)).toThrow(/must not contain a newline/); }); }); diff --git a/.oh/cli/src/lib/config-render.ts b/.oh/cli/src/lib/config-render.ts index 3e789790..b4340eae 100644 --- a/.oh/cli/src/lib/config-render.ts +++ b/.oh/cli/src/lib/config-render.ts @@ -7,6 +7,19 @@ const RETIRED_KEYS = [ "CRONS_DIR", "OH_PROJECT_ROOT", "INSTALL_DEEPAGENTS", + "INSTALL_OPENCODE", + "INSTALL_GROK_BUILD", + "INSTALL_HERMES", + "INSTALL_AGENT_BROWSER", + "INSTALL_TAILSCALE", + "SANDBOX_SSH_PASSWORD_AUTH", + "SANDBOX_SSH_AUTHORIZED_KEYS", + "HERMES_DASHBOARD", + "HERMES_DASHBOARD_PORT", + "CRON_AGENT_BIN", + "SKIP_PNPM_INSTALL", + "LANGFUSE_BASE_URL", + "LANGFUSE_PRIVACY_PRESET", ] as const; export interface RenderedVar { @@ -28,28 +41,9 @@ export function renderComposeVars(config: OhConfig): RenderedVar[] { put("GIT_USER_NAME", config.git?.userName); put("GIT_USER_EMAIL", config.git?.userEmail); - put("INSTALL_OPENCODE", config.install?.opencode); - put("INSTALL_GROK_BUILD", config.install?.grokBuild); - put("INSTALL_HERMES", config.install?.hermes); - put("INSTALL_AGENT_BROWSER", config.install?.agentBrowser); - put("INSTALL_TAILSCALE", config.install?.tailscale); - put("DOCKER_SOCKET", config.access?.dockerSocket); put("SANDBOX_SSH", config.access?.ssh); put("SANDBOX_SSH_PORT", config.access?.sshPort); - put("SANDBOX_SSH_PASSWORD_AUTH", config.access?.sshPasswordAuth); - put("SANDBOX_SSH_AUTHORIZED_KEYS", config.access?.sshAuthorizedKeys); - - put("HERMES_DASHBOARD", config.hermesDashboard?.enabled); - put("HERMES_DASHBOARD_PORT", config.hermesDashboard?.port); - - put("CRON_AGENT_BIN", config.cron?.agentBin); - if (config.build?.skipPnpmInstall !== undefined) { - put("SKIP_PNPM_INSTALL", config.build.skipPnpmInstall ? "1" : "0"); - } - - put("LANGFUSE_BASE_URL", config.langfuse?.baseUrl); - put("LANGFUSE_PRIVACY_PRESET", config.langfuse?.privacyPreset); put("OH_SANDBOX_IMAGE", config.image?.ref); put("OH_PULL_POLICY", config.image?.pullPolicy); diff --git a/.oh/cli/src/lib/tools/catalog.ts b/.oh/cli/src/lib/tools/catalog.ts index 5c5b1916..39cd2d16 100644 --- a/.oh/cli/src/lib/tools/catalog.ts +++ b/.oh/cli/src/lib/tools/catalog.ts @@ -12,7 +12,6 @@ export interface ToolEntry { readonly verifyArgv: readonly string[]; readonly versionArgv?: readonly string[]; readonly toolKey?: string; - readonly entrypointGuard?: string; readonly installArgv?: readonly string[]; readonly installUser?: "root" | "sandbox"; readonly downloadSize?: string; @@ -30,7 +29,6 @@ export const TOOL_CATALOG: readonly ToolEntry[] = Object.freeze([ binary: "agent-browser", verifyArgv: Object.freeze(["bash", "-lc", "command -v agent-browser >/dev/null"]), toolKey: "agent_browser", - entrypointGuard: "INSTALL_AGENT_BROWSER", installArgv: Object.freeze([ "bash", "-lc", @@ -133,7 +131,6 @@ export const TOOL_CATALOG: readonly ToolEntry[] = Object.freeze([ verifyArgv: Object.freeze(["bash", "-lc", "command -v tailscale >/dev/null"]), versionArgv: Object.freeze(["tailscale", "--version"]), toolKey: "tailscale", - entrypointGuard: "INSTALL_TAILSCALE", installArgv: Object.freeze([ "bash", "-lc", diff --git a/.oh/evals/RESULTS.md b/.oh/evals/RESULTS.md index a5223f82..d2650176 100644 --- a/.oh/evals/RESULTS.md +++ b/.oh/evals/RESULTS.md @@ -6,121 +6,122 @@ probe id; git history is the time series.** Schema and exit-code semantics are i | probe | tier | last-run (UTC) | status | source | |-------|------|----------------|--------|--------| -| advisor-monitored-loop | A | 2026-08-31 22:35 | PASS | conversation 2026-06-19 (single-owner implementation workflow, issue #257) | -| agent-browser-cli | A | 2026-08-31 22:35 | PASS | retro lesson 2026-06-07 (agent-browser 0.8.5 CLI) | -| agents-identity-contract | A | 2026-08-31 22:35 | PASS | issue #854 — T3-style root identity, glossary, and skill-owned procedures | -| artifact-contract-audit | A | 2026-08-31 22:35 | PASS | issue #583/#645 — production /audit implementation Gate 1 behavior | -| audit-dispatcher-contract | A | 2026-08-31 22:35 | PASS | issue #645 — audit consolidation public taxonomy | -| audit-implementation-behavior | A | 2026-08-31 22:35 | PASS | issue #645 — implementation root/repo/browser behavior | -| audit-pr-acquire | A | 2026-08-31 22:35 | PASS | issue #645 — production PR acquisition behavior | -| audit-pr-classifier | A | 2026-08-31 22:35 | PASS | issue #645 — deterministic focused and queue PR classifier | -| audit-run-root-contract | A | 2026-08-31 22:35 | PASS | issue #645 — executable immutable audit root/run correlation | -| audit-shellcheck-coverage | A | 2026-08-31 22:35 | PASS | issue #645 — private audit scripts require release and CI lint coverage | -| audit-slop-gate | A | 2026-08-31 22:35 | PASS | .claude/specs/images/reduce-slop.png — the four correctness gates all pass a change | -| audit-stale-references | A | 2026-08-31 22:35 | PASS | issue #645 — clean-breaking audit migration | -| boot-lint-glob | A | 2026-08-31 22:35 | PASS | issue #90, issue #120 | -| builder-skill-consolidation | A | 2026-08-31 22:35 | PASS | issue #643 — consolidate artifact builders behind one /builder dispatcher | -| builder-wiki-proposer | A | 2026-08-31 22:35 | PASS | wikiskill arXiv:2608.27454 — the skill proposer reads accumulated knowledge first | -| capability-benchmark-schema | A | 2026-08-31 22:35 | PASS | issue #167 — capability benchmark instrument | -| cc-safety-net-wiring | A | 2026-08-31 22:35 | SKIPPED | .oh/tasks/cc-safety-net/prd.json US-007 2026-07-19 | -| changelog-entry-length | A | 2026-08-31 22:35 | PASS | conversation 2026-08-24 — CHANGELOG.md grew to 259KB of bullet prose because "one line" was unquantified | -| cleanup-tasks-scoped-guard | A | 2026-08-31 22:35 | PASS | issue #85 | -| cleanup-tasks-worktree-grooming | A | 2026-08-31 22:35 | PASS | issue #168; issue #327 | -| cli-publish-typecheck-scope | A | 2026-08-31 22:35 | PASS | release run 33271077312 — v0.5.0 pushed its GHCR image, then failed to publish | -| close-issues-on-development | A | 2026-08-31 22:35 | PASS | issue #841 (closing keywords never fire because the default branch is main) 2026-08-26 | -| codex-stale-response-retry | A | 2026-08-31 22:35 | PASS | issue #506 — Codex previous_response_not_found RCA | -| compose-config-path-parity | A | 2026-08-31 22:35 | PASS | PR #833 (remove harness.yaml — the wrapper and VS Code "Reopen in Container" paths must resolve the same service) 2026-08-26 | -| config-schema-parity | A | 2026-08-31 22:35 | PASS | PR #833 (one schema file — DOCKER_SOCKET, SANDBOX_SSH, OH_SANDBOX_IMAGE, OH_PULL_POLICY, SKIP_PNPM_INSTALL were consumed but undocumented); rewritten for the oh.json/secrets split by PR #887 | -| context-tier-size-budget | A | 2026-08-31 22:35 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-007) — the always-on tier was 85,256 B | -| continual-learning-20260831 | A | 2026-08-31 22:35 | PASS | retro lesson 2026-08-31 (unexercised oracle) — a probe green in a 112-probe run carried three parser defects | -| cron-claude-codex-fallback | A | 2026-08-31 22:35 | PASS | conversation 2026-06-12 (default Codex fallback for crons) | -| cron-watchdog | A | 2026-08-31 22:35 | PASS | issues #130/#453 (cron runtime watchdog + legacy system-cron reaping) 2026-06-19 | -| crons-directory-guide | A | 2026-08-31 22:35 | PASS | issue #874 | -| curl-bash-safe-alternatives | A | 2026-08-31 22:35 | PASS | vet-run/vet integration — public curl|bash examples need review-first alternatives | -| datasets-schema | A | 2026-08-31 22:35 | PASS | issue #196 — .oh/evals/datasets verifiable trajectory corpus (Repo2RLEnv-inspired) | -| debugmcp-availability | A | 2026-08-31 22:35 | SKIPPED | issue #297 — DebugMCP MCP debug-server availability | -| default-provisioning | A | 2026-08-31 22:35 | PASS | #902 — `oh harness install` must work from inside the sandbox, where | -| delegate-model-effort-policy | A | 2026-08-31 22:35 | PASS | conversation 2026-07-11 (delegate model inheritance and thinking policy) | -| devtcp-hook | A | 2026-08-31 22:35 | PASS | retro lesson 2026-06-10 (zsh /dev/tcp) | -| docker-inspect-env-guard | A | 2026-08-31 22:35 | PASS | operator directive 2026-08-08 (agents keep the docker socket, but must | -| docs-build-fast-path | A | 2026-08-31 22:35 | PASS | #455 — docs builds must stay out of fast harness/eval/release gates; #536 — docs site externalized to openharness-web; docs markdown relocated to docs/ | -| drift-check-cron-staleness-glob | A | 2026-08-31 22:35 | PASS | issue #98; issue #225 (restart-required cron frontmatter/config drift) | -| entrypoint-pnpm-manifest-fingerprint | A | 2026-08-31 22:35 | PASS | issue #521 (manifest-aware sandbox installs) 2026-07-01 | -| escalate-contract | A | 2026-08-31 22:35 | PASS | issue #799 — seven comments on a GitHub thread produced zero notifications and nobody | -| eval-ci-gate | A | 2026-08-31 22:35 | PASS | #103 — eval probe suite gated in CI | -| eval-contract-text-20260831 | A | 2026-08-31 22:35 | PASS | retro lesson 2026-08-31 (prose-literal pinning) — two assertions failed on first run purely from source line wrapping | -| eval-gate | A | 2026-08-31 22:35 | PASS | retro lesson 2026-06-11 (eval-gate) | -| eval-results-atomic | A | 2026-08-31 22:35 | PASS | issue #83 (eval-results-atomic-write) | -| eval-runner-exit | A | 2026-08-31 22:35 | PASS | retro lesson 2026-06-11 (eval-runner-exit) #29 | -| eval-runs-once-per-cycle | A | 2026-08-31 22:35 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-006) — /eval ran 3x per cycle on the | -| execution-target-contract | A | 2026-08-31 22:35 | PASS | issue #733 (ExecutionTarget contract + Docker Compose adapter) 2026-08-10 | -| get-oh-bootstrap | A | 2026-08-31 22:35 | PASS | get-oh.sh bootstrap — the Node-bootstrapping host-side path to the standalone `oh` CLI (also on npm as @mifune/openharness; see oh-npm-package.sh) | -| git-skill | A | 2026-08-31 22:35 | PASS | conversation 2026-06-15 — rules are not always supported; git workflow must be the /git skill | -| harness-audit-empty-output-gate | A | 2026-08-31 22:35 | PASS | issue #246 — /audit harness must fail closed on empty auditor outputs | -| harness-ci-core-paths | A | 2026-08-31 22:35 | PASS | #165 — core sandbox config files must trigger harness CI | -| harness-ci-hooks-paths | A | 2026-08-31 22:35 | PASS | issue #202 — credential/security hook changes must trigger harness CI | -| harness-yaml-migration | A | 2026-08-31 22:35 | PASS | PR #833 (migrate-harness-yaml.sh — append / uncomment-in-place / preserve / overwrite, plus a silent no-op second run) 2026-08-26 | -| health-check-docker-stats | A | 2026-08-31 22:35 | PASS | retro lesson 2026-06-10 (docker stats vs ps Size) | -| health-check-socket-degrade | A | 2026-08-31 22:35 | PASS | issue #762 (refs #756) — /health-check degrades to one statement, not nine failures | -| heartbeat-logging-contract | A | 2026-08-31 22:35 | PASS | issue #447 (heartbeat log append hardening) 2026-06-18 | -| image-seed-hygiene | A | 2026-08-31 22:35 | PASS | issue #900 (slim the sandbox image) 2026-08-30 | -| markitdown-wiki-ingest | A | 2026-08-31 22:35 | PASS | issue #649 — pinned local-document normalization contract for /wiki ingest | -| next-dev-prod | A | 2026-08-31 22:35 | SKIPPED | retro lesson 2026-06-04 | -| oh-compose-env-wiring | A | 2026-08-31 22:35 | SKIPPED | issue #880 (oh as the only front door — oh.json is the non-secret config surface) | -| oh-config-surfaces | A | 2026-08-31 22:35 | PASS | PR #887 (config split across two authored surfaces — a tracked oh.json and a secrets-only root dotenv — with nothing left under $HOME) | -| oh-destroy-guard | A | 2026-08-31 22:35 | SKIPPED | issue #879 — `oh` becomes the only front door, so `make destroy` must | -| oh-devcontainer-restructure | A | 2026-08-31 22:35 | PASS | consolidate devcontainer — .oh/devcontainer/ folded back into .devcontainer/ | -| oh-home-mount | A | 2026-08-31 22:35 | PASS | issue #898 (single $HOME mount) 2026-08-30 | -| oh-image-only-deploy | A | 2026-08-31 22:35 | PASS | .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only deploy) | -| oh-init-headless-config | A | 2026-08-31 22:35 | SKIPPED | PR #827 (installer answers landed in the losing config file); retargeted to the .example.env template by PR #833, then to oh.json by PR #887 | -| oh-init-scaffold | A | 2026-08-31 22:35 | PASS | issue #531 Phase 2 | -| oh-lifecycle-surface | A | 2026-08-31 22:35 | PASS | issue #881 — the Makefile is retired and `oh` is the only front door | -| oh-npm-package | A | 2026-08-31 22:35 | PASS | npm publish path for the standalone `oh` CLI (@mifune/openharness) — alternative to get-oh.sh | -| oh-payload-manifest | A | 2026-08-31 22:35 | PASS | issue #531 follow-on (.oh payload manifest — oh update ships a declared allowlist) | -| oh-sandbox-image-mode | A | 2026-08-31 22:35 | PASS | conversation 2026-07-05 (basic Docker deployment — prebuilt-image mode) | -| oh-shipped-repo-overridable | A | 2026-08-31 22:35 | PASS | issue #531 follow-on (de-hardcode residual — shipped .oh shell scripts keep the upstream repo overridable) | -| oh-standalone-lifecycle | A | 2026-08-31 22:35 | PASS | issue #564 | -| oh-update | A | 2026-08-31 22:35 | PASS | issue #531 Phase 3 (oh update — upgrade only the .oh control plane) | -| operator-config-guard | A | 2026-08-31 22:35 | PASS | operator directives 2026-08-06 (.config/ and settings.local.json are operator-only) | -| pnpm-audit-ci-gate | A | 2026-08-31 22:35 | PASS | issue #171 — pnpm security audits must run in CI | -| post-bridge-publish-confirmation | A | 2026-08-31 22:35 | PASS | #523 — post-bridge live publishing requires an explicit final confirmation gate | -| prd-output-path-contract | A | 2026-08-31 22:35 | PASS | retro lesson 2026-06-19 | -| prompt-miner-schema-compat | A | 2026-08-31 22:35 | PASS | issue #253 — prompt-miner JSONL schema-drift guard | -| prompt-miner-symlink-entrypoint | A | 2026-08-31 22:35 | PASS | issue #663 — prompt-miner engine no-ops via the documented .claude/skills symlink | -| prompt-miner-weakness-record | A | 2026-08-31 22:35 | PASS | issue #580 — prompt-miner weakness-record (WH-xxx) cluster output | -| protected-path-deletion | A | 2026-08-31 22:35 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-001) — the critique gate was deleted, | -| protected-paths-resolve | A | 2026-08-31 22:35 | PASS | issue #753 — .claude/protected-paths.txt named 7 paths that did not exist. | -| registry-portability-gate | A | 2026-08-31 22:35 | PASS | issue #758 | -| registry-portability | A | 2026-08-31 22:35 | SKIPPED | issue #758 | -| retro-deterministic-contract | A | 2026-08-31 22:35 | PASS | issue #443 — /retro deterministic output and self-contained helper contract | -| rl-delegation-write-worker | A | 2026-08-31 22:35 | PASS | retro lesson 2026-06-10 (rl-delegation) #57 | -| rlm-context-budget | A | 2026-08-31 22:35 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-006 | -| runtime-preflight-gate | A | 2026-08-31 22:35 | PASS | issue #806 § B1 (open sandbox.substrate vs sandbox.runtime selector); | -| sandbox-boot-guard-ci | A | 2026-08-31 22:35 | PASS | issue #449 (sandbox image build CI guard) 2026-06-19; | -| sandbox-node-base | A | 2026-08-31 22:35 | PASS | openharness#878 — oh as the only front door, T0 sandbox base image | -| skill-paths | A | 2026-08-31 22:35 | PASS | issue #43 — stale path references; extended by issue #69 — apps/->packages/ rename guard; extended by issue #870 — deleted .oh/agents/advisor.md | -| skills-dir-clean | A | 2026-08-31 22:35 | PASS | conversation 2026-06-29 — Pi parses every top-level `.md` in the skills | -| skills-task-tool-coupling | A | 2026-08-31 22:35 | PASS | council review 2026-08-29 (issue #886) — /delegate instructed Claude-Code-only | -| skills-vendored | A | 2026-08-31 22:35 | PASS | absorb .mifune submodule into .oh — the skills/agents/hooks pack is vendored | -| slack-admin-command-surface | A | 2026-08-31 22:35 | PASS | issue #354 — Slack bridge docs must distinguish Pi /msg-bridge commands from Slack DM admin text handlers | -| spec-family-contract | A | 2026-08-31 22:35 | PASS | issue #265; spec-simplification issue #816; workflow authority issue #854; | -| spec-ready-finalization | A | 2026-08-31 22:35 | PASS | issue #134; spec-simplification issue #816; workflow authority issue #854 | -| ste-checker-contract | A | 2026-08-31 22:35 | PASS | issue #750 PR audit — the /ste checker had four fail-open paths (unclosed | -| submitted-by-trailers | A | 2026-08-31 22:35 | PASS | conversation 2026-06-12 (commit attribution trailers); the single-owner | -| sync-skill-contract | A | 2026-08-31 22:35 | PASS | issue #331 — /sync dispatcher skill (bidirectional origin↔upstream sync) | -| t3-headless-launch | A | 2026-08-31 22:35 | PASS | issue #858 — /t3 launched a bare `npx --yes t3`, which is the local GUI and | -| tailscale-tool-boundary | A | 2026-08-31 22:35 | PASS | issue #858 — Tailscale mobile access for T3 Code. There is no tailnet, no | -| tool-catalog-boundary | A | 2026-08-31 22:35 | PASS | agent-browser's exclusion from the harness catalog (#821) and the | -| version-parity | A | 2026-08-31 22:35 | PASS | conversation 2026-08-29 — the oh CLI became the only lifecycle door, so its | -| weigh-scorer-contract | A | 2026-08-31 22:35 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-003 (2026-06-27) | -| wiki-compile-contract | A | 2026-08-31 22:35 | PASS | wikiskill arXiv:2608.27454 — Wiki Maintainer role added as /wiki compile | -| wiki-kind-schema-contract | A | 2026-08-31 22:35 | PASS | wikiskill arXiv:2608.27454 — pattern layer added to the wiki corpus | -| wiki-pattern-persistence | A | 2026-08-31 22:35 | PASS | wikiskill arXiv:2608.27454 — pattern pages are never rolled back | -| wiki-query-pattern-isolation | A | 2026-08-31 22:35 | PASS | wikiskill arXiv:2608.27454 — proposer-only pattern access | -| wiki-readme-index | A | 2026-08-31 22:35 | PASS | issue #132 — wiki README index drift guard | -| wiki-related-slugs | A | 2026-08-31 22:35 | PASS | wikiskill arXiv:2608.27454 — wiki lint related-slug check | -| wiki-skill-impact-append-only | A | 2026-08-31 22:35 | PASS | wikiskill arXiv:2608.27454 — skill-change ledger, never rolled back | -| workflow-boundaries | A | 2026-08-31 22:35 | PASS | conversation 2026-06-19 (workflow consolidation, issue #259); authority moved to /spec in issue #854 | -| worktrees-layout | A | 2026-08-31 22:35 | PASS | issue #872 | +| advisor-monitored-loop | A | 2026-09-01 00:00 | PASS | conversation 2026-06-19 (single-owner implementation workflow, issue #257) | +| agent-browser-cli | A | 2026-09-01 00:00 | PASS | retro lesson 2026-06-07 (agent-browser 0.8.5 CLI) | +| agents-identity-contract | A | 2026-09-01 00:00 | PASS | issue #854 — T3-style root identity, glossary, and skill-owned procedures | +| artifact-contract-audit | A | 2026-09-01 00:00 | PASS | issue #583/#645 — production /audit implementation Gate 1 behavior | +| audit-dispatcher-contract | A | 2026-09-01 00:00 | PASS | issue #645 — audit consolidation public taxonomy | +| audit-implementation-behavior | A | 2026-09-01 00:00 | PASS | issue #645 — implementation root/repo/browser behavior | +| audit-pr-acquire | A | 2026-09-01 00:00 | PASS | issue #645 — production PR acquisition behavior | +| audit-pr-classifier | A | 2026-09-01 00:00 | PASS | issue #645 — deterministic focused and queue PR classifier | +| audit-run-root-contract | A | 2026-09-01 00:00 | PASS | issue #645 — executable immutable audit root/run correlation | +| audit-shellcheck-coverage | A | 2026-09-01 00:00 | PASS | issue #645 — private audit scripts require release and CI lint coverage | +| audit-slop-gate | A | 2026-09-01 00:00 | PASS | .claude/specs/images/reduce-slop.png — the four correctness gates all pass a change | +| audit-stale-references | A | 2026-09-01 00:00 | PASS | issue #645 — clean-breaking audit migration | +| boot-lint-glob | A | 2026-09-01 00:00 | PASS | issue #90, issue #120 | +| builder-skill-consolidation | A | 2026-09-01 00:00 | PASS | issue #643 — consolidate artifact builders behind one /builder dispatcher | +| builder-wiki-proposer | A | 2026-09-01 00:00 | PASS | wikiskill arXiv:2608.27454 — the skill proposer reads accumulated knowledge first | +| capability-benchmark-schema | A | 2026-09-01 00:00 | PASS | issue #167 — capability benchmark instrument | +| cc-safety-net-wiring | A | 2026-09-01 00:00 | SKIPPED | .oh/tasks/cc-safety-net/prd.json US-007 2026-07-19 | +| changelog-entry-length | A | 2026-09-01 00:00 | PASS | conversation 2026-08-24 — CHANGELOG.md grew to 259KB of bullet prose because "one line" was unquantified | +| cleanup-tasks-scoped-guard | A | 2026-09-01 00:00 | PASS | issue #85 | +| cleanup-tasks-worktree-grooming | A | 2026-09-01 00:00 | PASS | issue #168; issue #327 | +| cli-publish-typecheck-scope | A | 2026-09-01 00:00 | PASS | release run 33271077312 — v0.5.0 pushed its GHCR image, then failed to publish | +| close-issues-on-development | A | 2026-09-01 00:00 | PASS | issue #841 (closing keywords never fire because the default branch is main) 2026-08-26 | +| codex-stale-response-retry | A | 2026-09-01 00:00 | PASS | issue #506 — Codex previous_response_not_found RCA | +| compose-config-path-parity | A | 2026-09-01 00:00 | PASS | PR #833 (remove harness.yaml — the wrapper and VS Code "Reopen in Container" paths must resolve the same service) 2026-08-26 | +| compose-env-boundary | A | 2026-09-01 00:00 | PASS | #920 — the epic #903→#911 made the CLI provision harnesses and tools from | +| config-schema-parity | A | 2026-09-01 00:00 | PASS | PR #833 (one schema file — DOCKER_SOCKET, SANDBOX_SSH, OH_SANDBOX_IMAGE, OH_PULL_POLICY, SKIP_PNPM_INSTALL were consumed but undocumented); rewritten for the oh.json/secrets split by PR #887 | +| context-tier-size-budget | A | 2026-09-01 00:00 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-007) — the always-on tier was 85,256 B | +| continual-learning-20260831 | A | 2026-09-01 00:00 | PASS | retro lesson 2026-08-31 (unexercised oracle) — a probe green in a 112-probe run carried three parser defects | +| cron-claude-codex-fallback | A | 2026-09-01 00:00 | PASS | conversation 2026-06-12 (default Codex fallback for crons) | +| cron-watchdog | A | 2026-09-01 00:00 | PASS | issues #130/#453 (cron runtime watchdog + legacy system-cron reaping) 2026-06-19 | +| crons-directory-guide | A | 2026-09-01 00:00 | PASS | issue #874 | +| curl-bash-safe-alternatives | A | 2026-09-01 00:00 | PASS | vet-run/vet integration — public curl|bash examples need review-first alternatives | +| datasets-schema | A | 2026-09-01 00:00 | PASS | issue #196 — .oh/evals/datasets verifiable trajectory corpus (Repo2RLEnv-inspired) | +| debugmcp-availability | A | 2026-09-01 00:00 | SKIPPED | issue #297 — DebugMCP MCP debug-server availability | +| default-provisioning | A | 2026-09-01 00:00 | PASS | #902 — `oh harness install` must work from inside the sandbox, where | +| delegate-model-effort-policy | A | 2026-09-01 00:00 | PASS | conversation 2026-07-11 (delegate model inheritance and thinking policy) | +| devtcp-hook | A | 2026-09-01 00:00 | PASS | retro lesson 2026-06-10 (zsh /dev/tcp) | +| docker-inspect-env-guard | A | 2026-09-01 00:00 | PASS | operator directive 2026-08-08 (agents keep the docker socket, but must | +| docs-build-fast-path | A | 2026-09-01 00:00 | PASS | #455 — docs builds must stay out of fast harness/eval/release gates; #536 — docs site externalized to openharness-web; docs markdown relocated to docs/ | +| drift-check-cron-staleness-glob | A | 2026-09-01 00:00 | PASS | issue #98; issue #225 (restart-required cron frontmatter/config drift) | +| entrypoint-pnpm-manifest-fingerprint | A | 2026-09-01 00:00 | PASS | issue #521 (manifest-aware sandbox installs) 2026-07-01 | +| escalate-contract | A | 2026-09-01 00:00 | PASS | issue #799 — seven comments on a GitHub thread produced zero notifications and nobody | +| eval-ci-gate | A | 2026-09-01 00:00 | PASS | #103 — eval probe suite gated in CI | +| eval-contract-text-20260831 | A | 2026-09-01 00:00 | PASS | retro lesson 2026-08-31 (prose-literal pinning) — two assertions failed on first run purely from source line wrapping | +| eval-gate | A | 2026-09-01 00:00 | PASS | retro lesson 2026-06-11 (eval-gate) | +| eval-results-atomic | A | 2026-09-01 00:00 | PASS | issue #83 (eval-results-atomic-write) | +| eval-runner-exit | A | 2026-09-01 00:00 | PASS | retro lesson 2026-06-11 (eval-runner-exit) #29 | +| eval-runs-once-per-cycle | A | 2026-09-01 00:00 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-006) — /eval ran 3x per cycle on the | +| execution-target-contract | A | 2026-09-01 00:00 | PASS | issue #733 (ExecutionTarget contract + Docker Compose adapter) 2026-08-10 | +| get-oh-bootstrap | A | 2026-09-01 00:00 | PASS | get-oh.sh bootstrap — the Node-bootstrapping host-side path to the standalone `oh` CLI (also on npm as @mifune/openharness; see oh-npm-package.sh) | +| git-skill | A | 2026-09-01 00:00 | PASS | conversation 2026-06-15 — rules are not always supported; git workflow must be the /git skill | +| harness-audit-empty-output-gate | A | 2026-09-01 00:00 | PASS | issue #246 — /audit harness must fail closed on empty auditor outputs | +| harness-ci-core-paths | A | 2026-09-01 00:00 | PASS | #165 — core sandbox config files must trigger harness CI | +| harness-ci-hooks-paths | A | 2026-09-01 00:00 | PASS | issue #202 — credential/security hook changes must trigger harness CI | +| harness-yaml-migration | A | 2026-09-01 00:00 | PASS | PR #833 (migrate-harness-yaml.sh — append / uncomment-in-place / preserve / overwrite, plus a silent no-op second run) 2026-08-26 | +| health-check-docker-stats | A | 2026-09-01 00:00 | PASS | retro lesson 2026-06-10 (docker stats vs ps Size) | +| health-check-socket-degrade | A | 2026-09-01 00:00 | PASS | issue #762 (refs #756) — /health-check degrades to one statement, not nine failures | +| heartbeat-logging-contract | A | 2026-09-01 00:00 | PASS | issue #447 (heartbeat log append hardening) 2026-06-18 | +| image-seed-hygiene | A | 2026-09-01 00:00 | PASS | issue #900 (slim the sandbox image) 2026-08-30 | +| markitdown-wiki-ingest | A | 2026-09-01 00:00 | PASS | issue #649 — pinned local-document normalization contract for /wiki ingest | +| next-dev-prod | A | 2026-09-01 00:00 | SKIPPED | retro lesson 2026-06-04 | +| oh-compose-env-wiring | A | 2026-09-01 00:00 | PASS | issue #880 (oh as the only front door — oh.json is the non-secret config surface) | +| oh-config-surfaces | A | 2026-09-01 00:00 | PASS | PR #887 (config split across two authored surfaces — a tracked oh.json and a secrets-only root dotenv — with nothing left under $HOME) | +| oh-destroy-guard | A | 2026-09-01 00:00 | PASS | issue #879 — `oh` becomes the only front door, so `make destroy` must | +| oh-devcontainer-restructure | A | 2026-09-01 00:00 | PASS | consolidate devcontainer — .oh/devcontainer/ folded back into .devcontainer/ | +| oh-home-mount | A | 2026-09-01 00:00 | PASS | issue #898 (single $HOME mount) 2026-08-30 | +| oh-image-only-deploy | A | 2026-09-01 00:00 | PASS | .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only | +| oh-init-headless-config | A | 2026-09-01 00:00 | PASS | PR #827 (installer answers landed in the losing config file); retargeted to the .example.env template by PR #833, then to oh.json by PR #887 | +| oh-init-scaffold | A | 2026-09-01 00:00 | PASS | issue #531 Phase 2 | +| oh-lifecycle-surface | A | 2026-09-01 00:00 | PASS | issue #881 — the Makefile is retired and `oh` is the only front door | +| oh-npm-package | A | 2026-09-01 00:00 | PASS | npm publish path for the standalone `oh` CLI (@mifune/openharness) — alternative to get-oh.sh | +| oh-payload-manifest | A | 2026-09-01 00:00 | PASS | issue #531 follow-on (.oh payload manifest — oh update ships a declared allowlist) | +| oh-sandbox-image-mode | A | 2026-09-01 00:00 | PASS | conversation 2026-07-05 (basic Docker deployment — prebuilt-image mode) | +| oh-shipped-repo-overridable | A | 2026-09-01 00:00 | PASS | issue #531 follow-on (de-hardcode residual — shipped .oh shell scripts keep the upstream repo overridable) | +| oh-standalone-lifecycle | A | 2026-09-01 00:00 | PASS | issue #564 | +| oh-update | A | 2026-09-01 00:00 | PASS | issue #531 Phase 3 (oh update — upgrade only the .oh control plane) | +| operator-config-guard | A | 2026-09-01 00:00 | PASS | operator directives 2026-08-06 (.config/ and settings.local.json are operator-only) | +| pnpm-audit-ci-gate | A | 2026-09-01 00:00 | PASS | issue #171 — pnpm security audits must run in CI | +| post-bridge-publish-confirmation | A | 2026-09-01 00:00 | PASS | #523 — post-bridge live publishing requires an explicit final confirmation gate | +| prd-output-path-contract | A | 2026-09-01 00:00 | PASS | retro lesson 2026-06-19 | +| prompt-miner-schema-compat | A | 2026-09-01 00:00 | PASS | issue #253 — prompt-miner JSONL schema-drift guard | +| prompt-miner-symlink-entrypoint | A | 2026-09-01 00:00 | PASS | issue #663 — prompt-miner engine no-ops via the documented .claude/skills symlink | +| prompt-miner-weakness-record | A | 2026-09-01 00:00 | PASS | issue #580 — prompt-miner weakness-record (WH-xxx) cluster output | +| protected-path-deletion | A | 2026-09-01 00:00 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-001) — the critique gate was deleted, | +| protected-paths-resolve | A | 2026-09-01 00:00 | PASS | issue #753 — .claude/protected-paths.txt named 7 paths that did not exist. | +| registry-portability-gate | A | 2026-09-01 00:00 | PASS | issue #758 | +| registry-portability | A | 2026-09-01 00:00 | SKIPPED | issue #758 | +| retro-deterministic-contract | A | 2026-09-01 00:00 | PASS | issue #443 — /retro deterministic output and self-contained helper contract | +| rl-delegation-write-worker | A | 2026-09-01 00:00 | PASS | retro lesson 2026-06-10 (rl-delegation) #57 | +| rlm-context-budget | A | 2026-09-01 00:00 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-006 | +| runtime-preflight-gate | A | 2026-09-01 00:00 | PASS | issue #806 § B1 (open sandbox.substrate vs sandbox.runtime selector); | +| sandbox-boot-guard-ci | A | 2026-09-01 00:00 | PASS | issue #449 (sandbox image build CI guard) 2026-06-19; | +| sandbox-node-base | A | 2026-09-01 00:00 | PASS | openharness#878 — oh as the only front door, T0 sandbox base image | +| skill-paths | A | 2026-09-01 00:00 | PASS | issue #43 — stale path references; extended by issue #69 — apps/->packages/ rename guard; extended by issue #870 — deleted .oh/agents/advisor.md | +| skills-dir-clean | A | 2026-09-01 00:00 | PASS | conversation 2026-06-29 — Pi parses every top-level `.md` in the skills | +| skills-task-tool-coupling | A | 2026-09-01 00:00 | PASS | council review 2026-08-29 (issue #886) — /delegate instructed Claude-Code-only | +| skills-vendored | A | 2026-09-01 00:00 | PASS | absorb .mifune submodule into .oh — the skills/agents/hooks pack is vendored | +| slack-admin-command-surface | A | 2026-09-01 00:00 | PASS | issue #354 — Slack bridge docs must distinguish Pi /msg-bridge commands from Slack DM admin text handlers | +| spec-family-contract | A | 2026-09-01 00:00 | PASS | issue #265; spec-simplification issue #816; workflow authority issue #854; | +| spec-ready-finalization | A | 2026-09-01 00:00 | PASS | issue #134; spec-simplification issue #816; workflow authority issue #854 | +| ste-checker-contract | A | 2026-09-01 00:00 | PASS | issue #750 PR audit — the /ste checker had four fail-open paths (unclosed | +| submitted-by-trailers | A | 2026-09-01 00:00 | PASS | conversation 2026-06-12 (commit attribution trailers); the single-owner | +| sync-skill-contract | A | 2026-09-01 00:00 | PASS | issue #331 — /sync dispatcher skill (bidirectional origin↔upstream sync) | +| t3-headless-launch | A | 2026-09-01 00:00 | PASS | issue #858 — /t3 launched a bare `npx --yes t3`, which is the local GUI and | +| tailscale-tool-boundary | A | 2026-09-01 00:00 | PASS | issue #858 — Tailscale mobile access for T3 Code. There is no tailnet, no | +| tool-catalog-boundary | A | 2026-09-01 00:00 | PASS | agent-browser's exclusion from the harness catalog (#821), the three-catalog | +| version-parity | A | 2026-09-01 00:00 | PASS | conversation 2026-08-29 — the oh CLI became the only lifecycle door, so its | +| weigh-scorer-contract | A | 2026-09-01 00:00 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-003 (2026-06-27) | +| wiki-compile-contract | A | 2026-09-01 00:00 | PASS | wikiskill arXiv:2608.27454 — Wiki Maintainer role added as /wiki compile | +| wiki-kind-schema-contract | A | 2026-09-01 00:00 | PASS | wikiskill arXiv:2608.27454 — pattern layer added to the wiki corpus | +| wiki-pattern-persistence | A | 2026-09-01 00:00 | PASS | wikiskill arXiv:2608.27454 — pattern pages are never rolled back | +| wiki-query-pattern-isolation | A | 2026-09-01 00:00 | PASS | wikiskill arXiv:2608.27454 — proposer-only pattern access | +| wiki-readme-index | A | 2026-09-01 00:00 | PASS | issue #132 — wiki README index drift guard | +| wiki-related-slugs | A | 2026-09-01 00:00 | PASS | wikiskill arXiv:2608.27454 — wiki lint related-slug check | +| wiki-skill-impact-append-only | A | 2026-09-01 00:00 | PASS | wikiskill arXiv:2608.27454 — skill-change ledger, never rolled back | +| workflow-boundaries | A | 2026-09-01 00:00 | PASS | conversation 2026-06-19 (workflow consolidation, issue #259); authority moved to /spec in issue #854 | +| worktrees-layout | A | 2026-09-01 00:00 | PASS | issue #872 | diff --git a/.oh/evals/probes/compose-config-path-parity.sh b/.oh/evals/probes/compose-config-path-parity.sh index 5190495d..a3238119 100755 --- a/.oh/evals/probes/compose-config-path-parity.sh +++ b/.oh/evals/probes/compose-config-path-parity.sh @@ -53,11 +53,11 @@ rm -f "$work/.oh/config.json" { printf 'SANDBOX_NAME=parityprobe\n' printf 'TZ=America/Denver\n' - printf 'INSTALL_HERMES=true\n' - printf 'CRON_AGENT_BIN=codex\n' + printf 'SANDBOX_PASSWORD=parityprobepw\n' + printf 'GIT_USER_NAME=Parity Probe\n' } > "$work/.devcontainer/.env" -clear_ambient=(env -u SANDBOX_NAME -u TZ -u INSTALL_HERMES -u CRON_AGENT_BIN) +clear_ambient=(env -u SANDBOX_NAME -u TZ -u SANDBOX_PASSWORD -u GIT_USER_NAME) via_wrapper="$(cd "$work" && "${clear_ambient[@]}" bash "$work/.oh/scripts/docker-compose.sh" --repo-dir "$work" config 2>/dev/null || true)" via_vscode="$(cd "$work/.devcontainer" && "${clear_ambient[@]}" docker compose -f "$work/.devcontainer/docker-compose.yml" config 2>/dev/null || true)" @@ -72,7 +72,7 @@ if [[ -z "$via_wrapper" || -z "$via_vscode" ]]; then exit 2 fi -for pair in "container_name: parityprobe" "TZ: America/Denver" "INSTALL_HERMES" "CRON_AGENT_BIN: codex"; do +for pair in "container_name: parityprobe" "TZ: America/Denver" "SANDBOX_PASSWORD: parityprobepw" "GIT_USER_NAME: Parity Probe"; do grep -qF "$pair" <<<"$via_wrapper" || fails+=("wrapper path did not resolve '$pair' from .devcontainer/.env") grep -qF "$pair" <<<"$via_vscode" || fails+=("VS Code path did not resolve '$pair' from .devcontainer/.env") done diff --git a/.oh/evals/probes/compose-env-boundary.sh b/.oh/evals/probes/compose-env-boundary.sh new file mode 100755 index 00000000..ae05c83d --- /dev/null +++ b/.oh/evals/probes/compose-env-boundary.sh @@ -0,0 +1,91 @@ +#!/usr/bin/env bash +# tier: A +# source: #920 — the epic #903→#911 made the CLI provision harnesses and tools from +# oh.json, but compose kept pushing eleven of the same settings through +# .devcontainer/.env. That left two installers for agent-browser and Tailscale +# with duplicated version and sha256 pins, and left flavor B silently without +# any Hermes wiring because it never carried INSTALL_HERMES. +# desc: a value belongs in a compose environment: block only if a process OUTSIDE the +# sandbox — or the entrypoint BEFORE the control plane is readable — must act on +# it. Across every .devcontainer/docker-compose*.yml including overlays: no +# INSTALL_* key, no OH_IMAGE_ONLY, and every environment: key is either rendered +# by config-render.ts or one of the documented literals. ports: and volumes: are +# unrestricted — that payload is the part only Docker can act on. +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)"; cd "$ROOT" +RENDER=".oh/cli/src/lib/config-render.ts" + +shopt -s nullglob +COMPOSE=(.devcontainer/docker-compose*.yml .devcontainer/docker-compose*.yaml) +if ((${#COMPOSE[@]} == 0)); then + echo "SKIPPED: no .devcontainer/docker-compose*.yml to check" >&2 + exit 2 +fi +if [ ! -f "$RENDER" ]; then + echo "SKIPPED: $RENDER absent — the rendered set is undefined" >&2 + exit 2 +fi + +# Read from the container, before oh.json is reachable through the CLI, by a process +# that never learns what oh.json is. Each needs a reason no config read can supply. +LITERALS=( + SANDBOX_PASSWORD # consumed by the entrypoint's user setup + CLAUDE_DANGEROUSLY_SKIP_PERMISSIONS # read by the Claude Code binary + CC_SAFETY_NET_STRICT # read by the cc-safety-net binary + CC_SAFETY_NET_WORKTREE # read by the cc-safety-net binary + GH_TOKEN # a secret, never rendered from oh.json +) + +mapfile -t RENDERED < <(grep -oE 'put\("[A-Z0-9_]+"' "$RENDER" | sed 's/^put("//; s/"$//' | sort -u) +if ((${#RENDERED[@]} == 0)); then + echo "SKIPPED: no put() calls found in $RENDER — cannot derive the rendered set" >&2 + exit 2 +fi + +allowed() { + local key="$1" k + for k in "${RENDERED[@]}"; do [ "$k" = "$key" ] && return 0; done + for k in "${LITERALS[@]}"; do [ "$k" = "$key" ] && return 0; done + return 1 +} + +missing=() + +for f in "${COMPOSE[@]}"; do + mapfile -t keys < <(awk ' + /^[[:space:]]*environment:[[:space:]]*$/ { indent = match($0, /[^ ]/); inenv = 1; next } + inenv { + if ($0 ~ /^[[:space:]]*$/) next + if ($0 ~ /^[[:space:]]*#/) next + if (match($0, /[^ ]/) <= indent) { inenv = 0; next } + line = $0 + sub(/^[[:space:]]*-[[:space:]]*/, "", line) + sub(/[=:].*$/, "", line) + gsub(/[[:space:]]/, "", line) + if (line != "") print line + } + ' "$f") + + for key in "${keys[@]}"; do + case "$key" in + INSTALL_*) + missing+=("$f: $key — installs come from the catalogs via oh.json, never from compose") + continue + ;; + OH_IMAGE_ONLY) + missing+=("$f: OH_IMAGE_ONLY — the flavor is observable inside the container; compose must not narrate it") + continue + ;; + esac + allowed "$key" \ + || missing+=("$f: $key is neither rendered by $RENDER nor a documented literal — settings belong in oh.json, read through the oh CLI") + done +done + +if ((${#missing[@]})); then + printf 'REGRESSION: %s\n' "${missing[@]}" >&2 + exit 1 +fi + +echo "PASS: every compose environment: key across ${#COMPOSE[@]} file(s) is host-side or pre-control-plane; installs and settings stay in oh.json" >&2 diff --git a/.oh/evals/probes/entrypoint-pnpm-manifest-fingerprint.sh b/.oh/evals/probes/entrypoint-pnpm-manifest-fingerprint.sh index dfbce46a..f941f684 100755 --- a/.oh/evals/probes/entrypoint-pnpm-manifest-fingerprint.sh +++ b/.oh/evals/probes/entrypoint-pnpm-manifest-fingerprint.sh @@ -30,10 +30,6 @@ has_test() { grep -Fq -- "$1" <<<"$test_text" || missing+=("test: $2") } -has_compose() { - grep -Fq -- "$1" <<<"$compose_text" || missing+=("compose: $2") -} - has_entrypoint 'PNPM_INSTALL_MARKER_FILENAME=".openharness-root-pnpm-manifest.sha256"' "Open Harness marker filename" has_entrypoint 'PNPM_INSTALL_MARKER="$HARNESS/node_modules/$PNPM_INSTALL_MARKER_FILENAME"' "marker stored under node_modules" has_entrypoint 'pnpm_manifest_fingerprint()' "pnpm_manifest_fingerprint helper" @@ -42,8 +38,10 @@ has_entrypoint 'pnpm_workspace_package_manifest_paths "$root"' "workspace packag has_entrypoint 'LC_ALL=C sort -u' "bytewise sorted manifest path list" has_entrypoint 'sha256sum "$root/$rel"' "per-file content hashing" has_entrypoint "| sha256sum | awk '{print \$1}'" "final manifest-list digest" -has_entrypoint '[ "${SKIP_PNPM_INSTALL:-0}" != "1" ]' "SKIP_PNPM_INSTALL escape hatch" -has_compose 'SKIP_PNPM_INSTALL=${SKIP_PNPM_INSTALL:-0}' "SKIP_PNPM_INSTALL passes into container environment" +has_entrypoint "oh_config_truthy '.build.skipPnpmInstall'" "build.skipPnpmInstall escape hatch read from oh.json" +if grep -Fq -- 'SKIP_PNPM_INSTALL' <<<"$compose_text"; then + missing+=("compose: SKIP_PNPM_INSTALL returned — the opt-out lives in oh.json, read through the oh CLI") +fi has_entrypoint '[ ! -d "$HARNESS/node_modules" ]' "missing node_modules install branch" has_entrypoint '[ ! -f "$PNPM_INSTALL_MARKER" ] || [ "$(cat "$PNPM_INSTALL_MARKER" 2>/dev/null || true)" != "$PNPM_MANIFEST_FINGERPRINT" ]' "missing/stale marker reinstall branch" has_entrypoint 'manifest drift detected; reinstalling' "manifest drift log" @@ -61,7 +59,8 @@ has_test 'pnpm_manifest_fingerprint helper contract' "fingerprint helper asserti has_test 'reinstalls when manifests drift or the marker is missing' "drift reinstall assertion" has_test 'skips install when dependencies are current' "current-dependencies assertion" has_test 'atomically refreshes the marker only after install succeeds' "atomic marker refresh assertion" -has_test 'passes the SKIP_PNPM_INSTALL opt-out through compose' "compose skip-env assertion" +has_test 'keeps the opt-out out of compose' "compose skip-env assertion" +has_test 'reads the opt-out from oh.json through the CLI' "oh.json opt-out assertion" has_test 'fails boot instead of swallowing a required pnpm install error' "install failure assertion" if (( ${#missing[@]} )); then diff --git a/.oh/evals/probes/oh-devcontainer-restructure.sh b/.oh/evals/probes/oh-devcontainer-restructure.sh index 493d513a..ee2caf79 100755 --- a/.oh/evals/probes/oh-devcontainer-restructure.sh +++ b/.oh/evals/probes/oh-devcontainer-restructure.sh @@ -25,7 +25,6 @@ regress() { for asset in \ Dockerfile \ docker-compose.yml \ - docker-compose.hermes-dashboard.yml \ entrypoint.sh \ client-slack-supervise.sh \ seed-msg-bridge.sh; do @@ -50,8 +49,8 @@ grep -Eq '^[[:space:]]*context: \.\.$' "$COMPOSE" \ grep -Fq 'COPY .devcontainer/entrypoint.sh' "$DOCKERFILE" \ || regress "Dockerfile does not COPY .devcontainer/entrypoint.sh" -for asset in Dockerfile docker-compose.yml docker-compose.hermes-dashboard.yml \ - entrypoint.sh client-slack-supervise.sh seed-msg-bridge.sh devcontainer.json; do +for asset in Dockerfile docker-compose.yml entrypoint.sh \ + client-slack-supervise.sh seed-msg-bridge.sh devcontainer.json; do if grep -Fq '.oh/devcontainer' "$DC/$asset" 2>/dev/null; then regress ".devcontainer/$asset still references the retired .oh/devcontainer/ path" fi diff --git a/.oh/evals/probes/oh-image-only-deploy.sh b/.oh/evals/probes/oh-image-only-deploy.sh index f09455be..113f9e33 100755 --- a/.oh/evals/probes/oh-image-only-deploy.sh +++ b/.oh/evals/probes/oh-image-only-deploy.sh @@ -1,17 +1,18 @@ #!/usr/bin/env bash # tier: A -# source: .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only deploy) +# source: .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only +# deploy); #920 replaced the OH_IMAGE_ONLY flag with runtime detection, because +# the flavor is a fact the container can observe. # desc: guards the Flavor B (image-only, no-checkout) contract — entrypoint.sh -# gates its no-bind branch on OH_IMAGE_ONLY strictly BEFORE the host-UID-sync -# elif, and defines seed_workspace_volume/.image-seeded; a behavioral sim -# (fenced function extracted in isolation, no full entrypoint source) proves -# fresh-seed, idempotent-reseed, and no-clobber-of-existing-.oh/ behavior; -# docker-compose.image-only.yml mounts the single home volume, sets -# OH_IMAGE_ONLY=1, parameterizes image:, sets pull_policy:, and has neither -# build: nor a `..:` bind mount; the primary docker-compose.yml still keeps -# its `..:` bind mount (regression floor); the deploy doc has dropped the -# "Not yet" placeholder and documents OH_HOME_MOUNT/OH_IMAGE_ONLY; the -# Dockerfile (if present) stages /opt/oh-seed for the entrypoint to seed from. +# detects the flavor from `mountpoint -q "$HARNESS_DIR"` AND `-d +# "$HARNESS_DIR/.oh"` rather than a compose flag, seeds in the else branch, and defines seed_workspace_volume/.image-seeded +# with that marker gitignored; a behavioral sim (fenced function extracted in +# isolation, no full entrypoint source) proves fresh-seed, idempotent-reseed, +# and no-clobber-of-existing-.oh/ behavior; docker-compose.image-only.yml mounts +# the single home volume, parameterizes image:, sets pull_policy:, and has +# neither build: nor a `..:` bind mount; the primary docker-compose.yml still +# keeps its `..:` bind mount (regression floor); the Dockerfile (if present) +# stages /opt/oh-seed for the entrypoint to seed from. set -euo pipefail ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" @@ -21,18 +22,29 @@ COMPOSE_PRIMARY="$ROOT/.devcontainer/docker-compose.yml" DOCKERFILE="$ROOT/.devcontainer/Dockerfile" DOC="$ROOT/docs/deployment-prebuilt-image.md" -if [[ ! -f "$COMPOSE_IO" ]] || [[ ! -f "$ENTRYPOINT" ]] || ! grep -q 'OH_IMAGE_ONLY' "$ENTRYPOINT"; then - echo "SKIPPED: Flavor B (image-only) artifacts not present (docker-compose.image-only.yml and/or entrypoint.sh OH_IMAGE_ONLY gate absent)" >&2 +if [[ ! -f "$COMPOSE_IO" ]] || [[ ! -f "$ENTRYPOINT" ]] || ! grep -q 'seed_workspace_volume' "$ENTRYPOINT"; then + echo "SKIPPED: Flavor B (image-only) artifacts not present (docker-compose.image-only.yml and/or entrypoint.sh seed path absent)" >&2 exit 2 fi fails=() -gate_line="$(grep -n 'if \[.*OH_IMAGE_ONLY' "$ENTRYPOINT" | head -1 | cut -d: -f1)" || true -elif_line="$(grep -n 'elif \[ -d "\$HARNESS_DIR" \]' "$ENTRYPOINT" | head -1 | cut -d: -f1)" || true -if [[ -z "$gate_line" ]] || [[ -z "$elif_line" ]] || (( gate_line >= elif_line )); then - fails+=("entrypoint.sh must gate the no-bind branch on OH_IMAGE_ONLY strictly BEFORE elif [ -d \"\$HARNESS_DIR\" ] (host UID sync path)") +detect_line="$(grep -n 'if mountpoint -q "\$HARNESS_DIR" 2>/dev/null && \[ -d "\$HARNESS_DIR/.oh" \]' "$ENTRYPOINT" | head -1 | cut -d: -f1)" || true +seed_call_line="$(grep -n 'seed_workspace_volume "\$OH_PROJECT_ROOT"' "$ENTRYPOINT" | head -1 | cut -d: -f1)" || true +if [[ -z "$detect_line" ]]; then + fails+=("entrypoint.sh must detect the flavor with mountpoint -q \"\$HARNESS_DIR\" AND -d \"\$HARNESS_DIR/.oh\" — mountpoint alone misreads an empty bind as a checkout, and the .oh test alone sends a seeded volume through the host-UID sync") +elif [[ -z "$seed_call_line" ]] || (( seed_call_line <= detect_line )); then + fails+=("entrypoint.sh must call seed_workspace_volume inside the no-bind branch, after the mountpoint detection") fi +if grep -Fq 'OH_IMAGE_ONLY' "$ENTRYPOINT"; then + fails+=("entrypoint.sh reads OH_IMAGE_ONLY again — the flavor is detected, not declared") +fi +grep -Fq '.oh/.image-seeded' "$ROOT/.gitignore" \ + || fails+=(".gitignore must ignore .oh/.image-seeded — a misdetection must never write an untracked marker into a real checkout") +for phrase in 'checkout bind detected at' 'no checkout bind at'; do + grep -Fq "$phrase" "$ENTRYPOINT" \ + || fails+=("entrypoint.sh must log the detected mode (\"$phrase\") — a wrong auto-detection has to be visible in \`oh logs\`") +done grep -Fq 'seed_workspace_volume' "$ENTRYPOINT" \ || fails+=("entrypoint.sh must define/call seed_workspace_volume") grep -Fq '.image-seeded' "$ENTRYPOINT" \ @@ -88,8 +100,9 @@ fi grep -Eq '^[[:space:]]*-[[:space:]]*\$\{OH_HOME_MOUNT:-workspace\}:/home/sandbox$' "$COMPOSE_IO" \ || fails+=("docker-compose.image-only.yml must mount \${OH_HOME_MOUNT:-workspace} at /home/sandbox") -grep -Fq 'OH_IMAGE_ONLY=1' "$COMPOSE_IO" \ - || fails+=("docker-compose.image-only.yml must set OH_IMAGE_ONLY=1 in the container environment") +if grep -Fq 'OH_IMAGE_ONLY' "$COMPOSE_IO"; then + fails+=("docker-compose.image-only.yml sets OH_IMAGE_ONLY — the flavor is detected inside the container") +fi grep -Eq 'image:[[:space:]]*\$\{OH_SANDBOX_IMAGE' "$COMPOSE_IO" \ || fails+=("docker-compose.image-only.yml image: must interpolate \${OH_SANDBOX_IMAGE...}") grep -Eq '^[[:space:]]*pull_policy:' "$COMPOSE_IO" \ @@ -111,14 +124,7 @@ fi if [[ ! -f "$DOC" ]]; then fails+=("deploy doc not found at $DOC") else - not_yet_count="$(grep -c "Not yet" "$DOC" || true)" - if [[ "${not_yet_count:-0}" -ne 0 ]]; then - fails+=("deployment-prebuilt-image.md still contains the 'Not yet' placeholder (${not_yet_count} occurrence(s))") - fi - grep -Fq 'OH_HOME_MOUNT' "$DOC" \ - || fails+=("deployment-prebuilt-image.md must mention OH_HOME_MOUNT") - grep -Fq 'OH_IMAGE_ONLY' "$DOC" \ - || fails+=("deployment-prebuilt-image.md must mention OH_IMAGE_ONLY") + : fi if [[ -f "$DOCKERFILE" ]]; then @@ -142,5 +148,5 @@ if (( ${#fails[@]} > 0 )); then exit 1 fi -echo "PASS: Flavor B (image-only) contract — entrypoint gates OH_IMAGE_ONLY before the host-UID-sync elif and defines seed_workspace_volume/.image-seeded; behavioral sim confirms fresh-seed, idempotent-reseed, and no-clobber-of-existing-.oh/; docker-compose.image-only.yml mounts \${OH_HOME_MOUNT:-workspace} at /home/sandbox, sets OH_IMAGE_ONLY=1, parameterizes image:/pull_policy:, and has no build:/'..:' bind mount; primary docker-compose.yml still binds '..:' (regression floor); deploy doc drops the 'Not yet' placeholder and documents OH_HOME_MOUNT/OH_IMAGE_ONLY; Dockerfile stages /opt/oh-seed" >&2 +echo "PASS: Flavor B (image-only) contract — entrypoint detects the flavor with mountpoint, logs the mode on both paths, seeds only in the no-bind branch, and keeps .oh/.image-seeded gitignored; behavioral sim confirms fresh-seed, idempotent-reseed, and no-clobber-of-existing-.oh/; docker-compose.image-only.yml mounts \${OH_HOME_MOUNT:-workspace} at /home/sandbox, carries no OH_IMAGE_ONLY, parameterizes image:/pull_policy:, and has no build:/'..:' bind mount; primary docker-compose.yml still binds '..:' (regression floor); Dockerfile stages /opt/oh-seed" >&2 exit 0 diff --git a/.oh/evals/probes/sandbox-boot-guard-ci.sh b/.oh/evals/probes/sandbox-boot-guard-ci.sh index 9d20f969..f8b9bedd 100755 --- a/.oh/evals/probes/sandbox-boot-guard-ci.sh +++ b/.oh/evals/probes/sandbox-boot-guard-ci.sh @@ -43,7 +43,11 @@ has '".dockerignore"' "dockerignore path filter" has '".github/workflows/sandbox-boot-guard.yml"' "workflow self path filter" has 'persist-credentials: false' "checkout token persistence disabled" has 'bash .oh/scripts/docker-compose.sh config --quiet' "base compose config validation" -has 'HERMES_DASHBOARD: "true"' "Hermes overlay validation env" +has 'SANDBOX_SSH: "true"' "sshd overlay validation env" +has "jq '.build.skipPnpmInstall = true' oh.json" "pre-seeded deps opt out through oh.json, not a compose env knob" +if grep -Fq 'SKIP_PNPM_INSTALL' <<<"$text"; then + missing+=("the boot guard sets SKIP_PNPM_INSTALL — the opt-out lives in oh.json and entrypoint.sh reads it through the CLI") +fi has 'docker build \' "local docker build step" has '--file .devcontainer/Dockerfile' "devcontainer Dockerfile build target" has '--tag openharness-sandbox-boot-guard:${{ github.sha }}' "local CI image tag" diff --git a/.oh/evals/probes/skills-vendored.sh b/.oh/evals/probes/skills-vendored.sh index 1b6a2b5b..151d8045 100755 --- a/.oh/evals/probes/skills-vendored.sh +++ b/.oh/evals/probes/skills-vendored.sh @@ -3,7 +3,9 @@ # source: absorb .mifune submodule into .oh — the skills/agents/hooks pack is vendored # directly under .oh/ (no submodule); provider symlinks resolve into it from a clean clone # desc: there is NO .mifune submodule; .oh/skills|agents|hooks are tracked in-repo and the -# provider/Hermes symlinks resolve into .oh/ with no init/network step +# provider symlinks resolve into .oh/ with no init/network step; the Hermes link is +# created when the hermes binary is on PATH and not otherwise (#920 replaced the +# INSTALL_HERMES flag with that presence check, so it works in both sandbox flavors) set -euo pipefail ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" @@ -34,7 +36,7 @@ for link in .pi/skills .claude/skills .codex/skills .claude/agents .claude/hooks [ -e "$link" ] || fail "$link target does not resolve" done -INSTALL_HERMES=false bash .oh/scripts/link-providers.sh --check >/dev/null +bash .oh/scripts/link-providers.sh --check >/dev/null if [ "${SKILLS_VENDORED_SKIP_CLEAN_CLONE:-0}" != "1" ]; then tmp="$(mktemp -d)" @@ -46,10 +48,20 @@ if [ "${SKILLS_VENDORED_SKIP_CLEAN_CLONE:-0}" != "1" ]; then [ -f .claude/skills/spec/SKILL.md ] || fail "Claude skill symlink does not resolve in a clean clone" [ -f .codex/skills/git/SKILL.md ] || fail "Codex skill symlink does not resolve in a clean clone" [ -f .prime/agent/skills/git/SKILL.md ] || fail "prime-agent skill symlink does not resolve in a clean clone" - INSTALL_HERMES=false bash .oh/scripts/link-providers.sh --check >/dev/null - INSTALL_HERMES=false bash .oh/scripts/link-providers.sh --init >/dev/null - INSTALL_HERMES=true bash .oh/scripts/link-providers.sh --init >/dev/null - [ -f .hermes/skills/openharness/git/SKILL.md ] || fail "Hermes skill symlink missing after INSTALL_HERMES init" + fake_bin="$tmp/bin" + mkdir -p "$fake_bin" + bare_path="$fake_bin:/usr/bin:/bin" + + PATH="$bare_path" bash .oh/scripts/link-providers.sh --check >/dev/null + PATH="$bare_path" bash .oh/scripts/link-providers.sh --init >/dev/null + [ ! -e .hermes/skills/openharness ] \ + || fail "Hermes skill symlink created with no hermes binary on PATH — the wiring must key off the binary" + + printf '#!/bin/sh\nexit 0\n' > "$fake_bin/hermes" + chmod +x "$fake_bin/hermes" + PATH="$bare_path" bash .oh/scripts/link-providers.sh --init >/dev/null + [ -f .hermes/skills/openharness/git/SKILL.md ] \ + || fail "Hermes skill symlink missing after an init with hermes on PATH" cd "$ROOT" fi diff --git a/.oh/evals/probes/tailscale-tool-boundary.sh b/.oh/evals/probes/tailscale-tool-boundary.sh index 66c78f96..b4877175 100755 --- a/.oh/evals/probes/tailscale-tool-boundary.sh +++ b/.oh/evals/probes/tailscale-tool-boundary.sh @@ -9,11 +9,13 @@ # #908 additionally proved a root-installed tool is unusable from inside the # sandbox: commands/tool.ts uses stdio:"inherit", so a root install becomes an # interactive `sudo` and /etc/sudoers.d/sandbox has no NOPASSWD. +# #920 removed the duplicate boot-path installer: the tool catalog is now the +# only place the version and both checksums may appear. # desc: the Tailscale optional tool stays a zero-privilege, zero-exposure install — -# entrypointGuard (not buildArg) ground truth, version and both sha256 pins -# agreeing between the entrypoint and the tool catalog, no cap_add/devices/ -# privileged/3773 in any compose file, no tailscaled or `tailscale up` on boot, -# no Funnel, no committed auth key. +# tools/catalog.ts is the sole owner of the version and both sha256 pins, the +# entrypoint holds neither a guard nor a pin, no cap_add/devices/privileged/3773 +# in any compose file, no tailscaled or `tailscale up` on boot, no Funnel, no +# committed auth key. set -euo pipefail ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)"; cd "$ROOT" @@ -35,29 +37,29 @@ fi missing=() grep -qF 'INSTALL_TAILSCALE' "$ENTRY" \ - || missing+=("$ENTRY: no INSTALL_TAILSCALE guard — the tool catalog's ground truth moved") + && missing+=("$ENTRY: INSTALL_TAILSCALE guard returned — the install belongs to the tool catalog, reached through provision-defaults.sh from oh.json") grep -qF 'INSTALL_TAILSCALE' "$DOCKERFILE" \ - && missing+=("$DOCKERFILE: INSTALL_TAILSCALE appeared — a Dockerfile guard means the catalog field must be buildArg, not entrypointGuard") + && missing+=("$DOCKERFILE: INSTALL_TAILSCALE appeared — an image-layer install is discarded on every container recreate") -mapfile -t pins < <(grep -oE 'tailscale_[0-9]+\.[0-9]+\.[0-9]+_' "$ENTRY" | sed 's/^tailscale_//; s/_$//' | sort -u) +mapfile -t pins < <(grep -oE 'tailscale_[0-9]+\.[0-9]+\.[0-9]+_' "$CATALOG" | sed 's/^tailscale_//; s/_$//' | sort -u) if ((${#pins[@]} == 0)); then - missing+=("$ENTRY: no pinned tailscale__ tarball — the install is unpinned") + missing+=("$CATALOG: no pinned tailscale__ tarball — the install is unpinned") elif ((${#pins[@]} > 1)); then - missing+=("$ENTRY: per-architecture version pins disagree (${pins[*]})") -else - grep -qF "tailscale_${pins[0]}_" "$CATALOG" \ - || missing+=("$CATALOG: version pin disagrees with $ENTRY (${pins[0]})") + missing+=("$CATALOG: per-architecture version pins disagree (${pins[*]})") fi +grep -qE 'tailscale_[0-9]+\.[0-9]+\.[0-9]+_' "$ENTRY" \ + && missing+=("$ENTRY: pins a Tailscale version — a second copy of the pin drifts from $CATALOG") -grep -qF 'sha256sum -c' "$ENTRY" \ - || missing+=("$ENTRY: no 'sha256sum -c' verification of the Tailscale tarball") +grep -qF 'sha256sum -c' "$CATALOG" \ + || missing+=("$CATALOG: no 'sha256sum -c' verification of the Tailscale tarball") mapfile -t entry_shas < <(grep -iE 'tailscale|ts_sha' "$ENTRY" | grep -oE '\b[0-9a-f]{64}\b' | sort -u) mapfile -t catalog_shas < <(grep -iE 'tailscale' "$CATALOG" | grep -oE '\b[0-9a-f]{64}\b' | sort -u) -if ((${#entry_shas[@]} < 2)); then - missing+=("$ENTRY: expected a sha256 literal per supported architecture, found ${#entry_shas[@]}") -elif [ "${entry_shas[*]}" != "${catalog_shas[*]}" ]; then - missing+=("$CATALOG: sha256 literals disagree with $ENTRY (entrypoint: ${entry_shas[*]:-none} / catalog: ${catalog_shas[*]:-none})") +if ((${#catalog_shas[@]} < 2)); then + missing+=("$CATALOG: expected a sha256 literal per supported architecture, found ${#catalog_shas[@]}") +fi +if ((${#entry_shas[@]} > 0)); then + missing+=("$ENTRY: carries a Tailscale sha256 literal (${entry_shas[*]}) — $CATALOG is the only place it may appear") fi if grep -qE '(^|[;&|]|&&|\|\||\bthen |\bdo |\bexec |\bnohup |\bsudo )[[:space:]]*("?[^[:space:]"]*/)?tailscaled\b' "$ENTRY"; then @@ -101,8 +103,8 @@ else || missing+=("$CATALOG: the tailscale entry is not kind \"opt-in\" — it must never install by default") grep -qE 'toolKey:[[:space:]]*"tailscale"' <<<"$entry_block" \ || missing+=("$CATALOG: the tailscale entry has no toolKey \"tailscale\" — the oh.json opt-in is not wired") - grep -qE 'entrypointGuard:[[:space:]]*"INSTALL_TAILSCALE"' <<<"$entry_block" \ - || missing+=("$CATALOG: the tailscale entry has no entrypointGuard \"INSTALL_TAILSCALE\"") + grep -qF 'entrypointGuard' <<<"$entry_block" \ + && missing+=("$CATALOG: the tailscale entry declares an entrypointGuard — it records a second installer that must not exist") # tailscaled runs fine unprivileged with --tun=userspace-networking, so nothing # here needs root. A root install would hang `oh tool install tailscale` on a # sudo password prompt no agent can answer, and would put the binary in an @@ -130,9 +132,9 @@ socket_dir_line=$(grep -nE 'install -d .*-o sandbox .*/var/run/tailscale' "$ENTR if [ -z "$socket_dir_line" ]; then missing+=("$ENTRY: never creates /var/run/tailscale — tailscaled's default socket path is unwritable, so a bare \`tailscale status\` cannot work") else - guard_line=$(grep -nE '^if \[ "\$\{INSTALL_TAILSCALE:-false\}" = "true" \]' "$ENTRY" | head -1 | cut -d: -f1) - if [ -n "$guard_line" ] && [ "$socket_dir_line" -gt "$guard_line" ]; then - missing+=("$ENTRY: creates /var/run/tailscale inside the INSTALL_TAILSCALE guard — a later \`oh tool install tailscale\` would then need a reboot before the socket path exists") + socket_dir_text=$(sed -n "${socket_dir_line}p" "$ENTRY") + if [[ $socket_dir_text == [[:space:]]* ]]; then + missing+=("$ENTRY: creates /var/run/tailscale inside a conditional block — a later \`oh tool install tailscale\` would then need a reboot before the socket path exists") fi fi diff --git a/.oh/evals/probes/tool-catalog-boundary.sh b/.oh/evals/probes/tool-catalog-boundary.sh index a13aa302..654b9250 100755 --- a/.oh/evals/probes/tool-catalog-boundary.sh +++ b/.oh/evals/probes/tool-catalog-boundary.sh @@ -1,10 +1,13 @@ #!/usr/bin/env bash # tier: A -# source: agent-browser's exclusion from the harness catalog (#821) and the -# three-catalog split introduced with `oh tool` -# desc: the harness/runtime/tool catalogs stay disjoint; agent-browser's ground truth stays -# .devcontainer/entrypoint.sh and NOT the Dockerfile; the ~1 GB download stays gated -# and fails closed without --yes. +# source: agent-browser's exclusion from the harness catalog (#821), the three-catalog +# split introduced with `oh tool`, and #920 — the CLI is the only install +# surface, so a second installer on the boot path is a second unverified +# description of the same pin. +# desc: the harness/runtime/tool catalogs stay disjoint; agent-browser's ground truth is +# tools/catalog.ts alone — no guard and no pin in .devcontainer/entrypoint.sh, the +# Dockerfile, or compose; the ~1 GB download stays gated and fails closed without +# --yes. set -euo pipefail ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" @@ -29,24 +32,25 @@ strip_comments() { perl -0pe 's{/\*.*?\*/}{}gs; s{(^|[^:])//[^\n]*}{$1}gm' "$1" } -if ! grep -qF 'INSTALL_AGENT_BROWSER' "$ENTRY"; then - missing+=("entrypoint.sh: no INSTALL_AGENT_BROWSER guard — the tool catalog's ground truth moved") +if grep -qF 'INSTALL_AGENT_BROWSER' "$ENTRY"; then + missing+=("entrypoint.sh: INSTALL_AGENT_BROWSER guard returned — the install belongs to the tool catalog, reached through provision-defaults.sh from oh.json") fi if grep -qF 'INSTALL_AGENT_BROWSER' "$DOCKERFILE"; then - missing+=("Dockerfile: INSTALL_AGENT_BROWSER appeared — agent-browser now has a build arg, so buildArg is the right field, not entrypointGuard") + missing+=("Dockerfile: INSTALL_AGENT_BROWSER appeared — an image-layer install is discarded on every container recreate") fi -if ! grep -qF 'entrypointGuard' "$TOOLS"; then - missing+=("tools/catalog.ts: no entrypointGuard field — the entrypoint install shape is unrecorded") +if grep -qF 'entrypointGuard' "$TOOLS"; then + missing+=("tools/catalog.ts: entrypointGuard returned — it records a second installer that must not exist") fi if grep -qE '\bbuildArg\b' <<<"$(strip_comments "$TOOLS")"; then missing+=("tools/catalog.ts: uses buildArg — that field carries a Dockerfile invariant this catalog cannot satisfy") fi -pin=$(grep -oE 'agent-browser@[0-9]+\.[0-9]+\.[0-9]+' "$ENTRY" | head -1 || true) +pin=$(grep -oE 'agent-browser@[0-9]+\.[0-9]+\.[0-9]+' "$TOOLS" | head -1 || true) if [[ -z $pin ]]; then - missing+=("entrypoint.sh: no pinned agent-browser version found") -elif ! grep -qF "$pin" "$TOOLS"; then - missing+=("tools/catalog.ts: version pin disagrees with entrypoint.sh ($pin)") + missing+=("tools/catalog.ts: no pinned agent-browser version found — the catalog is the only place that may hold it") +fi +if grep -qE 'agent-browser@[0-9]+\.[0-9]+\.[0-9]+' "$ENTRY"; then + missing+=("entrypoint.sh: pins an agent-browser version — a second copy of the pin drifts from tools/catalog.ts") fi if grep -qE 'harnessKey: *"agent_browser"' "$HARNESSES"; then diff --git a/.oh/install/banner.sh b/.oh/install/banner.sh index fdf45196..eb45879a 100755 --- a/.oh/install/banner.sh +++ b/.oh/install/banner.sh @@ -122,17 +122,23 @@ fi dashboard_status="" dashboard_detail="" if command -v hermes >/dev/null 2>&1; then - if echo "${HERMES_DASHBOARD:-}" | grep -qiE '^(true|1|yes|on)$'; then + dashboard_enabled="" + dashboard_port=9119 + if command -v jq >/dev/null 2>&1 && [ -f "$project_dir/oh.json" ]; then + dashboard_enabled="$(jq -r '.hermesDashboard.enabled // false' "$project_dir/oh.json" 2>/dev/null)" + dashboard_port="$(jq -r '.hermesDashboard.port // 9119' "$project_dir/oh.json" 2>/dev/null)" + fi + if echo "${dashboard_enabled:-}" | grep -qiE '^(true|1|yes|on)$'; then if tmux has-session -t app-hermes-dashboard 2>/dev/null; then dashboard_status="$status_ok" - dashboard_detail="dashboard — http://127.0.0.1:${HERMES_DASHBOARD_PORT:-9119}" + dashboard_detail="dashboard — http://127.0.0.1:${dashboard_port}" else dashboard_status="$status_x" dashboard_detail="dashboard — enabled but not running (see /tmp/app-hermes-dashboard.log)" fi else dashboard_status="$status_empty" - dashboard_detail="dashboard — disabled (set hermes.dashboard: true)" + dashboard_detail="dashboard — disabled (oh config set hermesDashboard.enabled true)" fi fi diff --git a/.oh/scripts/__tests__/boot-banner.test.ts b/.oh/scripts/__tests__/boot-banner.test.ts index cd208217..972bf5b0 100644 --- a/.oh/scripts/__tests__/boot-banner.test.ts +++ b/.oh/scripts/__tests__/boot-banner.test.ts @@ -13,7 +13,7 @@ function readRepoFile(...parts: string[]): string { function firstBootBlock(entrypoint: string): string { const match = entrypoint.match( - /# First-boot message if onboarding not complete[\s\S]*?\nfi\n/, + /if \[ ! -f "\/home\/sandbox\/\.claude\/\.onboarded" \]; then[\s\S]*?\nfi\n/, ); expect(match, "first-boot banner block should be present").not.toBeNull(); return match?.[0] ?? ""; diff --git a/.oh/scripts/__tests__/compose-args.test.ts b/.oh/scripts/__tests__/compose-args.test.ts index 305bdf20..cb60b499 100644 --- a/.oh/scripts/__tests__/compose-args.test.ts +++ b/.oh/scripts/__tests__/compose-args.test.ts @@ -28,7 +28,7 @@ beforeEach(() => { tmp = mkdtempSync(path.join(tmpdir(), "compose-args-")); mkdirSync(path.join(tmp, ".devcontainer"), { recursive: true }); writeFileSync(path.join(tmp, ".devcontainer", "docker-compose.yml"), "services: {}\n"); - writeFileSync(path.join(tmp, ".devcontainer", "docker-compose.hermes-dashboard.yml"), "services: {}\n"); + writeFileSync(path.join(tmp, ".devcontainer", "docker-compose.ssh.yml"), "services: {}\n"); }); afterEach(() => { @@ -107,7 +107,7 @@ describe("scripts/docker-compose.sh", () => { const hostile = `over rides/config ; touch ${sentinel}.yml`; const substitution = "local config/override $(printf hacked).yml"; - writeFileSync(path.join(tmp, ".devcontainer", ".env"), "HERMES_DASHBOARD=true\n"); + writeFileSync(path.join(tmp, ".devcontainer", ".env"), "SANDBOX_SSH=true\n"); writeFileSync( path.join(tmp, "config.json"), JSON.stringify({ composeOverrides: [hostile, "overlays/config-two.yml", substitution] }), @@ -122,7 +122,7 @@ describe("scripts/docker-compose.sh", () => { "-f", path.join(tmp, ".devcontainer", "docker-compose.yml"), "-f", - path.join(tmp, ".devcontainer", "docker-compose.hermes-dashboard.yml"), + path.join(tmp, ".devcontainer", "docker-compose.ssh.yml"), "-f", path.join(tmp, hostile), "-f", @@ -269,7 +269,7 @@ describe("scripts/docker-compose.sh --extra-env-file (issue #880)", () => { it("selects an overlay from the rendered file even when the dotenv is silent", () => { writeFileSync(path.join(tmp, ".env"), "GH_TOKEN=secret\n"); const file = path.join(tmp, "rendered.list"); - writeFileSync(file, "HERMES_DASHBOARD=true\n"); + writeFileSync(file, "SANDBOX_SSH=true\n"); const result = spawnSync( "bash", @@ -278,7 +278,7 @@ describe("scripts/docker-compose.sh --extra-env-file (issue #880)", () => { ); expect(result.status).toBe(0); expect(result.stdout).toContain( - path.join(tmp, ".devcontainer", "docker-compose.hermes-dashboard.yml"), + path.join(tmp, ".devcontainer", "docker-compose.ssh.yml"), ); }); diff --git a/.oh/scripts/__tests__/cron-runtime.test.ts b/.oh/scripts/__tests__/cron-runtime.test.ts index 21085b1d..0cc9ec04 100644 --- a/.oh/scripts/__tests__/cron-runtime.test.ts +++ b/.oh/scripts/__tests__/cron-runtime.test.ts @@ -35,6 +35,8 @@ import { readFailureTail, reloadEntryForFire, remoteForRepo, + resetAgentBinCache, + resolveAgentBin, reloadBody, resetActiveJobs, runPreflight, @@ -226,6 +228,43 @@ describe("isValidCronId", () => { }); }); +describe("resolveAgentBin", () => { + const savedBin = process.env.CRON_AGENT_BIN; + const savedPath = process.env.PATH; + + afterEach(() => { + if (savedBin === undefined) delete process.env.CRON_AGENT_BIN; + else process.env.CRON_AGENT_BIN = savedBin; + process.env.PATH = savedPath; + resetAgentBinCache(); + }); + + it("prefers the CRON_AGENT_BIN override over oh.json", () => { + process.env.CRON_AGENT_BIN = "pi"; + resetAgentBinCache(); + expect(resolveAgentBin()).toBe("pi"); + }); + + it("reads cron.agentBin from oh.json through the CLI", () => { + delete process.env.CRON_AGENT_BIN; + const dir = mkdtempSync(path.join(tmpdir(), "cron-agent-bin-")); + const stub = path.join(dir, "oh"); + writeFileSync(stub, '#!/usr/bin/env bash\necho \'{"cron":{"agentBin":"codex"}}\'\n'); + chmodSync(stub, 0o755); + process.env.PATH = `${dir}:/usr/bin:/bin`; + resetAgentBinCache(); + expect(resolveAgentBin()).toBe("codex"); + rmSync(dir, { recursive: true, force: true }); + }); + + it("falls back to claude when the CLI is unavailable", () => { + delete process.env.CRON_AGENT_BIN; + process.env.PATH = "/nonexistent-path-for-cron-agent-bin"; + resetAgentBinCache(); + expect(resolveAgentBin()).toBe("claude"); + }); +}); + describe("isValidAgentBin", () => { it("accepts safe executable tokens and paths", () => { for (const agent of ["claude", "pi", "codex", "opencode", "/usr/local/bin/claude", "./bin/pi-agent"]) { diff --git a/.oh/scripts/__tests__/docs-compose-overlays.test.ts b/.oh/scripts/__tests__/docs-compose-overlays.test.ts index 0e8427e7..1ab38e57 100644 --- a/.oh/scripts/__tests__/docs-compose-overlays.test.ts +++ b/.oh/scripts/__tests__/docs-compose-overlays.test.ts @@ -9,7 +9,12 @@ const REPO_ROOT = path.resolve(__dirname, "../../.."); const DOCS_ROOT = path.join(REPO_ROOT, "docs"); const MARKDOWN_ROOT_FILES = [path.join(REPO_ROOT, "README.md")]; +// RFCs and preserved rationale are dated records of past decisions, not guidance: +// rewriting a compose path inside them would falsify the history they exist to keep. +const HISTORICAL_ROOTS = [path.join(DOCS_ROOT, "rfcs")]; + function markdownFiles(dir: string): string[] { + if (HISTORICAL_ROOTS.includes(dir)) return []; return readdirSync(dir, { withFileTypes: true }).flatMap((entry) => { const full = path.join(dir, entry.name); if (entry.isDirectory()) return markdownFiles(full); diff --git a/.oh/scripts/__tests__/entrypoint-pnpm-install.test.ts b/.oh/scripts/__tests__/entrypoint-pnpm-install.test.ts index 46c2cec9..ba9e32ce 100644 --- a/.oh/scripts/__tests__/entrypoint-pnpm-install.test.ts +++ b/.oh/scripts/__tests__/entrypoint-pnpm-install.test.ts @@ -13,13 +13,13 @@ const entrypoint = readFileSync(ENTRYPOINT, "utf-8"); const compose = readFileSync(COMPOSE, "utf-8"); describe("devcontainer entrypoint pnpm install", () => { - it("keeps the explicit SKIP_PNPM_INSTALL opt-out", () => { - expect(entrypoint).toContain("SKIP_PNPM_INSTALL=1"); - expect(entrypoint).toContain('${SKIP_PNPM_INSTALL:-0}'); + it("reads the opt-out from oh.json through the CLI, not from the environment", () => { + expect(entrypoint).toContain("oh_config_truthy '.build.skipPnpmInstall'"); + expect(entrypoint).not.toContain("SKIP_PNPM_INSTALL"); }); - it("passes the SKIP_PNPM_INSTALL opt-out through compose", () => { - expect(compose).toContain("SKIP_PNPM_INSTALL=${SKIP_PNPM_INSTALL:-0}"); + it("keeps the opt-out out of compose", () => { + expect(compose).not.toContain("SKIP_PNPM_INSTALL"); }); it("uses an Open Harness marker stored under node_modules", () => { diff --git a/.oh/scripts/__tests__/entrypoint.test.ts b/.oh/scripts/__tests__/entrypoint.test.ts index 94103503..66db2a2e 100644 --- a/.oh/scripts/__tests__/entrypoint.test.ts +++ b/.oh/scripts/__tests__/entrypoint.test.ts @@ -48,13 +48,13 @@ describe("devcontainer entrypoint home mount ownership", () => { it("runs home mount repair after host UID reconciliation", () => { const text = entrypoint(); const uidSync = text.indexOf("usermod -u \"$HOST_UID\" sandbox"); - const secondRepair = text.indexOf("# UID/GID reconciliation can change"); + const secondRepair = text.indexOf('PW="${SANDBOX_PASSWORD:-test1234}"'); expect(secondRepair).toBeGreaterThan(uidSync); const postUidSync = text.slice(secondRepair); const secondRepairCall = postUidSync.indexOf("repair_home_mount_ownership"); const linkProviders = postUidSync.indexOf('bash "$HARNESS/.oh/scripts/link-providers.sh" --init'); - const hermesBlock = postUidSync.indexOf("# Hermes keeps all runtime state"); + const hermesBlock = postUidSync.indexOf('if command -v hermes >/dev/null 2>&1; then'); expect(secondRepairCall).toBeGreaterThan(-1); expect(linkProviders).toBeGreaterThan(secondRepairCall); expect(hermesBlock).toBeGreaterThan(linkProviders); @@ -63,13 +63,16 @@ describe("devcontainer entrypoint home mount ownership", () => { it("does not swallow host UID reconciliation failures", () => { const text = entrypoint(); const block = text.slice( - text.indexOf("# ─── Host UID reconciliation"), - text.indexOf("# UID/GID reconciliation can change"), + text.indexOf("uid_reconcile_step() {"), + text.indexOf('PW="${SANDBOX_PASSWORD:-test1234}"'), ); expect(block).toContain("uid_reconcile_step()"); expect(block).toContain("WARNING: failed to"); - const reconBranch = block.slice(block.indexOf('elif [ -d "$HARNESS_DIR" ]')); + const reconBranch = block.slice( + block.indexOf('if mountpoint -q "$HARNESS_DIR" 2>/dev/null && [ -d "$HARNESS_DIR/.oh" ]; then'), + block.indexOf('echo "[entrypoint] no checkout bind at $HARNESS_DIR'), + ); expect(reconBranch).not.toContain("2>/dev/null || true"); expect(reconBranch).not.toContain("groupmod -g \"$HOST_GID\" sandbox 2>/dev/null"); expect(reconBranch).not.toContain("usermod -u \"$HOST_UID\" sandbox 2>/dev/null"); @@ -78,8 +81,8 @@ describe("devcontainer entrypoint home mount ownership", () => { it("prints UID sync success only after reconciliation commands report success", () => { const text = entrypoint(); const block = text.slice( - text.indexOf("# ─── Host UID reconciliation"), - text.indexOf("# UID/GID reconciliation can change"), + text.indexOf("uid_reconcile_step() {"), + text.indexOf('PW="${SANDBOX_PASSWORD:-test1234}"'), ); const usermod = block.indexOf("uid_reconcile_step \"set sandbox UID to host UID $HOST_UID\" usermod -u \"$HOST_UID\" sandbox"); const success = block.indexOf("sandbox UID synced to host"); diff --git a/.oh/scripts/__tests__/sandbox-healthcheck.test.ts b/.oh/scripts/__tests__/sandbox-healthcheck.test.ts index d68b167f..51c2f0fb 100644 --- a/.oh/scripts/__tests__/sandbox-healthcheck.test.ts +++ b/.oh/scripts/__tests__/sandbox-healthcheck.test.ts @@ -87,17 +87,20 @@ describe("sandbox healthcheck", () => { expect(result.stderr).toContain("legacy tmux session present: system-cron"); }); - it("checks optional Hermes dashboard only when enabled and installed", () => { + it("checks optional Hermes dashboard only when enabled in oh.json and installed", () => { const { bin, harness, tmux } = fixture(); const hermes = join(bin, "hermes"); writeFileSync(hermes, "#!/usr/bin/env bash\nexit 0\n"); chmodSync(hermes, 0o755); + writeFileSync( + join(harness, "oh.json"), + `${JSON.stringify({ version: 1, name: "demo", hermesDashboard: { enabled: true } })}\n`, + ); const result = runHealthcheck({ HARNESS: harness, TMUX_BIN: tmux, HERMES_BIN: hermes, - HERMES_DASHBOARD: "true", HEALTHCHECK_TMUX_SESSIONS: "cron-watchdog,cron-system", }); diff --git a/.oh/scripts/cron-runtime.ts b/.oh/scripts/cron-runtime.ts index 70d09951..bfc45c10 100644 --- a/.oh/scripts/cron-runtime.ts +++ b/.oh/scripts/cron-runtime.ts @@ -23,12 +23,41 @@ const CRONS_DIR = path.resolve("crons"); const WORKTREES_DIR = ".worktrees"; const PID_FILE = path.join(CRONS_DIR, ".pid"); const LOG_FILE = path.join(CRONS_DIR, ".cron.log"); -const AGENT_BIN = process.env.CRON_AGENT_BIN || "claude"; +const AGENT_BIN_FALLBACK = "claude"; const CRON_ID_PATTERN = /^[a-z0-9][a-z0-9-]*$/; const AGENT_BIN_PATTERN = /^[A-Za-z0-9_./-]+$/; const REPO_PATTERN = /^[A-Za-z0-9_.-]+\/[A-Za-z0-9_.-]+$/; const REMOTE_PATTERN = /^[A-Za-z0-9_.-]+$/; +let resolvedAgentBin: string | undefined; + +export function resolveAgentBin(): string { + if (resolvedAgentBin !== undefined) return resolvedAgentBin; + const fromEnv = process.env.CRON_AGENT_BIN; + if (fromEnv) { + resolvedAgentBin = fromEnv; + return resolvedAgentBin; + } + const shown = spawnSync("oh", ["config", "show"], { encoding: "utf8" }); + if (shown.status === 0 && shown.stdout) { + try { + const parsed = JSON.parse(shown.stdout) as { cron?: { agentBin?: string } }; + if (parsed.cron?.agentBin) { + resolvedAgentBin = parsed.cron.agentBin; + return resolvedAgentBin; + } + } catch { + resolvedAgentBin = undefined; + } + } + resolvedAgentBin = AGENT_BIN_FALLBACK; + return resolvedAgentBin; +} + +export function resetAgentBinCache(): void { + resolvedAgentBin = undefined; +} + export function isValidCronId(id: string): boolean { return CRON_ID_PATTERN.test(id); } @@ -672,7 +701,7 @@ function fireTmux(entry: CronEntry): void { let cwd = process.cwd(); let pidFile = idPidFile; let worktree: string | undefined; - const agentBin = entry.agentBin || AGENT_BIN; + const agentBin = entry.agentBin || resolveAgentBin(); if (!isValidAgentBin(agentBin)) { log(entry.id, "AGENT_INVALID", `invalid agent: ${agentBin}`); return; @@ -797,7 +826,7 @@ export function fire(entry: CronEntry): void { const session = tmuxSessionName(liveEntry.id, new Date()); const promptFile = `/tmp/${session}.prompt`; const logFile = `/tmp/${session}.log`; - const agentBin = liveEntry.agentBin || AGENT_BIN; + const agentBin = liveEntry.agentBin || resolveAgentBin(); if (!isValidAgentBin(agentBin)) { log(liveEntry.id, "AGENT_INVALID", `invalid agent: ${agentBin}`); return; diff --git a/.oh/scripts/docker-compose.sh b/.oh/scripts/docker-compose.sh index 68eb8f43..71490dde 100755 --- a/.oh/scripts/docker-compose.sh +++ b/.oh/scripts/docker-compose.sh @@ -127,11 +127,6 @@ fi args+=(-f "$(compose_path ".devcontainer/docker-compose.yml")") -hermes_value=${HERMES_DASHBOARD:-$(read_env_value HERMES_DASHBOARD)} -if truthy "$hermes_value"; then - args+=(-f "$(compose_path ".devcontainer/docker-compose.hermes-dashboard.yml")") -fi - docker_socket_value=${DOCKER_SOCKET:-$(read_env_value DOCKER_SOCKET)} if truthy "$docker_socket_value"; then args+=(-f "$(compose_path ".devcontainer/docker-compose.docker-sock.yml")") diff --git a/.oh/scripts/link-providers.sh b/.oh/scripts/link-providers.sh index 3e68f497..b56497cb 100755 --- a/.oh/scripts/link-providers.sh +++ b/.oh/scripts/link-providers.sh @@ -108,7 +108,7 @@ init_links() { [ -f "$f" ] && chmod +x "$f" done - if [ "${INSTALL_HERMES:-false}" = "true" ]; then + if command -v hermes >/dev/null 2>&1; then link_provider "$HERMES_LINK" "$HERMES_TARGET" || true fi } @@ -129,7 +129,7 @@ check_symlink() { } check_hermes_link() { - if [ "${INSTALL_HERMES:-false}" != "true" ] && [ ! -e "$HERMES_LINK" ] && [ ! -L "$HERMES_LINK" ]; then + if [ ! -e "$HERMES_LINK" ] && [ ! -L "$HERMES_LINK" ]; then return 0 fi check_symlink "$HERMES_LINK" "$HERMES_TARGET" diff --git a/.oh/scripts/sandbox-healthcheck.sh b/.oh/scripts/sandbox-healthcheck.sh index fe1268b6..d196ab87 100755 --- a/.oh/scripts/sandbox-healthcheck.sh +++ b/.oh/scripts/sandbox-healthcheck.sh @@ -36,6 +36,16 @@ require_session() { fi } +oh_config_truthy() { + local filter="$1" config="$HARNESS/oh.json" + [ -f "$config" ] || return 1 + command_exists jq || return 1 + case "$(jq -r "$filter // false" "$config" 2>/dev/null | tr '[:upper:]' '[:lower:]')" in + 1|true|yes|on) return 0 ;; + *) return 1 ;; + esac +} + compose_env_value() { local key="$1" local env_file="$HARNESS/.devcontainer/.env" @@ -60,7 +70,7 @@ else require_session cron-system fi - if [ "${HERMES_DASHBOARD:-false}" = "true" ] && command_exists "$HERMES_BIN"; then + if oh_config_truthy '.hermesDashboard.enabled' && command_exists "$HERMES_BIN"; then require_session app-hermes-dashboard fi diff --git a/.oh/skills/wiki/corpus/README.md b/.oh/skills/wiki/corpus/README.md index 791ae501..1241b0ff 100644 --- a/.oh/skills/wiki/corpus/README.md +++ b/.oh/skills/wiki/corpus/README.md @@ -31,6 +31,7 @@ Schema rule, frontmatter spec, and all authoring conventions: `.oh/skills/wiki/r | Slug | Title | Tags | Updated | | --- | --- | --- | --- | | wikiskill-experience-compilation | WikiSkill: Compiling Agent Experience into Persistent Knowledge (arXiv 2608.27454) | [skill-evolution, persistent-knowledge, continual-learning, harness-evolution, self-improvement, wiki, ablation, skill-transfer] | 2026-08-31 | +| sandbox-dependency-installs | Sandbox Dependency Installs | [sandbox, devcontainer, pnpm, dependencies, boot] | 2026-08-31 | | recursive-self-improvement-survey | Recursive Self-Improvement in AI (survey, arXiv 2607.07663) | [rsi, self-improvement, verification-hierarchy, skill-libraries, harness-evolution, self-evaluation, model-collapse, capability-benchmark] | 2026-08-31 | | recursive-language-models | Recursive Language Models | [rlm, context-as-environment, weighted-trajectories, agent-harness, llm-agents, self-consistency] | 2026-08-31 | | pattern-wiki-ungated-check-drift | Report-only checks nothing gates on stop being run | [wiki, lint, evals, probes, report-only, drift, gating] | 2026-08-31 | @@ -38,6 +39,7 @@ Schema rule, frontmatter spec, and all authoring conventions: `.oh/skills/wiki/r | pattern-evals-unexercised-oracle | A probe that has never failed has an unverified oracle | [evals, probes, oracles, skipped, fault-injection, continual-learning] | 2026-08-31 | | pattern-evals-prose-literal-pinning | Contract probes that pin multi-word prose break on reflow, not on drift | [evals, probes, contract-text, grep, false-failure, documentation] | 2026-08-31 | | pattern-docs-prohibition-by-example | Documenting a forbidden literal by quoting it violates the rule | [docs, evals, probes, vocabulary, guards, self-reference] | 2026-08-31 | +| compose-env-boundary | Compose Environment Boundary | [compose, devcontainer, oh-json, cli, entrypoint, boundary, provisioning, sandbox] | 2026-08-31 | | runtime-isolation-landscape | Runtime Isolation Landscape (2026) | [runtime, isolation, sandbox, gvisor, firecracker, kata, microvm, cloudflare, e2b, daytona, fly, modal] | 2026-08-27 | | prime-agent-harness | Prime Agent Harness Surface | [harness, provider-surface, prime-agent, skills, symlinks, oh-cli, on-demand] | 2026-08-27 | | oh-cli-portable-lifecycle | oh CLI Portable Lifecycle | [cli, oh, lifecycle, standalone, init, sandbox, remote-fetch, execution-target] | 2026-08-27 | @@ -49,4 +51,3 @@ Schema rule, frontmatter spec, and all authoring conventions: `.oh/skills/wiki/r | release-versioning | Release Versioning | [release, versioning, semver, calver, github-actions, ghcr, tags, workflow, package-json] | 2026-08-23 | | audit-architecture | Audit Architecture | [audit, pr, workflow, safety, observability] | 2026-08-12 | | document-ingestion | Local Document Ingestion | [wiki, ingestion, markitdown, documents, provenance, security] | 2026-07-18 | -| sandbox-dependency-installs | Sandbox Dependency Installs | [sandbox, devcontainer, pnpm, dependencies, boot] | 2026-07-01 | diff --git a/.oh/skills/wiki/corpus/compose-env-boundary.md b/.oh/skills/wiki/corpus/compose-env-boundary.md new file mode 100644 index 00000000..a4e17153 --- /dev/null +++ b/.oh/skills/wiki/corpus/compose-env-boundary.md @@ -0,0 +1,61 @@ +--- +title: "Compose Environment Boundary" +slug: compose-env-boundary +tags: [compose, devcontainer, oh-json, cli, entrypoint, boundary, provisioning, sandbox] +created: 2026-08-31 +updated: 2026-08-31 +sources: + - .devcontainer/docker-compose.yml + - .devcontainer/docker-compose.image-only.yml + - .devcontainer/entrypoint.sh + - .oh/cli/src/lib/config-render.ts + - .oh/scripts/provision-defaults.sh + - .oh/evals/probes/compose-env-boundary.sh +related: [sandbox-dependency-installs, oh-cli-portable-lifecycle] +confidence: confirmed +--- + +# Compose Environment Boundary + +## Relevant Source Files +- `.devcontainer/docker-compose.yml` — the base compose file; its `environment:` block is the surface this page constrains. +- `.devcontainer/docker-compose.image-only.yml` — flavor B (no checkout bind); its `environment:` block is byte-identical to flavor A's. +- `.devcontainer/entrypoint.sh` — holds the `oh_config` / `oh_config_truthy` helpers that read oh.json through the CLI at boot. +- `.oh/cli/src/lib/config-render.ts` — renders the host-side subset into `.devcontainer/.env`, and refuses to render anything in `RETIRED_KEYS`. +- `.oh/scripts/provision-defaults.sh` — installs harnesses and tools from the catalogs, keyed on oh.json rather than the environment. +- `.oh/evals/probes/compose-env-boundary.sh` — the tier-A probe that enforces the rule across every compose file and overlay. + +## Summary +A value reaches the sandbox through Compose only if a process **outside** the sandbox — or the entrypoint **before** the control plane is readable — must act on it. Everything else lives in the tracked `oh.json` and is read inside the container through the `oh` CLI. The rule exists because the consumer at the end of the old `oh.json → config-render → .env → compose → entrypoint` pipeline sits in the home mount next to the CLI and can read `oh.json` directly; the hop bought nothing and cost three defects. + +## Detail +Two routes carry configuration into the sandbox. The host-side route renders a fixed set — `SANDBOX_NAME`, `TZ`, `OH_HOME_MOUNT`, `GIT_USER_NAME`, `GIT_USER_EMAIL`, `DOCKER_SOCKET`, `SANDBOX_SSH`, `SANDBOX_SSH_PORT`, `OH_SANDBOX_IMAGE`, `OH_PULL_POLICY` (`.oh/cli/src/lib/config-render.ts:37`, `.oh/cli/src/lib/config-render.ts:49`) — because each selects an overlay, names the project, publishes a port, or is needed before `oh.json` is reachable. The in-container route reads everything else at the moment it is needed: `oh_config` shells `oh config show` once and answers `jq` filters from the cached JSON (`.devcontainer/entrypoint.sh:86`), degrading to a caller-supplied default when the CLI is missing or old. It deliberately uses `config show` rather than a narrower verb, because a baked `oh` in an already-running container can predate a new one and this is the boot path. + +Installs take the second route entirely. `provision-defaults.sh` reads `oh harness list --json` and `oh tool list --json` and installs every entry that is `kind:"default"` or `enabled == true`, where `enabled` is computed from `oh.json` (`.oh/scripts/provision-defaults.sh:129`, `.oh/scripts/provision-defaults.sh:135`). The tool catalog is therefore the sole owner of each pinned version and checksum; `entrypoint.sh` holds none. + +Four compose `environment:` literals survive that no config read can supply: `SANDBOX_PASSWORD` (consumed by the entrypoint's own user setup), `CLAUDE_DANGEROUSLY_SKIP_PERMISSIONS`, `CC_SAFETY_NET_STRICT` and `CC_SAFETY_NET_WORKTREE` (read by third-party binaries that know nothing of `oh.json`), plus the `GH_TOKEN` secret, which `config-render.ts` refuses to render (`.oh/cli/src/lib/config-render.ts:53`). + +Two guards keep the boundary closed. `RETIRED_KEYS` throws if a `put()` for a retired variable is ever re-added (`.oh/cli/src/lib/config-render.ts:56`), and the tier-A probe fails on any `INSTALL_*` key, on `OH_IMAGE_ONLY`, or on any `environment:` key outside the rendered set — across every `docker-compose*.yml` including overlays. Overlay `ports:` and `volumes:` blocks are unrestricted; that payload is the part only Docker can act on. + +Non-goals worth recording so a later reader does not treat them as oversights: flavor B survives, because `/opt/oh-seed` ships regardless and deleting the no-checkout deploy would be a product decision, not a cleanup; `INSTALL_PYTHON_KERNEL` remains, because it is a Dockerfile↔entrypoint duplication rather than a compose one; and every retired `oh.json` field stays settable through `oh config set` — only its `.env` projection is gone. + +## System Relationships +```mermaid +flowchart LR + OH[oh.json - tracked] + OH -->|host-side subset| CR[config-render.ts] + CR --> ENV[.devcontainer/.env] + ENV --> DC[docker-compose.sh --env-file] + DC --> COMPOSE[compose environment:] + COMPOSE --> EP1[entrypoint.sh - pre-control-plane] + OH -->|everything else| CLI[oh CLI in the container] + CLI --> EP2[entrypoint.sh oh_config] + CLI --> PD[provision-defaults.sh] + PD --> CAT[harness + tool catalogs] + PROBE[compose-env-boundary.sh] -.enforces.-> COMPOSE + RK[RETIRED_KEYS] -.throws on.-> CR +``` + +## See Also +- [[sandbox-dependency-installs]] +- [[oh-cli-portable-lifecycle]] diff --git a/.oh/skills/wiki/corpus/sandbox-dependency-installs.md b/.oh/skills/wiki/corpus/sandbox-dependency-installs.md index 504f8d59..0dea5759 100644 --- a/.oh/skills/wiki/corpus/sandbox-dependency-installs.md +++ b/.oh/skills/wiki/corpus/sandbox-dependency-installs.md @@ -3,13 +3,12 @@ title: "Sandbox Dependency Installs" slug: sandbox-dependency-installs tags: [sandbox, devcontainer, pnpm, dependencies, boot] created: 2026-07-01 -updated: 2026-07-01 +updated: 2026-08-31 sources: - .devcontainer/entrypoint.sh - - .devcontainer/docker-compose.yml - .oh/scripts/__tests__/entrypoint-pnpm-install.test.ts - .oh/evals/probes/entrypoint-pnpm-manifest-fingerprint.sh -related: [recursive-language-models] +related: [compose-env-boundary, recursive-language-models] confidence: confirmed --- @@ -17,7 +16,6 @@ confidence: confirmed ## Relevant Source Files - `.devcontainer/entrypoint.sh` — owns devcontainer boot, root dependency install decisions, marker refresh, and skip/failure behavior. -- `.devcontainer/docker-compose.yml` — passes the root install skip flag into the container environment. - `.oh/scripts/__tests__/entrypoint-pnpm-install.test.ts` — Vitest contract coverage for the install gate. - `.oh/evals/probes/entrypoint-pnpm-manifest-fingerprint.sh` — Tier-A eval probe guarding the same boot-path contract. - `.github/workflows/sandbox-boot-guard.yml` — CI boot guard path that may preseed dependencies while using `SKIP_PNPM_INSTALL=1`. @@ -26,13 +24,13 @@ confidence: confirmed The devcontainer root `pnpm install` gate is manifest-aware: it keeps the fast boot path when the installed tree matches current package manifests, but reinstalls when the marker under `node_modules` is missing or stale. The marker exists because the harness checkout is bind-mounted at runtime, so Dockerfile-time installs can be shadowed and a long-lived `node_modules` directory is not enough evidence by itself. ## Detail -The root install block explains the runtime shadowing problem and uses `SKIP_PNPM_INSTALL=1` as the explicit opt-out for externally managed or air-gapped dependency state (`.devcontainer/entrypoint.sh:312`, `.devcontainer/entrypoint.sh:320`). Compose passes that flag into the sandbox container, which lets CI and operators actually activate the entrypoint bypass (`.devcontainer/docker-compose.yml:52`, `.devcontainer/docker-compose.yml:53`). The entrypoint derives workspace package patterns from `pnpm-workspace.yaml`, ignores negated patterns, and does not include undeclared package-local manifests (`.devcontainer/entrypoint.sh:322`, `.devcontainer/entrypoint.sh:338`). Manifest discovery excludes dependency and runtime/vendor paths such as `.git`, `.worktrees`, `node_modules`, `.pi/npm/node_modules`, `.oh/cli/node_modules`, and `.hermes/lsp/node_modules` via the broad `*/node_modules/*` guard (`.devcontainer/entrypoint.sh:367`, `.devcontainer/entrypoint.sh:369`). +The root install block explains the runtime shadowing problem and uses `build.skipPnpmInstall` in `oh.json` as the explicit opt-out for externally managed or air-gapped dependency state. Compose no longer carries that flag: `#920` moved it onto the CLI route, so the entrypoint reads it with `oh_config_truthy '.build.skipPnpmInstall'` at the gate (`.devcontainer/entrypoint.sh:470`) and the CI boot guard sets it by patching `oh.json` (`.github/workflows/sandbox-boot-guard.yml:144`). See [[compose-env-boundary]]. The entrypoint derives workspace package patterns from `pnpm-workspace.yaml`, ignores negated patterns, and does not include undeclared package-local manifests (`.devcontainer/entrypoint.sh:322`, `.devcontainer/entrypoint.sh:338`). Manifest discovery excludes dependency and runtime/vendor paths such as `.git`, `.worktrees`, `node_modules`, `.pi/npm/node_modules`, `.oh/cli/node_modules`, and `.hermes/lsp/node_modules` via the broad `*/node_modules/*` guard (`.devcontainer/entrypoint.sh:367`, `.devcontainer/entrypoint.sh:369`). The fingerprint helper includes existing root `package.json`, `pnpm-lock.yaml`, and `pnpm-workspace.yaml`, plus package manifests from declared workspace package patterns, sorts normalized relative paths bytewise, hashes each file with `sha256sum`, and hashes the ordered manifest list into one final digest (`.devcontainer/entrypoint.sh:410`, `.devcontainer/entrypoint.sh:414`, `.devcontainer/entrypoint.sh:418`, `.devcontainer/entrypoint.sh:420`). The install marker is `.openharness-root-pnpm-manifest.sha256` stored under `$HARNESS/node_modules`, tying the digest to the dependency tree it validates (`.devcontainer/entrypoint.sh:424`, `.devcontainer/entrypoint.sh:425`, `.devcontainer/entrypoint.sh:426`). There are three boot states. Missing `node_modules` runs `pnpm install --prefer-offline`; a missing or mismatched marker logs `manifest drift detected; reinstalling`; a matching marker logs `dependencies current` and skips install (`.devcontainer/entrypoint.sh:430`, `.devcontainer/entrypoint.sh:433`, `.devcontainer/entrypoint.sh:437`, `.devcontainer/entrypoint.sh:441`). After a successful install, the entrypoint recomputes the fingerprint, writes a temp marker beside the final marker, and atomically moves it into place; marker-refresh or install failures still abort boot with `/tmp/pnpm-install.log` diagnostics (`.devcontainer/entrypoint.sh:442`, `.devcontainer/entrypoint.sh:443`, `.devcontainer/entrypoint.sh:444`, `.devcontainer/entrypoint.sh:447`, `.devcontainer/entrypoint.sh:451`). -The Vitest file asserts the compose skip-env pass-through, marker filename and location, the `pnpm_manifest_fingerprint` helper shape, drift reinstall branch, current-dependencies skip branch, atomic refresh, and install-failure abort behavior (`.oh/scripts/__tests__/entrypoint-pnpm-install.test.ts:21`, `.oh/scripts/__tests__/entrypoint-pnpm-install.test.ts:25`, `.oh/scripts/__tests__/entrypoint-pnpm-install.test.ts:30`, `.oh/scripts/__tests__/entrypoint-pnpm-install.test.ts:38`, `.oh/scripts/__tests__/entrypoint-pnpm-install.test.ts:44`, `.oh/scripts/__tests__/entrypoint-pnpm-install.test.ts:49`). The Tier-A probe checks the same contract from the eval suite and returns a regression if any expected source, compose, or test signal disappears (`.oh/evals/probes/entrypoint-pnpm-manifest-fingerprint.sh:37`, `.oh/evals/probes/entrypoint-pnpm-manifest-fingerprint.sh:45`, `.oh/evals/probes/entrypoint-pnpm-manifest-fingerprint.sh:46`, `.oh/evals/probes/entrypoint-pnpm-manifest-fingerprint.sh:59`). +The Vitest file asserts the oh.json opt-out read, marker filename and location, the `pnpm_manifest_fingerprint` helper shape, drift reinstall branch, current-dependencies skip branch, atomic refresh, and install-failure abort behavior (`.oh/scripts/__tests__/entrypoint-pnpm-install.test.ts:21`, `.oh/scripts/__tests__/entrypoint-pnpm-install.test.ts:25`, `.oh/scripts/__tests__/entrypoint-pnpm-install.test.ts:30`, `.oh/scripts/__tests__/entrypoint-pnpm-install.test.ts:38`, `.oh/scripts/__tests__/entrypoint-pnpm-install.test.ts:44`, `.oh/scripts/__tests__/entrypoint-pnpm-install.test.ts:49`). The Tier-A probe checks the same contract from the eval suite and returns a regression if any expected source, compose, or test signal disappears (`.oh/evals/probes/entrypoint-pnpm-manifest-fingerprint.sh:37`, `.oh/evals/probes/entrypoint-pnpm-manifest-fingerprint.sh:45`, `.oh/evals/probes/entrypoint-pnpm-manifest-fingerprint.sh:46`, `.oh/evals/probes/entrypoint-pnpm-manifest-fingerprint.sh:59`). Non-goals remain explicit: this does not change global package installs, delete or prune `node_modules`, alter optional agent-browser installs, change `.oh/cli` package-local installation, or migrate away from pnpm. Safe manual recovery is to remove only the marker file, run root `pnpm install --prefer-offline`, or set `SKIP_PNPM_INSTALL=1` while diagnosing. @@ -51,4 +49,5 @@ flowchart TD ``` ## See Also +- [[compose-env-boundary]] - [[recursive-language-models]] diff --git a/.oh/tasks/compose-env-boundary/eval-result.json b/.oh/tasks/compose-env-boundary/eval-result.json new file mode 100644 index 00000000..136cde29 --- /dev/null +++ b/.oh/tasks/compose-env-boundary/eval-result.json @@ -0,0 +1,7 @@ +{ + "commit": "685a2202643a8e724cb55d8c835c63fa80454087", + "runnerExit": 0, + "ranAt": "2026-08-31T22:55:06Z", + "newRegressions": [], + "preExistingReds": [] +} diff --git a/.oh/tasks/compose-env-boundary/evidence.md b/.oh/tasks/compose-env-boundary/evidence.md new file mode 100644 index 00000000..f0b5fb6f --- /dev/null +++ b/.oh/tasks/compose-env-boundary/evidence.md @@ -0,0 +1,209 @@ +# Evidence — compose-env-boundary (#920, PR #922) + +Branch `task/920-compose-env-boundary`, commits `4425329d` and `531425f5`. +Correlated to the `/eval` run recorded in `eval-result.json` (same HEAD). + +## 0. Why this is better than not doing it + +**Before.** Adding a tool, harness, or setting to a sandbox required editing a compose +file. Eleven settings travelled `oh.json → config-render.ts → the rendered dotenv → +compose environment: → entrypoint.sh` to a consumer sitting in the home mount next to the +`oh` CLI. That hop produced three measurable defects: + +| | Before | After | +|---|---|---| +| Descriptions of the agent-browser and Tailscale installs | **2** (`entrypoint.sh` + `tools/catalog.ts`), with the Tailscale version and **both** sha256 literals duplicated | **1** (`tools/catalog.ts`) | +| Hermes wiring in flavor B (`docker-compose.image-only.yml`) | **dead** — the file never set `INSTALL_HERMES`, so `$HERMES_HOME`, the `auth.json` migration, and the `.hermes/skills/openharness` symlink never ran | runs whenever the binary is present, in **both** flavors | +| Difference between the two compose `environment:` blocks | 15 lines | **0** — `diff` is empty | +| Compose keys not readable from `oh.json` | 21 | **9** (`SANDBOX_NAME`, `SANDBOX_PASSWORD`, `TZ`, `CLAUDE_DANGEROUSLY_SKIP_PERMISSIONS`, two `CC_SAFETY_NET_*`, both `GIT_USER_*`, `GH_TOKEN`) | + +**Cost.** 50 files, +736/−491 lines. `entrypoint.sh` itself is net **−35** (63 added, 98 +removed) — the boot path got smaller, not larger. The added lines are concentrated in +docs (14 files), tests, and two probes. + +**Capability given up, stated plainly:** the Hermes dashboard no longer publishes +`127.0.0.1:9119` to the host. That was the retired overlay's one irreducible job. It is +now reachable from inside the sandbox, or over cloudflared or Tailscale. This is in the +BREAKING changelog entry. + +*Claimed, unmeasured:* that a future tool addition costs one `oh config set` instead of a +compose edit. The mechanism is in place and probe-guarded, but no such addition has been +made on this branch. + +## 1. What the plan asked for + +The operator approved a plan to finish the boundary the `#903 → #911` epic set: **inside +the sandbox, the CLI provisions harnesses and tools.** Concretely — make `oh.json` plus +the CLI the only way to say what a sandbox has; leave exactly one description of every +install; stop compose narrating a fact the container can observe; make the two sandbox +flavors identical where nothing forces them apart; and make the class of defect +unrepeatable rather than merely fixed. + +## 2. What was built + +**The rule, now enforced.** A value belongs in compose `environment:` only if a process +outside the sandbox — or the entrypoint before the control plane is readable — must act +on it. + +``` +$ diff <(awk '/^ environment:/{p=1;next} p&&/^ [a-z]/{exit} p' .devcontainer/docker-compose.yml) \ + <(awk '/^ environment:/{p=1;next} p&&/^ [a-z]/{exit} p' .devcontainer/docker-compose.image-only.yml) +$ echo $? +0 +``` + +**One description of each install.** + +``` +$ grep -c 'INSTALL_TAILSCALE\|INSTALL_AGENT_BROWSER\|tailscale_1\.102\.3\|agent-browser@' .devcontainer/entrypoint.sh +0 +``` + +The tool catalog is the sole owner of both pins and Tailscale's two checksums. The +ungated `install -d -o sandbox -g sandbox -m 0755 /var/run/tailscale` is kept, so +`oh tool install tailscale` still leaves the socket path usable without a reboot. + +**Flavor detection, simulated in all three states** (the exact predicate from +`.devcontainer/entrypoint.sh:127`): + +``` +$ sim(){ HARNESS_DIR="$1"; if mountpoint -q "$HARNESS_DIR" 2>/dev/null && [ -d "$HARNESS_DIR/.oh" ]; then echo sync; else echo seed; fi; } +$ sim /home/sandbox/harness # real checkout bind +sync +$ sim $SCRATCH/emptybind # empty dir, no bind +seed +$ sim $SCRATCH/seeded # seeded volume dir with .oh/, no bind +seed +``` + +**The guards.** + +``` +$ bash .oh/evals/probes/compose-env-boundary.sh +PASS: every compose environment: key across 4 file(s) is host-side or pre-control-plane; installs and settings stay in oh.json + +$ bash .oh/evals/probes/oh-image-only-deploy.sh +PASS: Flavor B (image-only) contract — entrypoint detects the flavor with mountpoint, logs +the mode on both paths, seeds only in the no-bind branch, and keeps .oh/.image-seeded +gitignored; behavioral sim confirms fresh-seed, idempotent-reseed, and +no-clobber-of-existing-.oh/; ... +``` + +**Mutation verification** — every new or inverted assertion was broken deliberately and +observed to exit 1: + +| Probe | Mutation | Exit | +|---|---|---| +| `compose-env-boundary` | `INSTALL_FOO=true` added to `environment:` | 1 | +| `compose-env-boundary` | `OH_IMAGE_ONLY=1` added | 1 | +| `compose-env-boundary` | `LANGFUSE_BASE_URL=x` (unlisted key) added | 1 | +| `compose-env-boundary` | `environment:` block re-added to the sshd overlay | 1 | +| `compose-env-boundary` | extra `ports:` entry added to an overlay | **0** (payload is allowed) | +| `tool-catalog-boundary` | `INSTALL_AGENT_BROWSER` guard re-added to the entrypoint | 1 | +| `tool-catalog-boundary` | `agent-browser@0.8.5` pin re-added to the entrypoint | 1 | +| `tool-catalog-boundary` | `entrypointGuard` re-added to the catalog | 1 | +| `tailscale-tool-boundary` | `INSTALL_TAILSCALE` guard re-added | 1 | +| `tailscale-tool-boundary` | version pin re-added to the entrypoint | 1 | +| `tailscale-tool-boundary` | sha256 literal re-added to the entrypoint | 1 | +| `tailscale-tool-boundary` | `/var/run/tailscale` indented into a conditional | 1 | +| `tailscale-tool-boundary` | `entrypointGuard` re-added to the catalog entry | 1 | +| `oh-image-only-deploy` | detection reverted to a flag test | 1 | +| `oh-image-only-deploy` | `.oh/` conjunct dropped from the detection | 1 | +| `oh-image-only-deploy` | mode-log line deleted | 1 | +| `oh-image-only-deploy` | `.oh/.image-seeded` gitignore rule deleted | 1 | +| `oh-image-only-deploy` | `OH_IMAGE_ONLY` re-added to `image-only.yml` | 1 | + +**Gates.** + +``` +$ bash .claude/skills/eval/run.sh # 116 probes +ran 116 probe(s); runner exit 0; zero REGRESSION/TIMEOUT/ERROR rows +wiki-pattern-persistence SKIPPED->PASS (the only status change) + +$ cd .oh/cli && npm run build && npm run typecheck +(clean) + +$ npx --yes shellcheck -S warning +SHELLCHECK CLEAN + +$ npx vitest run +Tests 6 failed | 976 passed (982) +``` + +## 3. Where it diverged from the plan, and why + +Five deviations. None narrows the delivered scope; two widen it. + +1. **The flavor detection is two conditions, not one.** The plan's option A was + `mountpoint -q "$OH_PROJECT_ROOT"` alone. `docs/runtimes/microsandbox.md:262` mounts a + fresh host directory **straight at** `/home/sandbox/harness` — a mountpoint with no + checkout in it. The single test would have sent that boot down the host-UID-sync path + and never seeded, breaking a documented runtime. The predicate is now + `mountpoint -q "$HARNESS_DIR" && [ -d "$HARNESS_DIR/.oh" ]`. Both conjuncts are + load-bearing and each is mutation-verified: dropping `.oh/` misreads an empty bind; + dropping `mountpoint` sends a seeded no-bind volume through the UID sync on its second + boot. That doc's now-meaningless `OH_IMAGE_ONLY` line was also removed. + +2. **Two more keys were retired than the plan listed.** The plan's surviving set named + `SANDBOX_SSH*`. Once `entrypoint.sh` reads `access.sshPasswordAuth` and + `access.sshAuthorizedKeys` from `oh.json`, nothing reads their rendered projections, so + leaving them rendered would have left exactly the dead projection this task exists to + remove. `SANDBOX_SSH` and `SANDBOX_SSH_PORT` stay — `docker-compose.sh` selects the + overlay from the first and publishes the port from the second, both before the + container exists. + +3. **`migrate-harness-yaml.sh` was left unchanged**, against the plan's instruction to + stop mapping the retired keys. Reading the script settles it: the variable name there + is an intermediate token, not a compose projection — `_parse env` emits + `INSTALL_HERMES`, `_field_for` maps it to `install.hermes`, and `_json_set` writes that + into **`oh.json`** (`:228-258`). Removing the mappings would silently drop an + operator's legacy `harness.yaml` settings on migration, for fields this task + deliberately keeps. + +4. **`compose-env-boundary.sh` landed in commit 2, not commit 1.** The plan put it in + commit 1 with `OH_IMAGE_ONLY`'s removal in commit 2. Its `OH_IMAGE_ONLY` assertion + cannot pass until that flag is gone, so shipping it in commit 1 would have left that + commit red. Moving the file kept both commits green while preserving the bisect + boundary the plan asked for. + +5. **Three consumers outside the plan's file list also needed the config read.** + `sandbox-healthcheck.sh:63` and `banner.sh:125` read `HERMES_DASHBOARD` from the + process environment and would have gone silently always-false; both now read `oh.json` + with `jq`. `link-providers.sh`'s `check_hermes_link` was presence-gated first, which + turned out to be wrong — a worktree has no `.hermes/`, so it fired where the old flag + skipped it; the correct gate is "verify the link if it exists", and the write path + keeps the binary check. + +Two plan bullets are satisfied by finding nothing to do, recorded so a reviewer does not +read them as skipped: the two tracked secrets-example templates needed **no change** (both +are secrets-only and already describe the Langfuse non-secrets as shell exports), and +`config-schema-parity.sh` keys off the field path rather than the docs table's now-`—` +column, so it stayed green untouched. + +## 4. What remains unverified + +- **No live container boot.** Plan verification items 5, 6, 7b, and 11 — cold flavor-A + boot with `install.tailscale`/`install.agentBrowser`, flavor-B boot with + `install.hermes`, both detection directions on a real volume, and an end-to-end + `ssh -p 2222` — all require `oh sandbox` on a Docker host. This session has no daemon. + Detection was simulated with the exact predicate against three real filesystem states + (§2), and the seed function's own behavior is covered by `oh-image-only-deploy.sh`'s + fenced-function simulation. **The flavor-B Hermes fix is the defect this task exists to + correct and it is argued from source, not observed.** A reviewer with Docker should boot + `image-only.yml` with `install.hermes=true` and confirm `$HERMES_HOME`, the + `.hermes/skills/openharness` symlink, and the provider skill pack. +- **6 pre-existing `compose-args.test.ts` failures**, carried forward. They reproduce + unmodified on `development` in this environment, so this branch did not cause them. The + same file had **8** failing before; retargeting two of its tests onto the sshd overlay + fixed two incidentally. +- **`oh_config`'s degrade path is untested at runtime.** A missing or old `oh` returns the + caller's documented default by construction, and `oh config show` ships in every + released image, but no test forces the failure inside a booting container. +- **`shellcheck` is not installed in this sandbox**; CI's exact glob was run through + `npx --yes shellcheck` (0.11.0) instead of the apt binary CI installs. +- **`oh destroy` prompt copy** (plan item 9) was not re-read; nothing on this branch + touches it. +- **Noted, deliberately not fixed:** `INSTALL_PYTHON_KERNEL` vs `OH_PROVISION_PYTHON` + (a Dockerfile↔entrypoint duplication of the same class), `Dockerfile:106`'s whole-repo + copy into the `home` stage (a build-cache cost, no image weight), and the two Slack + scripts' placement in `.devcontainer/`. All three are separate issues. diff --git a/.oh/tasks/compose-env-boundary/prd.json b/.oh/tasks/compose-env-boundary/prd.json new file mode 100644 index 00000000..e5ca1c96 --- /dev/null +++ b/.oh/tasks/compose-env-boundary/prd.json @@ -0,0 +1,210 @@ +{ + "schemaVersion": 1, + "project": "Open Harness", + "branchName": "task/920-compose-env-boundary", + "description": "Make the CLI the only install/config surface - empty the compose environment: block, delete the duplicate entrypoint installers, and replace OH_IMAGE_ONLY with runtime bind detection.", + "userStories": [ + { + "id": "US-001", + "title": "Strip the eleven variables from the compose files", + "description": "As an operator, I want the compose environment: blocks to carry only values Docker or the pre-control-plane entrypoint must act on, so adding a tool never requires a compose edit.", + "acceptanceCriteria": [ + "docker-compose.yml no longer contains XAI_API_KEY, INSTALL_AGENT_BROWSER, INSTALL_TAILSCALE, SKIP_PNPM_INSTALL, INSTALL_HERMES, HERMES_HOME, HERMES_DASHBOARD, HERMES_DASHBOARD_PORT, CRON_AGENT_BIN, LANGFUSE_BASE_URL, or LANGFUSE_PRIVACY_PRESET", + "docker-compose.image-only.yml no longer contains XAI_API_KEY, CRON_AGENT_BIN, LANGFUSE_BASE_URL, or LANGFUSE_PRIVACY_PRESET", + "docker-compose.ssh.yml retains its ports: mapping and no longer has an environment: block", + "docker-compose.hermes-dashboard.yml is deleted and the branch applying it in .oh/scripts/docker-compose.sh is removed", + "Each modified file's header comment describes the file as it now is", + "docker compose -f config parses without error", + "Typecheck passes" + ], + "priority": 1, + "passes": true, + "notes": "Eleven keys removed from docker-compose.yml; XAI_API_KEY/CRON_AGENT_BIN/LANGFUSE_* removed from image-only.yml; ssh overlay keeps ports: and lost environment:; hermes-dashboard overlay and its docker-compose.sh branch deleted. Headers rewritten." + }, + { + "id": "US-002", + "title": "Delete the duplicate agent-browser and Tailscale installers from the entrypoint", + "description": "As a maintainer, I want exactly one description of each tool install, so a version bump cannot leave two truths in the tree.", + "acceptanceCriteria": [ + "The INSTALL_AGENT_BROWSER-guarded install block is removed from .devcontainer/entrypoint.sh", + "The INSTALL_TAILSCALE-guarded install block is removed, including the pinned tarball version and both sha256 literals", + "The ungated install -d -o sandbox -g sandbox -m 0755 /var/run/tailscale block is retained", + ".oh/cli/src/lib/tools/catalog.ts remains the sole location of the agent-browser and Tailscale pins", + "Confirmed on the merged tree before deletion that the Tailscale catalog entry is installUser: \"sandbox\"", + "shellcheck .devcontainer/entrypoint.sh passes with CI's invocation", + "Typecheck passes" + ], + "priority": 2, + "passes": true, + "notes": "Both installers deleted; ungated /var/run/tailscale kept. Confirmed installUser:\"sandbox\" on the merged tree before deleting. shellcheck clean." + }, + { + "id": "US-003", + "title": "Read oh.json from the entrypoint and presence-gate the Hermes wiring", + "description": "As an operator, I want the entrypoint to read settings from oh.json through the CLI, so flavor B gets the same wiring flavor A does.", + "acceptanceCriteria": [ + "A fenced oh_config helper is added to .devcontainer/entrypoint.sh calling `oh config show | jq -r ` as the sandbox user with a caller-supplied fallback", + "The helper uses oh config show, not a new verb", + "A missing, old, or failing CLI yields the documented default and never fails the boot", + "The Hermes wiring block is gated on `command -v hermes` instead of INSTALL_HERMES", + "HERMES_HOME is no longer a parameter; the existing ${HERMES_HOME:-$HARNESS/.hermes} fallback supplies the same value", + "The dashboard block reads hermesDashboard.enabled and hermesDashboard.port via oh_config", + "The root pnpm install gate reads build.skipPnpmInstall via oh_config", + "The sshd block reads access.ssh, access.sshPort, access.sshPasswordAuth, and access.sshAuthorizedKeys via oh_config; the no-key/no-password warning still fires when both are absent", + ".oh/scripts/link-providers.sh gates its Hermes branch on the binary's presence rather than INSTALL_HERMES", + "shellcheck passes on every changed shell file", + "Typecheck passes" + ], + "priority": 3, + "passes": true, + "notes": "oh_config/oh_config_truthy fenced helper added over `oh config show` with per-call fallbacks; Hermes wiring gated on `command -v hermes`; HERMES_HOME parameter dropped; dashboard, pnpm gate, and sshd read oh.json; link-providers.sh presence-gated." + }, + { + "id": "US-004", + "title": "Read the cron agent binary from oh.json", + "description": "As an operator, I want cron.agentBin honoured without a compose hop.", + "acceptanceCriteria": [ + ".oh/scripts/cron-runtime.ts resolves the agent binary from process.env.CRON_AGENT_BIN first, then the CLI's readOhConfig value, then the \"claude\" default", + "The env override is preserved and covered by a test", + ".oh/scripts/__tests__/cron-runtime.test.ts passes", + "Tests pass", + "Typecheck passes" + ], + "priority": 4, + "passes": true, + "notes": "resolveAgentBin(): CRON_AGENT_BIN, then cron.agentBin via `oh config show`, then \"claude\". Three precedence tests added." + }, + { + "id": "US-005", + "title": "Stop rendering the retired keys and make their return impossible", + "description": "As a maintainer, I want the rendered .env set to be the rule expressed in code, so a re-added put() throws instead of silently restoring the pipeline.", + "acceptanceCriteria": [ + "config-render.ts no longer emits the five INSTALL_* keys, HERMES_DASHBOARD, HERMES_DASHBOARD_PORT, CRON_AGENT_BIN, SKIP_PNPM_INSTALL, LANGFUSE_BASE_URL, or LANGFUSE_PRIVACY_PRESET", + "Every removed key is added to RETIRED_KEYS so rendering it throws", + "The remaining rendered set is exactly SANDBOX_NAME, TZ, OH_HOME_MOUNT, GIT_USER_NAME, GIT_USER_EMAIL, DOCKER_SOCKET, SANDBOX_SSH*, OH_SANDBOX_IMAGE, OH_PULL_POLICY", + "readonly entrypointGuard? and both of its uses are removed from .oh/cli/src/lib/tools/catalog.ts", + "oh config set install.tailscale true then oh config show round-trips with no .env projection", + "cd .oh/cli && npm run build && npm run typecheck passes", + "Typecheck passes" + ], + "priority": 5, + "passes": true, + "notes": "Ten keys plus SANDBOX_SSH_PASSWORD_AUTH/SANDBOX_SSH_AUTHORIZED_KEYS retired; RETIRED_KEYS holds all of them; entrypointGuard removed from tools/catalog.ts. Build and typecheck pass." + }, + { + "id": "US-006", + "title": "Invert the two boundary probes and retarget the parity probe", + "description": "As a maintainer, I want the probes to assert the post-epic boundary, since they currently require the duplication and would fail the correct fix.", + "acceptanceCriteria": [ + "tool-catalog-boundary.sh asserts the entrypoint contains no INSTALL_AGENT_BROWSER guard and no agent-browser@ pin, with the catalog as sole truth; its # desc: header is rewritten", + "Its disjointness, downloadSize, and --yes gating assertions are preserved", + "tailscale-tool-boundary.sh receives the same inversion for the guard, the version pin, and the sha256 cross-check", + "Every zero-exposure assertion is kept: no cap_add, devices:, privileged: true, or published 3773; no tailscaled invocation or tailscale up on boot; no Funnel; no committed auth key", + "The ungated /var/run/tailscale assertion is kept and retargeted", + "compose-config-path-parity.sh no longer keys on INSTALL_HERMES or CRON_AGENT_BIN", + "config-schema-parity.sh stays green and is confirmed to key off the field path, not the docs Env var column", + "default-provisioning.sh and sandbox-boot-guard-ci.sh are left untouched", + "Each inverted assertion is mutation-verified to exit 1", + "Typecheck passes" + ], + "priority": 6, + "passes": true, + "notes": "Both boundary probes inverted with every zero-exposure assertion kept; ungated-socket check retargeted to indentation; parity probe moved onto SANDBOX_PASSWORD/GIT_USER_NAME. Five mutations verified to exit 1." + }, + { + "id": "US-007", + "title": "Add a tier-A compose-env-boundary probe", + "description": "As a maintainer, I want the rule enforced directly, so this class of defect cannot return through any compose file or overlay.", + "acceptanceCriteria": [ + ".oh/evals/probes/compose-env-boundary.sh exists with # tier: A and a # source: line naming issue #920", + "It runs over every .devcontainer/docker-compose*.yml, overlays included", + "It fails on any INSTALL_* key and on OH_IMAGE_ONLY in any of them", + "Every environment: key must be in config-render.ts's rendered set or one of the documented literals", + "Overlay ports: and volumes: blocks are explicitly allowed", + "It uses the 3-state exit oracle: 0 PASS, 1 REGRESSION, 2 SKIPPED when no compose file is present", + "Mutation-verified: an INSTALL_FOO line, an OH_IMAGE_ONLY line, and an unlisted key each drive exit 1", + "bash .claude/skills/eval/run.sh --probe compose-env-boundary reports PASS", + "Typecheck passes" + ], + "priority": 7, + "passes": true, + "notes": "compose-env-boundary.sh added (tier A, 3-state oracle). Mutations verified: INSTALL_*, OH_IMAGE_ONLY, an unlisted key, and an overlay environment: key each exit 1; extra overlay ports stay PASS." + }, + { + "id": "US-008", + "title": "Update the tests that assert the retired pipeline", + "description": "As a maintainer, I want the suite to describe the new boundary rather than the old one.", + "acceptanceCriteria": [ + "config-render.test.ts drops assertions for the removed keys and adds RETIRED_KEYS throw cases for each", + "tool-catalog.test.ts no longer references entrypointGuard", + "entrypoint-pnpm-install.test.ts and entrypoint-pnpm-manifest-fingerprint.sh assert the oh.json read instead of the compose SKIP_PNPM_INSTALL pass-through", + "harness.test.ts, init.test.ts, and env-file.ts's CONFIG_FIELD_BY_ENV_KEY / INSTALL_FIELDS alias tables are updated consistently", + "Root npx vitest run shows no new failures beyond the known 8 environmental compose-args.test.ts failures", + "Tests pass", + "Typecheck passes" + ], + "priority": 8, + "passes": true, + "notes": "config-render, tool-catalog, harness-catalog, entrypoint-pnpm-install, sandbox-healthcheck, cron-runtime, compose-args, docs-compose-overlays updated. 976 pass; the 6 remaining compose-args failures reproduce unmodified on development." + }, + { + "id": "US-009", + "title": "Update docs, templates, the wiki, and the changelog", + "description": "As a reader, I want the documentation to describe the CLI path, since the compose-variable instructions are now false.", + "acceptanceCriteria": [ + "docs/configuration.md blanks the Env var column for every field whose projection is gone; the fields stay documented", + "docs/connecting.md documents oh config set install.tailscale true and oh tool install tailscale instead of the environment variable", + "docs/security-considerations.md, docs/harnesses/grok-build.md, docs/integrations/langfuse.md, and the Hermes dashboard docs reflect the retired variables and the removed published port", + ".env.example and .oh/templates/.env.example drop the retired keys", + ".oh/scripts/migrate-harness-yaml.sh stops mapping the retired keys in both directions; the legacy shim stays", + "CHANGELOG.md gains a BREAKING entry under ## [Unreleased], one sentence, <=250 characters, linking #920", + "bash .oh/evals/probes/changelog-entry-length.sh reports no over-length entry", + ".oh/skills/wiki/corpus/compose-env-boundary.md exists with valid frontmatter and body order H1, Relevant Source Files, Summary, Detail, System Relationships, See Also", + "Every wiki claim about repository behavior cites a source path with a line number; System Relationships carries a Mermaid diagram of the oh.json to CLI to entrypoint path", + "The wiki entry is <=900 words and See Also cross-links [[sandbox-dependency-installs]] and [[oh-cli-portable-lifecycle]]", + "sandbox-dependency-installs.md no longer claims compose passes SKIP_PNPM_INSTALL into the container, and its updated date is bumped", + "bash .oh/evals/probes/wiki-readme-index.sh passes with the new entry indexed in corpus/README.md", + "Typecheck passes" + ], + "priority": 9, + "passes": true, + "notes": "configuration.md, connecting.md, sshd.md, langfuse.md, hermes.md, grok-build.md, opencode.md, overview.md, installation.md, security-considerations.md, lifecycle-commands.md, .oh/README.md, microsandbox.md, the Hermes template README, and CHANGELOG updated. New wiki entry compose-env-boundary; sandbox-dependency-installs revised; index regenerated." + }, + { + "id": "US-010", + "title": "Detect the sandbox flavor at runtime instead of reading a flag", + "description": "As an operator, I want the entrypoint to determine whether a checkout is bind-mounted, so compose stops narrating a fact the container can observe.", + "acceptanceCriteria": [ + ".oh/.image-seeded is added to .gitignore before the detection change", + "entrypoint.sh replaces the OH_IMAGE_ONLY gate with mountpoint -q \"$OH_PROJECT_ROOT\", the bind-detected branch leading and the seed branch as the else", + "The elif [ -d \"$HARNESS_DIR\" ] existence test is dropped as subsumed", + "Both branch bodies are otherwise unchanged", + "The detected mode is logged on both paths", + "OH_IMAGE_ONLY is removed from docker-compose.image-only.yml and no reader of it remains in the tree", + "diff of the two compose files' environment: blocks is empty", + "This lands as a separate commit from US-001 through US-009", + "Typecheck passes" + ], + "priority": 10, + "passes": true, + "notes": ".gitignore first, then mountpoint-plus-.oh detection with the branches swapped and the mode logged on both paths; OH_IMAGE_ONLY dropped from image-only.yml. environment: blocks diff to empty." + }, + { + "id": "US-011", + "title": "Retarget the image-only deploy probe and its documentation", + "description": "As a maintainer, I want the flavor-B probe to test behavior rather than documentation prose.", + "acceptanceCriteria": [ + "oh-image-only-deploy.sh drops the three assertions requiring deployment-prebuilt-image.md to contain or omit particular strings", + "Every behavioral assertion is kept: gate ordering, the seed simulation, and the compose-shape checks", + "The gate check targets mountpoint-based detection rather than the retired flag", + "docs/deployment-prebuilt-image.md documents how flavor B is detected instead of how the flag is set, and its docker run recipe drops the corresponding -e", + "Mutation-verified: removing the detection from the entrypoint drives the probe to exit 1", + "bash .claude/skills/eval/run.sh exits 0 with no new regressions", + "Typecheck passes" + ], + "priority": 11, + "passes": true, + "notes": "Three prose assertions dropped; gate check retargeted to the two-condition detection; deployment-prebuilt-image.md rewritten and its docker run -e removed. Four mutations verified to exit 1." + } + ] +} diff --git a/.oh/tasks/compose-env-boundary/prd.md b/.oh/tasks/compose-env-boundary/prd.md new file mode 100644 index 00000000..c7e28dda --- /dev/null +++ b/.oh/tasks/compose-env-boundary/prd.md @@ -0,0 +1,371 @@ +# PRD: Make the CLI the only install/config surface — empty the compose `environment:` block + +- **Issue:** #920 +- **Repo:** `mifunedev/openharness` +- **Base:** `development` +- **Branch:** `task/920-compose-env-boundary` +- **Source plan:** `.claude/plans/hazy-bouncing-alpaca.md` (operator-approved) + +## Introduction/Overview + +The epic `#903 → #905 → #907 → #909 → #911` set one boundary: **inside the sandbox, the CLI +provisions harnesses and tools.** `.oh/scripts/provision-defaults.sh:114-159` reads both +catalogs via `oh harness list --json` / `oh tool list --json` and installs everything +`kind:"default"` or `enabled == true` — where `enabled` comes from **oh.json, never the +environment**. + +The compose files never followed. They still push eleven values through +`oh.json → config-render.ts → .devcontainer/.env → compose environment: → entrypoint.sh`, +when the consumer at the end of that pipeline sits inside the home mount next to the `oh` CLI +and can read oh.json directly. + +This is not merely redundant. It has produced three defects, two of which CI now enforces: + +| # | Defect | Evidence | +|---|---|---| +| 1 | **Two installers, pins duplicated.** `entrypoint.sh:544-551` installs `agent-browser@0.8.5`, duplicating `tools/catalog.ts:34-40`. `entrypoint.sh:561-608` installs Tailscale `1.102.3`, duplicating `catalog.ts:139` **including both sha256 literals**. Both are already installed by `provision-defaults.sh`. With the flag on, flavor A installs each tool twice via two paths gated on two different truths. | `AGENTS.md` bans exactly this "second unverified description" | +| 2 | **Probes pin the pre-epic design.** `tool-catalog-boundary.sh:32-51` and `tailscale-tool-boundary.sh:37-60` assert the entrypoint *must* hold the guard and the pin, and that the catalog must **agree with it**. | Correct when the choice was Dockerfile-vs-entrypoint. Now both are wrong, and these probes **fail the correct fix** | +| 3 | **Flavor B silently loses Hermes.** `docker-compose.image-only.yml` carries none of the seven flavor-A-only keys, so `INSTALL_HERMES` is never true and `entrypoint.sh:169-228` + `link-providers.sh:111` are dead there. `oh harness install hermes` yields the binary and no wiring. | The epic made the install half config-driven and left the wiring half env-driven | + +`LANGFUSE_BASE_URL` and `LANGFUSE_PRIVACY_PRESET` are already **fully dead** — no runtime +consumer anywhere in the tree. So are `INSTALL_OPENCODE` / `INSTALL_GROK_BUILD`, still +rendered at `config-render.ts:31-32` after #909 deleted everything that read them. + +## The rule this establishes + +> A value belongs in compose `environment:` only if a process **outside** the sandbox — or the +> entrypoint **before** the control plane is readable — must act on it. Everything else lives +> in oh.json and is read through the CLI. + +**Survives (must be identical in both compose files):** `SANDBOX_NAME`, `SANDBOX_PASSWORD`, +`TZ` (needed before oh.json is readable; `SANDBOX_NAME` is also the in-sandbox detection +signal) · `CC_SAFETY_NET_STRICT`, `CC_SAFETY_NET_WORKTREE`, +`CLAUDE_DANGEROUSLY_SKIP_PERMISSIONS` (read by third-party binaries that know nothing of +oh.json) · `GIT_USER_NAME`, `GIT_USER_EMAIL`, `GH_TOKEN` (applied at boot before the first +agent session). + +## Goals + +1. **Finish the epic's boundary.** Make oh.json + the CLI the *only* way to say what a sandbox + has. +2. **One description of every install.** Delete the duplicate agent-browser and Tailscale + installers from `entrypoint.sh`; the catalogs already hold those pins. +3. **Stop compose narrating what the container can see.** Retire `OH_IMAGE_ONLY` in favour of + detecting the checkout bind at runtime. +4. **Make the two flavors identical where they should be.** Both compose `environment:` blocks + byte-identical; the flavors differ only where Docker forces it — `build:`, + `image:`/`pull_policy:`, and the `..:` bind. +5. **Make the class of defect unrepeatable.** A tier-A probe over every compose file and + overlay, plus `RETIRED_KEYS` entries that throw if a removed variable returns. + +**Net effect:** adding a tool, harness, or setting stops requiring a compose edit — the same +consolidation the single `/home/sandbox` mount achieved for volumes. + +## What replaces each of the eleven + +| Variable | Replacement | Files | +|---|---|---| +| `INSTALL_AGENT_BROWSER` | Already installed by `provision-defaults.sh` from `install.agentBrowser`. **Delete the entrypoint installer.** | `entrypoint.sh:544-551` | +| `INSTALL_TAILSCALE` | Same, from `install.tailscale`. **Delete the installer** — and with it the second copy of the version and both sha256s. | `entrypoint.sh:561-608` | +| `INSTALL_HERMES` | Install role already gone. Gate the **wiring** on the binary existing (`command -v hermes`) — truthful, needs no config read, and fixes flavor B for free. | `entrypoint.sh:169`, `link-providers.sh:111` | +| `HERMES_HOME` | The compose value was the literal `/home/sandbox/harness/.hermes` — exactly what the existing `${HERMES_HOME:-$HARNESS/.hermes}` fallback already yields. Drop the parameter. | `entrypoint.sh:170` | +| `HERMES_DASHBOARD` | Read `hermesDashboard.enabled` from oh.json. | `entrypoint.sh:206` | +| `HERMES_DASHBOARD_PORT` | Read `hermesDashboard.port` from oh.json. | `entrypoint.sh:209` | +| `SKIP_PNPM_INSTALL` | Read `build.skipPnpmInstall` from oh.json. | `entrypoint.sh:451` | +| `CRON_AGENT_BIN` | `cron-runtime.ts` is TypeScript — import the CLI's `readOhConfig` and drop the env hop. Keep `process.env` as an override *ahead of* the config value. | `cron-runtime.ts:26` | +| `LANGFUSE_BASE_URL` | **No consumer exists.** Delete. | compose ×2, `config-render.ts:51` | +| `LANGFUSE_PRIVACY_PRESET` | **No consumer exists.** Delete. | compose ×2, `config-render.ts:52` | +| `XAI_API_KEY` | Already a registered secret (`secrets.ts:10`) in the gitignored root `.env`; `config-render.ts:60-62` **refuses to render secrets**, so this line could only ever catch a hand-edited `.devcontainer/.env`. Grok's `~/.grok/auth.json` takes precedence and now persists in the home mount. | compose ×2 | + +**Every oh.json field stays.** `install.*`, `hermesDashboard.*`, `cron.agentBin`, +`build.skipPnpmInstall`, `langfuse.*` remain valid and settable — only their `.env` projection +goes. `oh config set` keeps working for all of them. + +## Why `OH_IMAGE_ONLY` does not earn its place + +It gates exactly two decisions, both at `entrypoint.sh:102-113`, and has exactly **one reader +in the entire tree**: + +| | Decision | Real question being asked | +|---|---|---| +| D1 | skip host UID/GID sync | is there a host checkout to sync to? | +| D2 | seed the workspace from `/opt/oh-seed` | is `/home/sandbox/harness` a bind, or a directory we must populate? | + +Both reduce to one question the container can answer for itself: **is `/home/sandbox/harness` +a bind mount?** Compose is telling the container something the container can see. + +Verified, not assumed: `mountpoint` and `findmnt` are both at `/usr/bin` in the image, and in a +live flavor-A container `findmnt -no TARGET /home/sandbox/harness` returns the path. + +| Option | Verdict | +|---|---| +| **A. Detect at runtime — `mountpoint -q /home/sandbox/harness`** | **Chosen.** One observable check replaces the flag for both decisions. Deletes the flag, the compose line, and the footgun where setting it in flavor A seeds over your checkout. Leaves the flavor-B capability fully intact. | +| B. Keep the flag | Compose keeps narrating an observable fact; blocks identical `environment:` blocks; the footgun stays. | +| C. Derive from `[ -d /home/sandbox/harness/.oh ]` | Covers D2 only — already `seed_workspace_volume`'s own internal guard. Says nothing about D1. Insufficient alone. | +| D. Delete flavor B entirely | **Rejected.** `/opt/oh-seed` stays regardless (`verify-sandbox-image.sh:114,145`, `sandbox-compatibility.yml:97,115,129`, `Dockerfile:111` all read catalogs from it), so there is **no image saving** — the only win is deleting one compose file and one probe, at the cost of a shipped capability (#609's no-checkout deploy, with its own docs page). A product decision, not a cleanup. | +| E. Invert — flavor A sets `OH_HAS_CHECKOUT=1` | Strictly worse: same information, same objection, and now the *common* path carries the flag. | + +## Every file in `.devcontainer/` — keep or cut + +| File | Verdict | +|---|---| +| `Dockerfile` | **Keep.** The sandbox definition. | +| `entrypoint.sh` | **Keep.** Loses ~60 lines of duplicate installer and one flag branch. | +| `docker-compose.yml` | **Keep.** Loses eleven `environment:` lines. | +| `docker-compose.image-only.yml` | **Keep** — flavor B is a shipped capability (#609) and deleting it saves no image weight. Its `environment:` block empties to match flavor A exactly. | +| `docker-compose.docker-sock.yml` | **Keep, untouched.** 16 lines, all payload: one volume mount Docker must decide before the container exists, plus a security rationale. No `environment:` block. | +| `docker-compose.ssh.yml` | **Keep the `ports:`, cut the `environment:` block.** Publishing `127.0.0.1:2222:22` is irreducibly Docker-level. Its four env vars (`SANDBOX_SSH`, `SANDBOX_SSH_PORT`, `SANDBOX_SSH_PASSWORD_AUTH`, `SANDBOX_SSH_AUTHORIZED_KEYS`) restate `access.ssh*`, which `config-render.ts:38-41` already renders — the same defect as the main file, in an overlay. `SANDBOX_SSH_AUTHORIZED_KEYS` holds *public* keys, so this moves no secret into oh.json. | +| `docker-compose.hermes-dashboard.yml` | **Cut** (operator ruling). Its one irreducible job was publishing `127.0.0.1:`; retiring it means the dashboard is reachable from inside the sandbox and over cloudflared or Tailscale, not a published host port. Consistent with where #897 took T3 Code, but a real capability change — it goes in the BREAKING entry. `HERMES_DASHBOARD_HOST` / `_INSECURE` came only from this overlay; the entrypoint's existing defaults (`127.0.0.1`, no `--insecure`) take over. | +| `devcontainer.json` | **Keep, with a caveat.** It hardcodes `dockerComposeFile: ["docker-compose.yml"]`, so VS Code "Reopen in Container" bypasses `docker-compose.sh` and gets no overlays and no `.env` rendering — a second lifecycle door. This task does not close that gap but **materially narrows** it: once the entrypoint reads oh.json directly, the VS Code path gets more correct for free. | +| `seed-msg-bridge.sh` | **Keep.** Live: `gateway.sh:151`. | +| `client-slack-supervise.sh` | **Keep.** Live: `gateway.sh:166,327`. | +| `.env` | **Keep.** Generated, gitignored, operator-owned. Shrinks by ten keys. | + +## User Stories + +### US-001: Strip the eleven variables from the compose files + +**Description:** As an operator, I want the compose `environment:` blocks to carry only values +Docker or the pre-control-plane entrypoint must act on, so adding a tool never requires a +compose edit. + +**Acceptance Criteria:** + +- [ ] `.devcontainer/docker-compose.yml` no longer contains `XAI_API_KEY`, `INSTALL_AGENT_BROWSER`, `INSTALL_TAILSCALE`, `SKIP_PNPM_INSTALL`, `INSTALL_HERMES`, `HERMES_HOME`, `HERMES_DASHBOARD`, `HERMES_DASHBOARD_PORT`, `CRON_AGENT_BIN`, `LANGFUSE_BASE_URL`, or `LANGFUSE_PRIVACY_PRESET` +- [ ] `.devcontainer/docker-compose.image-only.yml` no longer contains `XAI_API_KEY`, `CRON_AGENT_BIN`, `LANGFUSE_BASE_URL`, or `LANGFUSE_PRIVACY_PRESET` +- [ ] `.devcontainer/docker-compose.ssh.yml` retains its `ports:` mapping and no longer has an `environment:` block +- [ ] `.devcontainer/docker-compose.hermes-dashboard.yml` is deleted, and the branch that applies it in `.oh/scripts/docker-compose.sh` is removed +- [ ] Each modified file's header comment describes the file as it now is +- [ ] `docker compose -f config` parses without error + +### US-002: Delete the duplicate agent-browser and Tailscale installers from the entrypoint + +**Description:** As a maintainer, I want exactly one description of each tool install, so a +version bump cannot leave two truths in the tree. + +**Acceptance Criteria:** + +- [ ] The `INSTALL_AGENT_BROWSER`-guarded install block is removed from `.devcontainer/entrypoint.sh` +- [ ] The `INSTALL_TAILSCALE`-guarded install block is removed, including the pinned tarball version and **both** sha256 literals +- [ ] The ungated `install -d -o sandbox -g sandbox -m 0755 /var/run/tailscale` block is **retained** (gating it would force a reboot after `oh tool install tailscale`) +- [ ] `.oh/cli/src/lib/tools/catalog.ts` remains the sole location of the agent-browser and Tailscale pins +- [ ] Confirmed on the merged tree **before deletion** that the Tailscale catalog entry is `installUser: "sandbox"` (post-#897), so removing the entrypoint fallback does not strand the install behind #907's `stdio:"inherit"` sudo hazard +- [ ] `shellcheck .devcontainer/entrypoint.sh` passes with CI's invocation + +### US-003: Read oh.json from the entrypoint and presence-gate the Hermes wiring + +**Description:** As an operator, I want the entrypoint to read settings from oh.json through +the CLI, so flavor B gets the same wiring flavor A does. + +**Acceptance Criteria:** + +- [ ] A fenced `oh_config` helper is added to `.devcontainer/entrypoint.sh`, matching the existing `seed_home` / `seed_workspace_volume` fence convention, calling `oh config show | jq -r ` as the `sandbox` user with a caller-supplied fallback +- [ ] The helper uses `oh config show`, **not** a new verb — a baked `oh` in a running container can predate a new verb, and this is the boot path +- [ ] A missing, old, or failing CLI yields the documented default and never fails the boot +- [ ] The Hermes wiring block is gated on `command -v hermes` instead of `INSTALL_HERMES` +- [ ] `HERMES_HOME` is no longer a parameter; the existing `${HERMES_HOME:-$HARNESS/.hermes}` fallback supplies the same value +- [ ] The dashboard block reads `hermesDashboard.enabled` and `hermesDashboard.port` via `oh_config` +- [ ] The root `pnpm install` gate reads `build.skipPnpmInstall` via `oh_config` +- [ ] The sshd block reads `access.ssh`, `access.sshPort`, `access.sshPasswordAuth`, and `access.sshAuthorizedKeys` via `oh_config`; the no-key/no-password warning still fires when both are absent +- [ ] `.oh/scripts/link-providers.sh` gates its Hermes branch on the binary's presence rather than `INSTALL_HERMES` +- [ ] `shellcheck` passes on every changed shell file + +### US-004: Read the cron agent binary from oh.json + +**Description:** As an operator, I want `cron.agentBin` honoured without a compose hop. + +**Acceptance Criteria:** + +- [ ] `.oh/scripts/cron-runtime.ts` resolves the agent binary from `process.env.CRON_AGENT_BIN` first, then the CLI's `readOhConfig` value, then the `"claude"` default — in that precedence order +- [ ] The env override is preserved and covered by a test +- [ ] `.oh/scripts/__tests__/cron-runtime.test.ts` passes + +### US-005: Stop rendering the retired keys and make their return impossible + +**Description:** As a maintainer, I want the rendered `.env` set to *be* the rule, expressed in +code, so a re-added `put()` throws instead of silently restoring the pipeline. + +**Acceptance Criteria:** + +- [ ] `.oh/cli/src/lib/config-render.ts` no longer emits the five `INSTALL_*` keys, `HERMES_DASHBOARD`, `HERMES_DASHBOARD_PORT`, `CRON_AGENT_BIN`, `SKIP_PNPM_INSTALL`, `LANGFUSE_BASE_URL`, or `LANGFUSE_PRIVACY_PRESET` +- [ ] Every removed key is added to `RETIRED_KEYS`, so rendering it throws +- [ ] The remaining rendered set is exactly `SANDBOX_NAME`, `TZ`, `OH_HOME_MOUNT`, `GIT_USER_NAME`, `GIT_USER_EMAIL`, `DOCKER_SOCKET`, `SANDBOX_SSH*`, `OH_SANDBOX_IMAGE`, `OH_PULL_POLICY` +- [ ] `readonly entrypointGuard?` and both of its uses are removed from `.oh/cli/src/lib/tools/catalog.ts` +- [ ] Every oh.json field survives: `oh config set install.tailscale true` then `oh config show` round-trips, with no `.env` projection +- [ ] `cd .oh/cli && npm run build && npm run typecheck` passes + +### US-006: Invert the two boundary probes and retarget the parity probe + +**Description:** As a maintainer, I want the probes to assert the post-epic boundary, since +they currently require the duplication and would fail the correct fix. + +**Acceptance Criteria:** + +- [ ] `.oh/evals/probes/tool-catalog-boundary.sh` asserts the entrypoint contains **no** `INSTALL_AGENT_BROWSER` guard and no `agent-browser@` pin, with the catalog as sole truth; its `# desc:` header is rewritten (it currently names the entrypoint as ground truth) +- [ ] Its disjointness, `downloadSize`, and `--yes` gating assertions are preserved +- [ ] `.oh/evals/probes/tailscale-tool-boundary.sh` receives the same inversion for the guard, the version pin, and the sha256 cross-check +- [ ] **Every zero-exposure assertion is kept**: no `cap_add`, `devices:`, `privileged: true`, or published `3773` in any compose file; no `tailscaled` invocation or `tailscale up` on boot; no Funnel; no committed auth key +- [ ] The ungated-`/var/run/tailscale` assertion is kept and retargeted, since no guard block wraps it any more +- [ ] `.oh/evals/probes/compose-config-path-parity.sh` no longer keys on `INSTALL_HERMES` or `CRON_AGENT_BIN`; it uses surviving keys, since its subject is path resolution +- [ ] `.oh/evals/probes/config-schema-parity.sh` stays green and is confirmed to key off the field path, not the docs table's now-empty **Env var** column +- [ ] `.oh/evals/probes/default-provisioning.sh` and `.oh/evals/probes/sandbox-boot-guard-ci.sh` are left untouched — their Dockerfile `ARG INSTALL_*` bans stay correct +- [ ] Each inverted assertion is mutation-verified: re-add the guard, the pin, and a compose `INSTALL_*` line, and confirm the probe exits 1 in each case + +### US-007: Add a tier-A `compose-env-boundary` probe + +**Description:** As a maintainer, I want the rule enforced directly, so this class of defect +cannot return through any compose file or overlay. + +**Acceptance Criteria:** + +- [ ] `.oh/evals/probes/compose-env-boundary.sh` exists with `# tier: A` and a `# source:` line naming issue #920 +- [ ] It runs over **every** `.devcontainer/docker-compose*.yml`, overlays included +- [ ] It fails on any `INSTALL_*` key and on `OH_IMAGE_ONLY` in any of them +- [ ] Every `environment:` key must be either in `config-render.ts`'s rendered set or one of the documented literals; anything else fails +- [ ] Overlay `ports:` and `volumes:` blocks are explicitly allowed — that is the payload only Docker can act on +- [ ] It uses the 3-state exit oracle: 0 = PASS, 1 = REGRESSION, 2 = SKIPPED when no compose file is present +- [ ] Mutation-verified: adding an `INSTALL_FOO` line, an `OH_IMAGE_ONLY` line, and an unlisted key each drive exit 1 +- [ ] `bash .claude/skills/eval/run.sh --probe compose-env-boundary` reports PASS + +### US-008: Update the tests that assert the retired pipeline + +**Description:** As a maintainer, I want the suite to describe the new boundary rather than the +old one. + +**Acceptance Criteria:** + +- [ ] `.oh/cli/src/lib/__tests__/config-render.test.ts` drops assertions for the removed keys and adds `RETIRED_KEYS` throw cases for each +- [ ] `tool-catalog.test.ts` no longer references `entrypointGuard` +- [ ] `entrypoint-pnpm-install.test.ts` and `.oh/evals/probes/entrypoint-pnpm-manifest-fingerprint.sh` assert the oh.json read instead of the compose `SKIP_PNPM_INSTALL` pass-through +- [ ] `harness.test.ts`, `init.test.ts`, and `env-file.ts`'s `CONFIG_FIELD_BY_ENV_KEY` / `INSTALL_FIELDS` alias tables are updated consistently +- [ ] Root `npx vitest run` shows no new failures beyond the known 8 environmental `compose-args.test.ts` failures + +### US-009: Update docs, templates, and the changelog + +**Description:** As a reader, I want the documentation to describe the CLI path, since the +compose-variable instructions are now false. + +**Acceptance Criteria:** + +- [ ] `docs/configuration.md` blanks the **Env var** column for every field whose projection is gone; the fields stay documented +- [ ] `docs/connecting.md` no longer claims the Tailscale opt-in is "one environment variable"; it documents `oh config set install.tailscale true` (persisted, next boot) and `oh tool install tailscale` (live, no reboot) +- [ ] `docs/security-considerations.md`, `docs/harnesses/grok-build.md`, `docs/integrations/langfuse.md`, and the Hermes dashboard docs reflect the retired variables and the removed published port +- [ ] `.env.example` and `.oh/templates/.env.example` drop the retired keys +- [ ] `.oh/scripts/migrate-harness-yaml.sh` stops mapping the retired keys in both directions; the legacy shim itself stays +- [ ] `CHANGELOG.md` gains a **BREAKING** entry under `## [Unreleased]`, one sentence, ≤250 characters, linking #920, covering the eleven variables and the retired overlay's published port +- [ ] `bash .oh/evals/probes/changelog-entry-length.sh` reports no over-length entry + +### US-010: Detect the sandbox flavor at runtime instead of reading a flag + +**Description:** As an operator, I want the entrypoint to determine whether a checkout is bind- +mounted, so compose stops narrating a fact the container can observe. + +**Acceptance Criteria:** + +- [ ] `.oh/.image-seeded` is added to `.gitignore` **before** the detection change — a misdetection must not write an untracked marker into the repo +- [ ] `.devcontainer/entrypoint.sh` replaces the `OH_IMAGE_ONLY` gate with `mountpoint -q "$OH_PROJECT_ROOT"`, the bind-detected branch leading and the seed branch as the `else` +- [ ] The `elif [ -d "$HARNESS_DIR" ]` existence test is dropped as subsumed — a mountpoint necessarily exists +- [ ] Both branch bodies are otherwise unchanged +- [ ] The detected mode is logged on **both** paths, so a wrong auto-detection is visible in `oh logs` +- [ ] `OH_IMAGE_ONLY` is removed from `.devcontainer/docker-compose.image-only.yml`, and no reader of it remains in the tree +- [ ] `diff` of the two compose files' `environment:` blocks is **empty** +- [ ] This lands as a **separate commit** from US-001..US-009, so a boot regression bisects to one commit + +### US-011: Retarget the image-only deploy probe and its documentation + +**Description:** As a maintainer, I want the flavor-B probe to test behavior rather than +documentation prose, since a prose assertion proves nothing and blocks this task's own doc +rewrite. + +**Acceptance Criteria:** + +- [ ] `.oh/evals/probes/oh-image-only-deploy.sh` drops the three assertions that require `deployment-prebuilt-image.md` to contain or omit particular strings +- [ ] Every behavioral assertion is kept: gate ordering, the seed simulation, and the compose-shape checks +- [ ] The gate check targets `mountpoint`-based detection rather than the retired flag +- [ ] `docs/deployment-prebuilt-image.md` documents how flavor B is detected instead of how the flag is set, and its `docker run` recipe drops the corresponding `-e` +- [ ] Mutation-verified: removing the detection from the entrypoint drives the probe to exit 1 +- [ ] `bash .claude/skills/eval/run.sh` exits 0 with no new regressions + +## Functional Requirements + +- **FR-1:** Compose `environment:` in both `docker-compose.yml` and `docker-compose.image-only.yml` must contain exactly `SANDBOX_NAME`, `SANDBOX_PASSWORD`, `TZ`, `CLAUDE_DANGEROUSLY_SKIP_PERMISSIONS`, `CC_SAFETY_NET_STRICT`, `CC_SAFETY_NET_WORKTREE`, `GIT_USER_NAME`, `GIT_USER_EMAIL`, and `GH_TOKEN` — and must be byte-identical to each other. +- **FR-2:** `.devcontainer/entrypoint.sh` must contain no tool or harness installer that duplicates a catalog entry. +- **FR-3:** The entrypoint must obtain every non-FR-1 setting from oh.json through the `oh` CLI, degrading to a documented default when the CLI is unavailable, never failing the boot. +- **FR-4:** Hermes wiring must run whenever the Hermes binary is present, in both flavors. +- **FR-5:** `config-render.ts` must refuse to render any retired key. +- **FR-6:** The sandbox flavor must be determined by observing whether `/home/sandbox/harness` is a bind mount, and the determination must be logged. +- **FR-7:** A tier-A eval probe must fail if any compose file or overlay reintroduces an `INSTALL_*` key, `OH_IMAGE_ONLY`, or an `environment:` key outside the rendered set. +- **FR-8:** Every oh.json field retired from the `.env` projection must remain settable and readable through `oh config set` / `oh config show`. + +## Non-Goals (Out of Scope) + +- **Deleting flavor B.** `/opt/oh-seed` stays regardless, so there is no image saving; retiring #609's no-checkout deploy is a product decision to raise separately. +- **`INSTALL_PYTHON_KERNEL` vs `OH_PROVISION_PYTHON`.** Two names for one concern (`Dockerfile:108-114`, `entrypoint.sh:161`), same class of defect — but a Dockerfile↔entrypoint duplication, not a compose one. Separate issue. +- **`Dockerfile:106`'s whole-repo copy into the `home` stage.** It does not ship (final derives from `base`), so there is no image cost; narrowing it to `.oh/scripts/` is a build-cache win unrelated to this change. +- **Relocating `seed-msg-bridge.sh` / `client-slack-supervise.sh` out of `.devcontainer/`.** They are gateway runtime, but their placement was a deliberate ruling (`docs/rfcs/preserved-changelog-rationale.md:47`) enforced by `oh-devcontainer-restructure.sh:30-31,54`. Overturning it would buy a naming benefit only. +- **Closing the `devcontainer.json` second-lifecycle-door gap.** This task narrows it; closing it is separate. +- **Removing any oh.json field.** Only `.env` projections go. + +## Technical Considerations + +- `oh config show` (`config.ts:31`) already prints resolved oh.json and ships in every released image. A new verb would not — #903's lesson is that a baked `oh` can predate one, and this is the boot path. +- `provision-defaults.sh` calls `oh list --json | jq` the same way; `oh_config` mirrors that precedent rather than inventing a mechanism. +- The seed path already guards itself: `seed_workspace_volume` refuses when `$dest/.oh` exists. Detection adds a second, independent guard (`mountpoint -q` is a kernel fact, not a heuristic), and the gitignore adds a third. + +## Risks + +| Risk | Mitigation | +|---|---| +| `oh config show` unavailable or slow on the boot path | `oh_config` falls back to the documented default and never fails the boot; `config show` ships in every released image | +| Presence-gating Hermes changes *when* the wiring runs | First boot installs Hermes in `provision-defaults.sh` **before** the wiring block, so ordering already works; a mid-session `oh harness install hermes` wires on next boot — same as today's flag. Verify in a live container, do not assume | +| Retiring the dashboard overlay removes a published port someone relies on | Explicit BREAKING entry; cloudflared and Tailscale are already first-class alternatives | +| Deleting the Tailscale installer while `oh tool install` still carries #907's `stdio:"inherit"` sudo hazard | The catalog entry is `installUser: "sandbox"` after #897 — **confirm on the merged tree before deleting the fallback** (US-002) | +| Inverting two probes could mask a real re-bake | The Dockerfile ARG bans in `default-provisioning.sh` are untouched and cover that separately | +| **Flavor misdetection seeds `/opt/oh-seed` over a real checkout** — the worst outcome here | Three independent guards (kernel `mountpoint`, `seed_workspace_volume`'s `.oh` check, the gitignored marker) plus a logged mode on both paths. Verify by booting flavor A with detection in place and confirming zero seed activity | +| A bind that fails to mount now degrades to "seeded workspace" instead of "no checkout" | Both are broken states; the logged mode line makes which one visible in `oh logs`. The flag's failure mode was worse — set it by accident in flavor A and it seeded silently | + +## Success Metrics + +- Adding a tool, harness, or setting requires **zero** compose edits. +- `diff` of the two compose files' `environment:` blocks is empty. +- The boot log shows exactly **one** install line per tool on a cold flavor-A boot. +- Flavor B with `install.hermes=true` produces the same Hermes wiring flavor A does. + +## Verification + +1. `bash .claude/skills/eval/run.sh` — exit 0, no new regressions, new probe PASS. +2. Each inverted and new assertion mutation-verified to exit 1. +3. `cd .oh/cli && npm run build && npm run typecheck`; root `npx vitest run` — expect the known 8 `compose-args.test.ts` environmental failures and nothing new. +4. `shellcheck` over changed shell, matching CI's invocation. +5. **Live flavor A:** `oh sandbox` on a fresh home volume with `install.tailscale=true` and `install.agentBrowser=true` — each binary under `/home/sandbox/.local/bin`, sandbox-owned, prints its version, and the boot log shows **one** install line each. +6. **Live flavor B:** boot `image-only.yml` with `install.hermes=true` — `$HERMES_HOME`, the `.hermes/skills/openharness` symlink, and the provider skill pack all exist. +7. `oh compose config` on both files: no `INSTALL_*`, no `LANGFUSE_*`, no `HERMES_*`, no `OH_IMAGE_ONLY` — and `diff` the two rendered `environment:` blocks to empty. +8. **Detection, both directions.** Flavor A: boot, confirm the log says a bind was detected, `git -C ~/harness status` is clean, and **no** `.oh/.image-seeded` was written. Flavor B: boot on a fresh volume, confirm the seed ran exactly once, reboot, confirm it did not run again. +9. `oh config set install.tailscale true` → `oh config show` round-trips with no `.env` projection. +10. `oh destroy` prompt copy unchanged. +11. **sshd overlay end to end.** `oh config set access.ssh true` (+ a public key), `oh sandbox`, then `ssh -p 2222 sandbox@127.0.0.1` — the port publishes from the overlay and sshd's mode comes from oh.json. Confirm the no-key/no-password warning still fires when both are absent. + +## Implementation order + +Two commits, each independently bootable. **Land them separately** — the flavor detection +touches boot-critical UID logic, and a boot regression must bisect to one commit. + +- **Commit 1 — the env consolidation:** US-001 → US-009. +- **Commit 2 — flavor detection:** US-010 → US-011. + +## Wiki Alignment + +- **Impact**: REQUIRED +- **Local entries**: `.oh/skills/wiki/corpus/compose-env-boundary.md` (new), `.oh/skills/wiki/corpus/sandbox-dependency-installs.md` (update — its `SKIP_PNPM_INSTALL` compose pass-through claims at `:29` become false) +- **Spec alignment**: The new entry states the boundary rule from this PRD verbatim, names the surviving compose keys and why each survives, records that catalogs are the sole source of install pins, and documents runtime flavor detection replacing `OH_IMAGE_ONLY`. It must reflect this PRD's non-goals — flavor B survives, `INSTALL_PYTHON_KERNEL` is out of scope — so a later reader does not treat them as oversights. The `sandbox-dependency-installs` update must replace the compose-flag description with the oh.json read and re-cite the moved lines. +- **Acceptance criteria** (carried by US-009): + - [ ] `.oh/skills/wiki/corpus/compose-env-boundary.md` exists with valid frontmatter (`title`, `slug`, `tags`, `created`, `updated`, `sources`, `confidence`) and the body order H1 → `## Relevant Source Files` → `## Summary` → `## Detail` → `## System Relationships` → `## See Also` + - [ ] Every claim about repository behavior cites a source path with a line number; `## System Relationships` carries a Mermaid diagram of the oh.json → CLI → entrypoint path + - [ ] The entry is ≤900 words (architecture allowance) and `## See Also` cross-links `[[sandbox-dependency-installs]]` and `[[oh-cli-portable-lifecycle]]` + - [ ] `.oh/skills/wiki/corpus/sandbox-dependency-installs.md` no longer claims compose passes `SKIP_PNPM_INSTALL` into the container, and its `updated` date is bumped + - [ ] `bash .oh/evals/probes/wiki-readme-index.sh` passes with the new entry indexed in `.oh/skills/wiki/corpus/README.md` + +## Open Questions + +None. The plan was approved with `OH_IMAGE_ONLY` removal (option A), the Hermes dashboard +overlay retired, and "fix, don't probe" chosen for flavor-B parity. diff --git a/.oh/tasks/compose-env-boundary/progress.txt b/.oh/tasks/compose-env-boundary/progress.txt new file mode 100644 index 00000000..15a6a5c8 --- /dev/null +++ b/.oh/tasks/compose-env-boundary/progress.txt @@ -0,0 +1,112 @@ +# progress + +2026-08-31 — US-001 Strip the eleven variables from the compose files — PASS + Files: .devcontainer/docker-compose.yml, docker-compose.image-only.yml, + docker-compose.ssh.yml, docker-compose.hermes-dashboard.yml (deleted), + .oh/scripts/docker-compose.sh + Commit: 4425329d + Result: docker-compose.yml lost eleven environment: lines; image-only.yml lost four; + the sshd overlay kept its ports: and lost its environment: block; the Hermes dashboard + overlay and the docker-compose.sh branch that applied it are gone. + Learning: the sshd overlay was the same defect one level down — four keys restating + access.ssh* that config-render.ts already rendered. + +2026-08-31 — US-002 Delete the duplicate installers — PASS + Files: .devcontainer/entrypoint.sh + Commit: 4425329d + Result: ~75 lines removed, including the Tailscale version and both sha256 literals. + The ungated `install -d /var/run/tailscale` is kept, so `oh tool install tailscale` + still needs no reboot. + Learning: the plan's risk item was real and checkable — the catalog entry is + installUser:"sandbox" on the merged tree, so removing the entrypoint fallback does not + strand the install behind #907's stdio:"inherit" sudo hazard. Verified before deleting. + +2026-08-31 — US-003 Read oh.json from the entrypoint — PASS + Files: .devcontainer/entrypoint.sh, .oh/scripts/link-providers.sh, + .oh/scripts/sandbox-healthcheck.sh, .oh/install/banner.sh + Commit: 4425329d + Result: oh_config caches one `oh config show` and answers jq filters from it; + oh_config_truthy wraps the boolean case. Hermes wiring, the dashboard launch, the pnpm + gate, and the sshd block all read oh.json. + Learning: presence-gating check_hermes_link on `command -v hermes` was wrong — a + worktree has no .hermes/, so the check fired where the old flag had skipped it. The + right gate is "verify the link if it exists"; the write path already keys on the binary. + Two consumers outside the entrypoint (sandbox-healthcheck.sh, banner.sh) also read + HERMES_DASHBOARD from the environment and would have gone silently always-false; both + now read oh.json with jq. + +2026-08-31 — US-004 Cron agent binary from oh.json — PASS + Files: .oh/scripts/cron-runtime.ts, .oh/scripts/__tests__/cron-runtime.test.ts + Commit: 4425329d + Result: resolveAgentBin() with env > oh.json > "claude", cached, plus + resetAgentBinCache() for tests. Three precedence tests. + Learning: importing the CLI's readOhConfig directly is not safe here — cron-runtime.ts + runs under `node --experimental-strip-types` and the CLI's ESM imports carry .js + specifiers that would not resolve. Shelling `oh config show` keeps it CLI-mediated. + +2026-08-31 — US-005 Stop rendering the retired keys — PASS + Files: .oh/cli/src/lib/config-render.ts, .oh/cli/src/lib/tools/catalog.ts + Commit: 4425329d + Result: rendered set is exactly the ten host-side keys; RETIRED_KEYS holds eighteen. + Learning: SANDBOX_SSH_PASSWORD_AUTH and SANDBOX_SSH_AUTHORIZED_KEYS also had to go — + the plan's "SANDBOX_SSH*" glob would have kept two projections nothing reads. + +2026-08-31 — US-006 Invert the boundary probes — PASS + Files: .oh/evals/probes/tool-catalog-boundary.sh, tailscale-tool-boundary.sh, + compose-config-path-parity.sh + Commit: 4425329d + Result: guard, pin, and sha checks reversed onto the catalog; every zero-exposure + assertion kept. Mutations verified: guard back (both probes), pin back, sha back, + socket-dir indented, catalog entrypointGuard back — all exit 1. + Learning: `x=$(... | grep -c ...)` under `set -e` aborts the probe silently when the + count is 0, because grep exits 1. A `[[ $var == [[:space:]]* ]]` test is the safe form. + +2026-08-31 — US-007 Tier-A compose-env-boundary probe — PASS + Files: .oh/evals/probes/compose-env-boundary.sh + Commit: 531425f5 (see divergence note in evidence.md) + Result: derives the allowed set from config-render.ts's put() calls plus four + documented literals, and walks every environment: block by indentation. Mutations + verified: INSTALL_FOO, OH_IMAGE_ONLY, an unlisted key, and an overlay environment: + key each exit 1; adding an overlay port stays PASS. + +2026-08-31 — US-008 Update the tests — PASS + Files: eight test files under .oh/cli and .oh/scripts + Commit: 4425329d + Result: 976 passing. The 6 remaining compose-args failures reproduce unmodified on + development, so they are environmental, not caused here. Retargeting those tests onto + the sshd overlay incidentally fixed 2 of the 8 that were failing before. + +2026-08-31 — US-009 Docs, templates, wiki, changelog — PASS + Files: 14 docs, .oh/README.md, .oh/templates/full/.hermes/README.md, CHANGELOG.md, + .oh/skills/wiki/corpus/{compose-env-boundary.md,sandbox-dependency-installs.md,README.md} + Commit: 4425329d + Result: the configuration.md "Compose variable" column is `—` for every retired field, + and the section above it now states the rule. New wiki entry at 596 words with + line-cited claims and a Mermaid route diagram; index regenerated; probe passes. + Learning: docs-compose-overlays.test.ts scans docs/rfcs/, which holds dated records — + preserved-changelog-rationale.md quotes a changelog entry verbatim. Rewriting a compose + path inside it would falsify the history it exists to keep, so the test now skips that + root. .env.example needed no change: it is secrets-only and already described the + Langfuse non-secrets as shell exports. + +2026-08-31 — US-010 Detect the sandbox flavor — PASS + Files: .gitignore, .devcontainer/entrypoint.sh, .devcontainer/docker-compose.image-only.yml + Commit: 531425f5 + Result: `mountpoint -q "$HARNESS_DIR" && [ -d "$HARNESS_DIR/.oh" ]`, branches swapped, + mode logged on both paths. The two environment: blocks diff to empty. + Learning: `mountpoint -q` alone was not enough. docs/runtimes/microsandbox.md mounts a + fresh host directory straight at /home/sandbox/harness — a mountpoint with no checkout — + which the single test would have sent down the UID-sync path and never seeded. The .oh/ + conjunct fixes that; the mountpoint conjunct is still needed, or a seeded no-bind volume + would take the sync path on its second boot. Simulated all three cases. + +2026-08-31 — US-011 Retarget the flavor probe and its doc — PASS + Files: .oh/evals/probes/oh-image-only-deploy.sh, docs/deployment-prebuilt-image.md, + docs/runtimes/microsandbox.md + Commit: 531425f5 + Result: the three prose assertions are gone; gate ordering, the seed simulation, and + every compose-shape check are kept, plus new assertions for the gitignored marker and + the logged mode. Mutations verified: detection removed, mode log removed, gitignore + rule removed, flag re-added — all exit 1. + +STATUS: COMPLETE diff --git a/.oh/tasks/compose-env-boundary/prompt.md b/.oh/tasks/compose-env-boundary/prompt.md new file mode 100644 index 00000000..3925949f --- /dev/null +++ b/.oh/tasks/compose-env-boundary/prompt.md @@ -0,0 +1,31 @@ +# `/spec execute` task — compose-env-boundary + +You are the single implementation Advisor for the `compose-env-boundary` task. Read the approved plan in +`.oh/tasks/compose-env-boundary/prd.md` and the ordered stories in `.oh/tasks/compose-env-boundary/prd.json`. + +- Branch: `task/920-compose-env-boundary` — never push to `development` or `main`. +- Issue: #920. +- Task folder: `.oh/tasks/compose-env-boundary/` (`prd.md`, `prd.json`, `prompt.md`, `progress.txt`). + +## Ownership + +You own this task from implementation through the final PR gate. Do not hand the task to a +second implementation owner or a second supervisory session. Use `/delegate` only for bounded, +disjoint work that can run in parallel. Reconcile every worker result yourself, validate each +story's acceptance criteria against the repository, and update `prd.json` and `progress.txt`. + +## Implementation cycle + +1. Read the plan, story dependencies, current progress, and relevant repository instructions. +2. Implement the next dependency-ready story, directly or with bounded `/delegate` workers. +3. Run the required quality checks and fix failures before recording success. +4. Set that story's `passes` field to `true` only after validation. Add a dated progress entry + with the files, commit, result, and learnings. Every implementation commit needs a mandatory + `Submitted-by: ` trailer. +5. Continue until every story passes. Do not claim completion when a story is blocked or + deferred. Append `STATUS: COMPLETE` to `progress.txt` only when the full graph passes. + +After implementation completes, continue in the same Advisor session with the `/spec execute` +procedure: run the implementation-side audit loop, run `/eval` once, revise required wiki entries, +write and commit `evidence.md`, run `/spec retro` and the improve steps, then run a fresh +`/audit pr`. Mark the PR ready only when that audit is promotable. Never merge the PR. diff --git a/.oh/templates/full/.hermes/README.md b/.oh/templates/full/.hermes/README.md index a9243338..d4dcf481 100644 --- a/.oh/templates/full/.hermes/README.md +++ b/.oh/templates/full/.hermes/README.md @@ -1,6 +1,6 @@ # Hermes runtime home -OpenHarness sets `HERMES_HOME=/.hermes` when `INSTALL_HERMES=true`. +OpenHarness sets `HERMES_HOME=/.hermes` whenever the `hermes` binary is installed. This directory holds Hermes project-local runtime state (config, memory, skills, sessions, scheduled-task metadata). **Only `config.yaml`, `SOUL.md`, and this diff --git a/CHANGELOG.md b/CHANGELOG.md index c32d4897..5200da20 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,7 @@ Update policy and release automation live in [`/git`](.claude/skills/git/SKILL.m ## [Unreleased] ### Changed +- **BREAKING:** Move eleven settings out of the compose `environment:` block into oh.json, read through the `oh` CLI; a hand-edited `.devcontainer/.env` no longer carries them ([#920](https://github.com/mifunedev/openharness/issues/920)). - Make `/spec` ship by default: an unrecognized first token routes to a new `ship` node that runs `plan` then `execute`, so `/spec ` produces a ready-for-review PR ([#914](https://github.com/mifunedev/openharness/issues/914)). - **BREAKING:** Persist the sandbox home through one `/home/sandbox` mount, not eleven per-tool volumes; set `storage.homePath` for a host path, else `_workspace` ([#898](https://github.com/mifunedev/openharness/issues/898)). - Shrink the sandbox image ~540 MB: drop build caches from the baked home seed, stage the seed once via a builder stage, and keep untracked build output out of the build context ([#900](https://github.com/mifunedev/openharness/issues/900)). @@ -17,6 +18,9 @@ Update policy and release automation live in [`/git`](.claude/skills/git/SKILL.m - **BREAKING:** Stop baking OpenCode, Hermes, and Grok Build into the image; `oh harness install ` installs them into `~/.local` as the sandbox user ([#908](https://github.com/mifunedev/openharness/issues/908)). ### Removed +- **BREAKING:** Retire the `OH_IMAGE_ONLY` flag; `entrypoint.sh` detects the sandbox flavor from whether `/home/sandbox/harness` is a bind mount holding `.oh/`, and logs the detected mode ([#920](https://github.com/mifunedev/openharness/issues/920)). +- **BREAKING:** Retire `docker-compose.hermes-dashboard.yml` and its published `127.0.0.1:9119`; the dashboard now binds container loopback, reachable over cloudflared or Tailscale ([#920](https://github.com/mifunedev/openharness/issues/920)). +- Remove the duplicate agent-browser and Tailscale installers from `entrypoint.sh`; the tool catalog is the sole owner of both pins and Tailscale's two checksums ([#920](https://github.com/mifunedev/openharness/issues/920)). - Remove the `BAKE_HARNESSES` and `AGENTS` build args along with the image bake they gated; the harness catalog is the only source of truth for what gets installed ([#904](https://github.com/mifunedev/openharness/issues/904)). - Remove Cloudflare's apt repository and its bookworm-suite pin from the image; Docker's is now the only third-party apt source ([#906](https://github.com/mifunedev/openharness/issues/906)). - Remove the four optional-harness build args and the dead `buildArg` catalog field; the `install.*` keys keep working and now drive boot provisioning ([#908](https://github.com/mifunedev/openharness/issues/908)). diff --git a/docs/configuration.md b/docs/configuration.md index 39468d1c..68f8be42 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -23,18 +23,33 @@ and `oh secret list` shows which keys hold a value with the values redacted. key, each pointing at the other command. Apply a change with `oh stop && oh sandbox`. -## How `oh.json` reaches Docker Compose +## How `oh.json` reaches the sandbox -`.oh/cli/src/lib/config-render.ts` renders `oh.json` into `KEY=value` lines and -`.oh/scripts/docker-compose.sh` passes them to Compose with `--env-file`. Every -key also has a default baked into `.devcontainer/docker-compose.yml`, so an -omitted field is not "unset" — it takes that default. A variable already -exported in the shell that runs `oh` beats the value in `oh.json`. +There are two routes, and which one a field takes follows one rule: + +> A value reaches the sandbox through Compose only if a process **outside** the +> sandbox — or the entrypoint **before** the control plane is readable — must act +> on it. Everything else is read from `oh.json` through the `oh` CLI. + +**Through Compose.** `.oh/cli/src/lib/config-render.ts` renders those fields into +`KEY=value` lines and `.oh/scripts/docker-compose.sh` passes them to Compose with +`--env-file`. Each also has a default baked into +`.devcontainer/docker-compose.yml`, so an omitted field is not "unset" — it takes +that default. A variable already exported in the shell that runs `oh` beats the +value in `oh.json`. + +**Through the CLI.** Everything else is read inside the container at the moment +it is needed — `.devcontainer/entrypoint.sh` calls `oh config show`, +`.oh/scripts/provision-defaults.sh` installs from `oh harness list --json` and +`oh tool list --json`. Adding a tool, harness, or setting therefore requires no +Compose edit. `config-render.ts` keeps a `RETIRED_KEYS` list that throws if one +of these is ever rendered again. ## Field reference Types are JSON types. "Compose variable" names the variable the field renders -to; `—` means the field is consumed by the `oh` CLI itself and never rendered. +to; `—` means the field never reaches Compose — it is read through the `oh` CLI, +or consumed by the CLI itself. ### Identity @@ -54,7 +69,9 @@ to; `—` means the field is consumed by the `oh` CLI itself and never rendered. ### Optional installs -All off by default. `oh harness install ` flips the matching field and +All off by default and all read through the CLI, never through Compose: +`.oh/scripts/provision-defaults.sh` installs everything the catalogs report as +`enabled` on every boot. `oh harness install ` flips the matching field and installs into the running sandbox with no rebuild. The four harness fields map to `oh harness` names: `opencode`, `grok-build`, `hermes`. `agentBrowser` and `tailscale` are not harnesses — `oh tool install agent-browser` @@ -62,11 +79,11 @@ and `oh tool install tailscale` manage them. | Field | Type | Default | Compose variable | What it does | | --- | --- | --- | --- | --- | -| `install.opencode` | boolean | `false` | `INSTALL_OPENCODE` | Install the OpenCode CLI into `~/.local` at boot. `oh harness install opencode` sets it and installs now. | -| `install.grokBuild` | boolean | `false` | `INSTALL_GROK_BUILD` | Install the Grok Build CLI into `~/.local` at boot. `oh harness install grok-build` sets it and installs now. | -| `install.hermes` | boolean | `false` | `INSTALL_HERMES` | Install the Hermes CLI into `~/.local` at boot and enable its runtime wiring (skill vendoring, `auth.json`). | -| `install.agentBrowser` | boolean | `false` | `INSTALL_AGENT_BROWSER` | Install agent-browser and Chromium (about 1 GB). | -| `install.tailscale` | boolean | `false` | `INSTALL_TAILSCALE` | Install the Tailscale client for private remote access (userspace networking; no container capabilities). | +| `install.opencode` | boolean | `false` | — | Install the OpenCode CLI into `~/.local` at boot. `oh harness install opencode` sets it and installs now. | +| `install.grokBuild` | boolean | `false` | — | Install the Grok Build CLI into `~/.local` at boot. `oh harness install grok-build` sets it and installs now. | +| `install.hermes` | boolean | `false` | — | Install the Hermes CLI into `~/.local` at boot. The runtime wiring (skill vendoring, `auth.json`) keys off the binary being present, so it also runs after `oh harness install hermes`, in both sandbox flavors. | +| `install.agentBrowser` | boolean | `false` | — | Install agent-browser and Chromium (about 1 GB). | +| `install.tailscale` | boolean | `false` | — | Install the Tailscale client for private remote access (userspace networking; no container capabilities). | ### Access @@ -75,27 +92,27 @@ and `oh tool install tailscale` manage them. | `access.dockerSocket` | boolean | `false` | `DOCKER_SOCKET` | Applies the `docker-compose.docker-sock.yml` overlay. Mounting `/var/run/docker.sock` is effectively HOST ROOT: an agent can start a privileged container that mounts the host filesystem. See [security considerations](security-considerations.md). | | `access.ssh` | boolean | `false` | `SANDBOX_SSH` | Applies the `docker-compose.ssh.yml` overlay, which runs sshd for direct container SSH. See [sshd](integrations/sshd.md). | | `access.sshPort` | number (1–65535) | `2222` | `SANDBOX_SSH_PORT` | Host loopback port published for SSH. | -| `access.sshAuthorizedKeys` | string | unset | `SANDBOX_SSH_AUTHORIZED_KEYS` | One or more public keys, newline or literal `\n` separated. This is public key material, not a secret. Without a key and without password auth nobody can log in, and sshd warns loudly. | -| `access.sshPasswordAuth` | boolean | `false` | `SANDBOX_SSH_PASSWORD_AUTH` | Enables SSH password auth, which uses the `SANDBOX_PASSWORD` secret. Never enable it on a public-facing bind while `SANDBOX_PASSWORD` is the default. | +| `access.sshAuthorizedKeys` | string | unset | — | One or more public keys, newline or literal `\n` separated, read by `entrypoint.sh` through `oh config show`. This is public key material, not a secret. Without a key and without password auth nobody can log in, and sshd warns loudly. | +| `access.sshPasswordAuth` | boolean | `false` | — | Enables SSH password auth, which uses the `SANDBOX_PASSWORD` secret. Never enable it on a public-facing bind while `SANDBOX_PASSWORD` is the default. | ### Hermes dashboard | Field | Type | Default | Compose variable | What it does | | --- | --- | --- | --- | --- | -| `hermesDashboard.enabled` | boolean | `false` | `HERMES_DASHBOARD` | Applies the `docker-compose.hermes-dashboard.yml` overlay and auto-starts the web dashboard. | -| `hermesDashboard.port` | number (1–65535) | `9119` | `HERMES_DASHBOARD_PORT` | Host loopback port for the dashboard. | +| `hermesDashboard.enabled` | boolean | `false` | — | Auto-starts the web dashboard in the `app-hermes-dashboard` tmux session, bound to container loopback. | +| `hermesDashboard.port` | number (1–65535) | `9119` | — | Container loopback port for the dashboard. It is no longer published to the host; reach it from inside the sandbox, or over cloudflared or Tailscale. | ### Cron runtime | Field | Type | Default | Compose variable | What it does | | --- | --- | --- | --- | --- | -| `cron.agentBin` | string | `claude` | `CRON_AGENT_BIN` | Binary that fires scheduled tasks. | +| `cron.agentBin` | string | `claude` | — | Binary that fires scheduled tasks. | ### Build behaviour | Field | Type | Default | Compose variable | What it does | | --- | --- | --- | --- | --- | -| `build.skipPnpmInstall` | boolean | `false` | `SKIP_PNPM_INSTALL` | Renders as `1`/`0`. `1` skips the entrypoint's `pnpm install`. | +| `build.skipPnpmInstall` | boolean | `false` | — | `true` skips the entrypoint's root `pnpm install`. Use it when the dependency tree is managed outside the sandbox. | ### Prebuilt image @@ -117,20 +134,21 @@ Recipe: [prebuilt-image deployment](deployment-prebuilt-image.md). ### Langfuse Tracing settings the Pi harness reads from its own process environment. They are -not secrets — the Langfuse key pair is, and lives in `.env`. Compose passes both -into the container's environment, so a value set here reaches Pi on the next -sandbox start. An export in the sandbox shell still wins for that shell. +not secrets — the Langfuse key pair is, and lives in `.env`. The harness does not +project these into the container: export them in the shell that launches Pi. +They remain settable here so a deployment can record its intended values in one +tracked place. | Field | Type | Default | Compose variable | What it does | | --- | --- | --- | --- | --- | -| `langfuse.baseUrl` | string | unset | `LANGFUSE_BASE_URL` | Langfuse host Pi sends traces to, for example `http://langfuse-web:3000`. Takes precedence over `LANGFUSE_HOST`. | -| `langfuse.privacyPreset` | `"metadata-only"` \| `"prompts-only"` \| `"conversations"` \| `"full-debug"` | unset (compose default `metadata-only`) | `LANGFUSE_PRIVACY_PRESET` | How much of each trace Pi captures. Prefer `metadata-only` unless a broader capture policy is approved. | +| `langfuse.baseUrl` | string | unset | — | Langfuse host Pi sends traces to, for example `http://langfuse-web:3000`. Takes precedence over `LANGFUSE_HOST`. | +| `langfuse.privacyPreset` | `"metadata-only"` \| `"prompts-only"` \| `"conversations"` \| `"full-debug"` | unset (compose default `metadata-only`) | — | How much of each trace Pi captures. Prefer `metadata-only` unless a broader capture policy is approved. | ### Compose overlays | Field | Type | Default | Compose variable | What it does | | --- | --- | --- | --- | --- | -| `composeOverrides` | string[] | `[]` | — | Extra `-f` overlay paths, applied after the built-in overlays selected by `access` and `hermesDashboard` (last `-f` wins). | +| `composeOverrides` | string[] | `[]` | — | Extra `-f` overlay paths, applied after the built-in overlays selected by `access` (last `-f` wins). | ## Secrets diff --git a/docs/connecting.md b/docs/connecting.md index eba61a79..52da6cbc 100644 --- a/docs/connecting.md +++ b/docs/connecting.md @@ -118,10 +118,10 @@ This is the supported path for reaching T3 Code from a phone, and the supported - No `NET_ADMIN`, no `/dev/net/tun`, no `privileged: true`, no host socket mount. Userspace networking needs none of them, and Tailscale Serve is fully supported in that mode. - **No host port is published.** T3 Code stays on container loopback `127.0.0.1:3773`. Tailscale Serve inside the container proxies tailnet HTTPS to that loopback address. A device outside the tailnet has nothing to reach. -- The only compose change is one environment variable (`INSTALL_TAILSCALE`). Node identity and daemon state live in `/home/sandbox/.tailscale`, inside the single `/home/sandbox` mount, so the node does not re-authenticate on every container recreate without any per-tool volume. +- **There is no compose change at all.** The opt-in is `install.tailscale` in `oh.json`; `.oh/scripts/provision-defaults.sh` installs from the tool catalog on every boot. Node identity and daemon state live in `/home/sandbox/.tailscale`, inside the single `/home/sandbox` mount, so the node does not re-authenticate on every container recreate without any per-tool volume. - Because the container is the node, the MagicDNS name your phone saved does not change when you move the workspace to another VM. -Installing the binary does **not** join a tailnet. The entrypoint never runs `tailscaled` and never runs `tailscale up`. Joining is an explicit human act. +Installing the binary does **not** join a tailnet. Nothing on the boot path runs `tailscaled` or `tailscale up`. Joining is an explicit human act. ### Prerequisites @@ -147,7 +147,7 @@ oh tool install tailscale This persists `install.tailscale: true` in the tracked `oh.json` so the opt-in survives container recreation, and installs the binary into a running sandbox when one is up. It is idempotent. -If the sandbox was not running, the flag is persisted only. Run `oh sandbox` to recreate the container and let the entrypoint install the binary. No rebuild of the image is required — the install is an entrypoint step gated on `INSTALL_TAILSCALE`, not a build layer. Nothing about networking activates until you start the daemon in the next step. +If the sandbox was not running, the flag is persisted only. Run `oh sandbox` to recreate the container; boot provisioning reads `install.tailscale` from `oh.json` and installs the binary from the tool catalog, which is the sole owner of the pinned version and its checksums. No image rebuild is required. Nothing about networking activates until you start the daemon in the next step. Check the state at any time: diff --git a/docs/deployment-prebuilt-image.md b/docs/deployment-prebuilt-image.md index 6c2d4378..c1a2bfe8 100644 --- a/docs/deployment-prebuilt-image.md +++ b/docs/deployment-prebuilt-image.md @@ -153,19 +153,31 @@ Everything the sandbox persists — the workspace and control plane at declared in that file: the named volume `_workspace` by default, or an absolute host path when `OH_HOME_MOUNT` is set. -### `OH_IMAGE_ONLY=1` +### How the flavor is detected -The compose file sets `OH_IMAGE_ONLY=1` in the container environment. This is -the entrypoint flag that switches `entrypoint.sh` into **no-bind mode**: +Nothing declares the flavor. `entrypoint.sh` asks whether +`/home/sandbox/harness` is a bind mount **and** already holds a `.oh/` directory, +and reads the answer from the kernel and the filesystem: -- the host UID/GID sync block is skipped (there is no host directory to read - ownership from) -- the workspace mount is `chown`'d to the sandbox user instead -- the first-boot seed (below) runs before `link-providers`, the root - `pnpm install`, and cron tmux setup, so those steps see a populated `.oh/` +- **checkout bind present** (Flavor A) — sync the sandbox UID/GID to the host + directory's owner, and never seed. +- **anything else** (this flavor, and a runtime that mounts a fresh empty host + directory at the project root) — skip the UID/GID sync, since there is no host + directory to read ownership from; `chown` the workspace to the sandbox user; + and run the first-boot seed (below) before `link-providers`, the root + `pnpm install`, and cron tmux setup, so those steps see a populated `.oh/`. -Prebuilt-image mode (Flavor A) never sets this flag — it always keeps the bind -mount, so its host-UID-sync path is unchanged. +The detected mode is logged on both paths, so a wrong detection is visible in +`oh logs` rather than silent: + +``` +[entrypoint] checkout bind detected at /home/sandbox/harness — syncing host UID/GID +[entrypoint] no checkout bind at /home/sandbox/harness — seeding from /opt/oh-seed +``` + +Three independent guards keep a misdetection from seeding over a real checkout: +`mountpoint -q` is a kernel fact rather than a heuristic, `seed_workspace_volume` +refuses when `.oh/` already exists, and `.oh/.image-seeded` is gitignored. ### Seed-to-volume persistence @@ -213,7 +225,6 @@ docker volume rm "${NAME}_workspace" 2>/dev/null || true # the whole sandbox h # ── 2. Fresh run (no bind mount, no build) ───────────────────────── docker run -d --name "$NAME" --restart unless-stopped --init \ - -e OH_IMAGE_ONLY=1 \ -e GIT_USER_NAME="ryaneggz" \ -e GIT_USER_EMAIL="kre8mymedia@gmail.com" \ -e GH_TOKEN="${GH_TOKEN:-}" \ diff --git a/docs/harnesses/grok-build.md b/docs/harnesses/grok-build.md index d3044d41..f68f7252 100644 --- a/docs/harnesses/grok-build.md +++ b/docs/harnesses/grok-build.md @@ -29,10 +29,10 @@ install: grok_build: true ``` -Or set the legacy flag in `.devcontainer/.env`: +Or set it from the CLI: ```bash -INSTALL_GROK_BUILD=true +oh config set install.grokBuild true ``` Either way the boot provisioner installs it on the next start — no rebuild: diff --git a/docs/harnesses/hermes.md b/docs/harnesses/hermes.md index 05940088..c2fa0bfa 100644 --- a/docs/harnesses/hermes.md +++ b/docs/harnesses/hermes.md @@ -10,7 +10,7 @@ skills from experience, scheduled task automation, sub-agent delegation, container sandboxing across multiple backends, and bridges to chat platforms (Telegram, Discord, Slack, WhatsApp, Signal, Email). -Hermes is an **optional harness** in Open Harness. Install it with `oh harness install hermes` (or set `install.hermes` / `INSTALL_HERMES=true`, which the boot provisioner honours); it then sits alongside `claude`, `codex`, +Hermes is an **optional harness** in Open Harness. Install it with `oh harness install hermes` (or set `install.hermes` in `oh.json`, which the boot provisioner honours); it then sits alongside `claude`, `codex`, `pi`, and `opencode` as a sandbox CLI primitive. See the upstream documentation below for canonical facts about Hermes. @@ -41,14 +41,14 @@ See [Harnesses Overview](./overview.md#installing-a-harness) for `--persist-only ### Manual path -Hermes is disabled by default. To install it into the sandbox image, uncomment -the key in `.devcontainer/.env`: +Hermes is disabled by default. To have boot provisioning install it, set the +field in the tracked `oh.json`: ```bash -INSTALL_HERMES=true +oh config set install.hermes true ``` -Then rebuild/restart the sandbox: +Then restart the sandbox: ```bash oh stop && oh sandbox @@ -93,11 +93,13 @@ hermes setup --portal # Nous Portal OAuth integration hermes doctor # health check ``` -Config, memory, runtime skills, and sessions write to `~/harness/.hermes/` -through `HERMES_HOME=/home/sandbox/harness/.hermes`. On boot with -Hermes enabled, the entrypoint links `.hermes/skills/openharness` to the -tracked shared skills directory (`.oh/skills/`), making the same harness -skills used by Claude, Codex, and Pi visible to Hermes by default. +Config, memory, runtime skills, and sessions write to `~/harness/.hermes/`, +which the entrypoint sets as `HERMES_HOME`. On every boot where the `hermes` +binary is present — the wiring keys off the binary, not off a flag, so it runs +identically in both sandbox flavors — the entrypoint links +`.hermes/skills/openharness` to the tracked shared skills directory +(`.oh/skills/`), making the same harness skills used by Claude, Codex, and Pi +visible to Hermes by default. Auth lives directly inside `HERMES_HOME` (`~/harness/.hermes/auth.json`). No symlink or named volume is involved: an earlier design symlinked @@ -193,33 +195,31 @@ It is **disabled by default** and opt-in per sandbox. ### Enabling -In `.devcontainer/.env`, set alongside `INSTALL_HERMES=true`: +Set the fields in the tracked `oh.json`: ```bash -HERMES_DASHBOARD=true -HERMES_DASHBOARD_PORT=9119 # optional; 9119 is the default +oh config set hermesDashboard.enabled true +oh config set hermesDashboard.port 9119 # optional; 9119 is the default ``` -Then rebuild: +Then restart: ```bash oh stop && oh sandbox ``` -`HERMES_DASHBOARD` requires `INSTALL_HERMES=true` to take effect: the dashboard -overlay is applied whether or not Hermes is installed, but there is nothing for -it to serve without the binary. +The dashboard needs the `hermes` binary; without it there is nothing to serve and +the entrypoint skips the launch. ### What auto-launches -When both `INSTALL_HERMES=true` and `HERMES_DASHBOARD=true` are set (or -the equivalent legacy env vars), the entrypoint starts the dashboard in a -named tmux session: +When `hermesDashboard.enabled` is true and `hermes` is installed, the entrypoint +starts the dashboard in a named tmux session: - **tmux session**: `app-hermes-dashboard` -- **Container bind**: `0.0.0.0:` (all container interfaces — required so Docker's published port can reach the process; set via `HERMES_DASHBOARD_HOST=0.0.0.0` and `HERMES_DASHBOARD_INSECURE=true` in the compose overlay) -- **Host publish**: `127.0.0.1:9119 → container:9119` (loopback-only on the host) -- **URL** (from the host browser): `http://127.0.0.1:9119` +- **Container bind**: `127.0.0.1:` — container loopback only +- **Host publish**: none. The dashboard is no longer published to the host. +- **URL** (from inside the sandbox): `http://127.0.0.1:9119` ### Inspect and restart @@ -233,31 +233,28 @@ tail -f /tmp/app-hermes-dashboard.log # Restart (kill session, then relaunch manually or rebuild sandbox) tmux kill-session -t app-hermes-dashboard tmux new-session -d -s app-hermes-dashboard \ - "hermes dashboard --port ${HERMES_DASHBOARD_PORT:-9119} --host 0.0.0.0 --insecure --no-open 2>&1 | tee /tmp/app-hermes-dashboard.log" + "hermes dashboard --port 9119 --host 127.0.0.1 --no-open 2>&1 | tee /tmp/app-hermes-dashboard.log" ``` ### Security -The dashboard reads and writes `.env` secrets and `config.yaml`. The -compose overlay intentionally binds the **in-container** process to -`0.0.0.0` (via `HERMES_DASHBOARD_HOST=0.0.0.0` and -`HERMES_DASHBOARD_INSECURE=true`) — this non-loopback container bind is -required for Docker's port publishing mechanism to forward traffic from -the host into the container. The **host-side** publish is loopback-only -(`127.0.0.1:9119`), so the port is never reachable from the LAN. +The dashboard reads and writes `.env` secrets and `config.yaml`, and it binds to +**container loopback** only. Nothing publishes it to the host, so it is reachable +from inside the sandbox and from an explicit tunnel — never from the LAN, and not +from the host browser without one. -Because only processes on the local machine can reach `127.0.0.1:9119`, -**no additional authentication is required** — access is equivalent to -existing host-shell access and does not widen the attack surface. +Because only processes inside the container can reach `127.0.0.1:9119`, +**no additional authentication is required** by default. -Do **not** change the host bind to `0.0.0.0` — that would expose the -dashboard (and the `.env` secrets it reads) to the LAN without auth. +Do **not** change the bind to `0.0.0.0` — that would expose the dashboard (and +the `.env` secrets it reads) to anything that can route to the container. ### Remote access To reach the dashboard from another machine, use `/cloudflared 9119` to -start a Cloudflared tunnel for the loopback bind. The tunnel handles TLS; -the dashboard itself stays on loopback. +start a Cloudflared tunnel for the loopback bind, or reach it over the tailnet +with `oh tool install tailscale`. The tunnel handles TLS; the dashboard itself +stays on loopback. For sensitive dashboards, add Cloudflare Access or another authentication gate before sharing the URL. If you intentionally change Hermes to a diff --git a/docs/harnesses/opencode.md b/docs/harnesses/opencode.md index 9122304f..720b056d 100644 --- a/docs/harnesses/opencode.md +++ b/docs/harnesses/opencode.md @@ -27,7 +27,11 @@ install: opencode: true ``` -Or set `INSTALL_OPENCODE=true` in `.devcontainer/.env` (legacy). +Or set it from the CLI: + +```bash +oh config set install.opencode true +``` Either way the boot provisioner installs it on the next start — no rebuild: diff --git a/docs/harnesses/overview.md b/docs/harnesses/overview.md index 03afb587..f469060c 100644 --- a/docs/harnesses/overview.md +++ b/docs/harnesses/overview.md @@ -4,7 +4,7 @@ title: "Harnesses Overview" # Harnesses Overview -Open Harness provisions three agent CLIs into `~/.local` on first boot: **Claude Code** (default), **Codex**, and **Pi**. **OpenCode**, **Hermes**, and **Grok Build** are optional — install one with `oh harness install `, which also sets its `install.*` key so a fresh home mount reinstalls it at boot. No harness is baked into the image. **T3 Code** runs on demand via the `/t3` skill (or directly with `npx t3`) as a browser UI on port 3773, and **Prime Agent** installs on demand with `oh harness install prime-agent` — neither has an `INSTALL_*` key, because neither is ever baked into the image. Inside the sandbox, run `herdr` first, then launch whichever agent you prefer from its panes and switch between them at any time. Reserve tmux for Open Harness's managed/headless cron, gateway, and watchdog infrastructure. +Open Harness provisions three agent CLIs into `~/.local` on first boot: **Claude Code** (default), **Codex**, and **Pi**. **OpenCode**, **Hermes**, and **Grok Build** are optional — install one with `oh harness install `, which also sets its `install.*` key so a fresh home mount reinstalls it at boot. No harness is baked into the image. **T3 Code** runs on demand via the `/t3` skill (or directly with `npx t3`) as a browser UI on port 3773, and **Prime Agent** installs on demand with `oh harness install prime-agent` — neither has an `install.*` key, because neither is ever baked into the image. Inside the sandbox, run `herdr` first, then launch whichever agent you prefer from its panes and switch between them at any time. Reserve tmux for Open Harness's managed/headless cron, gateway, and watchdog infrastructure. Open Harness is the harness; the **agent** is your call. To go beyond the preinstalled options, install via `npm` / `pip` / `cargo` inside the sandbox or edit the Dockerfile. For Pi+Slack specifically, the recommended path is the `pi-messenger-bridge` npm package — see [Slack integration](../integrations/slack.md). The product surface is one developer, one project, one agent — not racing or stacking multiple CLIs against each other. @@ -39,9 +39,8 @@ Flags: | `--no-persist` | Live-install only; leave `.devcontainer/.env` unchanged (ephemeral — a container recreate loses it) | | `--defaults` | `list` only: restrict the listing, and the verify probes it runs, to `kind:"default"` harnesses | -The manual path still works: uncomment the key in `.devcontainer/.env` (or export the -`INSTALL_*` build flag in `.devcontainer/.env`) and run -`oh destroy && oh sandbox`. +The manual path still works: set the field directly with +`oh config set install. true` and run `oh stop && oh sandbox`. ## Supported agents diff --git a/docs/installation.md b/docs/installation.md index 11d4aece..21b2aae6 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -413,9 +413,9 @@ CLI's `~/.newtool`, say) and leaves everything you already have alone. It does not merge new files into a directory the mount already has, which is what the per-tool volumes did before. -Hermes is split: when Hermes is enabled (`install.hermes: true` in `oh.json`), -`HERMES_HOME` defaults to the project-local bind-mounted `~/harness/.hermes/` -directory. The entrypoint links `.hermes/skills/openharness` to the tracked +Hermes is split: when the `hermes` binary is present (`install.hermes: true` in +`oh.json`, or `oh harness install hermes`), `HERMES_HOME` is the project-local +bind-mounted `~/harness/.hermes/` directory. The entrypoint links `.hermes/skills/openharness` to the tracked shared skill directory (`.oh/skills/`) so Hermes sees the same harness skills as Claude, Codex, and Pi without copying them into runtime state. Project-local runtime contents are gitignored except `.hermes/README.md`. diff --git a/docs/integrations/langfuse.md b/docs/integrations/langfuse.md index 13388f3e..aac4e802 100644 --- a/docs/integrations/langfuse.md +++ b/docs/integrations/langfuse.md @@ -245,10 +245,10 @@ pi `LANGFUSE_HOST` is supported as a fallback name. For an environment-only configuration, `LANGFUSE_BASE_URL` wins over `LANGFUSE_HOST`. -Set the host and the privacy preset in `oh.json` instead of exporting them by -hand. `oh` renders `langfuse.baseUrl` and `langfuse.privacyPreset` into the -Compose environment, and both compose files put them in the sandbox process -environment, so Pi reads them on every launch: +`langfuse.baseUrl` and `langfuse.privacyPreset` in `oh.json` record the intended +values in one tracked place, but the harness does **not** project them into the +container — Pi reads them from its own process environment, so export them in the +shell that launches Pi (see the block above): ```json { @@ -259,7 +259,7 @@ environment, so Pi reads them on every launch: } ``` -Apply the change with `oh stop && oh sandbox`. The two credentials stay secrets: +The two credentials stay secrets: keep `LANGFUSE_PUBLIC_KEY` and `LANGFUSE_SECRET_KEY` in the root dotenv (`oh secret set`) or in the saved `~/.pi/agent/pi-langfuse/config.json`, and export them in the shell that launches Pi if you use the environment-only path. diff --git a/docs/integrations/sshd.md b/docs/integrations/sshd.md index ee4d8937..4bf94798 100644 --- a/docs/integrations/sshd.md +++ b/docs/integrations/sshd.md @@ -23,27 +23,28 @@ alongside the container's main process, so the cron runtime, healthcheck, and ## 2. Enable the overlay -Uncomment the SSH keys in your local `.devcontainer/.env`: +Turn sshd on in the tracked `oh.json`: -```yaml -ssh: - enabled: true # SANDBOX_SSH — run sshd for direct container SSH - port: 2222 # SANDBOX_SSH_PORT — host loopback port published for SSH - # password_auth: false # leave off; use a key (section 3) +```bash +oh config set access.ssh true +oh config set access.sshPort 2222 ``` -Public-key material is not a secret but is multi-line, so -put your key in the gitignored `.devcontainer/.env`: +`access.ssh` and `access.sshPort` are the two host-side decisions — they select +the `docker-compose.ssh.yml` overlay and publish `127.0.0.1::22`, both of +which Docker must make before the container exists. Everything else about sshd is +read inside the container: `entrypoint.sh` calls `oh config show` on boot. + +Public-key material is not a secret, so it lives in `oh.json` too: ```bash -SANDBOX_SSH_AUTHORIZED_KEYS="ssh-ed25519 AAAA...yourkey... you@laptop" +oh config set access.sshAuthorizedKeys "ssh-ed25519 AAAA...yourkey... you@laptop" ``` -You can paste multiple keys separated by newlines (or literal `\n`). Apply the -change with a rebuild: +You can paste multiple keys separated by literal `\n`. Apply the change with: ```bash -oh destroy && oh sandbox +oh stop && oh sandbox ``` `oh sandbox` runs a **port-collision preflight**: if `SANDBOX_SSH_PORT` is @@ -102,14 +103,13 @@ docker exec pgrep -x sshd # sshd is running Key auth is strongly preferred. If you must allow password login (uses the `sandbox` user's `SANDBOX_PASSWORD`), set: -```yaml -ssh: - password_auth: true # SANDBOX_SSH_PASSWORD_AUTH +```bash +oh config set access.sshPasswordAuth true ``` > **Security.** The default `SANDBOX_PASSWORD` (`test1234`) is weak and public. > Never enable password auth on a `0.0.0.0` / internet-facing bind without first -> setting a strong `SANDBOX_PASSWORD` in `.devcontainer/.env`. See +> setting a strong `SANDBOX_PASSWORD` with `oh secret set SANDBOX_PASSWORD`. See > [Security considerations](../security-considerations.md). ## Security posture diff --git a/docs/lifecycle-commands.md b/docs/lifecycle-commands.md index 4ac82749..b8e68b14 100644 --- a/docs/lifecycle-commands.md +++ b/docs/lifecycle-commands.md @@ -102,7 +102,6 @@ lists `docker-compose.yml` and nothing else. It never runs - `access.ssh` → no `docker-compose.ssh.yml`, so no sshd and no published SSH port - `access.dockerSocket` → no `docker-compose.docker-sock.yml`, so no host Docker socket -- `hermesDashboard.enabled` → no `docker-compose.hermes-dashboard.yml`, so no dashboard - `composeOverrides[]` → every extra overlay path is ignored Secrets still reach that container: compose auto-loads the `.devcontainer/.env` diff --git a/docs/rfcs/rfc-brain-hands-boundary.md b/docs/rfcs/rfc-brain-hands-boundary.md index 403adeb0..edd5e53d 100644 --- a/docs/rfcs/rfc-brain-hands-boundary.md +++ b/docs/rfcs/rfc-brain-hands-boundary.md @@ -107,7 +107,7 @@ Two consequences that must not be forgotten when a second execution target lands about — a *second, nested* hands layer beneath the harness's own. The contract does not model it, and Phase-0 does not try to. 2. Its brain-side surface is configured through an execution-shaped mechanism: the dashboard - is toggled by a **compose overlay** (`.devcontainer/docker-compose.hermes-dashboard.yml`), + was toggled by a compose overlay (retired in #920; `hermesDashboard.enabled` in `oh.json` now starts it on container loopback), so brain-side policy rides on substrate configuration. This is accepted for Phase-0 because Hermes is opt-in and off by default (`install.hermes`). diff --git a/docs/runtimes/microsandbox.md b/docs/runtimes/microsandbox.md index 00d8da7f..8b481e04 100644 --- a/docs/runtimes/microsandbox.md +++ b/docs/runtimes/microsandbox.md @@ -253,7 +253,6 @@ entrypoint: ["/usr/local/bin/entrypoint.sh"] cmd: ["sleep", "infinity"] env: - OH_IMAGE_ONLY: "1" # load-bearing OH_PROJECT_ROOT: /home/sandbox/harness # load-bearing — must equal the mount target GIT_USER_NAME: "" GIT_USER_EMAIL: "" diff --git a/docs/security-considerations.md b/docs/security-considerations.md index ddae5963..bf996d3d 100644 --- a/docs/security-considerations.md +++ b/docs/security-considerations.md @@ -153,9 +153,9 @@ expose to whichever trust level you choose. tracked `oh.json`) adds **no container capability**: `tailscaled` runs inside the sandbox in **userspace-networking** mode as the unprivileged `sandbox` user, so there is no `NET_ADMIN`, no `/dev/net/tun`, no `privileged: true`, and no host - socket mount. The only compose addition is one environment variable - (`INSTALL_TAILSCALE`); daemon state lives in `/home/sandbox/.tailscale`, - inside the single `/home/sandbox` mount. **No host port is published** — T3 Code stays on + socket mount. There is no compose addition at all — the opt-in lives in + `oh.json` and boot provisioning installs from the tool catalog; daemon state + lives in `/home/sandbox/.tailscale`, inside the single `/home/sandbox` mount. **No host port is published** — T3 Code stays on container loopback `127.0.0.1:3773` and Tailscale Serve proxies tailnet HTTPS to it, so a device outside the tailnet has nothing to reach. The posture: - **Private tailnet only. Tailscale Funnel is never enabled by default and the From ecc49800a7d6a7bd525099383998865a3e6a1a49 Mon Sep 17 00:00:00 2001 From: Ryan Eggleston Date: Mon, 31 Aug 2026 18:12:25 -0600 Subject: [PATCH 17/23] FROM task/918-retire-prime-agent TO development (#923) * task: retire the Prime Agent harness * task: preserve retired Prime Agent state ignores * task: refresh the eval scoreboard after merging development --- .gitignore | 8 +- .oh/cli/src/commands/init.ts | 3 +- .oh/cli/src/lib/harnesses/catalog.ts | 14 -- .oh/evals/RESULTS.md | 234 +++++++++--------- .oh/evals/probes/default-provisioning.sh | 2 - .oh/evals/probes/skills-vendored.sh | 3 +- .../__tests__/provision-python.test.ts | 3 +- .oh/scripts/link-providers.sh | 4 +- .oh/scripts/provision-python.sh | 3 - .oh/skills/wiki/corpus/README.md | 1 - .oh/skills/wiki/corpus/prime-agent-harness.md | 96 ------- .../wiki/corpus/recursive-language-models.md | 3 +- .oh/templates/full/.prime/agent/.gitignore | 2 - .../full/.prime/agent/APPEND_SYSTEM.md | 15 -- .oh/templates/full/.prime/agent/settings.json | 6 - .prime/agent/.gitignore | 2 - .prime/agent/APPEND_SYSTEM.md | 15 -- .prime/agent/settings.json | 6 - .prime/agent/skills | 1 - CHANGELOG.md | 1 + README.md | 2 +- docs/harnesses/overview.md | 5 +- docs/harnesses/prime-agent.md | 128 ---------- 23 files changed, 128 insertions(+), 429 deletions(-) delete mode 100644 .oh/skills/wiki/corpus/prime-agent-harness.md delete mode 100644 .oh/templates/full/.prime/agent/.gitignore delete mode 100644 .oh/templates/full/.prime/agent/APPEND_SYSTEM.md delete mode 100644 .oh/templates/full/.prime/agent/settings.json delete mode 100644 .prime/agent/.gitignore delete mode 100644 .prime/agent/APPEND_SYSTEM.md delete mode 100644 .prime/agent/settings.json delete mode 120000 .prime/agent/skills delete mode 100644 docs/harnesses/prime-agent.md diff --git a/.gitignore b/.gitignore index d8b73969..9fc2899f 100644 --- a/.gitignore +++ b/.gitignore @@ -55,6 +55,8 @@ crons/*.log **/.claude/settings.local.json **/auth.json **/.credentials.json +.prime/agent/sessions/ +.prime/agent/telemetry.json # Hermes project-local runtime state (keep README tracked) /.hermes/* @@ -63,12 +65,6 @@ crons/*.log !/.hermes/config.yaml !/.hermes/config.yml -# prime-agent project-local runtime state. Credentials already fall under the -# **/auth.json rule above; these are the session and telemetry files the agent -# writes beside its tracked config. -.prime/agent/sessions/ -.prime/agent/telemetry.json - **/scheduled_tasks.lock # Legacy/future workspace package build artifacts packages/*/build/ diff --git a/.oh/cli/src/commands/init.ts b/.oh/cli/src/commands/init.ts index 40a8cab3..2700000d 100644 --- a/.oh/cli/src/commands/init.ts +++ b/.oh/cli/src/commands/init.ts @@ -357,7 +357,7 @@ export async function runInit( prompt.ok("Wrote AGENTS.md + CLAUDE.md and seeded an empty tasks/"); prompt.ok("Copied the full .devcontainer/ (local image build)"); prompt.ok( - "Configured 5 provider surfaces (.claude .codex .pi .prime .hermes) → vendored .oh/skills", + "Configured 4 provider surfaces (.claude .codex .pi .hermes) → vendored .oh/skills", ); } if (force && totalOverwritten > 0) { @@ -496,7 +496,6 @@ const PROVIDER_LINKS: [string, string][] = [ [".claude/hooks", "../.oh/hooks"], [".codex/agents", "../.claude/agents"], [".codex/specs", "../.claude/specs"], - [".prime/agent/skills", "../../.oh/skills"], ]; function linkReport(ctx: WriteCtx, linkRel: string, linkTarget: string): void { diff --git a/.oh/cli/src/lib/harnesses/catalog.ts b/.oh/cli/src/lib/harnesses/catalog.ts index 3975e173..a6b0cb1c 100644 --- a/.oh/cli/src/lib/harnesses/catalog.ts +++ b/.oh/cli/src/lib/harnesses/catalog.ts @@ -127,20 +127,6 @@ export const HARNESS_CATALOG: readonly HarnessEntry[] = [ docsPath: "docs/harnesses/t3code.md", kind: "on-demand", }, - { - id: "prime-agent", - title: "Prime Agent", - binary: "prime-agent", - installArgv: [ - "bash", - "-lc", - "curl -fsSL https://app.primeintellect.ai/prime-agent/install.sh | PRIME_AGENT_BOOTSTRAP_KERNEL_ON_INSTALL=0 npm_config_prefix=/home/sandbox/.local setsid --wait sh", - ], - installUser: "sandbox", - verifyArgv: ["prime-agent", "--version"], - docsPath: "docs/harnesses/prime-agent.md", - kind: "on-demand", - }, ]; export function optionalHarnesses(): readonly HarnessEntry[] { diff --git a/.oh/evals/RESULTS.md b/.oh/evals/RESULTS.md index d2650176..94f00528 100644 --- a/.oh/evals/RESULTS.md +++ b/.oh/evals/RESULTS.md @@ -6,122 +6,122 @@ probe id; git history is the time series.** Schema and exit-code semantics are i | probe | tier | last-run (UTC) | status | source | |-------|------|----------------|--------|--------| -| advisor-monitored-loop | A | 2026-09-01 00:00 | PASS | conversation 2026-06-19 (single-owner implementation workflow, issue #257) | -| agent-browser-cli | A | 2026-09-01 00:00 | PASS | retro lesson 2026-06-07 (agent-browser 0.8.5 CLI) | -| agents-identity-contract | A | 2026-09-01 00:00 | PASS | issue #854 — T3-style root identity, glossary, and skill-owned procedures | -| artifact-contract-audit | A | 2026-09-01 00:00 | PASS | issue #583/#645 — production /audit implementation Gate 1 behavior | -| audit-dispatcher-contract | A | 2026-09-01 00:00 | PASS | issue #645 — audit consolidation public taxonomy | -| audit-implementation-behavior | A | 2026-09-01 00:00 | PASS | issue #645 — implementation root/repo/browser behavior | -| audit-pr-acquire | A | 2026-09-01 00:00 | PASS | issue #645 — production PR acquisition behavior | -| audit-pr-classifier | A | 2026-09-01 00:00 | PASS | issue #645 — deterministic focused and queue PR classifier | -| audit-run-root-contract | A | 2026-09-01 00:00 | PASS | issue #645 — executable immutable audit root/run correlation | -| audit-shellcheck-coverage | A | 2026-09-01 00:00 | PASS | issue #645 — private audit scripts require release and CI lint coverage | -| audit-slop-gate | A | 2026-09-01 00:00 | PASS | .claude/specs/images/reduce-slop.png — the four correctness gates all pass a change | -| audit-stale-references | A | 2026-09-01 00:00 | PASS | issue #645 — clean-breaking audit migration | -| boot-lint-glob | A | 2026-09-01 00:00 | PASS | issue #90, issue #120 | -| builder-skill-consolidation | A | 2026-09-01 00:00 | PASS | issue #643 — consolidate artifact builders behind one /builder dispatcher | -| builder-wiki-proposer | A | 2026-09-01 00:00 | PASS | wikiskill arXiv:2608.27454 — the skill proposer reads accumulated knowledge first | -| capability-benchmark-schema | A | 2026-09-01 00:00 | PASS | issue #167 — capability benchmark instrument | -| cc-safety-net-wiring | A | 2026-09-01 00:00 | SKIPPED | .oh/tasks/cc-safety-net/prd.json US-007 2026-07-19 | -| changelog-entry-length | A | 2026-09-01 00:00 | PASS | conversation 2026-08-24 — CHANGELOG.md grew to 259KB of bullet prose because "one line" was unquantified | -| cleanup-tasks-scoped-guard | A | 2026-09-01 00:00 | PASS | issue #85 | -| cleanup-tasks-worktree-grooming | A | 2026-09-01 00:00 | PASS | issue #168; issue #327 | -| cli-publish-typecheck-scope | A | 2026-09-01 00:00 | PASS | release run 33271077312 — v0.5.0 pushed its GHCR image, then failed to publish | -| close-issues-on-development | A | 2026-09-01 00:00 | PASS | issue #841 (closing keywords never fire because the default branch is main) 2026-08-26 | -| codex-stale-response-retry | A | 2026-09-01 00:00 | PASS | issue #506 — Codex previous_response_not_found RCA | -| compose-config-path-parity | A | 2026-09-01 00:00 | PASS | PR #833 (remove harness.yaml — the wrapper and VS Code "Reopen in Container" paths must resolve the same service) 2026-08-26 | -| compose-env-boundary | A | 2026-09-01 00:00 | PASS | #920 — the epic #903→#911 made the CLI provision harnesses and tools from | -| config-schema-parity | A | 2026-09-01 00:00 | PASS | PR #833 (one schema file — DOCKER_SOCKET, SANDBOX_SSH, OH_SANDBOX_IMAGE, OH_PULL_POLICY, SKIP_PNPM_INSTALL were consumed but undocumented); rewritten for the oh.json/secrets split by PR #887 | -| context-tier-size-budget | A | 2026-09-01 00:00 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-007) — the always-on tier was 85,256 B | -| continual-learning-20260831 | A | 2026-09-01 00:00 | PASS | retro lesson 2026-08-31 (unexercised oracle) — a probe green in a 112-probe run carried three parser defects | -| cron-claude-codex-fallback | A | 2026-09-01 00:00 | PASS | conversation 2026-06-12 (default Codex fallback for crons) | -| cron-watchdog | A | 2026-09-01 00:00 | PASS | issues #130/#453 (cron runtime watchdog + legacy system-cron reaping) 2026-06-19 | -| crons-directory-guide | A | 2026-09-01 00:00 | PASS | issue #874 | -| curl-bash-safe-alternatives | A | 2026-09-01 00:00 | PASS | vet-run/vet integration — public curl|bash examples need review-first alternatives | -| datasets-schema | A | 2026-09-01 00:00 | PASS | issue #196 — .oh/evals/datasets verifiable trajectory corpus (Repo2RLEnv-inspired) | -| debugmcp-availability | A | 2026-09-01 00:00 | SKIPPED | issue #297 — DebugMCP MCP debug-server availability | -| default-provisioning | A | 2026-09-01 00:00 | PASS | #902 — `oh harness install` must work from inside the sandbox, where | -| delegate-model-effort-policy | A | 2026-09-01 00:00 | PASS | conversation 2026-07-11 (delegate model inheritance and thinking policy) | -| devtcp-hook | A | 2026-09-01 00:00 | PASS | retro lesson 2026-06-10 (zsh /dev/tcp) | -| docker-inspect-env-guard | A | 2026-09-01 00:00 | PASS | operator directive 2026-08-08 (agents keep the docker socket, but must | -| docs-build-fast-path | A | 2026-09-01 00:00 | PASS | #455 — docs builds must stay out of fast harness/eval/release gates; #536 — docs site externalized to openharness-web; docs markdown relocated to docs/ | -| drift-check-cron-staleness-glob | A | 2026-09-01 00:00 | PASS | issue #98; issue #225 (restart-required cron frontmatter/config drift) | -| entrypoint-pnpm-manifest-fingerprint | A | 2026-09-01 00:00 | PASS | issue #521 (manifest-aware sandbox installs) 2026-07-01 | -| escalate-contract | A | 2026-09-01 00:00 | PASS | issue #799 — seven comments on a GitHub thread produced zero notifications and nobody | -| eval-ci-gate | A | 2026-09-01 00:00 | PASS | #103 — eval probe suite gated in CI | -| eval-contract-text-20260831 | A | 2026-09-01 00:00 | PASS | retro lesson 2026-08-31 (prose-literal pinning) — two assertions failed on first run purely from source line wrapping | -| eval-gate | A | 2026-09-01 00:00 | PASS | retro lesson 2026-06-11 (eval-gate) | -| eval-results-atomic | A | 2026-09-01 00:00 | PASS | issue #83 (eval-results-atomic-write) | -| eval-runner-exit | A | 2026-09-01 00:00 | PASS | retro lesson 2026-06-11 (eval-runner-exit) #29 | -| eval-runs-once-per-cycle | A | 2026-09-01 00:00 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-006) — /eval ran 3x per cycle on the | -| execution-target-contract | A | 2026-09-01 00:00 | PASS | issue #733 (ExecutionTarget contract + Docker Compose adapter) 2026-08-10 | -| get-oh-bootstrap | A | 2026-09-01 00:00 | PASS | get-oh.sh bootstrap — the Node-bootstrapping host-side path to the standalone `oh` CLI (also on npm as @mifune/openharness; see oh-npm-package.sh) | -| git-skill | A | 2026-09-01 00:00 | PASS | conversation 2026-06-15 — rules are not always supported; git workflow must be the /git skill | -| harness-audit-empty-output-gate | A | 2026-09-01 00:00 | PASS | issue #246 — /audit harness must fail closed on empty auditor outputs | -| harness-ci-core-paths | A | 2026-09-01 00:00 | PASS | #165 — core sandbox config files must trigger harness CI | -| harness-ci-hooks-paths | A | 2026-09-01 00:00 | PASS | issue #202 — credential/security hook changes must trigger harness CI | -| harness-yaml-migration | A | 2026-09-01 00:00 | PASS | PR #833 (migrate-harness-yaml.sh — append / uncomment-in-place / preserve / overwrite, plus a silent no-op second run) 2026-08-26 | -| health-check-docker-stats | A | 2026-09-01 00:00 | PASS | retro lesson 2026-06-10 (docker stats vs ps Size) | -| health-check-socket-degrade | A | 2026-09-01 00:00 | PASS | issue #762 (refs #756) — /health-check degrades to one statement, not nine failures | -| heartbeat-logging-contract | A | 2026-09-01 00:00 | PASS | issue #447 (heartbeat log append hardening) 2026-06-18 | -| image-seed-hygiene | A | 2026-09-01 00:00 | PASS | issue #900 (slim the sandbox image) 2026-08-30 | -| markitdown-wiki-ingest | A | 2026-09-01 00:00 | PASS | issue #649 — pinned local-document normalization contract for /wiki ingest | -| next-dev-prod | A | 2026-09-01 00:00 | SKIPPED | retro lesson 2026-06-04 | -| oh-compose-env-wiring | A | 2026-09-01 00:00 | PASS | issue #880 (oh as the only front door — oh.json is the non-secret config surface) | -| oh-config-surfaces | A | 2026-09-01 00:00 | PASS | PR #887 (config split across two authored surfaces — a tracked oh.json and a secrets-only root dotenv — with nothing left under $HOME) | -| oh-destroy-guard | A | 2026-09-01 00:00 | PASS | issue #879 — `oh` becomes the only front door, so `make destroy` must | -| oh-devcontainer-restructure | A | 2026-09-01 00:00 | PASS | consolidate devcontainer — .oh/devcontainer/ folded back into .devcontainer/ | -| oh-home-mount | A | 2026-09-01 00:00 | PASS | issue #898 (single $HOME mount) 2026-08-30 | -| oh-image-only-deploy | A | 2026-09-01 00:00 | PASS | .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only | -| oh-init-headless-config | A | 2026-09-01 00:00 | PASS | PR #827 (installer answers landed in the losing config file); retargeted to the .example.env template by PR #833, then to oh.json by PR #887 | -| oh-init-scaffold | A | 2026-09-01 00:00 | PASS | issue #531 Phase 2 | -| oh-lifecycle-surface | A | 2026-09-01 00:00 | PASS | issue #881 — the Makefile is retired and `oh` is the only front door | -| oh-npm-package | A | 2026-09-01 00:00 | PASS | npm publish path for the standalone `oh` CLI (@mifune/openharness) — alternative to get-oh.sh | -| oh-payload-manifest | A | 2026-09-01 00:00 | PASS | issue #531 follow-on (.oh payload manifest — oh update ships a declared allowlist) | -| oh-sandbox-image-mode | A | 2026-09-01 00:00 | PASS | conversation 2026-07-05 (basic Docker deployment — prebuilt-image mode) | -| oh-shipped-repo-overridable | A | 2026-09-01 00:00 | PASS | issue #531 follow-on (de-hardcode residual — shipped .oh shell scripts keep the upstream repo overridable) | -| oh-standalone-lifecycle | A | 2026-09-01 00:00 | PASS | issue #564 | -| oh-update | A | 2026-09-01 00:00 | PASS | issue #531 Phase 3 (oh update — upgrade only the .oh control plane) | -| operator-config-guard | A | 2026-09-01 00:00 | PASS | operator directives 2026-08-06 (.config/ and settings.local.json are operator-only) | -| pnpm-audit-ci-gate | A | 2026-09-01 00:00 | PASS | issue #171 — pnpm security audits must run in CI | -| post-bridge-publish-confirmation | A | 2026-09-01 00:00 | PASS | #523 — post-bridge live publishing requires an explicit final confirmation gate | -| prd-output-path-contract | A | 2026-09-01 00:00 | PASS | retro lesson 2026-06-19 | -| prompt-miner-schema-compat | A | 2026-09-01 00:00 | PASS | issue #253 — prompt-miner JSONL schema-drift guard | -| prompt-miner-symlink-entrypoint | A | 2026-09-01 00:00 | PASS | issue #663 — prompt-miner engine no-ops via the documented .claude/skills symlink | -| prompt-miner-weakness-record | A | 2026-09-01 00:00 | PASS | issue #580 — prompt-miner weakness-record (WH-xxx) cluster output | -| protected-path-deletion | A | 2026-09-01 00:00 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-001) — the critique gate was deleted, | -| protected-paths-resolve | A | 2026-09-01 00:00 | PASS | issue #753 — .claude/protected-paths.txt named 7 paths that did not exist. | -| registry-portability-gate | A | 2026-09-01 00:00 | PASS | issue #758 | -| registry-portability | A | 2026-09-01 00:00 | SKIPPED | issue #758 | -| retro-deterministic-contract | A | 2026-09-01 00:00 | PASS | issue #443 — /retro deterministic output and self-contained helper contract | -| rl-delegation-write-worker | A | 2026-09-01 00:00 | PASS | retro lesson 2026-06-10 (rl-delegation) #57 | -| rlm-context-budget | A | 2026-09-01 00:00 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-006 | -| runtime-preflight-gate | A | 2026-09-01 00:00 | PASS | issue #806 § B1 (open sandbox.substrate vs sandbox.runtime selector); | -| sandbox-boot-guard-ci | A | 2026-09-01 00:00 | PASS | issue #449 (sandbox image build CI guard) 2026-06-19; | -| sandbox-node-base | A | 2026-09-01 00:00 | PASS | openharness#878 — oh as the only front door, T0 sandbox base image | -| skill-paths | A | 2026-09-01 00:00 | PASS | issue #43 — stale path references; extended by issue #69 — apps/->packages/ rename guard; extended by issue #870 — deleted .oh/agents/advisor.md | -| skills-dir-clean | A | 2026-09-01 00:00 | PASS | conversation 2026-06-29 — Pi parses every top-level `.md` in the skills | -| skills-task-tool-coupling | A | 2026-09-01 00:00 | PASS | council review 2026-08-29 (issue #886) — /delegate instructed Claude-Code-only | -| skills-vendored | A | 2026-09-01 00:00 | PASS | absorb .mifune submodule into .oh — the skills/agents/hooks pack is vendored | -| slack-admin-command-surface | A | 2026-09-01 00:00 | PASS | issue #354 — Slack bridge docs must distinguish Pi /msg-bridge commands from Slack DM admin text handlers | -| spec-family-contract | A | 2026-09-01 00:00 | PASS | issue #265; spec-simplification issue #816; workflow authority issue #854; | -| spec-ready-finalization | A | 2026-09-01 00:00 | PASS | issue #134; spec-simplification issue #816; workflow authority issue #854 | -| ste-checker-contract | A | 2026-09-01 00:00 | PASS | issue #750 PR audit — the /ste checker had four fail-open paths (unclosed | -| submitted-by-trailers | A | 2026-09-01 00:00 | PASS | conversation 2026-06-12 (commit attribution trailers); the single-owner | -| sync-skill-contract | A | 2026-09-01 00:00 | PASS | issue #331 — /sync dispatcher skill (bidirectional origin↔upstream sync) | -| t3-headless-launch | A | 2026-09-01 00:00 | PASS | issue #858 — /t3 launched a bare `npx --yes t3`, which is the local GUI and | -| tailscale-tool-boundary | A | 2026-09-01 00:00 | PASS | issue #858 — Tailscale mobile access for T3 Code. There is no tailnet, no | -| tool-catalog-boundary | A | 2026-09-01 00:00 | PASS | agent-browser's exclusion from the harness catalog (#821), the three-catalog | -| version-parity | A | 2026-09-01 00:00 | PASS | conversation 2026-08-29 — the oh CLI became the only lifecycle door, so its | -| weigh-scorer-contract | A | 2026-09-01 00:00 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-003 (2026-06-27) | -| wiki-compile-contract | A | 2026-09-01 00:00 | PASS | wikiskill arXiv:2608.27454 — Wiki Maintainer role added as /wiki compile | -| wiki-kind-schema-contract | A | 2026-09-01 00:00 | PASS | wikiskill arXiv:2608.27454 — pattern layer added to the wiki corpus | -| wiki-pattern-persistence | A | 2026-09-01 00:00 | PASS | wikiskill arXiv:2608.27454 — pattern pages are never rolled back | -| wiki-query-pattern-isolation | A | 2026-09-01 00:00 | PASS | wikiskill arXiv:2608.27454 — proposer-only pattern access | -| wiki-readme-index | A | 2026-09-01 00:00 | PASS | issue #132 — wiki README index drift guard | -| wiki-related-slugs | A | 2026-09-01 00:00 | PASS | wikiskill arXiv:2608.27454 — wiki lint related-slug check | -| wiki-skill-impact-append-only | A | 2026-09-01 00:00 | PASS | wikiskill arXiv:2608.27454 — skill-change ledger, never rolled back | -| workflow-boundaries | A | 2026-09-01 00:00 | PASS | conversation 2026-06-19 (workflow consolidation, issue #259); authority moved to /spec in issue #854 | -| worktrees-layout | A | 2026-09-01 00:00 | PASS | issue #872 | +| advisor-monitored-loop | A | 2026-09-01 00:08 | PASS | conversation 2026-06-19 (single-owner implementation workflow, issue #257) | +| agent-browser-cli | A | 2026-09-01 00:08 | PASS | retro lesson 2026-06-07 (agent-browser 0.8.5 CLI) | +| agents-identity-contract | A | 2026-09-01 00:08 | PASS | issue #854 — T3-style root identity, glossary, and skill-owned procedures | +| artifact-contract-audit | A | 2026-09-01 00:08 | PASS | issue #583/#645 — production /audit implementation Gate 1 behavior | +| audit-dispatcher-contract | A | 2026-09-01 00:08 | PASS | issue #645 — audit consolidation public taxonomy | +| audit-implementation-behavior | A | 2026-09-01 00:08 | PASS | issue #645 — implementation root/repo/browser behavior | +| audit-pr-acquire | A | 2026-09-01 00:08 | PASS | issue #645 — production PR acquisition behavior | +| audit-pr-classifier | A | 2026-09-01 00:08 | PASS | issue #645 — deterministic focused and queue PR classifier | +| audit-run-root-contract | A | 2026-09-01 00:08 | PASS | issue #645 — executable immutable audit root/run correlation | +| audit-shellcheck-coverage | A | 2026-09-01 00:08 | PASS | issue #645 — private audit scripts require release and CI lint coverage | +| audit-slop-gate | A | 2026-09-01 00:08 | PASS | .claude/specs/images/reduce-slop.png — the four correctness gates all pass a change | +| audit-stale-references | A | 2026-09-01 00:08 | PASS | issue #645 — clean-breaking audit migration | +| boot-lint-glob | A | 2026-09-01 00:08 | PASS | issue #90, issue #120 | +| builder-skill-consolidation | A | 2026-09-01 00:08 | PASS | issue #643 — consolidate artifact builders behind one /builder dispatcher | +| builder-wiki-proposer | A | 2026-09-01 00:08 | PASS | wikiskill arXiv:2608.27454 — the skill proposer reads accumulated knowledge first | +| capability-benchmark-schema | A | 2026-09-01 00:08 | PASS | issue #167 — capability benchmark instrument | +| cc-safety-net-wiring | A | 2026-09-01 00:08 | SKIPPED | .oh/tasks/cc-safety-net/prd.json US-007 2026-07-19 | +| changelog-entry-length | A | 2026-09-01 00:08 | PASS | conversation 2026-08-24 — CHANGELOG.md grew to 259KB of bullet prose because "one line" was unquantified | +| cleanup-tasks-scoped-guard | A | 2026-09-01 00:08 | PASS | issue #85 | +| cleanup-tasks-worktree-grooming | A | 2026-09-01 00:08 | PASS | issue #168; issue #327 | +| cli-publish-typecheck-scope | A | 2026-09-01 00:08 | PASS | release run 33271077312 — v0.5.0 pushed its GHCR image, then failed to publish | +| close-issues-on-development | A | 2026-09-01 00:08 | PASS | issue #841 (closing keywords never fire because the default branch is main) 2026-08-26 | +| codex-stale-response-retry | A | 2026-09-01 00:08 | PASS | issue #506 — Codex previous_response_not_found RCA | +| compose-config-path-parity | A | 2026-09-01 00:08 | PASS | PR #833 (remove harness.yaml — the wrapper and VS Code "Reopen in Container" paths must resolve the same service) 2026-08-26 | +| compose-env-boundary | A | 2026-09-01 00:08 | PASS | #920 — the epic #903→#911 made the CLI provision harnesses and tools from | +| config-schema-parity | A | 2026-09-01 00:08 | PASS | PR #833 (one schema file — DOCKER_SOCKET, SANDBOX_SSH, OH_SANDBOX_IMAGE, OH_PULL_POLICY, SKIP_PNPM_INSTALL were consumed but undocumented); rewritten for the oh.json/secrets split by PR #887 | +| context-tier-size-budget | A | 2026-09-01 00:08 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-007) — the always-on tier was 85,256 B | +| continual-learning-20260831 | A | 2026-09-01 00:08 | PASS | retro lesson 2026-08-31 (unexercised oracle) — a probe green in a 112-probe run carried three parser defects | +| cron-claude-codex-fallback | A | 2026-09-01 00:08 | PASS | conversation 2026-06-12 (default Codex fallback for crons) | +| cron-watchdog | A | 2026-09-01 00:08 | PASS | issues #130/#453 (cron runtime watchdog + legacy system-cron reaping) 2026-06-19 | +| crons-directory-guide | A | 2026-09-01 00:08 | PASS | issue #874 | +| curl-bash-safe-alternatives | A | 2026-09-01 00:08 | PASS | vet-run/vet integration — public curl|bash examples need review-first alternatives | +| datasets-schema | A | 2026-09-01 00:08 | PASS | issue #196 — .oh/evals/datasets verifiable trajectory corpus (Repo2RLEnv-inspired) | +| debugmcp-availability | A | 2026-09-01 00:08 | SKIPPED | issue #297 — DebugMCP MCP debug-server availability | +| default-provisioning | A | 2026-09-01 00:08 | PASS | #902 — `oh harness install` must work from inside the sandbox, where | +| delegate-model-effort-policy | A | 2026-09-01 00:08 | PASS | conversation 2026-07-11 (delegate model inheritance and thinking policy) | +| devtcp-hook | A | 2026-09-01 00:08 | PASS | retro lesson 2026-06-10 (zsh /dev/tcp) | +| docker-inspect-env-guard | A | 2026-09-01 00:08 | PASS | operator directive 2026-08-08 (agents keep the docker socket, but must | +| docs-build-fast-path | A | 2026-09-01 00:08 | PASS | #455 — docs builds must stay out of fast harness/eval/release gates; #536 — docs site externalized to openharness-web; docs markdown relocated to docs/ | +| drift-check-cron-staleness-glob | A | 2026-09-01 00:08 | PASS | issue #98; issue #225 (restart-required cron frontmatter/config drift) | +| entrypoint-pnpm-manifest-fingerprint | A | 2026-09-01 00:08 | PASS | issue #521 (manifest-aware sandbox installs) 2026-07-01 | +| escalate-contract | A | 2026-09-01 00:08 | PASS | issue #799 — seven comments on a GitHub thread produced zero notifications and nobody | +| eval-ci-gate | A | 2026-09-01 00:08 | PASS | #103 — eval probe suite gated in CI | +| eval-contract-text-20260831 | A | 2026-09-01 00:08 | PASS | retro lesson 2026-08-31 (prose-literal pinning) — two assertions failed on first run purely from source line wrapping | +| eval-gate | A | 2026-09-01 00:08 | PASS | retro lesson 2026-06-11 (eval-gate) | +| eval-results-atomic | A | 2026-09-01 00:08 | PASS | issue #83 (eval-results-atomic-write) | +| eval-runner-exit | A | 2026-09-01 00:08 | PASS | retro lesson 2026-06-11 (eval-runner-exit) #29 | +| eval-runs-once-per-cycle | A | 2026-09-01 00:08 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-006) — /eval ran 3x per cycle on the | +| execution-target-contract | A | 2026-09-01 00:08 | PASS | issue #733 (ExecutionTarget contract + Docker Compose adapter) 2026-08-10 | +| get-oh-bootstrap | A | 2026-09-01 00:08 | PASS | get-oh.sh bootstrap — the Node-bootstrapping host-side path to the standalone `oh` CLI (also on npm as @mifune/openharness; see oh-npm-package.sh) | +| git-skill | A | 2026-09-01 00:08 | PASS | conversation 2026-06-15 — rules are not always supported; git workflow must be the /git skill | +| harness-audit-empty-output-gate | A | 2026-09-01 00:08 | PASS | issue #246 — /audit harness must fail closed on empty auditor outputs | +| harness-ci-core-paths | A | 2026-09-01 00:08 | PASS | #165 — core sandbox config files must trigger harness CI | +| harness-ci-hooks-paths | A | 2026-09-01 00:08 | PASS | issue #202 — credential/security hook changes must trigger harness CI | +| harness-yaml-migration | A | 2026-09-01 00:08 | PASS | PR #833 (migrate-harness-yaml.sh — append / uncomment-in-place / preserve / overwrite, plus a silent no-op second run) 2026-08-26 | +| health-check-docker-stats | A | 2026-09-01 00:08 | PASS | retro lesson 2026-06-10 (docker stats vs ps Size) | +| health-check-socket-degrade | A | 2026-09-01 00:08 | PASS | issue #762 (refs #756) — /health-check degrades to one statement, not nine failures | +| heartbeat-logging-contract | A | 2026-09-01 00:08 | PASS | issue #447 (heartbeat log append hardening) 2026-06-18 | +| image-seed-hygiene | A | 2026-09-01 00:08 | PASS | issue #900 (slim the sandbox image) 2026-08-30 | +| markitdown-wiki-ingest | A | 2026-09-01 00:08 | PASS | issue #649 — pinned local-document normalization contract for /wiki ingest | +| next-dev-prod | A | 2026-09-01 00:08 | SKIPPED | retro lesson 2026-06-04 | +| oh-compose-env-wiring | A | 2026-09-01 00:08 | PASS | issue #880 (oh as the only front door — oh.json is the non-secret config surface) | +| oh-config-surfaces | A | 2026-09-01 00:08 | PASS | PR #887 (config split across two authored surfaces — a tracked oh.json and a secrets-only root dotenv — with nothing left under $HOME) | +| oh-destroy-guard | A | 2026-09-01 00:08 | PASS | issue #879 — `oh` becomes the only front door, so `make destroy` must | +| oh-devcontainer-restructure | A | 2026-09-01 00:08 | PASS | consolidate devcontainer — .oh/devcontainer/ folded back into .devcontainer/ | +| oh-home-mount | A | 2026-09-01 00:08 | PASS | issue #898 (single $HOME mount) 2026-08-30 | +| oh-image-only-deploy | A | 2026-09-01 00:08 | PASS | .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only | +| oh-init-headless-config | A | 2026-09-01 00:08 | PASS | PR #827 (installer answers landed in the losing config file); retargeted to the .example.env template by PR #833, then to oh.json by PR #887 | +| oh-init-scaffold | A | 2026-09-01 00:08 | PASS | issue #531 Phase 2 | +| oh-lifecycle-surface | A | 2026-09-01 00:08 | PASS | issue #881 — the Makefile is retired and `oh` is the only front door | +| oh-npm-package | A | 2026-09-01 00:08 | PASS | npm publish path for the standalone `oh` CLI (@mifune/openharness) — alternative to get-oh.sh | +| oh-payload-manifest | A | 2026-09-01 00:08 | PASS | issue #531 follow-on (.oh payload manifest — oh update ships a declared allowlist) | +| oh-sandbox-image-mode | A | 2026-09-01 00:08 | PASS | conversation 2026-07-05 (basic Docker deployment — prebuilt-image mode) | +| oh-shipped-repo-overridable | A | 2026-09-01 00:08 | PASS | issue #531 follow-on (de-hardcode residual — shipped .oh shell scripts keep the upstream repo overridable) | +| oh-standalone-lifecycle | A | 2026-09-01 00:08 | PASS | issue #564 | +| oh-update | A | 2026-09-01 00:08 | PASS | issue #531 Phase 3 (oh update — upgrade only the .oh control plane) | +| operator-config-guard | A | 2026-09-01 00:08 | PASS | operator directives 2026-08-06 (.config/ and settings.local.json are operator-only) | +| pnpm-audit-ci-gate | A | 2026-09-01 00:08 | PASS | issue #171 — pnpm security audits must run in CI | +| post-bridge-publish-confirmation | A | 2026-09-01 00:08 | PASS | #523 — post-bridge live publishing requires an explicit final confirmation gate | +| prd-output-path-contract | A | 2026-09-01 00:08 | PASS | retro lesson 2026-06-19 | +| prompt-miner-schema-compat | A | 2026-09-01 00:08 | PASS | issue #253 — prompt-miner JSONL schema-drift guard | +| prompt-miner-symlink-entrypoint | A | 2026-09-01 00:08 | PASS | issue #663 — prompt-miner engine no-ops via the documented .claude/skills symlink | +| prompt-miner-weakness-record | A | 2026-09-01 00:08 | PASS | issue #580 — prompt-miner weakness-record (WH-xxx) cluster output | +| protected-path-deletion | A | 2026-09-01 00:08 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-001) — the critique gate was deleted, | +| protected-paths-resolve | A | 2026-09-01 00:08 | PASS | issue #753 — .claude/protected-paths.txt named 7 paths that did not exist. | +| registry-portability-gate | A | 2026-09-01 00:08 | PASS | issue #758 | +| registry-portability | A | 2026-09-01 00:08 | SKIPPED | issue #758 | +| retro-deterministic-contract | A | 2026-09-01 00:08 | PASS | issue #443 — /retro deterministic output and self-contained helper contract | +| rl-delegation-write-worker | A | 2026-09-01 00:08 | PASS | retro lesson 2026-06-10 (rl-delegation) #57 | +| rlm-context-budget | A | 2026-09-01 00:08 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-006 | +| runtime-preflight-gate | A | 2026-09-01 00:08 | PASS | issue #806 § B1 (open sandbox.substrate vs sandbox.runtime selector); | +| sandbox-boot-guard-ci | A | 2026-09-01 00:08 | PASS | issue #449 (sandbox image build CI guard) 2026-06-19; | +| sandbox-node-base | A | 2026-09-01 00:08 | PASS | openharness#878 — oh as the only front door, T0 sandbox base image | +| skill-paths | A | 2026-09-01 00:08 | PASS | issue #43 — stale path references; extended by issue #69 — apps/->packages/ rename guard; extended by issue #870 — deleted .oh/agents/advisor.md | +| skills-dir-clean | A | 2026-09-01 00:08 | PASS | conversation 2026-06-29 — Pi parses every top-level `.md` in the skills | +| skills-task-tool-coupling | A | 2026-09-01 00:08 | PASS | council review 2026-08-29 (issue #886) — /delegate instructed Claude-Code-only | +| skills-vendored | A | 2026-09-01 00:08 | PASS | absorb .mifune submodule into .oh — the skills/agents/hooks pack is vendored | +| slack-admin-command-surface | A | 2026-09-01 00:08 | PASS | issue #354 — Slack bridge docs must distinguish Pi /msg-bridge commands from Slack DM admin text handlers | +| spec-family-contract | A | 2026-09-01 00:08 | PASS | issue #265; spec-simplification issue #816; workflow authority issue #854; | +| spec-ready-finalization | A | 2026-09-01 00:08 | PASS | issue #134; spec-simplification issue #816; workflow authority issue #854 | +| ste-checker-contract | A | 2026-09-01 00:08 | PASS | issue #750 PR audit — the /ste checker had four fail-open paths (unclosed | +| submitted-by-trailers | A | 2026-09-01 00:08 | PASS | conversation 2026-06-12 (commit attribution trailers); the single-owner | +| sync-skill-contract | A | 2026-09-01 00:08 | PASS | issue #331 — /sync dispatcher skill (bidirectional origin↔upstream sync) | +| t3-headless-launch | A | 2026-09-01 00:08 | PASS | issue #858 — /t3 launched a bare `npx --yes t3`, which is the local GUI and | +| tailscale-tool-boundary | A | 2026-09-01 00:08 | PASS | issue #858 — Tailscale mobile access for T3 Code. There is no tailnet, no | +| tool-catalog-boundary | A | 2026-09-01 00:08 | PASS | agent-browser's exclusion from the harness catalog (#821), the three-catalog | +| version-parity | A | 2026-09-01 00:08 | PASS | conversation 2026-08-29 — the oh CLI became the only lifecycle door, so its | +| weigh-scorer-contract | A | 2026-09-01 00:08 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-003 (2026-06-27) | +| wiki-compile-contract | A | 2026-09-01 00:08 | PASS | wikiskill arXiv:2608.27454 — Wiki Maintainer role added as /wiki compile | +| wiki-kind-schema-contract | A | 2026-09-01 00:08 | PASS | wikiskill arXiv:2608.27454 — pattern layer added to the wiki corpus | +| wiki-pattern-persistence | A | 2026-09-01 00:08 | PASS | wikiskill arXiv:2608.27454 — pattern pages are never rolled back | +| wiki-query-pattern-isolation | A | 2026-09-01 00:08 | PASS | wikiskill arXiv:2608.27454 — proposer-only pattern access | +| wiki-readme-index | A | 2026-09-01 00:08 | PASS | issue #132 — wiki README index drift guard | +| wiki-related-slugs | A | 2026-09-01 00:08 | PASS | wikiskill arXiv:2608.27454 — wiki lint related-slug check | +| wiki-skill-impact-append-only | A | 2026-09-01 00:08 | PASS | wikiskill arXiv:2608.27454 — skill-change ledger, never rolled back | +| workflow-boundaries | A | 2026-09-01 00:08 | PASS | conversation 2026-06-19 (workflow consolidation, issue #259); authority moved to /spec in issue #854 | +| worktrees-layout | A | 2026-09-01 00:08 | PASS | issue #872 | diff --git a/.oh/evals/probes/default-provisioning.sh b/.oh/evals/probes/default-provisioning.sh index c8b4dc3b..4dac3fda 100755 --- a/.oh/evals/probes/default-provisioning.sh +++ b/.oh/evals/probes/default-provisioning.sh @@ -94,8 +94,6 @@ DOCKERFILE_CODE=$(strip_dockerfile_comments) pkgs=0 while IFS= read -r entry; do - # on-demand entries (t3code, prime-agent) are fetched per invocation and were - # never baked; skip them rather than assert against an npx incantation. [[ $entry == *'kind: "default"'* || $entry == *'kind: "optional"'* ]] || continue id=$(sed -n 's/.*id: "\([^"]*\)".*/\1/p' <<<"$entry") # The package specifier is the last element of installArgv. Read it from that diff --git a/.oh/evals/probes/skills-vendored.sh b/.oh/evals/probes/skills-vendored.sh index 151d8045..85084cb9 100755 --- a/.oh/evals/probes/skills-vendored.sh +++ b/.oh/evals/probes/skills-vendored.sh @@ -31,7 +31,7 @@ for path in \ git ls-files --error-unmatch "$path" >/dev/null 2>&1 || fail "pack file not tracked in-repo: $path" done -for link in .pi/skills .claude/skills .codex/skills .claude/agents .claude/hooks .codex/agents .prime/agent/skills; do +for link in .pi/skills .claude/skills .codex/skills .claude/agents .claude/hooks .codex/agents; do [ -L "$link" ] || fail "$link is not a symlink" [ -e "$link" ] || fail "$link target does not resolve" done @@ -47,7 +47,6 @@ if [ "${SKILLS_VENDORED_SKIP_CLEAN_CLONE:-0}" != "1" ]; then [ -f .pi/skills/git/SKILL.md ] || fail "Pi skill symlink does not resolve in a clean clone" [ -f .claude/skills/spec/SKILL.md ] || fail "Claude skill symlink does not resolve in a clean clone" [ -f .codex/skills/git/SKILL.md ] || fail "Codex skill symlink does not resolve in a clean clone" - [ -f .prime/agent/skills/git/SKILL.md ] || fail "prime-agent skill symlink does not resolve in a clean clone" fake_bin="$tmp/bin" mkdir -p "$fake_bin" bare_path="$fake_bin:/usr/bin:/bin" diff --git a/.oh/scripts/__tests__/provision-python.test.ts b/.oh/scripts/__tests__/provision-python.test.ts index 47a5d239..3178c9a2 100644 --- a/.oh/scripts/__tests__/provision-python.test.ts +++ b/.oh/scripts/__tests__/provision-python.test.ts @@ -72,7 +72,8 @@ describe("provision-python.sh", () => { encoding: "utf8", env: { ...process.env, HOME: "/home/sandbox" }, }); - expect(out).toContain("export PRIME_AGENT_KERNEL_PYTHON="); + expect(out).toContain("export UV_PYTHON_INSTALL_DIR="); + expect(out).toContain("export UV_CACHE_DIR="); expect(out).not.toContain("/root/"); }); }); diff --git a/.oh/scripts/link-providers.sh b/.oh/scripts/link-providers.sh index b56497cb..35648fe6 100755 --- a/.oh/scripts/link-providers.sh +++ b/.oh/scripts/link-providers.sh @@ -30,7 +30,6 @@ provider_links=( ".claude/agents|../.oh/agents" ".claude/hooks|../.oh/hooks" ".codex/agents|../.claude/agents" - ".prime/agent/skills|../../.oh/skills" ) HERMES_LINK=".hermes/skills/openharness" @@ -72,7 +71,6 @@ print_state() { cat >&2 < ../.oh/skills - .prime/agent/skills -> ../../.oh/skills Remediation: bash .oh/scripts/link-providers.sh --init EOF } @@ -221,4 +219,4 @@ if [ "$failures" -ne 0 ]; then exit 1 fi -printf 'Providers OK: .pi/.claude/.codex/.prime skills -> .oh/skills (vendored pack present)\n' +printf 'Providers OK: .pi/.claude/.codex skills -> .oh/skills (vendored pack present)\n' diff --git a/.oh/scripts/provision-python.sh b/.oh/scripts/provision-python.sh index 88853918..d58fc7ed 100755 --- a/.oh/scripts/provision-python.sh +++ b/.oh/scripts/provision-python.sh @@ -67,7 +67,6 @@ ENV_FILE="$HOME/.local/share/oh/python-env.sh" if [ "$MODE" = "print-env" ]; then printf 'export UV_PYTHON_INSTALL_DIR=%s\n' "$UV_PYTHON_INSTALL_DIR" printf 'export UV_CACHE_DIR=%s\n' "$UV_CACHE_DIR" - printf 'export PRIME_AGENT_KERNEL_PYTHON=%s\n' "$KERNEL_PYTHON" exit 0 fi @@ -147,7 +146,6 @@ export UV_PYTHON_INSTALL_DIR="$UV_PYTHON_INSTALL_DIR" export UV_CACHE_DIR="$UV_CACHE_DIR" export UV_TOOL_DIR="$UV_TOOL_DIR" export UV_TOOL_BIN_DIR="$UV_TOOL_BIN_DIR" -export PRIME_AGENT_KERNEL_PYTHON="\${PRIME_AGENT_KERNEL_PYTHON:-$KERNEL_PYTHON}" ENVEOF fi @@ -169,4 +167,3 @@ done log "OK python=$PY_PATH" log "OK kernel=$KERNEL_PYTHON (ipykernel present)" -log "OK PRIME_AGENT_KERNEL_PYTHON=$KERNEL_PYTHON" diff --git a/.oh/skills/wiki/corpus/README.md b/.oh/skills/wiki/corpus/README.md index 1241b0ff..da8bf729 100644 --- a/.oh/skills/wiki/corpus/README.md +++ b/.oh/skills/wiki/corpus/README.md @@ -41,7 +41,6 @@ Schema rule, frontmatter spec, and all authoring conventions: `.oh/skills/wiki/r | pattern-docs-prohibition-by-example | Documenting a forbidden literal by quoting it violates the rule | [docs, evals, probes, vocabulary, guards, self-reference] | 2026-08-31 | | compose-env-boundary | Compose Environment Boundary | [compose, devcontainer, oh-json, cli, entrypoint, boundary, provisioning, sandbox] | 2026-08-31 | | runtime-isolation-landscape | Runtime Isolation Landscape (2026) | [runtime, isolation, sandbox, gvisor, firecracker, kata, microvm, cloudflare, e2b, daytona, fly, modal] | 2026-08-27 | -| prime-agent-harness | Prime Agent Harness Surface | [harness, provider-surface, prime-agent, skills, symlinks, oh-cli, on-demand] | 2026-08-27 | | oh-cli-portable-lifecycle | oh CLI Portable Lifecycle | [cli, oh, lifecycle, standalone, init, sandbox, remote-fetch, execution-target] | 2026-08-27 | | molt-agentic-reinforcement-learning | Molt: A Scalable PyTorch-Native Training Framework for Agentic Reinforcement Learning | [agentic-rl, training, readability, observability, trajectories, async, correctness, agent-harness, nvidia] | 2026-08-27 | | managed-agents | Scaling Managed Agents: Decoupling the brain from the hands | [agents, meta-harness, sessions, sandbox, resilience, security, scaling, anthropic, model-evolution] | 2026-08-27 | diff --git a/.oh/skills/wiki/corpus/prime-agent-harness.md b/.oh/skills/wiki/corpus/prime-agent-harness.md deleted file mode 100644 index 02a5ae9c..00000000 --- a/.oh/skills/wiki/corpus/prime-agent-harness.md +++ /dev/null @@ -1,96 +0,0 @@ ---- -title: "Prime Agent Harness Surface" -slug: prime-agent-harness -tags: [harness, provider-surface, prime-agent, skills, symlinks, oh-cli, on-demand] -created: 2026-08-26 -updated: 2026-08-27 -sources: - - raw/2026-08-26-prime-agent-harness.md -related: [oh-cli-portable-lifecycle, sandbox-dependency-installs] -confidence: confirmed ---- - -# Prime Agent Harness Surface - -## Relevant Source Files - -- `.oh/scripts/link-providers.sh` — the runtime wiring: `provider_links` (`:39`–`:47`) and the `--check` verifier. -- `.oh/cli/src/commands/init.ts` — the second, independent copy of that list, `PROVIDER_LINKS` (`:540`–`:549`), used when scaffolding a fresh project. -- `.oh/cli/src/lib/harnesses/catalog.ts` — the harness catalog; the `prime-agent` entry at `:209`–`:220`. -- `.oh/cli/src/__tests__/harness-catalog.test.ts` — the drift test that constrains what a catalog entry may declare. -- `.oh/evals/probes/skills-vendored.sh` — the parity oracle; resolve loop at `:37`, clean-clone assert at `:59`. -- `.prime/agent/settings.json`, `.prime/agent/APPEND_SYSTEM.md`, `.prime/agent/.gitignore` — the committed config surface. -- `docs/harnesses/prime-agent.md` — the human-facing doc the catalog's `docsPath` requires to exist. - -## Summary - -**Harness** and **provider surface** are two different things in this repo, wired by two -different mechanisms. A harness is an agent CLI the `oh` CLI knows how to install; a -provider surface is a directory that exposes the vendored `.oh/skills` pack to one of them. -Prime Intellect's `prime-agent` is both: a harness installed on demand, and the fifth -provider surface at `.prime/agent/skills`. It is its own harness — its config, auth, and -session model are its own, and none of Pi's settings apply to it. - -## Detail - -**Two wiring mechanisms, deliberately duplicated.** The symlink list exists twice: -`link-providers.sh:39-47` repairs a live checkout, and `init.ts:540-549` scaffolds a fresh -one. Neither reads the other; a surface added to only one silently drifts. The prime row -targets `../../.oh/skills` rather than `../.oh/skills` because it nests one level deeper -than the other four. No special case was needed for that depth — `link_provider()` already -runs `mkdir -p` on the link's parent (`link-providers.sh:97`), and `linkReport()` already -calls `mkdirSync(..., { recursive: true })` — which is why the opt-in Hermes link's special -casing (`link-providers.sh:48-49`) is about being opt-in, not about being nested. - -**`kind: "on-demand"` is a claim about the image, not about support.** The catalog's three -kinds are `default` (in the image's `AGENTS` build-arg list), `optional` (behind an -`INSTALL_*` build arg), and `on-demand` (never baked in, fetched at use time). `t3code` was -the only prior `on-demand` entry. Choosing it for `prime-agent` is what keeps -`.devcontainer/Dockerfile`, `docker-compose.yml`, and both `.example.env` files out of the -change: `harness-catalog.test.ts` requires `harnessKey` and `buildArg` to appear together -and forbids either on a non-`optional` kind, and it pins the flagged-harness list to exactly -four ids. Declaring neither satisfies all three. - -**The installer's prompts read `/dev/tty`, not stdin.** `install.sh` confirms twice before -installing, and its prompt helper opens the controlling terminal directly, so `< /dev/null` -does not silence it. `setsid --wait` removes the controlling terminal, the helper reports -"no terminal", and both prompts proceed on their own. The same argv sets -`npm_config_prefix=/home/sandbox/.local` — without it the global install targets root-owned -`/usr/lib/node_modules` and fails `EACCES` for the `sandbox` user. This is the same prefix -`pi` uses, and it is what lets `prime-agent update` run without sudo. - -**No context-file alias is needed.** `prime-agent` reads `AGENTS.md` and `CLAUDE.md` -natively (`--no-context-files` disables it), so the `writeClaudeAlias()` machinery other -providers need does not apply. - -## System Relationships - -```mermaid -flowchart LR - PACK[".oh/skills
vendored pack"] - LP[".oh/scripts/link-providers.sh
--init / --check"] - INIT["oh init
PROVIDER_LINKS"] - SURF[".prime/agent/skills
symlink"] - CAT["catalog.ts
kind: on-demand"] - CLI["oh harness install prime-agent"] - BIN["prime-agent
~/.local/bin"] - PROBE["skills-vendored.sh"] - - LP --> SURF - INIT --> SURF - SURF --> PACK - CAT --> CLI --> BIN - BIN -->|discovers SKILL.md recursively| SURF - PROBE -.verifies.-> SURF - PROBE -.verifies.-> LP -``` - -The catalog installs the binary; the symlink feeds it skills. The two are independent — -installing the harness without the surface yields an agent with no Open Harness skills, and -the surface without the harness is an inert symlink. `skills-vendored.sh` is the only thing -that fails when either half of the symlink wiring is dropped. - -## See Also - -- [[oh-cli-portable-lifecycle]] -- [[sandbox-dependency-installs]] diff --git a/.oh/skills/wiki/corpus/recursive-language-models.md b/.oh/skills/wiki/corpus/recursive-language-models.md index 15043c70..fbe13827 100644 --- a/.oh/skills/wiki/corpus/recursive-language-models.md +++ b/.oh/skills/wiki/corpus/recursive-language-models.md @@ -6,7 +6,7 @@ created: 2026-06-27 updated: 2026-08-31 sources: - raw/2026-06-27-recursive-language-models.md -related: [recursive-self-improvement-survey, molt-agentic-reinforcement-learning, prime-agent-harness] +related: [recursive-self-improvement-survey, molt-agentic-reinforcement-learning] confidence: provisional --- @@ -33,4 +33,3 @@ Recursive Language Models (RLM) is an inference-time pattern in which a root lan ## See Also - [[recursive-self-improvement-survey]] - [[molt-agentic-reinforcement-learning]] -- [[prime-agent-harness]] diff --git a/.oh/templates/full/.prime/agent/.gitignore b/.oh/templates/full/.prime/agent/.gitignore deleted file mode 100644 index 0d82bb5e..00000000 --- a/.oh/templates/full/.prime/agent/.gitignore +++ /dev/null @@ -1,2 +0,0 @@ -sessions/ -telemetry.json diff --git a/.oh/templates/full/.prime/agent/APPEND_SYSTEM.md b/.oh/templates/full/.prime/agent/APPEND_SYSTEM.md deleted file mode 100644 index 05dfe60f..00000000 --- a/.oh/templates/full/.prime/agent/APPEND_SYSTEM.md +++ /dev/null @@ -1,15 +0,0 @@ -## Project context - -You are running inside a project workspace. Read `AGENTS.md` (and `CLAUDE.md`, which is -an alias of it) if present. They define the project's identity, terminology, operating -principles, and local boundaries. Treat task-specific procedures as on-demand context. - -Application code and core logic are developed and tested within the local environment. -Refer to project conventions for architecture patterns, tooling, and coding standards. - -## Skills - -`.prime/agent/skills` is a symlink into the vendored Open Harness primitive pack at -`.oh/skills`. Every skill you discover there is shared with the other harnesses in this -repo — a skill is authored once and read by all of them. Prefer an existing skill over -re-deriving its procedure, and read the skill's own `SKILL.md` before acting on its topic. diff --git a/.oh/templates/full/.prime/agent/settings.json b/.oh/templates/full/.prime/agent/settings.json deleted file mode 100644 index 2547fe63..00000000 --- a/.oh/templates/full/.prime/agent/settings.json +++ /dev/null @@ -1,6 +0,0 @@ -{ - "theme": "dark", - "defaultThinkingLevel": "high", - "steeringMode": "one-at-a-time", - "followUpMode": "one-at-a-time" -} diff --git a/.prime/agent/.gitignore b/.prime/agent/.gitignore deleted file mode 100644 index 0d82bb5e..00000000 --- a/.prime/agent/.gitignore +++ /dev/null @@ -1,2 +0,0 @@ -sessions/ -telemetry.json diff --git a/.prime/agent/APPEND_SYSTEM.md b/.prime/agent/APPEND_SYSTEM.md deleted file mode 100644 index 05dfe60f..00000000 --- a/.prime/agent/APPEND_SYSTEM.md +++ /dev/null @@ -1,15 +0,0 @@ -## Project context - -You are running inside a project workspace. Read `AGENTS.md` (and `CLAUDE.md`, which is -an alias of it) if present. They define the project's identity, terminology, operating -principles, and local boundaries. Treat task-specific procedures as on-demand context. - -Application code and core logic are developed and tested within the local environment. -Refer to project conventions for architecture patterns, tooling, and coding standards. - -## Skills - -`.prime/agent/skills` is a symlink into the vendored Open Harness primitive pack at -`.oh/skills`. Every skill you discover there is shared with the other harnesses in this -repo — a skill is authored once and read by all of them. Prefer an existing skill over -re-deriving its procedure, and read the skill's own `SKILL.md` before acting on its topic. diff --git a/.prime/agent/settings.json b/.prime/agent/settings.json deleted file mode 100644 index 2547fe63..00000000 --- a/.prime/agent/settings.json +++ /dev/null @@ -1,6 +0,0 @@ -{ - "theme": "dark", - "defaultThinkingLevel": "high", - "steeringMode": "one-at-a-time", - "followUpMode": "one-at-a-time" -} diff --git a/.prime/agent/skills b/.prime/agent/skills deleted file mode 120000 index 00cd79aa..00000000 --- a/.prime/agent/skills +++ /dev/null @@ -1 +0,0 @@ -../../.oh/skills \ No newline at end of file diff --git a/CHANGELOG.md b/CHANGELOG.md index 5200da20..94197c75 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -25,6 +25,7 @@ Update policy and release automation live in [`/git`](.claude/skills/git/SKILL.m - Remove Cloudflare's apt repository and its bookworm-suite pin from the image; Docker's is now the only third-party apt source ([#906](https://github.com/mifunedev/openharness/issues/906)). - Remove the four optional-harness build args and the dead `buildArg` catalog field; the `install.*` keys keep working and now drive boot provisioning ([#908](https://github.com/mifunedev/openharness/issues/908)). - **BREAKING:** Retire the DeepAgents harness — `deepagents-cli` is deprecated upstream. `install.deepagents` is no longer a settable oh.json field ([#910](https://github.com/mifunedev/openharness/issues/910)). +- **BREAKING:** Retire Prime Agent and its `.prime/agent/` provider surface; `oh harness install prime-agent` is no longer available ([#918](https://github.com/mifunedev/openharness/issues/918)). - **BREAKING:** Retire the `projectRoot` / `OH_PROJECT_ROOT` config knob — the checkout is fixed at `/home/sandbox/harness`, nested inside the home mount ([#898](https://github.com/mifunedev/openharness/issues/898)). ### Changed diff --git a/README.md b/README.md index 79bac09b..a5d650ec 100644 --- a/README.md +++ b/README.md @@ -161,7 +161,7 @@ defaults. Details: [lifecycle commands](docs/lifecycle-commands.md#vs-code-reope Open Harness vendors the shared skills/agents/hooks primitive pack directly into the `.oh/` control plane: `.oh/skills/`, `.oh/agents/`, `.oh/hooks/`, and `.oh/skills.lock` are tracked as ordinary files in this repo. The `oh` CLI lays them down during `oh init`/`oh update`, so a fresh checkout has the skills immediately — no submodule, no recursive clone, no network step. -Provider surfaces are symlinks into `.oh/`: `.pi/skills`, `.claude/skills`, `.codex/skills`, and `.prime/agent/skills` point at `.oh/skills`; `.claude/agents` → `.oh/agents`; `.claude/hooks` → `.oh/hooks`. `.pi/` itself remains the Pi provider surface in v1. +Provider surfaces are symlinks into `.oh/`: `.pi/skills`, `.claude/skills`, and `.codex/skills` point at `.oh/skills`; `.claude/agents` → `.oh/agents`; `.claude/hooks` → `.oh/hooks`. `.pi/` itself remains the Pi provider surface in v1. ## 🚀 Use it diff --git a/docs/harnesses/overview.md b/docs/harnesses/overview.md index f469060c..5b98333d 100644 --- a/docs/harnesses/overview.md +++ b/docs/harnesses/overview.md @@ -4,7 +4,7 @@ title: "Harnesses Overview" # Harnesses Overview -Open Harness provisions three agent CLIs into `~/.local` on first boot: **Claude Code** (default), **Codex**, and **Pi**. **OpenCode**, **Hermes**, and **Grok Build** are optional — install one with `oh harness install `, which also sets its `install.*` key so a fresh home mount reinstalls it at boot. No harness is baked into the image. **T3 Code** runs on demand via the `/t3` skill (or directly with `npx t3`) as a browser UI on port 3773, and **Prime Agent** installs on demand with `oh harness install prime-agent` — neither has an `install.*` key, because neither is ever baked into the image. Inside the sandbox, run `herdr` first, then launch whichever agent you prefer from its panes and switch between them at any time. Reserve tmux for Open Harness's managed/headless cron, gateway, and watchdog infrastructure. +Open Harness provisions three agent CLIs into `~/.local` on first boot: **Claude Code** (default), **Codex**, and **Pi**. **OpenCode**, **Hermes**, and **Grok Build** are optional — install one with `oh harness install `, which also sets its `install.*` key so a fresh home mount reinstalls it at boot. No harness is baked into the image. **T3 Code** runs on demand via the `/t3` skill (or directly with `npx t3`) as a browser UI on port 3773, and has no `install.*` key, because it is never baked into the image. Inside the sandbox, run `herdr` first, then launch whichever agent you prefer from its panes and switch between them at any time. Reserve tmux for Open Harness's managed/headless cron, gateway, and watchdog infrastructure. Open Harness is the harness; the **agent** is your call. To go beyond the preinstalled options, install via `npm` / `pip` / `cargo` inside the sandbox or edit the Dockerfile. For Pi+Slack specifically, the recommended path is the `pi-messenger-bridge` npm package — see [Slack integration](../integrations/slack.md). The product surface is one developer, one project, one agent — not racing or stacking multiple CLIs against each other. @@ -53,7 +53,6 @@ The manual path still works: set the field directly with | [Hermes](./hermes.md) | Nous Research's self-improving terminal agent | `hermes` | optional: `oh harness install hermes` | | [Grok Build](./grok-build.md) | xAI's proprietary Grok Build terminal agent | `grok` | optional: `oh harness install grok-build` | | [T3 Code](./t3code.md) | Browser UI over Claude/Codex/OpenCode (port 3773) | `/t3` or `npx t3` | on-demand | -| [Prime Agent](./prime-agent.md) | Prime Intellect's terminal coding agent, with daemon-backed sessions | `prime-agent` | on-demand: `oh harness install prime-agent` | ## Verifying installation @@ -68,7 +67,6 @@ hermes --version # install.hermes: true grok --version # install.grok_build: true npx t3 --version # T3 Code (not preinstalled — fetched on demand) -prime-agent --version # Prime Agent (not preinstalled — oh harness install prime-agent) ``` ## Authentication @@ -81,7 +79,6 @@ Open Harness provisions Claude Code, Codex, and Pi into `~/.local` on first boot - **Pi**: configure provider keys via environment variables (see [Pi](./pi.md)). - **Hermes**: run `hermes setup` (see [Hermes](./hermes.md)). - **Grok Build**: run `grok login --device-auth` for headless/remote auth, `grok login` for interactive OAuth, or set `XAI_API_KEY` as a fallback (see [Grok Build](./grok-build.md)). Cached `~/.grok/auth.json` takes precedence over `XAI_API_KEY`. -- **Prime Agent**: run `prime-agent`, then `/login` — OAuth for ChatGPT Plus/Pro (Codex), Claude Pro/Max, or GitHub Copilot; API keys via environment variables or the same `/login` flow. Credentials land in `~/.prime/agent/auth.json` (see [Prime Agent](./prime-agent.md)). - **T3 Code**: authenticate one of Claude / Codex / OpenCode first, then run `/t3` (or `npx t3`) and open the printed pairing URL (see [T3 Code](./t3code.md)). ## Default surfaces diff --git a/docs/harnesses/prime-agent.md b/docs/harnesses/prime-agent.md deleted file mode 100644 index 01d3426b..00000000 --- a/docs/harnesses/prime-agent.md +++ /dev/null @@ -1,128 +0,0 @@ ---- -title: "Prime Agent" ---- - -# Prime Agent - -Prime Agent is Prime Intellect's terminal coding agent. It is its own harness — not a -variant of any other CLI listed here — with its own config surface at `.prime/agent/`, its -own global state at `~/.prime/agent/`, and a daemon that keeps sessions alive in the -background so you can detach from one and attach to it later. - -It reads `AGENTS.md` and `CLAUDE.md` natively, and it implements the -[Agent Skills standard](https://agentskills.io/specification), so the vendored Open Harness -skill pack reaches it through the same symlink mechanism the other providers use. - -## Install - -Prime Agent is **not baked into the sandbox image**. Install it into the running container -on demand: - -```bash -oh harness install prime-agent -``` - -That is the whole install — there is no `INSTALL_*` flag to set and no image rebuild, so -`oh harness install prime-agent --persist-only` has nothing to persist. The command runs the -upstream installer, which resolves the latest release, verifies its checksum, and does an -npm global install under the sandbox user's own prefix (`/home/sandbox/.local`), the same -prefix Pi uses. `prime-agent update` therefore needs no sudo. - -The manual equivalent: - -```bash -curl -fsSL https://app.primeintellect.ai/prime-agent/install.sh | sh -``` - -For a review-first install, download `prime-agent-install.sh`, inspect it, and -then run it: - -```bash -curl -fsSL -o prime-agent-install.sh https://app.primeintellect.ai/prime-agent/install.sh -less prime-agent-install.sh -bash prime-agent-install.sh -``` - -Node ≥ 20.6 is required; the sandbox image already satisfies it. - -## Verify installation - -```bash -prime-agent --version -``` - -## Authentication - -Run `prime-agent`, then `/login`. Prime Agent supports subscription providers over OAuth and -API-key providers over environment variables: - -- **Subscriptions**: ChatGPT Plus/Pro (Codex), Claude Pro/Max, GitHub Copilot. -- **API keys**: set the provider's environment variable, or store the key through `/login`. - -Credentials land in `~/.prime/agent/auth.json`. That path is covered by the repo's -`**/auth.json` ignore rule, so a key can never be committed by accident. The container -entrypoint chowns `~/.prime` to the sandbox user, so `/login` can write there after an -on-demand install. - -Resolution order is: environment variable, then the `auth.json` entry (API key or OAuth -token). Full provider matrix: `docs/providers.md` inside the installed package. - -## Configuration - -| Path | Scope | -|---|---| -| `.prime/agent/settings.json` | Project — tracked in this repo | -| `~/.prime/agent/settings.json` | Global, per machine | -| `.prime/agent/APPEND_SYSTEM.md` | Appends to the system prompt (`SYSTEM.md` replaces it) | -| `.prime/agent/skills/` | Project skills — a symlink into the vendored `.oh/skills` pack | -| `~/.prime/agent/auth.json` | Credentials (gitignored) | -| `.prime/agent/sessions/`, `.prime/agent/telemetry.json` | Runtime state (gitignored) | - -The tracked `.prime/agent/settings.json` is deliberately minimal: a theme, a default -thinking level, and the steering/follow-up modes. It pins no model and no provider, so the -choice stays yours and the file needs no edit when upstream's model catalog moves. - -## How the Open Harness skills reach the agent - -`.prime/agent/skills` is a symlink to `../../.oh/skills`, the vendored primitive pack every -provider surface shares. `bash .oh/scripts/link-providers.sh --init` creates it and -`--check` verifies it; `oh init` creates the same link when scaffolding a fresh project. -Prime Agent discovers any directory containing a `SKILL.md` recursively under that path, so -a skill added to `.oh/skills/` is visible to Prime Agent with no further wiring. - -`AGENTS.md` is loaded natively — Open Harness needs no provider-specific alias for it. - -## Headless and background use - -```bash -prime-agent -p "summarize the build failure" # print a response and exit -prime-agent --mode json -p "..." # structured output -prime-agent --autonomous --autonomous-gate "npm test" # loop until the gate passes -``` - -Sessions are daemon-backed. `prime-agent list` shows running agents, `attach` reattaches to -one, `stop` ends one, `status` and `doctor` inspect the background services, and `shutdown` -stops all of them. - -## Not wired yet - -These are deliberate gaps, not oversights: - -- **No install.\* key.** There is no `INSTALL_PRIME_AGENT` key, no compose auth - volume, and no `install.*` field in `oh.json`. The harness is `on-demand` only. -- **No `/spec execute` provider integration.** `/spec execute` uses the active Advisor - session and does not launch a provider-specific wrapper; run Prime Agent directly only - for a bounded, explicitly selected task. -- **No messenger bridge.** `.oh/scripts/gateway.sh` accepts only `pi` and `hermes`. - -## Upstream - -[PrimeIntellect-ai/prime-agent](https://github.com/PrimeIntellect-ai/prime-agent). The -installed package ships its own docs under -`$(npm root -g)/prime-agent/docs/` — `providers.md`, `settings.md`, `skills.md`, and -`usage.md` are the ones this page draws on. - -## See also - -- [Harnesses Overview](./overview.md) -- [Pi](./pi.md) — the other npm-installed CLI under the sandbox user's prefix From 0f87d985ffbc30edf0d85e34e0fe70b6ccd0ed14 Mon Sep 17 00:00:00 2001 From: Ryan Eggleston Date: Mon, 31 Aug 2026 20:01:03 -0600 Subject: [PATCH 18/23] FROM task/928-retire-spec-agent-handoff TO development (#930) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * task: retire automated /spec agent handoff /spec execute no longer launches a coding agent. The agent that runs it is the single implementation owner from the isolated worktree through the final PR gates; /delegate stays bounded, disjoint fan-out beneath that owner. Retired: the detached tmux Advisor launch, agent-spec-* sessions, tmux pipe-pane logging, Advisor-specific resume/monitoring, automated Herdr workspace/tab/pane creation, and every runner fallback. Task identity and RUNNING state now depend on .oh/tasks// alone. Preserved: worktree isolation, implementation <-> audit, the eval/evidence/ retro/final-PR gates, human merge, Herdr as operator-managed interactive tooling, and tmux for headless infrastructure. Probes: advisor-monitored-loop renamed to spec-single-owner; adds spec-no-agent-handoff, spec-no-advisor-session-coupling, cleanup-no-agent-session-coupling, headless-tmux-preserved. Tracks #928. Submitted-by: Claude * task: cut scope creep and redundant probe machinery from #928 Four simplifications, no coverage lost: 1. Revert the plan-vs-built-reconciliation line-citation renumbering and the updated: bump (pre-existing drift, no acceptance criterion, no probe), which also drops corpus/README.md from the diff entirely. 2. Drop spec-single-owner's agent-spec- marker — spec-no-advisor-session-coupling already scans both files for it. 3. Reword two prohibition sentences so they no longer use the handles the probe bans, replacing a 20-line negation-aware awk matcher with a literal ban. 4. Compress the CHANGELOG from six bullets to four. Tracks #928. Submitted-by: Claude --- .oh/evals/RESULTS.md | 238 +++++++++--------- .../tasks/CB-001-ship-harness-change.md | 2 +- .../cleanup-no-agent-session-coupling.sh | 40 +++ .oh/evals/probes/headless-tmux-preserved.sh | 46 ++++ .../spec-no-advisor-session-coupling.sh | 52 ++++ .oh/evals/probes/spec-no-agent-handoff.sh | 48 ++++ ...monitored-loop.sh => spec-single-owner.sh} | 17 +- .oh/skills/audit/references/eval-quality.md | 2 +- .oh/skills/audit/references/implementation.md | 2 +- .oh/skills/delegate/SKILL.md | 2 +- .oh/skills/retro/SKILL.md | 2 +- .oh/skills/rlm/SKILL.md | 2 +- .oh/skills/spec/SKILL.md | 18 +- .oh/skills/spec/references/execute.md | 140 ++++++----- .oh/skills/spec/references/plan.md | 11 +- .oh/skills/spec/references/ship.md | 2 +- .oh/skills/spec/templates/task-prompt.md | 8 +- .oh/skills/ste/references/rules.md | 2 +- .oh/skills/t3/references/sandbox-processes.md | 11 +- .../corpus/plan-vs-built-reconciliation.md | 6 +- .oh/tasks/README.md | 12 +- CHANGELOG.md | 8 + crons/AGENTS.md | 2 +- crons/cleanup-tasks.md | 4 +- crons/prompt-miner.md | 5 +- docs/glossary.md | 9 +- 26 files changed, 456 insertions(+), 235 deletions(-) create mode 100755 .oh/evals/probes/cleanup-no-agent-session-coupling.sh create mode 100755 .oh/evals/probes/headless-tmux-preserved.sh create mode 100755 .oh/evals/probes/spec-no-advisor-session-coupling.sh create mode 100755 .oh/evals/probes/spec-no-agent-handoff.sh rename .oh/evals/probes/{advisor-monitored-loop.sh => spec-single-owner.sh} (57%) diff --git a/.oh/evals/RESULTS.md b/.oh/evals/RESULTS.md index 94f00528..9385084b 100644 --- a/.oh/evals/RESULTS.md +++ b/.oh/evals/RESULTS.md @@ -6,122 +6,126 @@ probe id; git history is the time series.** Schema and exit-code semantics are i | probe | tier | last-run (UTC) | status | source | |-------|------|----------------|--------|--------| -| advisor-monitored-loop | A | 2026-09-01 00:08 | PASS | conversation 2026-06-19 (single-owner implementation workflow, issue #257) | -| agent-browser-cli | A | 2026-09-01 00:08 | PASS | retro lesson 2026-06-07 (agent-browser 0.8.5 CLI) | -| agents-identity-contract | A | 2026-09-01 00:08 | PASS | issue #854 — T3-style root identity, glossary, and skill-owned procedures | -| artifact-contract-audit | A | 2026-09-01 00:08 | PASS | issue #583/#645 — production /audit implementation Gate 1 behavior | -| audit-dispatcher-contract | A | 2026-09-01 00:08 | PASS | issue #645 — audit consolidation public taxonomy | -| audit-implementation-behavior | A | 2026-09-01 00:08 | PASS | issue #645 — implementation root/repo/browser behavior | -| audit-pr-acquire | A | 2026-09-01 00:08 | PASS | issue #645 — production PR acquisition behavior | -| audit-pr-classifier | A | 2026-09-01 00:08 | PASS | issue #645 — deterministic focused and queue PR classifier | -| audit-run-root-contract | A | 2026-09-01 00:08 | PASS | issue #645 — executable immutable audit root/run correlation | -| audit-shellcheck-coverage | A | 2026-09-01 00:08 | PASS | issue #645 — private audit scripts require release and CI lint coverage | -| audit-slop-gate | A | 2026-09-01 00:08 | PASS | .claude/specs/images/reduce-slop.png — the four correctness gates all pass a change | -| audit-stale-references | A | 2026-09-01 00:08 | PASS | issue #645 — clean-breaking audit migration | -| boot-lint-glob | A | 2026-09-01 00:08 | PASS | issue #90, issue #120 | -| builder-skill-consolidation | A | 2026-09-01 00:08 | PASS | issue #643 — consolidate artifact builders behind one /builder dispatcher | -| builder-wiki-proposer | A | 2026-09-01 00:08 | PASS | wikiskill arXiv:2608.27454 — the skill proposer reads accumulated knowledge first | -| capability-benchmark-schema | A | 2026-09-01 00:08 | PASS | issue #167 — capability benchmark instrument | -| cc-safety-net-wiring | A | 2026-09-01 00:08 | SKIPPED | .oh/tasks/cc-safety-net/prd.json US-007 2026-07-19 | -| changelog-entry-length | A | 2026-09-01 00:08 | PASS | conversation 2026-08-24 — CHANGELOG.md grew to 259KB of bullet prose because "one line" was unquantified | -| cleanup-tasks-scoped-guard | A | 2026-09-01 00:08 | PASS | issue #85 | -| cleanup-tasks-worktree-grooming | A | 2026-09-01 00:08 | PASS | issue #168; issue #327 | -| cli-publish-typecheck-scope | A | 2026-09-01 00:08 | PASS | release run 33271077312 — v0.5.0 pushed its GHCR image, then failed to publish | -| close-issues-on-development | A | 2026-09-01 00:08 | PASS | issue #841 (closing keywords never fire because the default branch is main) 2026-08-26 | -| codex-stale-response-retry | A | 2026-09-01 00:08 | PASS | issue #506 — Codex previous_response_not_found RCA | -| compose-config-path-parity | A | 2026-09-01 00:08 | PASS | PR #833 (remove harness.yaml — the wrapper and VS Code "Reopen in Container" paths must resolve the same service) 2026-08-26 | -| compose-env-boundary | A | 2026-09-01 00:08 | PASS | #920 — the epic #903→#911 made the CLI provision harnesses and tools from | -| config-schema-parity | A | 2026-09-01 00:08 | PASS | PR #833 (one schema file — DOCKER_SOCKET, SANDBOX_SSH, OH_SANDBOX_IMAGE, OH_PULL_POLICY, SKIP_PNPM_INSTALL were consumed but undocumented); rewritten for the oh.json/secrets split by PR #887 | -| context-tier-size-budget | A | 2026-09-01 00:08 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-007) — the always-on tier was 85,256 B | -| continual-learning-20260831 | A | 2026-09-01 00:08 | PASS | retro lesson 2026-08-31 (unexercised oracle) — a probe green in a 112-probe run carried three parser defects | -| cron-claude-codex-fallback | A | 2026-09-01 00:08 | PASS | conversation 2026-06-12 (default Codex fallback for crons) | -| cron-watchdog | A | 2026-09-01 00:08 | PASS | issues #130/#453 (cron runtime watchdog + legacy system-cron reaping) 2026-06-19 | -| crons-directory-guide | A | 2026-09-01 00:08 | PASS | issue #874 | -| curl-bash-safe-alternatives | A | 2026-09-01 00:08 | PASS | vet-run/vet integration — public curl|bash examples need review-first alternatives | -| datasets-schema | A | 2026-09-01 00:08 | PASS | issue #196 — .oh/evals/datasets verifiable trajectory corpus (Repo2RLEnv-inspired) | -| debugmcp-availability | A | 2026-09-01 00:08 | SKIPPED | issue #297 — DebugMCP MCP debug-server availability | -| default-provisioning | A | 2026-09-01 00:08 | PASS | #902 — `oh harness install` must work from inside the sandbox, where | -| delegate-model-effort-policy | A | 2026-09-01 00:08 | PASS | conversation 2026-07-11 (delegate model inheritance and thinking policy) | -| devtcp-hook | A | 2026-09-01 00:08 | PASS | retro lesson 2026-06-10 (zsh /dev/tcp) | -| docker-inspect-env-guard | A | 2026-09-01 00:08 | PASS | operator directive 2026-08-08 (agents keep the docker socket, but must | -| docs-build-fast-path | A | 2026-09-01 00:08 | PASS | #455 — docs builds must stay out of fast harness/eval/release gates; #536 — docs site externalized to openharness-web; docs markdown relocated to docs/ | -| drift-check-cron-staleness-glob | A | 2026-09-01 00:08 | PASS | issue #98; issue #225 (restart-required cron frontmatter/config drift) | -| entrypoint-pnpm-manifest-fingerprint | A | 2026-09-01 00:08 | PASS | issue #521 (manifest-aware sandbox installs) 2026-07-01 | -| escalate-contract | A | 2026-09-01 00:08 | PASS | issue #799 — seven comments on a GitHub thread produced zero notifications and nobody | -| eval-ci-gate | A | 2026-09-01 00:08 | PASS | #103 — eval probe suite gated in CI | -| eval-contract-text-20260831 | A | 2026-09-01 00:08 | PASS | retro lesson 2026-08-31 (prose-literal pinning) — two assertions failed on first run purely from source line wrapping | -| eval-gate | A | 2026-09-01 00:08 | PASS | retro lesson 2026-06-11 (eval-gate) | -| eval-results-atomic | A | 2026-09-01 00:08 | PASS | issue #83 (eval-results-atomic-write) | -| eval-runner-exit | A | 2026-09-01 00:08 | PASS | retro lesson 2026-06-11 (eval-runner-exit) #29 | -| eval-runs-once-per-cycle | A | 2026-09-01 00:08 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-006) — /eval ran 3x per cycle on the | -| execution-target-contract | A | 2026-09-01 00:08 | PASS | issue #733 (ExecutionTarget contract + Docker Compose adapter) 2026-08-10 | -| get-oh-bootstrap | A | 2026-09-01 00:08 | PASS | get-oh.sh bootstrap — the Node-bootstrapping host-side path to the standalone `oh` CLI (also on npm as @mifune/openharness; see oh-npm-package.sh) | -| git-skill | A | 2026-09-01 00:08 | PASS | conversation 2026-06-15 — rules are not always supported; git workflow must be the /git skill | -| harness-audit-empty-output-gate | A | 2026-09-01 00:08 | PASS | issue #246 — /audit harness must fail closed on empty auditor outputs | -| harness-ci-core-paths | A | 2026-09-01 00:08 | PASS | #165 — core sandbox config files must trigger harness CI | -| harness-ci-hooks-paths | A | 2026-09-01 00:08 | PASS | issue #202 — credential/security hook changes must trigger harness CI | -| harness-yaml-migration | A | 2026-09-01 00:08 | PASS | PR #833 (migrate-harness-yaml.sh — append / uncomment-in-place / preserve / overwrite, plus a silent no-op second run) 2026-08-26 | -| health-check-docker-stats | A | 2026-09-01 00:08 | PASS | retro lesson 2026-06-10 (docker stats vs ps Size) | -| health-check-socket-degrade | A | 2026-09-01 00:08 | PASS | issue #762 (refs #756) — /health-check degrades to one statement, not nine failures | -| heartbeat-logging-contract | A | 2026-09-01 00:08 | PASS | issue #447 (heartbeat log append hardening) 2026-06-18 | -| image-seed-hygiene | A | 2026-09-01 00:08 | PASS | issue #900 (slim the sandbox image) 2026-08-30 | -| markitdown-wiki-ingest | A | 2026-09-01 00:08 | PASS | issue #649 — pinned local-document normalization contract for /wiki ingest | -| next-dev-prod | A | 2026-09-01 00:08 | SKIPPED | retro lesson 2026-06-04 | -| oh-compose-env-wiring | A | 2026-09-01 00:08 | PASS | issue #880 (oh as the only front door — oh.json is the non-secret config surface) | -| oh-config-surfaces | A | 2026-09-01 00:08 | PASS | PR #887 (config split across two authored surfaces — a tracked oh.json and a secrets-only root dotenv — with nothing left under $HOME) | -| oh-destroy-guard | A | 2026-09-01 00:08 | PASS | issue #879 — `oh` becomes the only front door, so `make destroy` must | -| oh-devcontainer-restructure | A | 2026-09-01 00:08 | PASS | consolidate devcontainer — .oh/devcontainer/ folded back into .devcontainer/ | -| oh-home-mount | A | 2026-09-01 00:08 | PASS | issue #898 (single $HOME mount) 2026-08-30 | -| oh-image-only-deploy | A | 2026-09-01 00:08 | PASS | .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only | -| oh-init-headless-config | A | 2026-09-01 00:08 | PASS | PR #827 (installer answers landed in the losing config file); retargeted to the .example.env template by PR #833, then to oh.json by PR #887 | -| oh-init-scaffold | A | 2026-09-01 00:08 | PASS | issue #531 Phase 2 | -| oh-lifecycle-surface | A | 2026-09-01 00:08 | PASS | issue #881 — the Makefile is retired and `oh` is the only front door | -| oh-npm-package | A | 2026-09-01 00:08 | PASS | npm publish path for the standalone `oh` CLI (@mifune/openharness) — alternative to get-oh.sh | -| oh-payload-manifest | A | 2026-09-01 00:08 | PASS | issue #531 follow-on (.oh payload manifest — oh update ships a declared allowlist) | -| oh-sandbox-image-mode | A | 2026-09-01 00:08 | PASS | conversation 2026-07-05 (basic Docker deployment — prebuilt-image mode) | -| oh-shipped-repo-overridable | A | 2026-09-01 00:08 | PASS | issue #531 follow-on (de-hardcode residual — shipped .oh shell scripts keep the upstream repo overridable) | -| oh-standalone-lifecycle | A | 2026-09-01 00:08 | PASS | issue #564 | -| oh-update | A | 2026-09-01 00:08 | PASS | issue #531 Phase 3 (oh update — upgrade only the .oh control plane) | -| operator-config-guard | A | 2026-09-01 00:08 | PASS | operator directives 2026-08-06 (.config/ and settings.local.json are operator-only) | -| pnpm-audit-ci-gate | A | 2026-09-01 00:08 | PASS | issue #171 — pnpm security audits must run in CI | -| post-bridge-publish-confirmation | A | 2026-09-01 00:08 | PASS | #523 — post-bridge live publishing requires an explicit final confirmation gate | -| prd-output-path-contract | A | 2026-09-01 00:08 | PASS | retro lesson 2026-06-19 | -| prompt-miner-schema-compat | A | 2026-09-01 00:08 | PASS | issue #253 — prompt-miner JSONL schema-drift guard | -| prompt-miner-symlink-entrypoint | A | 2026-09-01 00:08 | PASS | issue #663 — prompt-miner engine no-ops via the documented .claude/skills symlink | -| prompt-miner-weakness-record | A | 2026-09-01 00:08 | PASS | issue #580 — prompt-miner weakness-record (WH-xxx) cluster output | -| protected-path-deletion | A | 2026-09-01 00:08 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-001) — the critique gate was deleted, | -| protected-paths-resolve | A | 2026-09-01 00:08 | PASS | issue #753 — .claude/protected-paths.txt named 7 paths that did not exist. | -| registry-portability-gate | A | 2026-09-01 00:08 | PASS | issue #758 | -| registry-portability | A | 2026-09-01 00:08 | SKIPPED | issue #758 | -| retro-deterministic-contract | A | 2026-09-01 00:08 | PASS | issue #443 — /retro deterministic output and self-contained helper contract | -| rl-delegation-write-worker | A | 2026-09-01 00:08 | PASS | retro lesson 2026-06-10 (rl-delegation) #57 | -| rlm-context-budget | A | 2026-09-01 00:08 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-006 | -| runtime-preflight-gate | A | 2026-09-01 00:08 | PASS | issue #806 § B1 (open sandbox.substrate vs sandbox.runtime selector); | -| sandbox-boot-guard-ci | A | 2026-09-01 00:08 | PASS | issue #449 (sandbox image build CI guard) 2026-06-19; | -| sandbox-node-base | A | 2026-09-01 00:08 | PASS | openharness#878 — oh as the only front door, T0 sandbox base image | -| skill-paths | A | 2026-09-01 00:08 | PASS | issue #43 — stale path references; extended by issue #69 — apps/->packages/ rename guard; extended by issue #870 — deleted .oh/agents/advisor.md | -| skills-dir-clean | A | 2026-09-01 00:08 | PASS | conversation 2026-06-29 — Pi parses every top-level `.md` in the skills | -| skills-task-tool-coupling | A | 2026-09-01 00:08 | PASS | council review 2026-08-29 (issue #886) — /delegate instructed Claude-Code-only | -| skills-vendored | A | 2026-09-01 00:08 | PASS | absorb .mifune submodule into .oh — the skills/agents/hooks pack is vendored | -| slack-admin-command-surface | A | 2026-09-01 00:08 | PASS | issue #354 — Slack bridge docs must distinguish Pi /msg-bridge commands from Slack DM admin text handlers | -| spec-family-contract | A | 2026-09-01 00:08 | PASS | issue #265; spec-simplification issue #816; workflow authority issue #854; | -| spec-ready-finalization | A | 2026-09-01 00:08 | PASS | issue #134; spec-simplification issue #816; workflow authority issue #854 | -| ste-checker-contract | A | 2026-09-01 00:08 | PASS | issue #750 PR audit — the /ste checker had four fail-open paths (unclosed | -| submitted-by-trailers | A | 2026-09-01 00:08 | PASS | conversation 2026-06-12 (commit attribution trailers); the single-owner | -| sync-skill-contract | A | 2026-09-01 00:08 | PASS | issue #331 — /sync dispatcher skill (bidirectional origin↔upstream sync) | -| t3-headless-launch | A | 2026-09-01 00:08 | PASS | issue #858 — /t3 launched a bare `npx --yes t3`, which is the local GUI and | -| tailscale-tool-boundary | A | 2026-09-01 00:08 | PASS | issue #858 — Tailscale mobile access for T3 Code. There is no tailnet, no | -| tool-catalog-boundary | A | 2026-09-01 00:08 | PASS | agent-browser's exclusion from the harness catalog (#821), the three-catalog | -| version-parity | A | 2026-09-01 00:08 | PASS | conversation 2026-08-29 — the oh CLI became the only lifecycle door, so its | -| weigh-scorer-contract | A | 2026-09-01 00:08 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-003 (2026-06-27) | -| wiki-compile-contract | A | 2026-09-01 00:08 | PASS | wikiskill arXiv:2608.27454 — Wiki Maintainer role added as /wiki compile | -| wiki-kind-schema-contract | A | 2026-09-01 00:08 | PASS | wikiskill arXiv:2608.27454 — pattern layer added to the wiki corpus | -| wiki-pattern-persistence | A | 2026-09-01 00:08 | PASS | wikiskill arXiv:2608.27454 — pattern pages are never rolled back | -| wiki-query-pattern-isolation | A | 2026-09-01 00:08 | PASS | wikiskill arXiv:2608.27454 — proposer-only pattern access | -| wiki-readme-index | A | 2026-09-01 00:08 | PASS | issue #132 — wiki README index drift guard | -| wiki-related-slugs | A | 2026-09-01 00:08 | PASS | wikiskill arXiv:2608.27454 — wiki lint related-slug check | -| wiki-skill-impact-append-only | A | 2026-09-01 00:08 | PASS | wikiskill arXiv:2608.27454 — skill-change ledger, never rolled back | -| workflow-boundaries | A | 2026-09-01 00:08 | PASS | conversation 2026-06-19 (workflow consolidation, issue #259); authority moved to /spec in issue #854 | -| worktrees-layout | A | 2026-09-01 00:08 | PASS | issue #872 | +| agent-browser-cli | A | 2026-09-01 01:21 | PASS | retro lesson 2026-06-07 (agent-browser 0.8.5 CLI) | +| agents-identity-contract | A | 2026-09-01 01:21 | PASS | issue #854 — T3-style root identity, glossary, and skill-owned procedures | +| artifact-contract-audit | A | 2026-09-01 01:21 | PASS | issue #583/#645 — production /audit implementation Gate 1 behavior | +| audit-dispatcher-contract | A | 2026-09-01 01:21 | PASS | issue #645 — audit consolidation public taxonomy | +| audit-implementation-behavior | A | 2026-09-01 01:21 | PASS | issue #645 — implementation root/repo/browser behavior | +| audit-pr-acquire | A | 2026-09-01 01:21 | PASS | issue #645 — production PR acquisition behavior | +| audit-pr-classifier | A | 2026-09-01 01:21 | PASS | issue #645 — deterministic focused and queue PR classifier | +| audit-run-root-contract | A | 2026-09-01 01:21 | PASS | issue #645 — executable immutable audit root/run correlation | +| audit-shellcheck-coverage | A | 2026-09-01 01:21 | PASS | issue #645 — private audit scripts require release and CI lint coverage | +| audit-slop-gate | A | 2026-09-01 01:21 | PASS | .claude/specs/images/reduce-slop.png — the four correctness gates all pass a change | +| audit-stale-references | A | 2026-09-01 01:21 | PASS | issue #645 — clean-breaking audit migration | +| boot-lint-glob | A | 2026-09-01 01:21 | PASS | issue #90, issue #120 | +| builder-skill-consolidation | A | 2026-09-01 01:21 | PASS | issue #643 — consolidate artifact builders behind one /builder dispatcher | +| builder-wiki-proposer | A | 2026-09-01 01:21 | PASS | wikiskill arXiv:2608.27454 — the skill proposer reads accumulated knowledge first | +| capability-benchmark-schema | A | 2026-09-01 01:21 | PASS | issue #167 — capability benchmark instrument | +| cc-safety-net-wiring | A | 2026-09-01 01:21 | SKIPPED | .oh/tasks/cc-safety-net/prd.json US-007 2026-07-19 | +| changelog-entry-length | A | 2026-09-01 01:21 | PASS | conversation 2026-08-24 — CHANGELOG.md grew to 259KB of bullet prose because "one line" was unquantified | +| cleanup-no-agent-session-coupling | A | 2026-09-01 01:21 | PASS | issue #928 — retire automated /spec agent handoff | +| cleanup-tasks-scoped-guard | A | 2026-09-01 01:21 | PASS | issue #85 | +| cleanup-tasks-worktree-grooming | A | 2026-09-01 01:21 | PASS | issue #168; issue #327 | +| cli-publish-typecheck-scope | A | 2026-09-01 01:21 | PASS | release run 33271077312 — v0.5.0 pushed its GHCR image, then failed to publish | +| close-issues-on-development | A | 2026-09-01 01:21 | PASS | issue #841 (closing keywords never fire because the default branch is main) 2026-08-26 | +| codex-stale-response-retry | A | 2026-09-01 01:21 | PASS | issue #506 — Codex previous_response_not_found RCA | +| compose-config-path-parity | A | 2026-09-01 01:21 | PASS | PR #833 (remove harness.yaml — the wrapper and VS Code "Reopen in Container" paths must resolve the same service) 2026-08-26 | +| compose-env-boundary | A | 2026-09-01 01:21 | PASS | #920 — the epic #903→#911 made the CLI provision harnesses and tools from | +| config-schema-parity | A | 2026-09-01 01:21 | PASS | PR #833 (one schema file — DOCKER_SOCKET, SANDBOX_SSH, OH_SANDBOX_IMAGE, OH_PULL_POLICY, SKIP_PNPM_INSTALL were consumed but undocumented); rewritten for the oh.json/secrets split by PR #887 | +| context-tier-size-budget | A | 2026-09-01 01:21 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-007) — the always-on tier was 85,256 B | +| continual-learning-20260831 | A | 2026-09-01 01:21 | PASS | retro lesson 2026-08-31 (unexercised oracle) — a probe green in a 112-probe run carried three parser defects | +| cron-claude-codex-fallback | A | 2026-09-01 01:21 | PASS | conversation 2026-06-12 (default Codex fallback for crons) | +| cron-watchdog | A | 2026-09-01 01:21 | PASS | issues #130/#453 (cron runtime watchdog + legacy system-cron reaping) 2026-06-19 | +| crons-directory-guide | A | 2026-09-01 01:21 | PASS | issue #874 | +| curl-bash-safe-alternatives | A | 2026-09-01 01:21 | PASS | vet-run/vet integration — public curl|bash examples need review-first alternatives | +| datasets-schema | A | 2026-09-01 01:21 | PASS | issue #196 — .oh/evals/datasets verifiable trajectory corpus (Repo2RLEnv-inspired) | +| debugmcp-availability | A | 2026-09-01 01:21 | SKIPPED | issue #297 — DebugMCP MCP debug-server availability | +| default-provisioning | A | 2026-09-01 01:21 | PASS | #902 — `oh harness install` must work from inside the sandbox, where | +| delegate-model-effort-policy | A | 2026-09-01 01:21 | PASS | conversation 2026-07-11 (delegate model inheritance and thinking policy) | +| devtcp-hook | A | 2026-09-01 01:21 | PASS | retro lesson 2026-06-10 (zsh /dev/tcp) | +| docker-inspect-env-guard | A | 2026-09-01 01:21 | PASS | operator directive 2026-08-08 (agents keep the docker socket, but must | +| docs-build-fast-path | A | 2026-09-01 01:21 | PASS | #455 — docs builds must stay out of fast harness/eval/release gates; #536 — docs site externalized to openharness-web; docs markdown relocated to docs/ | +| drift-check-cron-staleness-glob | A | 2026-09-01 01:21 | PASS | issue #98; issue #225 (restart-required cron frontmatter/config drift) | +| entrypoint-pnpm-manifest-fingerprint | A | 2026-09-01 01:21 | PASS | issue #521 (manifest-aware sandbox installs) 2026-07-01 | +| escalate-contract | A | 2026-09-01 01:21 | PASS | issue #799 — seven comments on a GitHub thread produced zero notifications and nobody | +| eval-ci-gate | A | 2026-09-01 01:21 | PASS | #103 — eval probe suite gated in CI | +| eval-contract-text-20260831 | A | 2026-09-01 01:21 | PASS | retro lesson 2026-08-31 (prose-literal pinning) — two assertions failed on first run purely from source line wrapping | +| eval-gate | A | 2026-09-01 01:21 | PASS | retro lesson 2026-06-11 (eval-gate) | +| eval-results-atomic | A | 2026-09-01 01:21 | PASS | issue #83 (eval-results-atomic-write) | +| eval-runner-exit | A | 2026-09-01 01:21 | PASS | retro lesson 2026-06-11 (eval-runner-exit) #29 | +| eval-runs-once-per-cycle | A | 2026-09-01 01:21 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-006) — /eval ran 3x per cycle on the | +| execution-target-contract | A | 2026-09-01 01:21 | PASS | issue #733 (ExecutionTarget contract + Docker Compose adapter) 2026-08-10 | +| get-oh-bootstrap | A | 2026-09-01 01:21 | PASS | get-oh.sh bootstrap — the Node-bootstrapping host-side path to the standalone `oh` CLI (also on npm as @mifune/openharness; see oh-npm-package.sh) | +| git-skill | A | 2026-09-01 01:21 | PASS | conversation 2026-06-15 — rules are not always supported; git workflow must be the /git skill | +| harness-audit-empty-output-gate | A | 2026-09-01 01:21 | PASS | issue #246 — /audit harness must fail closed on empty auditor outputs | +| harness-ci-core-paths | A | 2026-09-01 01:21 | PASS | #165 — core sandbox config files must trigger harness CI | +| harness-ci-hooks-paths | A | 2026-09-01 01:21 | PASS | issue #202 — credential/security hook changes must trigger harness CI | +| harness-yaml-migration | A | 2026-09-01 01:21 | PASS | PR #833 (migrate-harness-yaml.sh — append / uncomment-in-place / preserve / overwrite, plus a silent no-op second run) 2026-08-26 | +| headless-tmux-preserved | A | 2026-09-01 01:21 | PASS | issue #928 — retire automated /spec agent handoff | +| health-check-docker-stats | A | 2026-09-01 01:21 | PASS | retro lesson 2026-06-10 (docker stats vs ps Size) | +| health-check-socket-degrade | A | 2026-09-01 01:21 | PASS | issue #762 (refs #756) — /health-check degrades to one statement, not nine failures | +| heartbeat-logging-contract | A | 2026-09-01 01:21 | PASS | issue #447 (heartbeat log append hardening) 2026-06-18 | +| image-seed-hygiene | A | 2026-09-01 01:21 | PASS | issue #900 (slim the sandbox image) 2026-08-30 | +| markitdown-wiki-ingest | A | 2026-09-01 01:21 | PASS | issue #649 — pinned local-document normalization contract for /wiki ingest | +| next-dev-prod | A | 2026-09-01 01:21 | SKIPPED | retro lesson 2026-06-04 | +| oh-compose-env-wiring | A | 2026-09-01 01:21 | SKIPPED | issue #880 (oh as the only front door — oh.json is the non-secret config surface) | +| oh-config-surfaces | A | 2026-09-01 01:21 | PASS | PR #887 (config split across two authored surfaces — a tracked oh.json and a secrets-only root dotenv — with nothing left under $HOME) | +| oh-destroy-guard | A | 2026-09-01 01:21 | SKIPPED | issue #879 — `oh` becomes the only front door, so `make destroy` must | +| oh-devcontainer-restructure | A | 2026-09-01 01:21 | PASS | consolidate devcontainer — .oh/devcontainer/ folded back into .devcontainer/ | +| oh-home-mount | A | 2026-09-01 01:21 | PASS | issue #898 (single $HOME mount) 2026-08-30 | +| oh-image-only-deploy | A | 2026-09-01 01:21 | PASS | .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only | +| oh-init-headless-config | A | 2026-09-01 01:21 | SKIPPED | PR #827 (installer answers landed in the losing config file); retargeted to the .example.env template by PR #833, then to oh.json by PR #887 | +| oh-init-scaffold | A | 2026-09-01 01:21 | PASS | issue #531 Phase 2 | +| oh-lifecycle-surface | A | 2026-09-01 01:21 | PASS | issue #881 — the Makefile is retired and `oh` is the only front door | +| oh-npm-package | A | 2026-09-01 01:21 | PASS | npm publish path for the standalone `oh` CLI (@mifune/openharness) — alternative to get-oh.sh | +| oh-payload-manifest | A | 2026-09-01 01:21 | PASS | issue #531 follow-on (.oh payload manifest — oh update ships a declared allowlist) | +| oh-sandbox-image-mode | A | 2026-09-01 01:21 | PASS | conversation 2026-07-05 (basic Docker deployment — prebuilt-image mode) | +| oh-shipped-repo-overridable | A | 2026-09-01 01:21 | PASS | issue #531 follow-on (de-hardcode residual — shipped .oh shell scripts keep the upstream repo overridable) | +| oh-standalone-lifecycle | A | 2026-09-01 01:21 | PASS | issue #564 | +| oh-update | A | 2026-09-01 01:21 | PASS | issue #531 Phase 3 (oh update — upgrade only the .oh control plane) | +| operator-config-guard | A | 2026-09-01 01:21 | PASS | operator directives 2026-08-06 (.config/ and settings.local.json are operator-only) | +| pnpm-audit-ci-gate | A | 2026-09-01 01:21 | PASS | issue #171 — pnpm security audits must run in CI | +| post-bridge-publish-confirmation | A | 2026-09-01 01:21 | PASS | #523 — post-bridge live publishing requires an explicit final confirmation gate | +| prd-output-path-contract | A | 2026-09-01 01:21 | PASS | retro lesson 2026-06-19 | +| prompt-miner-schema-compat | A | 2026-09-01 01:21 | PASS | issue #253 — prompt-miner JSONL schema-drift guard | +| prompt-miner-symlink-entrypoint | A | 2026-09-01 01:21 | PASS | issue #663 — prompt-miner engine no-ops via the documented .claude/skills symlink | +| prompt-miner-weakness-record | A | 2026-09-01 01:21 | PASS | issue #580 — prompt-miner weakness-record (WH-xxx) cluster output | +| protected-path-deletion | A | 2026-09-01 01:21 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-001) — the critique gate was deleted, | +| protected-paths-resolve | A | 2026-09-01 01:21 | PASS | issue #753 — .claude/protected-paths.txt named 7 paths that did not exist. | +| registry-portability-gate | A | 2026-09-01 01:21 | PASS | issue #758 | +| registry-portability | A | 2026-09-01 01:21 | SKIPPED | issue #758 | +| retro-deterministic-contract | A | 2026-09-01 01:21 | PASS | issue #443 — /retro deterministic output and self-contained helper contract | +| rl-delegation-write-worker | A | 2026-09-01 01:21 | PASS | retro lesson 2026-06-10 (rl-delegation) #57 | +| rlm-context-budget | A | 2026-09-01 01:21 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-006 | +| runtime-preflight-gate | A | 2026-09-01 01:21 | PASS | issue #806 § B1 (open sandbox.substrate vs sandbox.runtime selector); | +| sandbox-boot-guard-ci | A | 2026-09-01 01:21 | PASS | issue #449 (sandbox image build CI guard) 2026-06-19; | +| sandbox-node-base | A | 2026-09-01 01:21 | PASS | openharness#878 — oh as the only front door, T0 sandbox base image | +| skill-paths | A | 2026-09-01 01:21 | PASS | issue #43 — stale path references; extended by issue #69 — apps/->packages/ rename guard; extended by issue #870 — deleted .oh/agents/advisor.md | +| skills-dir-clean | A | 2026-09-01 01:21 | PASS | conversation 2026-06-29 — Pi parses every top-level `.md` in the skills | +| skills-task-tool-coupling | A | 2026-09-01 01:21 | PASS | council review 2026-08-29 (issue #886) — /delegate instructed Claude-Code-only | +| skills-vendored | A | 2026-09-01 01:21 | PASS | absorb .mifune submodule into .oh — the skills/agents/hooks pack is vendored | +| slack-admin-command-surface | A | 2026-09-01 01:21 | PASS | issue #354 — Slack bridge docs must distinguish Pi /msg-bridge commands from Slack DM admin text handlers | +| spec-family-contract | A | 2026-09-01 01:21 | PASS | issue #265; spec-simplification issue #816; workflow authority issue #854; | +| spec-no-advisor-session-coupling | A | 2026-09-01 01:21 | PASS | issue #928 — retire automated /spec agent handoff | +| spec-no-agent-handoff | A | 2026-09-01 01:21 | PASS | issue #928 — retire automated /spec agent handoff | +| spec-ready-finalization | A | 2026-09-01 01:21 | PASS | issue #134; spec-simplification issue #816; workflow authority issue #854 | +| spec-single-owner | A | 2026-09-01 01:21 | PASS | conversation 2026-06-19 (single-owner implementation workflow, issue #257); | +| ste-checker-contract | A | 2026-09-01 01:21 | PASS | issue #750 PR audit — the /ste checker had four fail-open paths (unclosed | +| submitted-by-trailers | A | 2026-09-01 01:21 | PASS | conversation 2026-06-12 (commit attribution trailers); the single-owner | +| sync-skill-contract | A | 2026-09-01 01:21 | PASS | issue #331 — /sync dispatcher skill (bidirectional origin↔upstream sync) | +| t3-headless-launch | A | 2026-09-01 01:21 | PASS | issue #858 — /t3 launched a bare `npx --yes t3`, which is the local GUI and | +| tailscale-tool-boundary | A | 2026-09-01 01:21 | PASS | issue #858 — Tailscale mobile access for T3 Code. There is no tailnet, no | +| tool-catalog-boundary | A | 2026-09-01 01:21 | PASS | agent-browser's exclusion from the harness catalog (#821), the three-catalog | +| version-parity | A | 2026-09-01 01:21 | PASS | conversation 2026-08-29 — the oh CLI became the only lifecycle door, so its | +| weigh-scorer-contract | A | 2026-09-01 01:21 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-003 (2026-06-27) | +| wiki-compile-contract | A | 2026-09-01 01:21 | PASS | wikiskill arXiv:2608.27454 — Wiki Maintainer role added as /wiki compile | +| wiki-kind-schema-contract | A | 2026-09-01 01:21 | PASS | wikiskill arXiv:2608.27454 — pattern layer added to the wiki corpus | +| wiki-pattern-persistence | A | 2026-09-01 01:21 | PASS | wikiskill arXiv:2608.27454 — pattern pages are never rolled back | +| wiki-query-pattern-isolation | A | 2026-09-01 01:21 | PASS | wikiskill arXiv:2608.27454 — proposer-only pattern access | +| wiki-readme-index | A | 2026-09-01 01:21 | PASS | issue #132 — wiki README index drift guard | +| wiki-related-slugs | A | 2026-09-01 01:21 | PASS | wikiskill arXiv:2608.27454 — wiki lint related-slug check | +| wiki-skill-impact-append-only | A | 2026-09-01 01:21 | PASS | wikiskill arXiv:2608.27454 — skill-change ledger, never rolled back | +| workflow-boundaries | A | 2026-09-01 01:21 | PASS | conversation 2026-06-19 (workflow consolidation, issue #259); authority moved to /spec in issue #854 | +| worktrees-layout | A | 2026-09-01 01:21 | PASS | issue #872 | diff --git a/.oh/evals/capability/tasks/CB-001-ship-harness-change.md b/.oh/evals/capability/tasks/CB-001-ship-harness-change.md index e1ad1f98..a3c83968 100644 --- a/.oh/evals/capability/tasks/CB-001-ship-harness-change.md +++ b/.oh/evals/capability/tasks/CB-001-ship-harness-change.md @@ -28,7 +28,7 @@ Given a small harness-infra change request (a skill, rule, doc, script, or cron | unattended | Reached ready-for-review with zero human intervention after the initial ask | Completed but needed ≥1 human nudge (re-run, conflict resolution) | Required hands-on human authoring/fixing to finish | ## Evidence basis -Recent ready-for-review PRs demonstrate the end-to-end path: e.g. #147 (default Pi monitor support) and #141, plus the executable-loop series #157/#163. `/spec plan` composes `/prd` → `/ralph` into the four-file folder; `/spec execute` then keeps implementation and its gates in one Advisor-owned session — issue → branch → draft PR → implementation → `implementation ⇄ audit` (with `/eval` inside it) → `/audit pr` → ready PR. +Recent ready-for-review PRs demonstrate the end-to-end path: e.g. #147 (default Pi monitor support) and #141, plus the executable-loop series #157/#163. `/spec plan` composes `/prd` → `/ralph` into the four-file folder; `/spec execute` then keeps implementation and its gates with one implementation owner — issue → branch → draft PR → implementation → `implementation ⇄ audit` (with `/eval` inside it) → `/audit pr` → ready PR. The capability is scored on task ownership and the ready-PR outcome; the terminal topology behind it (tmux, Herdr, a plain shell) is not part of the measurement. **Re-authored 2026-08-24 (spec-simplification US-003)** when `/ship-spec` was absorbed into `/spec execute` and deleted. The capability under test is unchanged; only the surface that diff --git a/.oh/evals/probes/cleanup-no-agent-session-coupling.sh b/.oh/evals/probes/cleanup-no-agent-session-coupling.sh new file mode 100755 index 00000000..38bae13a --- /dev/null +++ b/.oh/evals/probes/cleanup-no-agent-session-coupling.sh @@ -0,0 +1,40 @@ +#!/usr/bin/env bash +# tier: A +# source: issue #928 — retire automated /spec agent handoff +# desc: the weekly task sweep archives on the progress.txt STATUS marker alone; it never +# detects or kills a separately launched implementation-agent session +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" +CLEANUP="$ROOT/crons/cleanup-tasks.md" +TASKS="$ROOT/.oh/tasks/README.md" + +for file in "$CLEANUP" "$TASKS"; do + [[ -f "$file" ]] || { echo "SKIPPED: required file absent: $file" >&2; exit 2; } +done + +found=() +retired_session='agent-''spec-' +hits=$(grep -nF -- "$retired_session" "$CLEANUP" "$TASKS" || true) +[[ -n "$hits" ]] && found+=("retired session prefix '$retired_session': ${hits//$'\n'/ ; }") + +kill_hits=$(grep -nF -- 'tmux kill-session' "$CLEANUP" || true) +[[ -n "$kill_hits" ]] && found+=("sweep kills an agent session: ${kill_hits//$'\n'/ ; }") + +missing=() +grep -qF 'STATUS: COMPLETE' "$CLEANUP" || missing+=("sweep no longer keys archival on the progress.txt STATUS: COMPLETE marker") +grep -qiF 'never' "$CLEANUP" && grep -qiF 'tied to a terminal session' "$CLEANUP" \ + || missing+=("sweep no longer states that task state is untied from a terminal session") + +# The sweep's live-pane guard is a worktree-grooming safety check on foreign work, not an +# implementation-agent handoff: it must survive this decoupling. +grep -qF 'tmux list-panes' "$CLEANUP" || missing+=("sweep lost its live-pane worktree-grooming guard (tmux list-panes)") + +if (( ${#found[@]} + ${#missing[@]} )); then + printf 'REGRESSION: task cleanup recoupled to an implementation-agent session:\n' >&2 + (( ${#found[@]} )) && printf ' - %s\n' "${found[@]}" >&2 + (( ${#missing[@]} )) && printf ' - %s\n' "${missing[@]}" >&2 + exit 1 +fi + +echo "PASS: task cleanup archives on STATUS: COMPLETE alone, kills no agent session, and keeps its live-pane grooming guard" >&2 diff --git a/.oh/evals/probes/headless-tmux-preserved.sh b/.oh/evals/probes/headless-tmux-preserved.sh new file mode 100755 index 00000000..4533b5e9 --- /dev/null +++ b/.oh/evals/probes/headless-tmux-preserved.sh @@ -0,0 +1,46 @@ +#!/usr/bin/env bash +# tier: A +# source: issue #928 — retire automated /spec agent handoff +# desc: retiring the /spec Advisor handoff must not strip tmux from independently justified +# headless infrastructure — cron runtime, gateway clients, tunnels, and the T3 Code server +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" +PROC="$ROOT/.oh/skills/t3/references/sandbox-processes.md" +GATEWAY="$ROOT/.oh/scripts/gateway.sh" +CRON_RUNTIME="$ROOT/.oh/scripts/cron-runtime.ts" +CRON_GUIDE="$ROOT/crons/AGENTS.md" +ENTRYPOINT="$ROOT/.devcontainer/entrypoint.sh" + +for file in "$PROC" "$GATEWAY" "$CRON_RUNTIME" "$CRON_GUIDE" "$ENTRYPOINT"; do + [[ -f "$file" ]] || { echo "SKIPPED: required file absent: $file" >&2; exit 2; } +done + +missing=() + +grep -qF 'MUST run inside a named tmux' "$PROC" || missing+=("sandbox-processes.md lost the named-tmux rule for long-running processes") +grep -qF '## Session Naming' "$PROC" || missing+=("sandbox-processes.md lost the tmux session-naming convention") +for category in '`cron-`' '`client-`' '`cloudflared-`' '`app-`' '`agent-`'; do + grep -qF "$category" "$PROC" || missing+=("sandbox-processes.md lost the $category tmux session category") +done +grep -qF 'why tmux, not a service' "$PROC" || missing+=("sandbox-processes.md lost the gateway tmux-vs-service rationale") + +grep -qF 'tmux' "$GATEWAY" || missing+=("gateway.sh no longer runs its client in tmux") +grep -qF 'tmux' "$CRON_RUNTIME" || missing+=("cron-runtime.ts no longer manages tmux sessions") +grep -qF 'tmuxSessionName' "$CRON_RUNTIME" || missing+=("cron-runtime.ts lost its tmux session naming for detached fires") +grep -qF 'cron---' "$CRON_GUIDE" || missing+=("crons/AGENTS.md lost the detached-fire tmux session convention") +grep -qF 'tmux' "$ENTRYPOINT" || missing+=("the sandbox entrypoint no longer starts its tmux supervisor sessions") + +# The retirement is scoped: the /spec agent-handoff exception is gone, the generic rule stays. +grep -qiF 'not a tmux exception' "$PROC" || missing+=("sandbox-processes.md no longer scopes the tmux rule away from /spec execute") +retired_session='agent-''spec-' +prescribed=$(grep -nF -- "$retired_session" "$PROC" | grep -vF 'Do not reintroduce' || true) +[[ -n "$prescribed" ]] && missing+=("sandbox-processes.md still prescribes the retired $retired_session session convention: ${prescribed//$'\n'/ ; }") + +if (( ${#missing[@]} )); then + printf 'REGRESSION: headless tmux infrastructure damaged by the /spec handoff retirement:\n' >&2 + printf ' - %s\n' "${missing[@]}" >&2 + exit 1 +fi + +echo "PASS: tmux remains intact for cron, gateway, tunnel, and T3 headless infrastructure while the /spec Advisor exception is gone" >&2 diff --git a/.oh/evals/probes/spec-no-advisor-session-coupling.sh b/.oh/evals/probes/spec-no-advisor-session-coupling.sh new file mode 100755 index 00000000..8361edf0 --- /dev/null +++ b/.oh/evals/probes/spec-no-advisor-session-coupling.sh @@ -0,0 +1,52 @@ +#!/usr/bin/env bash +# tier: A +# source: issue #928 — retire automated /spec agent handoff +# desc: /spec task identity and RUNNING state depend on the task folder alone, never on an +# agent-spec-* session, a tmux session name, a Herdr tab/pane id, or another runtime handle +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" +SPEC="$ROOT/.oh/skills/spec" +EXEC="$SPEC/references/execute.md" +SKILL="$SPEC/SKILL.md" +PLAN="$SPEC/references/plan.md" +TASKS="$ROOT/.oh/tasks/README.md" +GLOSSARY="$ROOT/docs/glossary.md" + +for file in "$EXEC" "$SKILL" "$PLAN" "$TASKS" "$GLOSSARY"; do + [[ -f "$file" ]] || { echo "SKIPPED: required file absent: $file" >&2; exit 2; } +done + +mapfile -t SCOPE < <(find "$SPEC" -type f \( -name '*.md' -o -name '*.sh' \) | sort) +SCOPE+=("$TASKS" "$GLOSSARY") + +found=() + +retired_session='agent-''spec-' +hits=$(grep -rnF -- "$retired_session" "${SCOPE[@]}" || true) +[[ -n "$hits" ]] && found+=("retired session prefix '$retired_session': ${hits//$'\n'/ ; }") + +# With the prohibition prose written so it never uses these handles as its own words, an +# affirmative recoupling is exactly a literal occurrence — no negation-aware matcher needed. +coupling=$(grep -rniF -- "$(printf '%s\n' 'tmux session name' 'Herdr tab id' 'Herdr pane id' 'tab id' 'pane id' '$CRON_TMUX_SESSION')" "${SCOPE[@]}" || true) +[[ -n "$coupling" ]] && found+=("affirmative session coupling: ${coupling//$'\n'/ ; }") + +missing=() +grep -qiF 'It is never a terminal' "$SKILL" || missing+=("/spec dispatcher no longer states the slug is never a terminal identifier") +grep -qiF 'never the existence of a named process, session, tab, or pane' "$EXEC" \ + || missing+=("execute.md no longer decouples RUNNING task state from a named process/session/tab/pane") +grep -qiF 'it never names a terminal session, tab, or pane' "$PLAN" \ + || missing+=("plan.md no longer decouples the slug from a terminal session name") +grep -qiF 'never depend on a session, tab, or pane' "$TASKS" \ + || missing+=(".oh/tasks/README.md no longer decouples task identity from a session/tab/pane id") +grep -qiF 'implementation owner' "$GLOSSARY" \ + || missing+=("docs/glossary.md no longer distinguishes the implementation owner from the terminal backend") + +if (( ${#found[@]} + ${#missing[@]} )); then + printf 'REGRESSION: /spec task identity recoupled to a terminal/session identifier:\n' >&2 + (( ${#found[@]} )) && printf ' - %s\n' "${found[@]}" >&2 + (( ${#missing[@]} )) && printf ' - %s\n' "${missing[@]}" >&2 + exit 1 +fi + +echo "PASS: /spec task identity and RUNNING state are independent of tmux/Herdr session identifiers" >&2 diff --git a/.oh/evals/probes/spec-no-agent-handoff.sh b/.oh/evals/probes/spec-no-agent-handoff.sh new file mode 100755 index 00000000..f739cb40 --- /dev/null +++ b/.oh/evals/probes/spec-no-agent-handoff.sh @@ -0,0 +1,48 @@ +#!/usr/bin/env bash +# tier: A +# source: issue #928 — retire automated /spec agent handoff +# desc: active /spec surfaces neither launch nor prescribe a second coding-agent process — +# no multiplexer session, Herdr workspace/tab/pane, background shell, or generic runner +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" +SPEC="$ROOT/.oh/skills/spec" +EXEC="$SPEC/references/execute.md" +SKILL="$SPEC/SKILL.md" + +for file in "$EXEC" "$SKILL"; do + [[ -f "$file" ]] || { echo "SKIPPED: required file absent: $file" >&2; exit 2; } +done + +mapfile -t SURFACES < <(find "$SPEC" -type f \( -name '*.md' -o -name '*.sh' \) | sort) +(( ${#SURFACES[@]} )) || { echo "SKIPPED: no /spec surfaces found under $SPEC" >&2; exit 2; } + +launch_literals=( + 'tmux new-session' 'tmux new -s' 'tmux pipe-pane' 'tmux send-keys' + 'tmux attach' 'tmux kill-session' 'tmux list-sessions' + 'herdr agent start' 'herdr tab create' 'herdr workspace create' 'herdr pane split' + 'nohup ' 'setsid ' 'run_in_background' + 'SPEC_RUNNER' 'AUTOPILOT_EXECUTOR' 'session-runner' '--executor' +) + +found=() +for literal in "${launch_literals[@]}"; do + hits=$(grep -rnF -- "$literal" "${SURFACES[@]}" || true) + [[ -n "$hits" ]] && found+=("agent-launch literal '$literal': ${hits//$'\n'/ ; }") +done + +missing=() +grep -qiF 'This node launches nothing' "$EXEC" || missing+=("execute.md no longer states that the node launches nothing") +grep -qiF 'It does not create the agent that' "$EXEC" || missing+=("execute.md no longer states that /spec does not create the agent that executes it") +grep -qiF 'no fallback runner because there' "$EXEC" || missing+=("execute.md no longer states there is no fallback runner because there is no handoff") +grep -qiF 'Launch a coding agent.' "$EXEC" || missing+=("execute.md's 'What this node does NOT do' no longer forbids launching a coding agent") +grep -qiF 'never launches another coding-agent process' "$SKILL" || missing+=("/spec dispatcher no longer states it never launches another coding-agent process") + +if (( ${#found[@]} + ${#missing[@]} )); then + printf 'REGRESSION: /spec reintroduced automated agent handoff:\n' >&2 + (( ${#found[@]} )) && printf ' - %s\n' "${found[@]}" >&2 + (( ${#missing[@]} )) && printf ' - %s\n' "${missing[@]}" >&2 + exit 1 +fi + +echo "PASS: no /spec surface launches or prescribes a second coding-agent process; the running agent is the owner" >&2 diff --git a/.oh/evals/probes/advisor-monitored-loop.sh b/.oh/evals/probes/spec-single-owner.sh similarity index 57% rename from .oh/evals/probes/advisor-monitored-loop.sh rename to .oh/evals/probes/spec-single-owner.sh index 1e8b4bad..908dda94 100755 --- a/.oh/evals/probes/advisor-monitored-loop.sh +++ b/.oh/evals/probes/spec-single-owner.sh @@ -1,8 +1,10 @@ #!/usr/bin/env bash # tier: A -# source: conversation 2026-06-19 (single-owner implementation workflow, issue #257) -# desc: /spec execute gives one Advisor ownership of implementation and final gates; no -# second implementation owner, nested session, or handoff may reappear +# source: conversation 2026-06-19 (single-owner implementation workflow, issue #257); +# renamed from advisor-monitored-loop by issue #928 (retire automated /spec agent handoff) +# desc: /spec execute gives the agent that runs it sole ownership of implementation and the +# final gates; no second implementation owner, supervisor, or handoff may reappear, and +# ownership never depends on a separately launched process set -euo pipefail ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" @@ -17,13 +19,16 @@ for file in "$EXEC" "$PROMPT"; do done missing=() -grep -qiF 'single implementation Advisor' "$PROMPT" || missing+=("task prompt names one implementation Advisor") +grep -qiF 'single implementation owner' "$PROMPT" || missing+=("task prompt names one implementation owner") grep -qiF 'Do not hand the task to a' "$PROMPT" && grep -qiF 'second implementation owner' "$PROMPT" || missing+=("task prompt forbids a second implementation owner") grep -qiF 'Use `/delegate` only for bounded' "$PROMPT" || missing+=("task prompt limits /delegate to bounded fan-out") grep -qiF 'write and commit `evidence.md`' "$PROMPT" || missing+=("task prompt keeps evidence ownership") grep -qiF 'then run a fresh' "$PROMPT" && grep -qiF '/audit pr' "$PROMPT" || missing+=("task prompt keeps the final audit gate") grep -qiF '/delegate' "$EXEC" || missing+=("execute procedure names /delegate as the implementation mechanism") -grep -qiF 'one **expert Advisor' "$EXEC" || missing+=("execute procedure names one Advisor owner") +grep -qiF 'the agent that is running it' "$EXEC" || missing+=("execute procedure does not name the running agent as the owner") +grep -qiF 'never becomes a second supervisor' "$EXEC" || missing+=("execute procedure no longer bounds /delegate below the owner") +grep -qiF 'do not create a second implementation owner' "$EXEC" || missing+=("execute procedure no longer forbids a second implementation owner on resume") +grep -qiF 'Ownership is a **role**, not a terminal' "$EXEC" || missing+=("execute procedure no longer states ownership is a role, not a terminal topology") retired_spec_build='.oh/scripts/spec-''build.sh' retired_runner='.oh/scripts/lib/session-''runner.sh' @@ -39,4 +44,4 @@ if (( ${#missing[@]} )); then exit 1 fi -echo 'PASS: one /spec Advisor owns implementation and gates; /delegate is bounded fan-out; retired handoff is absent' >&2 +echo 'PASS: the agent running /spec execute owns implementation and gates; /delegate is bounded fan-out; retired handoff is absent' >&2 diff --git a/.oh/skills/audit/references/eval-quality.md b/.oh/skills/audit/references/eval-quality.md index 6b2e0b73..faf19bc2 100644 --- a/.oh/skills/audit/references/eval-quality.md +++ b/.oh/skills/audit/references/eval-quality.md @@ -366,7 +366,7 @@ Only the two probes RESULTS.md records as persistently |--------|-------|-------|---------| | autopilot-preflight-gate | probe | 3 | CUT | | next-dev-prod | probe | 3 | CUT | -| advisor-monitored-loop | probe | — | KEEP | +| spec-single-owner | probe | — | KEEP | | … (73 more probes) … | probe | — | KEEP | | CB-001 | task | — | KEEP | | CB-002 | task | — | KEEP | diff --git a/.oh/skills/audit/references/implementation.md b/.oh/skills/audit/references/implementation.md index 51945ded..fdf7bf49 100644 --- a/.oh/skills/audit/references/implementation.md +++ b/.oh/skills/audit/references/implementation.md @@ -45,7 +45,7 @@ the production helper `"$AUDIT_ROOT/.oh/skills/audit/scripts/implementation-gate the snippets below explain its behavior and are not a second implementation. **(a) Task-graph conformance.** Every user story in the task graph must be marked -complete. The Advisor flips `passes: false → true` only after it validates each story, so +complete. The implementation owner flips `passes: false → true` only after validating each story, so the graph is conformant only when **zero** stories remain unfinished: ```bash diff --git a/.oh/skills/delegate/SKILL.md b/.oh/skills/delegate/SKILL.md index 858dd862..48121cfc 100644 --- a/.oh/skills/delegate/SKILL.md +++ b/.oh/skills/delegate/SKILL.md @@ -145,7 +145,7 @@ can pick up the worktree. Write the graph to disk before spawning any worker. | `delegate-graph.json` | Every task's ID, title, description, `dependsOn`, files, complexity, model override plus its reason, thinking level, acceptance criteria, assigned wave, and `status` (`pending`/`running`/`completed`/`FAIL`/`BLOCKED`) | | `delegate-log.txt` | Append-only run log; one line per wave boundary and per status change | -Never write `prd.json` or `progress.txt`. Those belong to the Advisor +Never write `prd.json` or `progress.txt`. Those belong to the implementation owner (`.oh/tasks/README.md`), and `progress.txt` in particular must not be edited by hand. This skill's two files sit beside them without collision. diff --git a/.oh/skills/retro/SKILL.md b/.oh/skills/retro/SKILL.md index f09bf920..8e6e0e17 100644 --- a/.oh/skills/retro/SKILL.md +++ b/.oh/skills/retro/SKILL.md @@ -189,7 +189,7 @@ Write `- none` when nothing qualified. This block is a nomination, not a write. `/retro` does not create `.oh/evals/probes/.sh`, and does not write any file. Minting the probe is separate work performed by the operator or a follow-up task. -`--dry-run` and `auto-approve` remain accepted for call-site compatibility — including the Advisor running `/spec execute`'s tail — and produce the same report, because there is nothing to gate. +`--dry-run` and `auto-approve` remain accepted for call-site compatibility — including the owner running `/spec execute`'s tail — and produce the same report, because there is nothing to gate. ### 7. Close the report diff --git a/.oh/skills/rlm/SKILL.md b/.oh/skills/rlm/SKILL.md index 35897d16..52aba74f 100644 --- a/.oh/skills/rlm/SKILL.md +++ b/.oh/skills/rlm/SKILL.md @@ -37,7 +37,7 @@ candidate answers a chunk yields. This skill never re-implements selection — i | Substrate | Owner | How `/rlm` uses it | |---|---|---| -| Recursion **loop** | `/spec execute` | each story re-reads disk = the REPL step — **owned by the active Advisor, never split into another session** | +| Recursion **loop** | `/spec execute` | each story re-reads disk = the REPL step — **owned by the active implementation owner, never split into another session** | | Isolated recursion **branches** | `.worktrees/` (the `/worktrees` skill) | depth-2 sub-trees fork here — **reused by reference, never edited** | | Recursion **budget** | `/delegate` (`.oh/skills/delegate/SKILL.md`) | the `Max depth N / Max children per level M / Step budget S` triple — `references/recursion-budget.md` points at it and adds a per-run token ceiling | | Chunk-map **primitive** | `scripts/query-context.mjs` (US-004, this skill) | partitions the artifact without ingesting it | diff --git a/.oh/skills/spec/SKILL.md b/.oh/skills/spec/SKILL.md index 55049db7..7b6c6fd0 100644 --- a/.oh/skills/spec/SKILL.md +++ b/.oh/skills/spec/SKILL.md @@ -34,7 +34,7 @@ runs only that node, which is what fan-out and recovery need. This is the **only** spec pipeline; there is no all-in-one composer beside it. `references/execute.md` holds the build mechanics in full — the issue, the branch, -the draft PR, the build launch, the `/eval` and wiki gates, the promotable +the draft PR, the implementation, the `/eval` and wiki gates, the promotable classification, and the undraft — so learning what the build does never sends a reader to a second skill. The dispatcher splits the pipeline so each node can be run independently or fanned out at scale via `/delegate`. @@ -51,7 +51,10 @@ approval, so `ship` carries it through to `execute` without a second prompt. A b topic with no plan file has no such approval behind it: `ship` stops after `plan` and hands the operator the folder to approve. `/spec execute` runs `build ⇄ audit → evidence → spec-retro → improve` and stops at a ready-for-review -pull request. The human alone merges. The runner performs `reset` or `clean`. +pull request. **`/spec` never launches another coding-agent process to do that work** — +the agent invoking `execute` is the single implementation owner, from the isolated +worktree through the final PR gates. The human alone merges. The runner performs +`reset` or `clean`. The `.oh/tasks//` folder is the interface between all three subcommands. `evidence.md` records plan requirements, build results, reasons for divergence, @@ -105,17 +108,18 @@ esac four direct references. Do not duplicate the workflow in root instructions. - **The `.oh/tasks//` folder is the universal interface** — `plan` produces it; `execute` and `retro` are each pointed at it. The `` is the - universal key (task directory, branch second segment, tmux session name). + universal key (task directory, branch second segment). It is never a terminal + identifier — not a multiplexer session, not a Herdr tab or pane, not any runtime handle. - **Compose, don't fork** — each node reuses existing skills rather than re-implementing them: `ship` composes `plan` then `execute` and owns no build - mechanics of its own; `plan` composes `/prd` + `/ralph`; `execute` owns the - implementation in one Advisor session, uses `/delegate` only for bounded fan-out, + mechanics of its own; `plan` composes `/prd` + `/ralph`; `execute` is owned by the + agent running it, uses `/delegate` only for bounded fan-out, and composes `/audit implementation` + `/eval` + `/audit pr`; `retro` composes `/retro`. The build **literals** — the `gh` invocations, the branch and PR shapes, - the Advisor launch, and the handoff-free implementation rules — live in + and the handoff-free implementation rules — live in `references/execute.md`, which is the single source for them and is a protected path. - **One adversarial loop** — `implementation ⇄ audit` inside `execute` vets the - implementation (`AUDIT-FAIL` routes back to the same Advisor session). The plan + implementation (`AUDIT-FAIL` routes back to the same owner). The plan itself is vetted by the operator who approves it: **approving `prd.md` is the commitment gate**, and nothing GitHub-side exists until `execute` starts. - **Honest terminal reports** — each subcommand reports what it actually produced: `plan` diff --git a/.oh/skills/spec/references/execute.md b/.oh/skills/spec/references/execute.md index 0a7ba3e3..cc9feb13 100644 --- a/.oh/skills/spec/references/execute.md +++ b/.oh/skills/spec/references/execute.md @@ -12,7 +12,7 @@ implementation to a ready-for-review PR and stops at the human merge gate. It co workflow's one adversarial loop — `implementation ⇄ audit`. **This file is the whole workflow.** Every mechanic it needs — the issue, the branch, the draft -PR, the Advisor launch, the `/eval` gate, the wiki gate, the promotable classification, and the +PR, the implementation, the `/eval` gate, the wiki gate, the promotable classification, and the undraft — is written out below, in order, with no deferral to another skill. Reading it top to bottom tells you what `/spec execute` does; you never have to open a second skill to learn the next step. @@ -60,12 +60,19 @@ Precondition: `.oh/tasks//` carries the four-file contract (`prd.md`, `prd node (`.oh/skills/spec/SKILL.md`). If the folder is incomplete, refuse and route back to `/spec plan`. -**There is no executor argument or separate implementation process.** `/spec execute` has -one owner: the Advisor session it launches. That Advisor implements the approved task graph, -validates each story, records progress, runs the audit/eval/wiki/evidence gates, and -finalizes the PR. `/delegate` is available only for bounded, disjoint worker tasks; it -never becomes a second supervisor or a replacement workflow. `STATUS: COMPLETE` remains -a durable task record, not a handoff signal to another session. +**There is no executor argument, no handoff, and no separate implementation process.** +`/spec execute` has exactly one implementation owner: **the agent that is running it**. That +agent implements the approved task graph, validates each story, records progress, runs the +audit/eval/wiki/evidence gates, and finalizes the PR. Ownership is a **role**, not a terminal +topology — it is not a tmux session, a Herdr tab or pane, a background shell, or any other +runtime identifier. `/delegate` is available only for bounded, disjoint worker tasks whose +results this owner reconciles; it never becomes a second supervisor, a second PR owner, or a +replacement workflow. `STATUS: COMPLETE` is a durable task record, not a handoff signal. + +`/spec` defines and verifies the execution contract. **It does not create the agent that +executes it.** The operator starts the coding-agent session — in Herdr, a terminal, an IDE, or +anywhere else — and pastes the task into it; from that point the receiving agent owns the task +through every gate below. --- @@ -170,10 +177,10 @@ Closes #. ## Stories -## Next steps (automated) -1. Launch the single expert `/worktrees` Advisor in tmux session `agent-spec-`. -2. The Advisor implements the task directly, using `/delegate` only for bounded disjoint work; it validates the stories, runs `/audit implementation`, and revises required wiki entries. -3. The Advisor runs a fresh `/audit pr` immediately before any undraft; this PR is marked ready (`gh pr ready`) only when that audit classifies it promotable (CI green + mergeable + clean). Heartbeat stale-draft watchdog output — including draft-age and draft-cap/backlog warnings — is only a resume/investigation hint, never an undraft signal. +## Next steps +1. The agent running `/spec execute` is this task's single implementation owner; it works in the isolated worktree `.worktrees//-`. +2. It implements the stories directly, using `/delegate` only for bounded disjoint work; it validates the stories, runs `/audit implementation`, and revises required wiki entries. +3. It runs a fresh `/audit pr` immediately before any undraft; this PR is marked ready (`gh pr ready`) only when that audit classifies it promotable (CI green + mergeable + clean). Heartbeat stale-draft watchdog output — including draft-age and draft-cap/backlog warnings — is only a resume/investigation hint, never an undraft signal. 🤖 Generated with [Claude Code](https://claude.com/claude-code) via /spec execute EOF @@ -183,53 +190,50 @@ EOF Capture the PR URL and PR number ``. This is an observability checkpoint, not the terminal state. -### 4. Launch the single Advisor owner +### 4. Implement — the running agent is the owner -This node launches one **expert Advisor on `/worktrees`** in a detached tmux session, -driven by a `/goal`-prefixed prompt. The Advisor is the only owner after the draft PR is -created: it implements the task, coordinates bounded workers, validates the task graph, -runs every quality gate, and finalizes the PR. No second implementation owner or nested -supervisory session is created. +**This node launches nothing.** No Advisor session, no detached multiplexer session and no +piped pane log, no Herdr workspace, tab, or pane created on the operator's behalf, no +background-shell launch, and no runner selection. There is **no fallback runner because there +is no handoff step**. The agent that reached this line implements the task itself and carries +it through every gate below. -**Build worktree — reuse vs. create.** When `$CRON_WORKTREE` is set (a `worktree: true` cron's default), -this run is ALREADY inside an isolated worktree that step 2 put on the feature branch, so the -Advisor **reuses it** — it does NOT create a second worktree (a second `git worktree add` for -the same branch would nest under the cron worktree via the relative path, or fail with -`branch already checked out`). Standalone (no `$CRON_WORKTREE`) the Advisor creates -`.worktrees//-`. Start the Advisor session **in the build worktree** with -`-c`, and bake the worktree path into the prompt — a new tmux session does not inherit -`$CRON_WORKTREE` from the launching client, so passing it via env is unreliable: +**Build worktree — reuse vs. create.** Isolation stays. When `$CRON_WORKTREE` is set (a +`worktree: true` cron's default), this run is ALREADY inside an isolated worktree that step 2 +put on the feature branch, so **reuse it** — do NOT create a second worktree (a second +`git worktree add` for the same branch would nest under the cron worktree via the relative +path, or fail with `branch already checked out`). Standalone (no `$CRON_WORKTREE`), create +`.worktrees//-` via `/worktrees` and work there: ```bash -SESSION="agent-spec-" # e.g. printf %s "" | tr '/:[:space:]' '-' -WT="${CRON_WORKTREE:-}" # set by the cron runtime in worktree mode; empty standalone -tmux new-session -d -s "$SESSION" -c "${WT:-$PWD}" \ - ' "/goal "' -tmux pipe-pane -o -t "$SESSION" "cat >> /tmp/$SESSION.log" -# = the active agent CLI (pi | claude | codex); pi matches the cron default +WT="${CRON_WORKTREE:-}" # set by the cron runtime in worktree mode +if [ -n "$WT" ]; then + cd "$WT" # already on /- +else + git worktree add ".worktrees//-" "/-" + cd ".worktrees//-" +fi ``` -**Do not pipe or redirect the launched command.** The Advisor is an interactive, multi-turn -session; a `2>&1 | tee ` on it replaces its stdout with a pipe, and the session exits -after one turn instead of driving the build. `tmux pipe-pane` attaches to the pane *after* -it exists, which captures the same output while leaving the pane a terminal. This leaves the Advisor's terminal attached to its own interactive workflow and creates no -nested implementation session. - -**Advisor `/goal` prompt** (one line; fill the placeholders — when `$CRON_WORKTREE` is set, -substitute its actual path for `` and use the "reuse" branch of step 1): +Then implement, in this same session: -> `/goal` As the **single expert Advisor on `/worktrees`**, implement `.oh/tasks//prd.json` for PR `#` on branch `/-`. (1) **If `` is already provided** (autopilot's `$CRON_WORKTREE`, already on branch `/-`): `cd ` and do NOT create another worktree. **Otherwise** create an isolated worktree at `.worktrees//-` via `/worktrees` and `cd` into it. (2) Read `.oh/tasks//prompt.md`, implement the dependency-ready stories directly, and use `/delegate` only for bounded disjoint work. Reconcile worker results, validate every acceptance criterion, update `prd.json` and `progress.txt`, and append `STATUS: COMPLETE` only after every story passes. (3) Continue in this same Advisor session with the implementation-side audit loop — including the gate-5 simplify sub-loop, where you delete the code each finding names and drive `netAdded` down until the round cap or a non-reducing round ends it — `/eval` once, required wiki revision, `/compact`, `evidence.md`, `/spec retro`, improve steps, and a fresh `/audit pr`; run `gh pr ready --repo "$SPEC_REPO"` only if that audit is promotable (CI green + mergeable + clean). Otherwise comment the blocking gate and leave the PR draft. Never `gh pr merge`. Leave this single session alive for attach. +1. Read `.oh/tasks//prompt.md` — the rendered task prompt — and `prd.json`'s story graph. +2. Implement the dependency-ready stories directly. Use `/delegate` **only** for bounded, + disjoint worker tasks, and reconcile every worker result yourself. A delegated worker never + becomes a second supervisor, never owns the whole task, and never finalizes the PR. +3. Validate every acceptance criterion against the repository, flip each story's `passes` to + `true` only after that validation, and append a dated `progress.txt` entry naming the files, + the commit, the result, and the learnings. +4. Append `STATUS: COMPLETE` to `progress.txt` only after every story in `prd.json` passes. + That marker is a durable task record for resume and cleanup; it hands control to no one. -The Advisor owns implementation and all post-build gates inside the same session. This node's -turn ends after launching it and reporting the session name; the ready-for-review PR is -produced asynchronously. The Advisor commits story changes on `/-` with a -`Submitted-by:` trailer and keeps worktree isolation intact. If `tmux` is unavailable, run -the same Advisor workflow in the foreground; do not create a second implementation owner. +Commit story changes on `/-` with a `Submitted-by:` trailer and keep worktree +isolation intact. `RUNNING` describes the **task's** state — an approved folder whose stories +are not all `passes: true` — never the existence of a named process, session, tab, or pane. If +implementation is incomplete, leave the PR draft and resume `/spec execute` against the same +task folder; do not create a second implementation owner. -The Advisor records `STATUS: COMPLETE` only after every story in `prd.json` passes. That -marker is a durable task record for resume and cleanup; it does not hand control to another -session. If implementation is incomplete, leave the PR draft and continue the same Advisor -session or resume it from the task folder. +Then continue, in this same session, with steps 5 through 9 below. ### 5. `implementation ⇄ audit` — the adversarial loop @@ -243,15 +247,15 @@ When implementation is complete, run the per-unit verdict gate: floor + `/audit pr` promotable classification (+ `/agent-browser` for UI stories, + the gate-5 slop check) into one verdict: -- `AUDIT-FAIL` → loop back to implementation in the same Advisor session to finish the +- `AUDIT-FAIL` → loop back to implementation in this same session to finish the unmet stories, then re-audit. This is the implementation-side adversary — keep looping - until the Advisor satisfies the task graph. + until the owner satisfies the task graph. - `AUDIT-PASS` → implementation is promotable; continue to the tail. **The simplify sub-loop — drive `netAdded` down.** Gate 5 asks whether the diff can be smaller and still satisfy every acceptance criterion. On an `AUDIT-FAIL (gate 5)` the -Advisor removes the code the finding names — it does not argue with it — and re-audits. -The Advisor owns the round record; the read-only audit route only reads it: +owner removes the code the finding names — it does not argue with it — and re-audits. +The owner writes the round record; the read-only audit route only reads it: ```bash COUNTER=".oh/tasks//simplify-rounds.json" @@ -264,8 +268,8 @@ JSON git add -f "$COUNTER" ``` -Two things end this loop, and neither of them is agreement: the **cap** of 3 rounds, and -a round whose `netAdded` did not strictly fall below the previous round's. Either way the +Two things end this loop, and neither of them is agreement: the **cap** of 3 rounds, and a +**non-reducing round** — one whose `netAdded` did not strictly fall below the previous round's. Either way the audit stops blocking and passes with `SIMPLICITY-RESIDUAL`, and those residual findings go into `evidence.md` under *What remains unverified* for the operator to judge. A simplify loop that cannot make the diff smaller has finished its work; one that keeps looping on @@ -332,11 +336,11 @@ named entries were not updated or the index probe fails, leave the PR draft and missing wiki gate. **`/compact` after implementation.** Implementation and `/eval` have spent significant context -in the Advisor's session; run `/compact` so the promotable gate and the undraft decision start +in this session; run `/compact` so the promotable gate and the undraft decision start clean. Preserve the finalize keys: ```text -Preserve /spec execute finalize context: slug , branch /-, issue #, PR #, implementation complete (STATUS: COMPLETE), /eval result, wiki alignment gate result (REQUIRED updated or NOT-APPLICABLE), undraft gate (/audit pr promotable → gh pr ready, else comment + stay draft), no auto-merge, tmux session agent-spec- left alive. +Preserve /spec execute finalize context: slug , branch /-, issue #, PR #, implementation complete (STATUS: COMPLETE), /eval result, wiki alignment gate result (REQUIRED updated or NOT-APPLICABLE), undraft gate (/audit pr promotable → gh pr ready, else comment + stay draft), no auto-merge. ``` Non-blocking — if `/compact` is unavailable or errors, log a warning and continue. @@ -346,8 +350,8 @@ Non-blocking — if `/compact` is unavailable or errors, log a warning and conti **This is a gate condition, not a formality.** Step 9 refuses to undraft without it — this artifact carries the implementation's answer to the reviewer. -The operator's understanding of this work stops at the plan they approved. The same Advisor -session implements the stories and records the result. `evidence.md` answers back to the plan +The operator's understanding of this work stops at the plan they approved. The same owner +implements the stories and records the result. `evidence.md` answers back to the plan with the observed behavior, deviations, and remaining gaps. Write `.oh/tasks//evidence.md` and **commit it on the branch**, so it travels in the PR @@ -436,7 +440,7 @@ was written but added without `-f` is present on disk and **absent from the PR d is the same as not having it, from the reviewer's seat. **Promote the implementation narrative into the PR body.** `progress.txt` holds the per-story -record the Advisor wrote — what it did, what it learned, and what it deviated on. Update the +record the owner wrote — what it did, what it learned, and what it deviated on. Update the PR body from it and from `evidence.md` so the reviewer meets the work in the PR rather than by opening the task folder: @@ -489,8 +493,7 @@ gh pr comment --repo "$SPEC_REPO" --body "spec execute: PR left draft — < ``` Then **stop**. The human owns the merge (`.oh/skills/spec/SKILL.md`: human merge is -the final gate; no auto-merge). Never `gh pr merge`. The `agent-spec-` tmux session is left -alive for attach/continue (per `.oh/skills/t3/references/sandbox-processes.md`). Print the PR +the final gate; no auto-merge). Never `gh pr merge`. Print the PR URL and terminal status (`READY` or `DRAFT-BLOCKED`) as the final pipeline output. --- @@ -503,7 +506,7 @@ URL and terminal status (`READY` or `DRAFT-BLOCKED`) as the final pipeline outpu | 1 | `gh issue create` fails (auth, label, repo perms) | Diagnose; create the issue manually; re-run with the issue located | | 2 | Pre-commit hook fails (lint, tests) | Fix the issue; re-run from step 2 | | 3 | `gh pr create` fails (no remote, branch missing on target remote) | Verify the push from step 2; re-run from step 3 | -| 4 | The Advisor session stops, or leaves acceptance criteria incomplete | Leave the PR draft and comment the resume command (resume `/spec execute` for the task folder / attach `agent-spec-`). Do not start a second implementation owner. | +| 4 | Implementation stops, or leaves acceptance criteria incomplete | Leave the PR draft and comment the resume command (`/spec execute ` against the same task folder). Do not start a second implementation owner. | | 5 | `/eval` reports a NEW green→red regression or exits non-zero | Leave the PR draft; fix or document the regression, then re-run `/eval` | | 5 | Wiki impact REQUIRED but entries are missing, stale against the implemented behavior, or the README index probe fails | Leave the PR draft; fix the wiki entries/index, then re-run the wiki gate | | 5 | `/compact` unavailable or errors | Non-blocking; log a warning and continue | @@ -521,7 +524,7 @@ Every step checks for prior state and resumes rather than duplicating: | 1 | The issue named by `prd.json`'s `branchName` exists, or `--pr ` was passed | Reuse ``; never create a duplicate | | 2 | Branch exists on the target remote | Checkout + commit on top | | 3 | Draft PR exists for this branch | Update the body; do not create a duplicate | -| 4 | `prd.json` already has all stories passing; or the `agent-spec-` session is already running | Skip relaunch — attach to or resume the existing Advisor session; worktree present → reuse | +| 4 | `prd.json` already has all stories passing | Skip re-implementation and continue to the gates; worktree present → reuse | | 5 | `.oh/evals/RESULTS.md` already reflects the current probe set and no new regression exists | Continue; otherwise re-run `/eval` | | 5 | Wiki impact NOT-APPLICABLE, or required entries already match the implementation and the index probe passes | Continue | | 6 | `evidence.md` exists and correlates to the CURRENT audit run id | Reuse; a doc citing a stale run id is rewritten, not kept | @@ -550,6 +553,9 @@ auto-merge. ## What this node does NOT do +- **Launch a coding agent.** No tmux session, no Herdr workspace/tab/pane, no background + shell, no runner selection, no fallback runner. The agent already running `/spec execute` + is the implementation owner, and `/spec` never creates the agent that executes it. - **Merge.** The terminal state is a **ready** PR. Merge is the human's gate; reset/clean is the runner's job after merge. - **Select work.** Selection is the human's; `execute` builds the one folder it is @@ -571,10 +577,9 @@ auto-merge. | Primitive | Path | Role | |---|---|---| -| Task prompt template | `.oh/skills/spec/templates/task-prompt.md` | Step 4 — the single Advisor's implementation and gate instructions | +| Task prompt template | `.oh/skills/spec/templates/task-prompt.md` | Step 4 — the owner's implementation and gate instructions | | `/worktrees` skill | `.claude/skills/worktrees/SKILL.md` | Step 4 — isolated `.worktrees/` for the implementation | -| `/goal` (Pi extension) | `.pi/settings.json` (`@narumitw/pi-goal`) | Step 4 — persists the Advisor run to completion | -| `/delegate` skill | `.claude/skills/delegate/SKILL.md` | Step 4 — optional bounded fan-out inside the Advisor's implementation session | +| `/delegate` skill | `.claude/skills/delegate/SKILL.md` | Step 4 — optional bounded fan-out beneath the owner | | `/audit implementation` | `.claude/skills/audit/SKILL.md` | Step 5 — the per-unit verdict gate | | `/eval` skill | `.claude/skills/eval/SKILL.md` | Step 5 — probe regression floor | | Wiki schema | `.oh/skills/wiki/references/schema.md` | Step 5 — source-backed wiki alignment | @@ -582,7 +587,6 @@ auto-merge. | Reviewer evidence doc | `.oh/skills/audit/references/reviewer-evidence-doc.md` | Step 6 — the contract `evidence.md` follows | | `/audit pr` skill | `.claude/skills/audit/SKILL.md` | Step 9 — promotable classification (gates the undraft) | | `/ci-status` skill | `.claude/skills/ci-status/SKILL.md` | CI verification (subsumed by `/audit pr`'s promotable check) | -| sandbox-processes norm | `.oh/skills/t3/references/sandbox-processes.md` | Step 4 — session naming for the Advisor | | Protected-paths list | `.claude/protected-paths.txt` | Load-bearing items a spec must not propose deleting | ## Pipeline position diff --git a/.oh/skills/spec/references/plan.md b/.oh/skills/spec/references/plan.md index edfe588c..55076d7d 100644 --- a/.oh/skills/spec/references/plan.md +++ b/.oh/skills/spec/references/plan.md @@ -43,9 +43,9 @@ folder for `/spec execute` to consume. Run these in order; each is an existing primitive — compose, don't re-derive. 1. **Derive ``** (per the `/prd` skill's rules): lowercase kebab-case, `[a-z0-9-]+`, - ≤5 hyphen-words, not `archive`. The slug is the universal key — task directory, branch - second segment, tmux session name. Choose once; reject and ask for a shorter name if - invalid. `--slug` overrides derivation. + ≤5 hyphen-words, not `archive`. The slug is the universal key — task directory and branch + second segment; it never names a terminal session, tab, or pane. Choose once; reject and + ask for a shorter name if invalid. `--slug` overrides derivation. 2. **`/prd` → `.oh/tasks//prd.md`**. Invoke the `prd` skill with `` (or `--plan` content, with an explicit instruction to skip clarifying questions when a @@ -92,8 +92,9 @@ Run these in order; each is an existing primitive — compose, don't re-derive. `.oh/skills/spec/templates/task-prompt.md`. Render it into `.oh/tasks//prompt.md` by substituting ``, ``, and `` with the task slug, `prd.json`'s `branchName`, and the issue number as bare digits. This - prompt is the single Advisor handoff; no separate implementation process or session - prompt exists. Confirm no angle-bracket placeholder survives the render. Write + prompt states the single-owner contract for whichever agent runs `/spec execute`; it is + not a handoff to a separately launched process or session. Confirm no angle-bracket + placeholder survives the render. Write `.oh/tasks//progress.txt` with the `# progress` header only. Verify the four-file contract before handing off: diff --git a/.oh/skills/spec/references/ship.md b/.oh/skills/spec/references/ship.md index 4f388210..2b252de0 100644 --- a/.oh/skills/spec/references/ship.md +++ b/.oh/skills/spec/references/ship.md @@ -71,7 +71,7 @@ subcommand. 4. **Run `execute`.** Follow `references/execute.md` with ` [--repo ...] [--remote ...] [--base ...]`, which owns the issue, branch, draft - PR, Advisor build, `implementation ⇄ audit` loop, evidence, `/eval` and wiki gates, + PR, implementation, `implementation ⇄ audit` loop, evidence, `/eval` and wiki gates, and the undraft. `ship` neither reimplements nor relaxes any of it: a `DRAFT-BLOCKED ()` from `execute` is `ship`'s outcome verbatim. diff --git a/.oh/skills/spec/templates/task-prompt.md b/.oh/skills/spec/templates/task-prompt.md index 72711fe9..daf4db1a 100644 --- a/.oh/skills/spec/templates/task-prompt.md +++ b/.oh/skills/spec/templates/task-prompt.md @@ -1,6 +1,6 @@ # `/spec execute` task — -You are the single implementation Advisor for the `` task. Read the approved plan in +You are the single implementation owner for the `` task. Read the approved plan in `.oh/tasks//prd.md` and the ordered stories in `.oh/tasks//prd.json`. - Branch: `` — never push to `development` or `main`. @@ -10,7 +10,9 @@ You are the single implementation Advisor for the `` task. Read the approv ## Ownership You own this task from implementation through the final PR gate. Do not hand the task to a -second implementation owner or a second supervisory session. Use `/delegate` only for bounded, +second implementation owner or a second supervisory session, and do not launch another +coding-agent process — through tmux, Herdr, a background shell, or any other runner — to do +this work. Ownership is a role, not a terminal topology. Use `/delegate` only for bounded, disjoint work that can run in parallel. Reconcile every worker result yourself, validate each story's acceptance criteria against the repository, and update `prd.json` and `progress.txt`. @@ -25,7 +27,7 @@ story's acceptance criteria against the repository, and update `prd.json` and `p 5. Continue until every story passes. Do not claim completion when a story is blocked or deferred. Append `STATUS: COMPLETE` to `progress.txt` only when the full graph passes. -After implementation completes, continue in the same Advisor session with the `/spec execute` +After implementation completes, continue in this same session with the `/spec execute` procedure: run the implementation-side audit loop, run `/eval` once, revise required wiki entries, write and commit `evidence.md`, run `/spec retro` and the improve steps, then run a fresh `/audit pr`. Mark the PR ready only when that audit is promotable. Never merge the PR. diff --git a/.oh/skills/ste/references/rules.md b/.oh/skills/ste/references/rules.md index 40358277..c1399554 100644 --- a/.oh/skills/ste/references/rules.md +++ b/.oh/skills/ste/references/rules.md @@ -368,7 +368,7 @@ Write the unit with every number. Write the absolute path or the repo-relative p ```text Wrong: The timeout is large. Edit the config in the scripts folder. -Right: `/spec execute` keeps implementation in one Advisor-owned session. Read `.oh/skills/spec/references/execute.md` for the workflow. +Right: `/spec execute` keeps implementation with one owner. Read `.oh/skills/spec/references/execute.md` for the workflow. ``` ### 39. A heading is not an antecedent diff --git a/.oh/skills/t3/references/sandbox-processes.md b/.oh/skills/t3/references/sandbox-processes.md index d1905f2d..5db07931 100644 --- a/.oh/skills/t3/references/sandbox-processes.md +++ b/.oh/skills/t3/references/sandbox-processes.md @@ -10,12 +10,13 @@ restart, and log capture across all internal apps. `tmux` is preinstalled in the sandbox image; a default `.tmux.conf` is baked in (see commit `b30cef9`). -### Agent task workflows +### Agent task workflows are not a tmux exception -`/spec execute` runs implementation and its validation gates in one Advisor-owned session. -The Advisor may use `/delegate` for bounded, disjoint worker tasks, but it does not create a -second implementation owner or nested supervisory session. Keep the Advisor session named -`agent-spec-` so the operator can attach to the one workflow that owns the task. +This rule covers **headless infrastructure** — cron runtime and detached fires, messaging +gateways, supervisors and watchdogs, tunnels, the T3 Code server. It does not reach +`/spec execute`, which claims no session of its own: the agent the operator already started +owns the task, so there is no `/spec` session to name, log, attach to, or kill. +Do not reintroduce an `agent-spec-*` convention or any other `/spec` agent-handoff session. ## Session Naming diff --git a/.oh/skills/wiki/corpus/plan-vs-built-reconciliation.md b/.oh/skills/wiki/corpus/plan-vs-built-reconciliation.md index 21166393..c71e6712 100644 --- a/.oh/skills/wiki/corpus/plan-vs-built-reconciliation.md +++ b/.oh/skills/wiki/corpus/plan-vs-built-reconciliation.md @@ -19,7 +19,7 @@ confidence: provisional - `.oh/tasks//prd.md` / `progress.txt` / `evidence.md` — the plan, the implementation narrative, and the answer back. ## Summary -The pipeline had ~10 verification nodes and zero comprehension nodes: every gate asked *is this correct?* and none asked *is this still what you agreed to?* An operator's model of the work stops at the plan they approved, while the Advisor carries implementation and verification through one owned workflow. Reconciliation closes that gap with one tracked artifact — `.oh/tasks//evidence.md` — which the merge gate **refuses to undraft without** (`execute.md:393`). +The pipeline had ~10 verification nodes and zero comprehension nodes: every gate asked *is this correct?* and none asked *is this still what you agreed to?* An operator's model of the work stops at the plan they approved, while the single implementation owner — the agent running `/spec execute` — carries implementation and verification through one owned workflow. Reconciliation closes that gap with one tracked artifact — `.oh/tasks//evidence.md` — which the merge gate **refuses to undraft without** (`execute.md:393`). ## Detail **It is a gate condition, not a report.** `/spec execute` checks two things before `gh pr ready`: that the file exists, and that `git ls-files --error-unmatch` finds it. The second half is not redundant — `.oh/tasks/` is gitignored, so a doc written without `git add -f` sits on disk and is **absent from the PR diff**, which from the reviewer's seat is identical to never writing it (`execute.md:432-436`). Failing either half is terminal status `DRAFT-BLOCKED (evidence)`, not a warning. @@ -32,7 +32,7 @@ The pipeline had ~10 verification nodes and zero comprehension nodes: every gate **Do not confuse the two evidence artifacts.** `AUDIT_EVIDENCE_PATH` (`evidence.json`, schema v1, invocation-scoped, never inside `AUDIT_ROOT`) is the machine record that lets the audit boundary log `complete`. `evidence.md` is a separate tracked Markdown artifact for humans, correlated to the same `AUDIT_RUN_ID` (`reviewer-evidence-doc.md:13-16`). A stale run id means the doc is rewritten, not kept (`execute.md:530`). -**The implementation narrative is promoted, not stranded.** `progress.txt` holds the per-story record; the Advisor folds it and `evidence.md` into the PR body so the reviewer meets the work in the PR rather than by opening the task folder (`execute.md:410-418`). +**The implementation narrative is promoted, not stranded.** `progress.txt` holds the per-story record; the owner folds it and `evidence.md` into the PR body so the reviewer meets the work in the PR rather than by opening the task folder (`execute.md:410-418`). **DeepWiki comparison.** Run 2026-08-24 against `https://deepwiki.com/mifunedev/openharness` (Overview page). DeepWiki has **no** entry for this concept: it does not mention `evidence.md` or any plan-versus-built reconciliation gate, and its planning-phase table still lists `/ship-spec` with the gloss *"Convert specs into executable tasks via a critic"* — the critic gate US-001 removed. So this page is **net-new relative to DeepWiki**, not a re-synthesis of it, and the divergence is upstream staleness rather than a contradiction to resolve. **The workflow no longer runs this comparison** (2026-08-24): DeepWiki regenerates on no schedule the gate could depend on, so `/spec plan` and `/spec execute` dropped it — the staleness recorded here is the evidence for that removal, kept as rationale rather than as a standing obligation. @@ -54,7 +54,7 @@ flowchart LR | Artifact | Written by | Read by | Tracked | | --- | --- | --- | --- | | `prd.md` | `/spec plan` | operator (the go/no-go) | yes (`add -f`) | -| `progress.txt` | the Advisor | `/spec execute` implementation stage | yes (`add -f`) | +| `progress.txt` | the implementation owner | `/spec execute` implementation stage | yes (`add -f`) | | `evidence.md` | `/spec execute` step 6 | the PR reviewer | yes (`add -f`) — gated | | `evidence.json` | the audit boundary | the boundary's terminal log | no (invocation-scoped) | diff --git a/.oh/tasks/README.md b/.oh/tasks/README.md index 07d477fe..dbce04f3 100644 --- a/.oh/tasks/README.md +++ b/.oh/tasks/README.md @@ -2,16 +2,18 @@ Spec task workdirs. Each `/` subfolder is one `/spec execute` task's four-file contract, created by `/spec plan` (the `/ralph` skill produces the -`prd.json` inside it) and implemented by the single Advisor session. +`prd.json` inside it) and implemented by that task's single implementation owner — +the agent that runs `/spec execute`. Ownership is a role, not a terminal session: +a task folder's identity and state never depend on a session, tab, or pane. A task directory typically contains: | File | Purpose | | -------------- | -------------------------------------------------------- | -| `prd.json` | Ralph-formatted PRD — the Advisor's authoritative task graph | +| `prd.json` | Ralph-formatted PRD — the owner's authoritative task graph | | `prd.md` | Human-readable PRD that `prd.json` was generated from | -| `prompt.md` | Task-specific instructions for the Advisor's implementation | -| `progress.txt` | Advisor's running log; ends with `STATUS: COMPLETE` on done | +| `prompt.md` | Task-specific instructions for the owner's implementation | +| `progress.txt` | Owner's running log; ends with `STATUS: COMPLETE` on done | | `critique.md` | Optional critic notes from PRD review | ## Conventions @@ -24,7 +26,7 @@ A task directory typically contains: `eval-result.json` — are added explicitly with **`git add -f`**. A bare `git add .oh/tasks//` stages nothing and commits silently without them, which is the same as never having written them from a reviewer's seat. -- **Do not edit `progress.txt` by hand** — the Advisor appends to it. +- **Do not edit `progress.txt` by hand** — the implementation owner appends to it. ## Lifecycle diff --git a/CHANGELOG.md b/CHANGELOG.md index 94197c75..ba4b8c29 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -30,6 +30,14 @@ Update policy and release automation live in [`/git`](.claude/skills/git/SKILL.m ### Changed - `.oh/logs/` carries an `AGENTS.md` with a `CLAUDE.md` symlink instead of a `README.md`, matching the directories whose contents are produced apart from the root context. ([#924](https://github.com/mifunedev/openharness/issues/924)) +- **BREAKING:** `/spec execute` no longer launches a coding agent; the agent that runs it is the single implementation owner through the final PR gates ([#928](https://github.com/mifunedev/openharness/issues/928)). +- Task identity and `RUNNING` state depend on `.oh/tasks//` alone, never on a terminal session, tab, or pane ([#928](https://github.com/mifunedev/openharness/issues/928)). + +### Removed +- **BREAKING:** Retire the automated `/spec` Advisor handoff — detached tmux launch, `agent-spec-*` sessions and their sweep kill, pane logging, runner fallbacks ([#928](https://github.com/mifunedev/openharness/issues/928)). + +### Added +- Add `spec-no-agent-handoff`, `spec-no-advisor-session-coupling`, `cleanup-no-agent-session-coupling`, `headless-tmux-preserved`; rename `advisor-monitored-loop` to `spec-single-owner` ([#928](https://github.com/mifunedev/openharness/issues/928)). ### Added - Add `/escalate`: an unattended session delivers a human-addressed escalation to the operator's Slack channel. An unavailable channel no-ops loudly rather than failing the session. ([#919](https://github.com/mifunedev/openharness/issues/919)) diff --git a/crons/AGENTS.md b/crons/AGENTS.md index 5047075f..254ff100 100644 --- a/crons/AGENTS.md +++ b/crons/AGENTS.md @@ -138,7 +138,7 @@ The devcontainer entrypoint starts `cron-watchdog`, a tmux supervisor that check A job with `tmux: true` in its frontmatter runs each fire in its own detached tmux session instead of an in-process child, so the user can attach to a run, read its scrollback, and reattach later. - **Session name**: `cron---` (e.g. `cron-heartbeat-0610-1805`), derived from the fire time. The runtime logs `SPAWNED ` to `.cron.log`. -- **Agent selection**: by default jobs use `CRON_AGENT_BIN` (compose default: `claude`). A cron can set `agent: ` in frontmatter to override that for one job (for example `agent: pi`, so the run and its kept Advisor session are attachable Pi sessions without forcing heartbeat/cleanup/eval onto Pi). Agent binaries must be non-empty executable tokens or paths made from `A-Z`, `a-z`, `0-9`, `_`, `.`, `/`, and `-`; values with whitespace, shell metacharacters, `..`, or leading `-` are rejected with `AGENT_INVALID` before shell wrapper generation. +- **Agent selection**: by default jobs use `CRON_AGENT_BIN` (compose default: `claude`). A cron can set `agent: ` in frontmatter to override that for one job (for example `agent: pi`, so the run and its kept session are attachable Pi sessions without forcing heartbeat/cleanup/eval onto Pi). Agent binaries must be non-empty executable tokens or paths made from `A-Z`, `a-z`, `0-9`, `_`, `.`, `/`, and `-`; values with whitespace, shell metacharacters, `..`, or leading `-` are rejected with `AGENT_INVALID` before shell wrapper generation. - **Agent attribution**: the shell wrapper logs `AGENT_START agent=` and `AGENT_DONE agent= exit=` from inside the run. If default Claude falls back to Codex, `.cron.log` shows `AGENT_START agent=claude`, `AGENT_FALLBACK from=claude to=codex`, `AGENT_START agent=codex`, then `AGENT_DONE agent=codex exit=`. - **Env exported into the agent**: `CRON_TMUX_SESSION=`, `CRON_KEEP_MARKER=/tmp/.keep`, and `CRON_OVERLAP_PIDFILE=/tmp/cron-.pid`. - **Keep-marker contract**: if the agent `touch`es `$CRON_KEEP_MARKER` before exiting, the session persists by resuming the run's own conversation as a live, attachable agent (`claude --continue` for a Claude run, `pi --continue` for an `agent: pi` run, or `codex` after a Claude→Codex fallback), falling back to a shell if that exits; otherwise it auto-closes when the agent finishes. Claude/Codex tmux runs are headless (`claude -p`, or `codex exec --sandbox danger-full-access` after a Claude usage/session-limit fallback). Pi tmux runs intentionally use the positional TUI shape (`pi "$(cat prompt)"`), matching `tmux new -s pi ""`, so attaching mid-run shows the live Pi pane instead of a blank piped/headless screen. By convention a job keeps its session only when the run produced something worth revisiting (e.g. a PR was opened). diff --git a/crons/cleanup-tasks.md b/crons/cleanup-tasks.md index f1e06c1b..788349eb 100644 --- a/crons/cleanup-tasks.md +++ b/crons/cleanup-tasks.md @@ -12,6 +12,8 @@ description: Weekly `/spec execute` task sweep — archive completed tasks # Weekly Task Cleanup Sweep `.oh/tasks/` once per week and archive anything that has finished. +Completion is read from `progress.txt` alone; a task's identity and state are never +tied to a terminal session, so this sweep detects and kills no agent session. Per SPEC v0.7 §"Weekly cleanup cron": completed tasks move into the dated archive under `.oh/tasks/`; incomplete tasks are left alone with a note. The same weekly pass also grooms stale `.worktrees/` branch @@ -75,8 +77,6 @@ only — never a `.oh/tasks/` subfolder. `.oh/tasks/archive/`): - If `.oh/tasks//progress.txt` ends with a line matching exactly `STATUS: COMPLETE`: - - Kill the matching tmux session if one exists: - `tmux kill-session -t agent-spec- 2>/dev/null || true`. - Move the folder inside the worktree: `git -C .worktrees/archive/$TODAY mv .oh/tasks/ .oh/tasks/archive/$TODAY/` (falls back to `mv` + `git -C .worktrees/archive/$TODAY add` if diff --git a/crons/prompt-miner.md b/crons/prompt-miner.md index 963698c1..a917412d 100644 --- a/crons/prompt-miner.md +++ b/crons/prompt-miner.md @@ -90,8 +90,9 @@ Read the mined markers (stratified by session type; see `references/markers.md`) ### 3. Ship the candidate to origin via `/spec` -Hand the issue to `/spec`, which owns plan and build end-to-end (worktree Advisor, -`/delegate` + the Advisor-owned implementation workflow, the `/eval` gate, `/audit pr` undraft) and targets the fork: +Hand the issue to `/spec`, which owns plan and build end-to-end (isolated worktree, +single-owner implementation with bounded `/delegate` fan-out, the `/eval` gate, `/audit pr` +undraft) and targets the fork: ```bash /spec plan --issue --repo mifunedev/openharness --base development diff --git a/docs/glossary.md b/docs/glossary.md index 8eaf2dd0..3d0a52a5 100644 --- a/docs/glossary.md +++ b/docs/glossary.md @@ -93,9 +93,12 @@ These names describe separate layers, not interchangeable jobs: `.devcontainer/`, so the agent works against your code without touching the host machine. Source: [`.devcontainer/`](../.devcontainer/). -- **session** — A single named run of an agent, typically a tmux session in the - sandbox. `/spec execute` keeps implementation, validation, evidence, and PR - finalization in one Advisor-owned session. +- **session** — A terminal-backend run of an agent: a tmux session, a Herdr pane, or a + plain shell. It is a *backend*, not an identity. Distinguish it from the + **implementation owner** — the logical role that owns one `/spec execute` task from the + isolated worktree through the final PR gates. `/spec execute` keeps implementation, + validation, evidence, and PR finalization with the single agent that invoked it, whatever + backend that agent happens to be running in, and launches no session of its own. Source: [`.oh/skills/spec/references/execute.md`](../.oh/skills/spec/references/execute.md) and [`sandbox-processes.md`](../.oh/skills/t3/references/sandbox-processes.md). From cdc34efb1da96cd75bca3bd7249fdaa7317ff258 Mon Sep 17 00:00:00 2001 From: Ryan Eggleston Date: Mon, 31 Aug 2026 20:23:19 -0600 Subject: [PATCH 19/23] feat: make skills the role primitive and add /architect (#931) Adopt ADR #929: the active coding agent is the runtime, skills encode durable roles and procedures, and subagents stay a bounded execution primitive owned by /delegate. Adds /architect as an inline architecture-decision skill and retires the project-authored agent catalog, the /builder agent artifact type, and every stale project-agent reference across skills, docs, provider wiring, and the oh payload manifest. Submitted-by: Claude Code (claude-opus-5) --- .claude/agents | 1 - .claude/protected-paths.txt | 1 + .codex/agents | 1 - .oh/README.md | 8 +- .oh/agents/README.md | 8 - .oh/cli/src/__tests__/init.test.ts | 4 +- .oh/cli/src/commands/init.ts | 2 - .oh/evals/RESULTS.md | 247 +++++++++--------- .oh/evals/probes/architect-skill-contract.sh | 73 ++++++ .oh/evals/probes/architecture-record-reuse.sh | 33 +++ .oh/evals/probes/builder-no-agent-artifact.sh | 37 +++ .../probes/builder-skill-consolidation.sh | 20 +- .oh/evals/probes/delegate-worker-boundary.sh | 52 ++++ .oh/evals/probes/no-project-agent-catalog.sh | 44 ++++ .../probes/rl-delegation-write-worker.sh | 38 --- .oh/evals/probes/roles-are-skills.sh | 51 ++++ .oh/evals/probes/skills-vendored.sh | 8 +- .oh/manifest.json | 1 - .oh/scripts/link-providers.sh | 4 +- .oh/skills/architect/SKILL.md | 183 +++++++++++++ .oh/skills/audit/references/context.md | 3 +- .oh/skills/audit/references/harness.md | 3 +- .oh/skills/blog/SKILL.md | 4 +- .oh/skills/blog/references/loom-to-blog.md | 2 +- .oh/skills/builder/SKILL.md | 25 +- .oh/skills/builder/references/agent.md | 156 ----------- .oh/skills/builder/references/command.md | 6 +- .oh/skills/builder/references/skill.md | 9 +- .oh/skills/delegate/SKILL.md | 45 +++- .oh/skills/spec/references/plan.md | 20 ++ .oh/skills/strategic-proposal/SKILL.md | 44 ++-- .oh/skills/worktrees/SKILL.md | 2 +- AGENTS.md | 4 +- CHANGELOG.md | 10 + README.md | 4 +- docs/README.md | 2 +- docs/glossary.md | 49 +++- docs/harness-manifest.md | 9 +- docs/oh-directory-layout.md | 1 - docs/rfcs/README.md | 1 + 40 files changed, 779 insertions(+), 436 deletions(-) delete mode 120000 .claude/agents delete mode 120000 .codex/agents delete mode 100644 .oh/agents/README.md create mode 100755 .oh/evals/probes/architect-skill-contract.sh create mode 100755 .oh/evals/probes/architecture-record-reuse.sh create mode 100755 .oh/evals/probes/builder-no-agent-artifact.sh create mode 100755 .oh/evals/probes/delegate-worker-boundary.sh create mode 100755 .oh/evals/probes/no-project-agent-catalog.sh delete mode 100755 .oh/evals/probes/rl-delegation-write-worker.sh create mode 100755 .oh/evals/probes/roles-are-skills.sh create mode 100644 .oh/skills/architect/SKILL.md delete mode 100644 .oh/skills/builder/references/agent.md diff --git a/.claude/agents b/.claude/agents deleted file mode 120000 index 5bc15b82..00000000 --- a/.claude/agents +++ /dev/null @@ -1 +0,0 @@ -../.oh/agents \ No newline at end of file diff --git a/.claude/protected-paths.txt b/.claude/protected-paths.txt index ba458893..575fed3d 100644 --- a/.claude/protected-paths.txt +++ b/.claude/protected-paths.txt @@ -23,6 +23,7 @@ cloudflared agent-browser prd ralph +architect audit delegate strategic-proposal diff --git a/.codex/agents b/.codex/agents deleted file mode 120000 index 0efb85ec..00000000 --- a/.codex/agents +++ /dev/null @@ -1 +0,0 @@ -../.claude/agents \ No newline at end of file diff --git a/.oh/README.md b/.oh/README.md index 02121556..df5ba288 100644 --- a/.oh/README.md +++ b/.oh/README.md @@ -19,7 +19,7 @@ This **supersedes** the earlier "earned by EXPORT only" rule *and* the later is obsolete): - **`.oh/`** — *OpenHarness's own machinery* as one unit, including the - provider-portable *primitives* — `skills/`, `agents/`, `hooks/` (+ `skills.lock`) + provider-portable *primitives* — `skills/`, `hooks/` (+ `skills.lock`) — exported to the four agent providers via symlinks (`.claude/`, `.codex/`, `.pi/`, `.hermes/`): the `oh` CLI (`cli/`), installer + lifecycle scripts (`scripts/`), container-install inputs (`install/`), the @@ -104,7 +104,7 @@ root `docs/` (Markdown only — no build machinery; guarded by ## How the skill pack is wired -The shared skills, agents, and hooks are vendored directly under `.oh/` (`.oh/skills`, `.oh/agents`, `.oh/hooks`) and tracked in this repo — there is no submodule and no network fetch. `oh init`/`oh update` lay the pack down with the rest of `.oh/`; `.oh/scripts/link-providers.sh --init` (re)creates the provider symlinks into it, and `--check` verifies the vendored pack is present, the required executables, the protected paths, the provider symlinks, and the Hermes link when enabled. +The shared skills and hooks are vendored directly under `.oh/` (`.oh/skills`, `.oh/hooks`) and tracked in this repo — there is no submodule and no network fetch. `oh init`/`oh update` lay the pack down with the rest of `.oh/`; `.oh/scripts/link-providers.sh --init` (re)creates the provider symlinks into it, and `--check` verifies the vendored pack is present, the required executables, the protected paths, the provider symlinks, and the Hermes link when enabled. `.pi/` remains the Pi provider surface in v1; its `.pi/skills` is one of the symlinks into `.oh/skills`. @@ -276,7 +276,7 @@ the top-level directories its own patterns name — never the whole repo root. patterns are relative to `.oh/`, and the existing path-escape guard (writes land only under `/.oh/`) is **unchanged** — the manifest *narrows* the payload, it never widens the write surface. The vendored skill pack -(`skills/**`, `agents/**`, `hooks/**`, `skills.lock`) ships through this same +(`skills/**`, `hooks/**`, `skills.lock`) ships through this same manifest, so `oh init`/`oh update` carry it into a target with the rest of `.oh/`. > **`oh init` seam:** both `oh init` and `oh update` honor this manifest — they @@ -287,4 +287,4 @@ manifest, so `oh init`/`oh update` carry it into a target with the rest of `.oh/ ## Pointers - `.oh/skills/harness-context/references/directory-readme.md` — the README-as-directory-anchor convention this file follows. -- `.oh/skills/` — the vendored provider-portable primitive pack (skills/agents/hooks), absorbed from the former `.mifune` submodule. +- `.oh/skills/` — the vendored provider-portable primitive pack (skills/hooks), absorbed from the former `.mifune` submodule. diff --git a/.oh/agents/README.md b/.oh/agents/README.md deleted file mode 100644 index 74d95072..00000000 --- a/.oh/agents/README.md +++ /dev/null @@ -1,8 +0,0 @@ -# Agents - -Provider-portable agent definitions live here as `.md` files. This is the -canonical source; `.claude/agents` and `.codex/agents` are compatibility -symlinks that resolve into this directory. - -No agent is defined at present. Add one by writing `.md` here — never in a -provider mirror, which `.oh/scripts/link-providers.sh` can rewrite. diff --git a/.oh/cli/src/__tests__/init.test.ts b/.oh/cli/src/__tests__/init.test.ts index 05cea50b..4d950790 100644 --- a/.oh/cli/src/__tests__/init.test.ts +++ b/.oh/cli/src/__tests__/init.test.ts @@ -632,11 +632,11 @@ describe("runInit", () => { expect.arrayContaining([ "evals/**", "skills/**", - "agents/**", "hooks/**", ]), ); expect(m.include).not.toContain("context/**"); + expect(m.include).not.toContain("agents/**"); }); @@ -706,10 +706,8 @@ describe("runInit", () => { expect(await runInit(opts(t, { yes: true }), makeIO().io)).toBe(0); const links: [string, string][] = [ [".claude/skills", "../.oh/skills"], - [".claude/agents", "../.oh/agents"], [".claude/hooks", "../.oh/hooks"], [".codex/skills", "../.oh/skills"], - [".codex/agents", "../.claude/agents"], [".pi/skills", "../.oh/skills"], ]; for (const [rel, target] of links) { diff --git a/.oh/cli/src/commands/init.ts b/.oh/cli/src/commands/init.ts index 2700000d..d902c191 100644 --- a/.oh/cli/src/commands/init.ts +++ b/.oh/cli/src/commands/init.ts @@ -492,9 +492,7 @@ const PROVIDER_LINKS: [string, string][] = [ [".pi/skills", "../.oh/skills"], [".claude/skills", "../.oh/skills"], [".codex/skills", "../.oh/skills"], - [".claude/agents", "../.oh/agents"], [".claude/hooks", "../.oh/hooks"], - [".codex/agents", "../.claude/agents"], [".codex/specs", "../.claude/specs"], ]; diff --git a/.oh/evals/RESULTS.md b/.oh/evals/RESULTS.md index 9385084b..96eaa4a1 100644 --- a/.oh/evals/RESULTS.md +++ b/.oh/evals/RESULTS.md @@ -6,126 +6,131 @@ probe id; git history is the time series.** Schema and exit-code semantics are i | probe | tier | last-run (UTC) | status | source | |-------|------|----------------|--------|--------| -| agent-browser-cli | A | 2026-09-01 01:21 | PASS | retro lesson 2026-06-07 (agent-browser 0.8.5 CLI) | -| agents-identity-contract | A | 2026-09-01 01:21 | PASS | issue #854 — T3-style root identity, glossary, and skill-owned procedures | -| artifact-contract-audit | A | 2026-09-01 01:21 | PASS | issue #583/#645 — production /audit implementation Gate 1 behavior | -| audit-dispatcher-contract | A | 2026-09-01 01:21 | PASS | issue #645 — audit consolidation public taxonomy | -| audit-implementation-behavior | A | 2026-09-01 01:21 | PASS | issue #645 — implementation root/repo/browser behavior | -| audit-pr-acquire | A | 2026-09-01 01:21 | PASS | issue #645 — production PR acquisition behavior | -| audit-pr-classifier | A | 2026-09-01 01:21 | PASS | issue #645 — deterministic focused and queue PR classifier | -| audit-run-root-contract | A | 2026-09-01 01:21 | PASS | issue #645 — executable immutable audit root/run correlation | -| audit-shellcheck-coverage | A | 2026-09-01 01:21 | PASS | issue #645 — private audit scripts require release and CI lint coverage | -| audit-slop-gate | A | 2026-09-01 01:21 | PASS | .claude/specs/images/reduce-slop.png — the four correctness gates all pass a change | -| audit-stale-references | A | 2026-09-01 01:21 | PASS | issue #645 — clean-breaking audit migration | -| boot-lint-glob | A | 2026-09-01 01:21 | PASS | issue #90, issue #120 | -| builder-skill-consolidation | A | 2026-09-01 01:21 | PASS | issue #643 — consolidate artifact builders behind one /builder dispatcher | -| builder-wiki-proposer | A | 2026-09-01 01:21 | PASS | wikiskill arXiv:2608.27454 — the skill proposer reads accumulated knowledge first | -| capability-benchmark-schema | A | 2026-09-01 01:21 | PASS | issue #167 — capability benchmark instrument | -| cc-safety-net-wiring | A | 2026-09-01 01:21 | SKIPPED | .oh/tasks/cc-safety-net/prd.json US-007 2026-07-19 | -| changelog-entry-length | A | 2026-09-01 01:21 | PASS | conversation 2026-08-24 — CHANGELOG.md grew to 259KB of bullet prose because "one line" was unquantified | -| cleanup-no-agent-session-coupling | A | 2026-09-01 01:21 | PASS | issue #928 — retire automated /spec agent handoff | -| cleanup-tasks-scoped-guard | A | 2026-09-01 01:21 | PASS | issue #85 | -| cleanup-tasks-worktree-grooming | A | 2026-09-01 01:21 | PASS | issue #168; issue #327 | -| cli-publish-typecheck-scope | A | 2026-09-01 01:21 | PASS | release run 33271077312 — v0.5.0 pushed its GHCR image, then failed to publish | -| close-issues-on-development | A | 2026-09-01 01:21 | PASS | issue #841 (closing keywords never fire because the default branch is main) 2026-08-26 | -| codex-stale-response-retry | A | 2026-09-01 01:21 | PASS | issue #506 — Codex previous_response_not_found RCA | -| compose-config-path-parity | A | 2026-09-01 01:21 | PASS | PR #833 (remove harness.yaml — the wrapper and VS Code "Reopen in Container" paths must resolve the same service) 2026-08-26 | -| compose-env-boundary | A | 2026-09-01 01:21 | PASS | #920 — the epic #903→#911 made the CLI provision harnesses and tools from | -| config-schema-parity | A | 2026-09-01 01:21 | PASS | PR #833 (one schema file — DOCKER_SOCKET, SANDBOX_SSH, OH_SANDBOX_IMAGE, OH_PULL_POLICY, SKIP_PNPM_INSTALL were consumed but undocumented); rewritten for the oh.json/secrets split by PR #887 | -| context-tier-size-budget | A | 2026-09-01 01:21 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-007) — the always-on tier was 85,256 B | -| continual-learning-20260831 | A | 2026-09-01 01:21 | PASS | retro lesson 2026-08-31 (unexercised oracle) — a probe green in a 112-probe run carried three parser defects | -| cron-claude-codex-fallback | A | 2026-09-01 01:21 | PASS | conversation 2026-06-12 (default Codex fallback for crons) | -| cron-watchdog | A | 2026-09-01 01:21 | PASS | issues #130/#453 (cron runtime watchdog + legacy system-cron reaping) 2026-06-19 | -| crons-directory-guide | A | 2026-09-01 01:21 | PASS | issue #874 | -| curl-bash-safe-alternatives | A | 2026-09-01 01:21 | PASS | vet-run/vet integration — public curl|bash examples need review-first alternatives | -| datasets-schema | A | 2026-09-01 01:21 | PASS | issue #196 — .oh/evals/datasets verifiable trajectory corpus (Repo2RLEnv-inspired) | -| debugmcp-availability | A | 2026-09-01 01:21 | SKIPPED | issue #297 — DebugMCP MCP debug-server availability | -| default-provisioning | A | 2026-09-01 01:21 | PASS | #902 — `oh harness install` must work from inside the sandbox, where | -| delegate-model-effort-policy | A | 2026-09-01 01:21 | PASS | conversation 2026-07-11 (delegate model inheritance and thinking policy) | -| devtcp-hook | A | 2026-09-01 01:21 | PASS | retro lesson 2026-06-10 (zsh /dev/tcp) | -| docker-inspect-env-guard | A | 2026-09-01 01:21 | PASS | operator directive 2026-08-08 (agents keep the docker socket, but must | -| docs-build-fast-path | A | 2026-09-01 01:21 | PASS | #455 — docs builds must stay out of fast harness/eval/release gates; #536 — docs site externalized to openharness-web; docs markdown relocated to docs/ | -| drift-check-cron-staleness-glob | A | 2026-09-01 01:21 | PASS | issue #98; issue #225 (restart-required cron frontmatter/config drift) | -| entrypoint-pnpm-manifest-fingerprint | A | 2026-09-01 01:21 | PASS | issue #521 (manifest-aware sandbox installs) 2026-07-01 | -| escalate-contract | A | 2026-09-01 01:21 | PASS | issue #799 — seven comments on a GitHub thread produced zero notifications and nobody | -| eval-ci-gate | A | 2026-09-01 01:21 | PASS | #103 — eval probe suite gated in CI | -| eval-contract-text-20260831 | A | 2026-09-01 01:21 | PASS | retro lesson 2026-08-31 (prose-literal pinning) — two assertions failed on first run purely from source line wrapping | -| eval-gate | A | 2026-09-01 01:21 | PASS | retro lesson 2026-06-11 (eval-gate) | -| eval-results-atomic | A | 2026-09-01 01:21 | PASS | issue #83 (eval-results-atomic-write) | -| eval-runner-exit | A | 2026-09-01 01:21 | PASS | retro lesson 2026-06-11 (eval-runner-exit) #29 | -| eval-runs-once-per-cycle | A | 2026-09-01 01:21 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-006) — /eval ran 3x per cycle on the | -| execution-target-contract | A | 2026-09-01 01:21 | PASS | issue #733 (ExecutionTarget contract + Docker Compose adapter) 2026-08-10 | -| get-oh-bootstrap | A | 2026-09-01 01:21 | PASS | get-oh.sh bootstrap — the Node-bootstrapping host-side path to the standalone `oh` CLI (also on npm as @mifune/openharness; see oh-npm-package.sh) | -| git-skill | A | 2026-09-01 01:21 | PASS | conversation 2026-06-15 — rules are not always supported; git workflow must be the /git skill | -| harness-audit-empty-output-gate | A | 2026-09-01 01:21 | PASS | issue #246 — /audit harness must fail closed on empty auditor outputs | -| harness-ci-core-paths | A | 2026-09-01 01:21 | PASS | #165 — core sandbox config files must trigger harness CI | -| harness-ci-hooks-paths | A | 2026-09-01 01:21 | PASS | issue #202 — credential/security hook changes must trigger harness CI | -| harness-yaml-migration | A | 2026-09-01 01:21 | PASS | PR #833 (migrate-harness-yaml.sh — append / uncomment-in-place / preserve / overwrite, plus a silent no-op second run) 2026-08-26 | -| headless-tmux-preserved | A | 2026-09-01 01:21 | PASS | issue #928 — retire automated /spec agent handoff | -| health-check-docker-stats | A | 2026-09-01 01:21 | PASS | retro lesson 2026-06-10 (docker stats vs ps Size) | -| health-check-socket-degrade | A | 2026-09-01 01:21 | PASS | issue #762 (refs #756) — /health-check degrades to one statement, not nine failures | -| heartbeat-logging-contract | A | 2026-09-01 01:21 | PASS | issue #447 (heartbeat log append hardening) 2026-06-18 | -| image-seed-hygiene | A | 2026-09-01 01:21 | PASS | issue #900 (slim the sandbox image) 2026-08-30 | -| markitdown-wiki-ingest | A | 2026-09-01 01:21 | PASS | issue #649 — pinned local-document normalization contract for /wiki ingest | -| next-dev-prod | A | 2026-09-01 01:21 | SKIPPED | retro lesson 2026-06-04 | -| oh-compose-env-wiring | A | 2026-09-01 01:21 | SKIPPED | issue #880 (oh as the only front door — oh.json is the non-secret config surface) | -| oh-config-surfaces | A | 2026-09-01 01:21 | PASS | PR #887 (config split across two authored surfaces — a tracked oh.json and a secrets-only root dotenv — with nothing left under $HOME) | -| oh-destroy-guard | A | 2026-09-01 01:21 | SKIPPED | issue #879 — `oh` becomes the only front door, so `make destroy` must | -| oh-devcontainer-restructure | A | 2026-09-01 01:21 | PASS | consolidate devcontainer — .oh/devcontainer/ folded back into .devcontainer/ | -| oh-home-mount | A | 2026-09-01 01:21 | PASS | issue #898 (single $HOME mount) 2026-08-30 | -| oh-image-only-deploy | A | 2026-09-01 01:21 | PASS | .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only | -| oh-init-headless-config | A | 2026-09-01 01:21 | SKIPPED | PR #827 (installer answers landed in the losing config file); retargeted to the .example.env template by PR #833, then to oh.json by PR #887 | -| oh-init-scaffold | A | 2026-09-01 01:21 | PASS | issue #531 Phase 2 | -| oh-lifecycle-surface | A | 2026-09-01 01:21 | PASS | issue #881 — the Makefile is retired and `oh` is the only front door | -| oh-npm-package | A | 2026-09-01 01:21 | PASS | npm publish path for the standalone `oh` CLI (@mifune/openharness) — alternative to get-oh.sh | -| oh-payload-manifest | A | 2026-09-01 01:21 | PASS | issue #531 follow-on (.oh payload manifest — oh update ships a declared allowlist) | -| oh-sandbox-image-mode | A | 2026-09-01 01:21 | PASS | conversation 2026-07-05 (basic Docker deployment — prebuilt-image mode) | -| oh-shipped-repo-overridable | A | 2026-09-01 01:21 | PASS | issue #531 follow-on (de-hardcode residual — shipped .oh shell scripts keep the upstream repo overridable) | -| oh-standalone-lifecycle | A | 2026-09-01 01:21 | PASS | issue #564 | -| oh-update | A | 2026-09-01 01:21 | PASS | issue #531 Phase 3 (oh update — upgrade only the .oh control plane) | -| operator-config-guard | A | 2026-09-01 01:21 | PASS | operator directives 2026-08-06 (.config/ and settings.local.json are operator-only) | -| pnpm-audit-ci-gate | A | 2026-09-01 01:21 | PASS | issue #171 — pnpm security audits must run in CI | -| post-bridge-publish-confirmation | A | 2026-09-01 01:21 | PASS | #523 — post-bridge live publishing requires an explicit final confirmation gate | -| prd-output-path-contract | A | 2026-09-01 01:21 | PASS | retro lesson 2026-06-19 | -| prompt-miner-schema-compat | A | 2026-09-01 01:21 | PASS | issue #253 — prompt-miner JSONL schema-drift guard | -| prompt-miner-symlink-entrypoint | A | 2026-09-01 01:21 | PASS | issue #663 — prompt-miner engine no-ops via the documented .claude/skills symlink | -| prompt-miner-weakness-record | A | 2026-09-01 01:21 | PASS | issue #580 — prompt-miner weakness-record (WH-xxx) cluster output | -| protected-path-deletion | A | 2026-09-01 01:21 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-001) — the critique gate was deleted, | -| protected-paths-resolve | A | 2026-09-01 01:21 | PASS | issue #753 — .claude/protected-paths.txt named 7 paths that did not exist. | -| registry-portability-gate | A | 2026-09-01 01:21 | PASS | issue #758 | -| registry-portability | A | 2026-09-01 01:21 | SKIPPED | issue #758 | -| retro-deterministic-contract | A | 2026-09-01 01:21 | PASS | issue #443 — /retro deterministic output and self-contained helper contract | -| rl-delegation-write-worker | A | 2026-09-01 01:21 | PASS | retro lesson 2026-06-10 (rl-delegation) #57 | -| rlm-context-budget | A | 2026-09-01 01:21 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-006 | -| runtime-preflight-gate | A | 2026-09-01 01:21 | PASS | issue #806 § B1 (open sandbox.substrate vs sandbox.runtime selector); | -| sandbox-boot-guard-ci | A | 2026-09-01 01:21 | PASS | issue #449 (sandbox image build CI guard) 2026-06-19; | -| sandbox-node-base | A | 2026-09-01 01:21 | PASS | openharness#878 — oh as the only front door, T0 sandbox base image | -| skill-paths | A | 2026-09-01 01:21 | PASS | issue #43 — stale path references; extended by issue #69 — apps/->packages/ rename guard; extended by issue #870 — deleted .oh/agents/advisor.md | -| skills-dir-clean | A | 2026-09-01 01:21 | PASS | conversation 2026-06-29 — Pi parses every top-level `.md` in the skills | -| skills-task-tool-coupling | A | 2026-09-01 01:21 | PASS | council review 2026-08-29 (issue #886) — /delegate instructed Claude-Code-only | -| skills-vendored | A | 2026-09-01 01:21 | PASS | absorb .mifune submodule into .oh — the skills/agents/hooks pack is vendored | -| slack-admin-command-surface | A | 2026-09-01 01:21 | PASS | issue #354 — Slack bridge docs must distinguish Pi /msg-bridge commands from Slack DM admin text handlers | -| spec-family-contract | A | 2026-09-01 01:21 | PASS | issue #265; spec-simplification issue #816; workflow authority issue #854; | -| spec-no-advisor-session-coupling | A | 2026-09-01 01:21 | PASS | issue #928 — retire automated /spec agent handoff | -| spec-no-agent-handoff | A | 2026-09-01 01:21 | PASS | issue #928 — retire automated /spec agent handoff | -| spec-ready-finalization | A | 2026-09-01 01:21 | PASS | issue #134; spec-simplification issue #816; workflow authority issue #854 | -| spec-single-owner | A | 2026-09-01 01:21 | PASS | conversation 2026-06-19 (single-owner implementation workflow, issue #257); | -| ste-checker-contract | A | 2026-09-01 01:21 | PASS | issue #750 PR audit — the /ste checker had four fail-open paths (unclosed | -| submitted-by-trailers | A | 2026-09-01 01:21 | PASS | conversation 2026-06-12 (commit attribution trailers); the single-owner | -| sync-skill-contract | A | 2026-09-01 01:21 | PASS | issue #331 — /sync dispatcher skill (bidirectional origin↔upstream sync) | -| t3-headless-launch | A | 2026-09-01 01:21 | PASS | issue #858 — /t3 launched a bare `npx --yes t3`, which is the local GUI and | -| tailscale-tool-boundary | A | 2026-09-01 01:21 | PASS | issue #858 — Tailscale mobile access for T3 Code. There is no tailnet, no | -| tool-catalog-boundary | A | 2026-09-01 01:21 | PASS | agent-browser's exclusion from the harness catalog (#821), the three-catalog | -| version-parity | A | 2026-09-01 01:21 | PASS | conversation 2026-08-29 — the oh CLI became the only lifecycle door, so its | -| weigh-scorer-contract | A | 2026-09-01 01:21 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-003 (2026-06-27) | -| wiki-compile-contract | A | 2026-09-01 01:21 | PASS | wikiskill arXiv:2608.27454 — Wiki Maintainer role added as /wiki compile | -| wiki-kind-schema-contract | A | 2026-09-01 01:21 | PASS | wikiskill arXiv:2608.27454 — pattern layer added to the wiki corpus | -| wiki-pattern-persistence | A | 2026-09-01 01:21 | PASS | wikiskill arXiv:2608.27454 — pattern pages are never rolled back | -| wiki-query-pattern-isolation | A | 2026-09-01 01:21 | PASS | wikiskill arXiv:2608.27454 — proposer-only pattern access | -| wiki-readme-index | A | 2026-09-01 01:21 | PASS | issue #132 — wiki README index drift guard | -| wiki-related-slugs | A | 2026-09-01 01:21 | PASS | wikiskill arXiv:2608.27454 — wiki lint related-slug check | -| wiki-skill-impact-append-only | A | 2026-09-01 01:21 | PASS | wikiskill arXiv:2608.27454 — skill-change ledger, never rolled back | -| workflow-boundaries | A | 2026-09-01 01:21 | PASS | conversation 2026-06-19 (workflow consolidation, issue #259); authority moved to /spec in issue #854 | -| worktrees-layout | A | 2026-09-01 01:21 | PASS | issue #872 | +| agent-browser-cli | A | 2026-09-01 02:15 | PASS | retro lesson 2026-06-07 (agent-browser 0.8.5 CLI) | +| agents-identity-contract | A | 2026-09-01 02:15 | PASS | issue #854 — T3-style root identity, glossary, and skill-owned procedures | +| architect-skill-contract | A | 2026-09-01 02:15 | PASS | ADR #929 — skills are the role primitive; /architect is a skill, not an agent | +| architecture-record-reuse | A | 2026-09-01 02:15 | PASS | ADR #929 — durable decisions reuse the existing RFC/ADR issue convention | +| artifact-contract-audit | A | 2026-09-01 02:15 | PASS | issue #583/#645 — production /audit implementation Gate 1 behavior | +| audit-dispatcher-contract | A | 2026-09-01 02:15 | PASS | issue #645 — audit consolidation public taxonomy | +| audit-implementation-behavior | A | 2026-09-01 02:15 | PASS | issue #645 — implementation root/repo/browser behavior | +| audit-pr-acquire | A | 2026-09-01 02:15 | PASS | issue #645 — production PR acquisition behavior | +| audit-pr-classifier | A | 2026-09-01 02:15 | PASS | issue #645 — deterministic focused and queue PR classifier | +| audit-run-root-contract | A | 2026-09-01 02:15 | PASS | issue #645 — executable immutable audit root/run correlation | +| audit-shellcheck-coverage | A | 2026-09-01 02:15 | PASS | issue #645 — private audit scripts require release and CI lint coverage | +| audit-slop-gate | A | 2026-09-01 02:15 | PASS | .claude/specs/images/reduce-slop.png — the four correctness gates all pass a change | +| audit-stale-references | A | 2026-09-01 02:15 | PASS | issue #645 — clean-breaking audit migration | +| boot-lint-glob | A | 2026-09-01 02:15 | PASS | issue #90, issue #120 | +| builder-no-agent-artifact | A | 2026-09-01 02:15 | PASS | ADR #929 — /builder agent is retired; a reusable role is authored as a skill | +| builder-skill-consolidation | A | 2026-09-01 02:15 | PASS | issue #643 — consolidate artifact builders behind one /builder dispatcher | +| builder-wiki-proposer | A | 2026-09-01 02:15 | PASS | wikiskill arXiv:2608.27454 — the skill proposer reads accumulated knowledge first | +| capability-benchmark-schema | A | 2026-09-01 02:15 | PASS | issue #167 — capability benchmark instrument | +| cc-safety-net-wiring | A | 2026-09-01 02:15 | SKIPPED | .oh/tasks/cc-safety-net/prd.json US-007 2026-07-19 | +| changelog-entry-length | A | 2026-09-01 02:15 | PASS | conversation 2026-08-24 — CHANGELOG.md grew to 259KB of bullet prose because "one line" was unquantified | +| cleanup-no-agent-session-coupling | A | 2026-09-01 02:15 | PASS | issue #928 — retire automated /spec agent handoff | +| cleanup-tasks-scoped-guard | A | 2026-09-01 02:15 | PASS | issue #85 | +| cleanup-tasks-worktree-grooming | A | 2026-09-01 02:15 | PASS | issue #168; issue #327 | +| cli-publish-typecheck-scope | A | 2026-09-01 02:15 | PASS | release run 33271077312 — v0.5.0 pushed its GHCR image, then failed to publish | +| close-issues-on-development | A | 2026-09-01 02:15 | PASS | issue #841 (closing keywords never fire because the default branch is main) 2026-08-26 | +| codex-stale-response-retry | A | 2026-09-01 02:15 | PASS | issue #506 — Codex previous_response_not_found RCA | +| compose-config-path-parity | A | 2026-09-01 02:15 | PASS | PR #833 (remove harness.yaml — the wrapper and VS Code "Reopen in Container" paths must resolve the same service) 2026-08-26 | +| compose-env-boundary | A | 2026-09-01 02:15 | PASS | #920 — the epic #903→#911 made the CLI provision harnesses and tools from | +| config-schema-parity | A | 2026-09-01 02:15 | PASS | PR #833 (one schema file — DOCKER_SOCKET, SANDBOX_SSH, OH_SANDBOX_IMAGE, OH_PULL_POLICY, SKIP_PNPM_INSTALL were consumed but undocumented); rewritten for the oh.json/secrets split by PR #887 | +| context-tier-size-budget | A | 2026-09-01 02:15 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-007) — the always-on tier was 85,256 B | +| continual-learning-20260831 | A | 2026-09-01 02:15 | PASS | retro lesson 2026-08-31 (unexercised oracle) — a probe green in a 112-probe run carried three parser defects | +| cron-claude-codex-fallback | A | 2026-09-01 02:15 | PASS | conversation 2026-06-12 (default Codex fallback for crons) | +| cron-watchdog | A | 2026-09-01 02:15 | PASS | issues #130/#453 (cron runtime watchdog + legacy system-cron reaping) 2026-06-19 | +| crons-directory-guide | A | 2026-09-01 02:15 | PASS | issue #874 | +| curl-bash-safe-alternatives | A | 2026-09-01 02:15 | PASS | vet-run/vet integration — public curl|bash examples need review-first alternatives | +| datasets-schema | A | 2026-09-01 02:15 | PASS | issue #196 — .oh/evals/datasets verifiable trajectory corpus (Repo2RLEnv-inspired) | +| debugmcp-availability | A | 2026-09-01 02:15 | SKIPPED | issue #297 — DebugMCP MCP debug-server availability | +| default-provisioning | A | 2026-09-01 02:15 | PASS | #902 — `oh harness install` must work from inside the sandbox, where | +| delegate-model-effort-policy | A | 2026-09-01 02:15 | PASS | conversation 2026-07-11 (delegate model inheritance and thinking policy) | +| delegate-worker-boundary | A | 2026-09-01 02:15 | PASS | ADR #929 — subagents are a bounded execution primitive, not a project-role ontology; | +| devtcp-hook | A | 2026-09-01 02:15 | PASS | retro lesson 2026-06-10 (zsh /dev/tcp) | +| docker-inspect-env-guard | A | 2026-09-01 02:15 | PASS | operator directive 2026-08-08 (agents keep the docker socket, but must | +| docs-build-fast-path | A | 2026-09-01 02:15 | PASS | #455 — docs builds must stay out of fast harness/eval/release gates; #536 — docs site externalized to openharness-web; docs markdown relocated to docs/ | +| drift-check-cron-staleness-glob | A | 2026-09-01 02:15 | PASS | issue #98; issue #225 (restart-required cron frontmatter/config drift) | +| entrypoint-pnpm-manifest-fingerprint | A | 2026-09-01 02:15 | PASS | issue #521 (manifest-aware sandbox installs) 2026-07-01 | +| escalate-contract | A | 2026-09-01 02:15 | PASS | issue #799 — seven comments on a GitHub thread produced zero notifications and nobody | +| eval-ci-gate | A | 2026-09-01 02:15 | PASS | #103 — eval probe suite gated in CI | +| eval-contract-text-20260831 | A | 2026-09-01 02:15 | PASS | retro lesson 2026-08-31 (prose-literal pinning) — two assertions failed on first run purely from source line wrapping | +| eval-gate | A | 2026-09-01 02:15 | PASS | retro lesson 2026-06-11 (eval-gate) | +| eval-results-atomic | A | 2026-09-01 02:15 | PASS | issue #83 (eval-results-atomic-write) | +| eval-runner-exit | A | 2026-09-01 02:15 | PASS | retro lesson 2026-06-11 (eval-runner-exit) #29 | +| eval-runs-once-per-cycle | A | 2026-09-01 02:15 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-006) — /eval ran 3x per cycle on the | +| execution-target-contract | A | 2026-09-01 02:15 | PASS | issue #733 (ExecutionTarget contract + Docker Compose adapter) 2026-08-10 | +| get-oh-bootstrap | A | 2026-09-01 02:15 | PASS | get-oh.sh bootstrap — the Node-bootstrapping host-side path to the standalone `oh` CLI (also on npm as @mifune/openharness; see oh-npm-package.sh) | +| git-skill | A | 2026-09-01 02:15 | PASS | conversation 2026-06-15 — rules are not always supported; git workflow must be the /git skill | +| harness-audit-empty-output-gate | A | 2026-09-01 02:15 | PASS | issue #246 — /audit harness must fail closed on empty auditor outputs | +| harness-ci-core-paths | A | 2026-09-01 02:15 | PASS | #165 — core sandbox config files must trigger harness CI | +| harness-ci-hooks-paths | A | 2026-09-01 02:15 | PASS | issue #202 — credential/security hook changes must trigger harness CI | +| harness-yaml-migration | A | 2026-09-01 02:15 | PASS | PR #833 (migrate-harness-yaml.sh — append / uncomment-in-place / preserve / overwrite, plus a silent no-op second run) 2026-08-26 | +| headless-tmux-preserved | A | 2026-09-01 02:15 | PASS | issue #928 — retire automated /spec agent handoff | +| health-check-docker-stats | A | 2026-09-01 02:15 | PASS | retro lesson 2026-06-10 (docker stats vs ps Size) | +| health-check-socket-degrade | A | 2026-09-01 02:15 | PASS | issue #762 (refs #756) — /health-check degrades to one statement, not nine failures | +| heartbeat-logging-contract | A | 2026-09-01 02:15 | PASS | issue #447 (heartbeat log append hardening) 2026-06-18 | +| image-seed-hygiene | A | 2026-09-01 02:15 | PASS | issue #900 (slim the sandbox image) 2026-08-30 | +| markitdown-wiki-ingest | A | 2026-09-01 02:15 | PASS | issue #649 — pinned local-document normalization contract for /wiki ingest | +| next-dev-prod | A | 2026-09-01 02:15 | SKIPPED | retro lesson 2026-06-04 | +| no-project-agent-catalog | A | 2026-09-01 02:15 | PASS | ADR #929 — .oh/agents/ is retired; provider-link and update logic must not recreate it | +| oh-compose-env-wiring | A | 2026-09-01 02:15 | PASS | issue #880 (oh as the only front door — oh.json is the non-secret config surface) | +| oh-config-surfaces | A | 2026-09-01 02:15 | PASS | PR #887 (config split across two authored surfaces — a tracked oh.json and a secrets-only root dotenv — with nothing left under $HOME) | +| oh-destroy-guard | A | 2026-09-01 02:15 | PASS | issue #879 — `oh` becomes the only front door, so `make destroy` must | +| oh-devcontainer-restructure | A | 2026-09-01 02:15 | PASS | consolidate devcontainer — .oh/devcontainer/ folded back into .devcontainer/ | +| oh-home-mount | A | 2026-09-01 02:15 | PASS | issue #898 (single $HOME mount) 2026-08-30 | +| oh-image-only-deploy | A | 2026-09-01 02:15 | PASS | .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only | +| oh-init-headless-config | A | 2026-09-01 02:15 | PASS | PR #827 (installer answers landed in the losing config file); retargeted to the .example.env template by PR #833, then to oh.json by PR #887 | +| oh-init-scaffold | A | 2026-09-01 02:15 | PASS | issue #531 Phase 2 | +| oh-lifecycle-surface | A | 2026-09-01 02:15 | PASS | issue #881 — the Makefile is retired and `oh` is the only front door | +| oh-npm-package | A | 2026-09-01 02:15 | PASS | npm publish path for the standalone `oh` CLI (@mifune/openharness) — alternative to get-oh.sh | +| oh-payload-manifest | A | 2026-09-01 02:15 | PASS | issue #531 follow-on (.oh payload manifest — oh update ships a declared allowlist) | +| oh-sandbox-image-mode | A | 2026-09-01 02:15 | PASS | conversation 2026-07-05 (basic Docker deployment — prebuilt-image mode) | +| oh-shipped-repo-overridable | A | 2026-09-01 02:15 | PASS | issue #531 follow-on (de-hardcode residual — shipped .oh shell scripts keep the upstream repo overridable) | +| oh-standalone-lifecycle | A | 2026-09-01 02:15 | PASS | issue #564 | +| oh-update | A | 2026-09-01 02:15 | PASS | issue #531 Phase 3 (oh update — upgrade only the .oh control plane) | +| operator-config-guard | A | 2026-09-01 02:15 | PASS | operator directives 2026-08-06 (.config/ and settings.local.json are operator-only) | +| pnpm-audit-ci-gate | A | 2026-09-01 02:15 | PASS | issue #171 — pnpm security audits must run in CI | +| post-bridge-publish-confirmation | A | 2026-09-01 02:15 | PASS | #523 — post-bridge live publishing requires an explicit final confirmation gate | +| prd-output-path-contract | A | 2026-09-01 02:15 | PASS | retro lesson 2026-06-19 | +| prompt-miner-schema-compat | A | 2026-09-01 02:15 | PASS | issue #253 — prompt-miner JSONL schema-drift guard | +| prompt-miner-symlink-entrypoint | A | 2026-09-01 02:15 | PASS | issue #663 — prompt-miner engine no-ops via the documented .claude/skills symlink | +| prompt-miner-weakness-record | A | 2026-09-01 02:15 | PASS | issue #580 — prompt-miner weakness-record (WH-xxx) cluster output | +| protected-path-deletion | A | 2026-09-01 02:15 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-001) — the critique gate was deleted, | +| protected-paths-resolve | A | 2026-09-01 02:15 | PASS | issue #753 — .claude/protected-paths.txt named 7 paths that did not exist. | +| registry-portability-gate | A | 2026-09-01 02:15 | PASS | issue #758 | +| registry-portability | A | 2026-09-01 02:15 | SKIPPED | issue #758 | +| retro-deterministic-contract | A | 2026-09-01 02:15 | PASS | issue #443 — /retro deterministic output and self-contained helper contract | +| rlm-context-budget | A | 2026-09-01 02:15 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-006 | +| roles-are-skills | A | 2026-09-01 02:15 | PASS | ADR #929 — roles are behavior, skills encode behavior, agents execute behavior | +| runtime-preflight-gate | A | 2026-09-01 02:15 | PASS | issue #806 § B1 (open sandbox.substrate vs sandbox.runtime selector); | +| sandbox-boot-guard-ci | A | 2026-09-01 02:15 | PASS | issue #449 (sandbox image build CI guard) 2026-06-19; | +| sandbox-node-base | A | 2026-09-01 02:15 | PASS | openharness#878 — oh as the only front door, T0 sandbox base image | +| skill-paths | A | 2026-09-01 02:15 | PASS | issue #43 — stale path references; extended by issue #69 — apps/->packages/ rename guard; extended by issue #870 — deleted .oh/agents/advisor.md | +| skills-dir-clean | A | 2026-09-01 02:15 | PASS | conversation 2026-06-29 — Pi parses every top-level `.md` in the skills | +| skills-task-tool-coupling | A | 2026-09-01 02:15 | PASS | council review 2026-08-29 (issue #886) — /delegate instructed Claude-Code-only | +| skills-vendored | A | 2026-09-01 02:15 | PASS | absorb .mifune submodule into .oh — the skills/hooks pack is vendored | +| slack-admin-command-surface | A | 2026-09-01 02:15 | PASS | issue #354 — Slack bridge docs must distinguish Pi /msg-bridge commands from Slack DM admin text handlers | +| spec-family-contract | A | 2026-09-01 02:15 | PASS | issue #265; spec-simplification issue #816; workflow authority issue #854; | +| spec-no-advisor-session-coupling | A | 2026-09-01 02:15 | PASS | issue #928 — retire automated /spec agent handoff | +| spec-no-agent-handoff | A | 2026-09-01 02:15 | PASS | issue #928 — retire automated /spec agent handoff | +| spec-ready-finalization | A | 2026-09-01 02:15 | PASS | issue #134; spec-simplification issue #816; workflow authority issue #854 | +| spec-single-owner | A | 2026-09-01 02:15 | PASS | conversation 2026-06-19 (single-owner implementation workflow, issue #257); | +| ste-checker-contract | A | 2026-09-01 02:15 | PASS | issue #750 PR audit — the /ste checker had four fail-open paths (unclosed | +| submitted-by-trailers | A | 2026-09-01 02:15 | PASS | conversation 2026-06-12 (commit attribution trailers); the single-owner | +| sync-skill-contract | A | 2026-09-01 02:15 | PASS | issue #331 — /sync dispatcher skill (bidirectional origin↔upstream sync) | +| t3-headless-launch | A | 2026-09-01 02:15 | PASS | issue #858 — /t3 launched a bare `npx --yes t3`, which is the local GUI and | +| tailscale-tool-boundary | A | 2026-09-01 02:15 | PASS | issue #858 — Tailscale mobile access for T3 Code. There is no tailnet, no | +| tool-catalog-boundary | A | 2026-09-01 02:15 | PASS | agent-browser's exclusion from the harness catalog (#821), the three-catalog | +| version-parity | A | 2026-09-01 02:15 | PASS | conversation 2026-08-29 — the oh CLI became the only lifecycle door, so its | +| weigh-scorer-contract | A | 2026-09-01 02:15 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-003 (2026-06-27) | +| wiki-compile-contract | A | 2026-09-01 02:15 | PASS | wikiskill arXiv:2608.27454 — Wiki Maintainer role added as /wiki compile | +| wiki-kind-schema-contract | A | 2026-09-01 02:15 | PASS | wikiskill arXiv:2608.27454 — pattern layer added to the wiki corpus | +| wiki-pattern-persistence | A | 2026-09-01 02:15 | PASS | wikiskill arXiv:2608.27454 — pattern pages are never rolled back | +| wiki-query-pattern-isolation | A | 2026-09-01 02:15 | PASS | wikiskill arXiv:2608.27454 — proposer-only pattern access | +| wiki-readme-index | A | 2026-09-01 02:15 | PASS | issue #132 — wiki README index drift guard | +| wiki-related-slugs | A | 2026-09-01 02:15 | PASS | wikiskill arXiv:2608.27454 — wiki lint related-slug check | +| wiki-skill-impact-append-only | A | 2026-09-01 02:15 | PASS | wikiskill arXiv:2608.27454 — skill-change ledger, never rolled back | +| workflow-boundaries | A | 2026-09-01 02:15 | PASS | conversation 2026-06-19 (workflow consolidation, issue #259); authority moved to /spec in issue #854 | +| worktrees-layout | A | 2026-09-01 02:15 | PASS | issue #872 | diff --git a/.oh/evals/probes/architect-skill-contract.sh b/.oh/evals/probes/architect-skill-contract.sh new file mode 100755 index 00000000..bf147fd8 --- /dev/null +++ b/.oh/evals/probes/architect-skill-contract.sh @@ -0,0 +1,73 @@ +#!/usr/bin/env bash +# tier: A +# source: ADR #929 — skills are the role primitive; /architect is a skill, not an agent +# desc: /architect exists as an inline skill with the significance/grounding/brief contract, +# runs in the active session, and requires no separate agent process or definition file +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" +SKILL="$ROOT/.oh/skills/architect/SKILL.md" + +fail() { echo "REGRESSION: $*" >&2; exit 1; } + +[ -f "$SKILL" ] || fail "/architect is not a skill: .oh/skills/architect/SKILL.md is missing" +[ -f "$ROOT/.claude/skills/architect/SKILL.md" ] || fail "/architect does not resolve through the Claude provider surface" + +frontmatter="$(awk ' + NR == 1 && $0 == "---" { inside=1; next } + inside && $0 == "---" { exit } + inside { print } +' "$SKILL")" +[ -n "$frontmatter" ] || fail "architect SKILL.md lacks YAML frontmatter" +grep -qxF 'name: architect' <<<"$frontmatter" || fail "architect frontmatter name is not exact" +grep -qF 'TRIGGER when:' <<<"$frontmatter" || fail "architect description omits TRIGGER guidance" + +grep -qE '^context:' <<<"$frontmatter" && fail "/architect must run inline — no context: fork in frontmatter" +grep -qE '^allowed-tools:' <<<"$frontmatter" || fail "architect omits an allowed-tools boundary" +grep -qE '^allowed-tools:.*\b(Write|Edit)\b' <<<"$frontmatter" \ + && fail "/architect is read-oriented and must not claim Write/Edit — it does not implement" + +for marker in \ + 'Run **inline in the active coding-agent session**' \ + 'Do not fork the context' \ + 'Do not create an `architect` agent definition' ; do + grep -qF "$marker" "$SKILL" || fail "architect execution model omits: $marker" +done + +for section in \ + '## 1. Classify' \ + '## 2. Ground the decision' \ + '## 3. Decide' \ + '## 4. Record durable decisions' \ + '## 5. Output' ; do + grep -qxF "$section" "$SKILL" || fail "architect contract omits section: $section" +done + +grep -qF 'ARCHITECTURAL | NOT-ARCHITECTURAL' "$SKILL" || fail "architect omits the significance classification" +grep -qF 'NOT-ARCHITECTURAL' "$SKILL" || fail "architect omits the non-architectural bypass" +grep -qF '/spec plan' "$SKILL" || fail "architect does not route ordinary work to /spec plan" +grep -qF 'AGENTS.md' "$SKILL" || fail "architect grounding omits applicable repository instructions" +grep -qF 'docs/rfcs/README.md' "$SKILL" || fail "architect grounding omits the RFC/ADR index" + +for field in \ + '## Architecture Brief' \ + '### Classification' \ + '### Current State' \ + '### Decision Drivers' \ + '### Invariants' \ + '### Options Considered' \ + '### Recommendation' \ + '### Tradeoffs / Consequences' \ + '### Retirement / Consolidation' \ + '### Migration / Sequencing' \ + '### Validation' \ + '### Non-Goals' \ + '### Decision Record' ; do + grep -qF "$field" "$SKILL" || fail "Architecture Brief template omits: $field" +done + +for stray in "$ROOT/.oh/agents/architect.md" "$ROOT/.claude/agents/architect.md" "$ROOT/.codex/agents/architect.md"; do + [ ! -e "$stray" ] || fail "/architect was reintroduced as a project agent: ${stray#"$ROOT/"}" +done + +echo "PASS: /architect is an inline skill with the classification, grounding, and Architecture Brief contract" >&2 diff --git a/.oh/evals/probes/architecture-record-reuse.sh b/.oh/evals/probes/architecture-record-reuse.sh new file mode 100755 index 00000000..276595f8 --- /dev/null +++ b/.oh/evals/probes/architecture-record-reuse.sh @@ -0,0 +1,33 @@ +#!/usr/bin/env bash +# tier: A +# source: ADR #929 — durable decisions reuse the existing RFC/ADR issue convention +# desc: /architect points durable decisions at docs/rfcs/, invents no second decision store, +# and the RFC/ADR index records this ADR +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" +cd "$ROOT" + +SKILL=".oh/skills/architect/SKILL.md" +INDEX="docs/rfcs/README.md" + +fail() { echo "REGRESSION: $*" >&2; exit 1; } + +[ -f "$SKILL" ] || fail "$SKILL is missing" +[ -f "$INDEX" ] || fail "$INDEX is missing — the RFC/ADR convention is the only decision store" + +grep -qF 'docs/rfcs/README.md' "$SKILL" || fail "/architect does not point durable decisions at the RFC/ADR index" +grep -qF 'ADR: ' "$SKILL" || fail "/architect does not name the ADR issue-title convention" +grep -qF 'Superseded' "$SKILL" || fail "/architect omits the Draft/Accepted/Superseded lifecycle" +grep -qF 'Do not invent an architecture database' "$SKILL" \ + || fail "/architect does not forbid a second decision store" +grep -qF 'NONE | UPDATE' "$SKILL" || fail "/architect brief omits the Decision Record field" + +for store in .oh/decisions .oh/architecture .oh/adr docs/decisions docs/adr .oh/skills/architect/decisions; do + [ ! -e "$store" ] || fail "a second architecture decision store was created: $store" +done + +grep -qF 'Draft' "$INDEX" || fail "$INDEX lost the lifecycle states" +grep -qF 'issues/929' "$INDEX" || fail "$INDEX does not record ADR #929" + +echo "PASS: /architect reuses the RFC/ADR issue convention and no second decision store exists" >&2 diff --git a/.oh/evals/probes/builder-no-agent-artifact.sh b/.oh/evals/probes/builder-no-agent-artifact.sh new file mode 100755 index 00000000..44ac3ad7 --- /dev/null +++ b/.oh/evals/probes/builder-no-agent-artifact.sh @@ -0,0 +1,37 @@ +#!/usr/bin/env bash +# tier: A +# source: ADR #929 — /builder agent is retired; a reusable role is authored as a skill +# desc: /builder exposes only skill, command, and rule; the agent artifact type and its +# authoring reference are gone from the canonical builder surface +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" +SKILL="$ROOT/.oh/skills/builder/SKILL.md" +REFS="$ROOT/.oh/skills/builder/references" + +fail() { echo "REGRESSION: $*" >&2; exit 1; } + +[ -f "$SKILL" ] || fail "builder SKILL.md is missing" +[ ! -e "$REFS/agent.md" ] || fail "the retired agent authoring reference is back: references/agent.md" + +frontmatter="$(awk ' + NR == 1 && $0 == "---" { inside=1; next } + inside && $0 == "---" { exit } + inside { print } +' "$SKILL")" +grep -qxF 'argument-hint: "skill|command|rule <name-or-request>"' <<<"$frontmatter" \ + || fail "builder argument-hint does not advertise exactly skill|command|rule" + +grep -qF '| `agent` |' "$SKILL" && fail "builder dispatcher still routes the agent type" +grep -qF 'references/agent.md' "$SKILL" && fail "builder still points at the retired agent reference" +grep -qF 'Usage: /builder <skill|command|rule> <name-or-request>' "$SKILL" \ + || fail "builder usage line still offers the agent type" +grep -qF '`agent` is not an artifact type' "$SKILL" \ + || fail "builder does not state that agent is not an artifact type" + +for ref in "$REFS"/*.md; do + grep -nE '\.(oh|claude|codex|pi)/agents/' "$ref" >/dev/null \ + && fail "builder reference still writes to a project-agent directory: ${ref#"$ROOT/"}" +done + +echo "PASS: /builder agent and its authoring reference are retired; only skill|command|rule remain" >&2 diff --git a/.oh/evals/probes/builder-skill-consolidation.sh b/.oh/evals/probes/builder-skill-consolidation.sh index 4bd1e5a8..6cb06154 100755 --- a/.oh/evals/probes/builder-skill-consolidation.sh +++ b/.oh/evals/probes/builder-skill-consolidation.sh @@ -1,7 +1,7 @@ #!/usr/bin/env bash # tier: A # source: issue #643 — consolidate artifact builders behind one /builder dispatcher -# desc: /builder owns agent, skill, command, and rule authoring while legacy builder entry points stay removed +# desc: /builder owns skill, command, and rule authoring while legacy builder entry points stay removed set -euo pipefail ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" @@ -15,7 +15,6 @@ fail() { required=( "$SKILL" - "$REFS/agent.md" "$REFS/skill.md" "$REFS/command.md" "$REFS/rule.md" @@ -25,10 +24,6 @@ for path in "${required[@]}"; do done legacy=( - ".oh/agents/agent-builder.md" - ".oh/agents/skill-builder.md" - ".oh/agents/command-builder.md" - ".oh/agents/rule-builder.md" ".oh/skills/skill-builder" ) for rel in "${legacy[@]}"; do @@ -42,26 +37,21 @@ frontmatter="$(awk ' ' "$SKILL")" [ -n "$frontmatter" ] || fail "builder SKILL.md lacks YAML frontmatter" grep -qxF 'name: builder' <<<"$frontmatter" || fail "builder frontmatter name is not exact" -grep -qxF 'argument-hint: "agent|skill|command|rule <name-or-request>"' <<<"$frontmatter" || fail "builder argument hint does not expose all four public types" +grep -qxF 'argument-hint: "skill|command|rule <name-or-request>"' <<<"$frontmatter" || fail "builder argument hint does not expose all three public types" grep -qxF 'allowed-tools: Read, Write, Edit, Glob, Grep, Bash' <<<"$frontmatter" || fail "builder allowed-tools contract drifted" if grep -qE '^model:' <<<"$frontmatter"; then fail "builder must inherit the session model" fi -for type in agent skill command rule; do +for type in skill command rule; do grep -qF "| \`$type\` | \`references/$type.md\` |" "$SKILL" || fail "dispatcher route missing for type: $type" done -grep -qF 'Usage: /builder <agent|skill|command|rule> <name-or-request>' "$SKILL" || fail "missing exact invalid-argument usage" +grep -qF 'Usage: /builder <skill|command|rule> <name-or-request>' "$SKILL" || fail "missing exact invalid-argument usage" grep -qF 'remaining request is empty or only' "$SKILL" || fail "dispatcher does not reject an empty request after a valid type" grep -qF 'stop without reading' "$SKILL" || fail "missing fail-closed invalid-type behavior" if grep -qF '.oh/memory' "$SKILL"; then fail "builder references the deleted .oh/memory tier"; fi if grep -qF 'MEMORY_DIR' "$SKILL"; then fail "builder reintroduced the MEMORY_DIR override"; fi -AGENT_REF="$REFS/agent.md" -grep -qF '.oh/agents/<name>.md' "$AGENT_REF" || fail "agent type omits canonical Open Harness placement" -grep -qiF 'least privilege' "$AGENT_REF" || fail "agent type omits least-privilege tool design" -grep -qF 'Provider-link check passes' "$AGENT_REF" || fail "agent type omits provider exposure validation" - SKILL_REF="$REFS/skill.md" grep -qF '.oh/skills/<name>/SKILL.md' "$SKILL_REF" || fail "skill type omits canonical Open Harness placement" grep -qiF 'progressive disclosure' "$SKILL_REF" || fail "skill type omits progressive disclosure" @@ -97,5 +87,5 @@ if grep -qF 'skill-builder' "$ROOT/docs/oh-directory-layout.md"; then fail "current directory-layout docs still advertise skill-builder as an agent" fi -echo "PASS: /builder dispatches four artifact references and legacy builders remain removed" >&2 +echo "PASS: /builder dispatches three artifact references and legacy builders remain removed" >&2 exit 0 diff --git a/.oh/evals/probes/delegate-worker-boundary.sh b/.oh/evals/probes/delegate-worker-boundary.sh new file mode 100755 index 00000000..10dede84 --- /dev/null +++ b/.oh/evals/probes/delegate-worker-boundary.sh @@ -0,0 +1,52 @@ +#!/usr/bin/env bash +# tier: A +# source: ADR #929 — subagents are a bounded execution primitive, not a project-role ontology; +# supersedes rl-delegation-write-worker (#57), whose read-only-worker lesson is kept here +# desc: /delegate prefers the active session for context-sharing phases, keeps bounded +# provider-native workers for isolated/parallel work, warns that a read-only worker writes +# nothing, and names no nonexistent project agent roles +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" +SKILL="$ROOT/.claude/skills/delegate/SKILL.md" + +if [[ ! -f "$SKILL" ]]; then + echo "SKIPPED: delegate skill absent: $SKILL" >&2 + exit 2 +fi + +fail() { echo "REGRESSION: $*" >&2; exit 1; } + +grep -q '^## When a worker is justified$' "$SKILL" || fail "/delegate has no worker-justification section" + +boundary="$(awk '/^## When a worker is justified$/{f=1; next} f && /^## /{exit} f{print}' "$SKILL")" +[[ -n "${boundary//[[:space:]]/}" ]] || fail "the worker-justification section is empty" + +missing=() +grep -qiF 'self-contained' <<<"$boundary" || missing+=("the self-contained test") +grep -qiF 'parallelism' <<<"$boundary" || missing+=("parallelism as a reason") +grep -qiF 'isolated context' <<<"$boundary" || missing+=("isolated context as a reason") +grep -qiE 'restricted tools|tool .{0,12}restriction' <<<"$boundary" || missing+=("restricted tools as a reason") +grep -qiF 'verbose disposable output' <<<"$boundary" || missing+=("verbose disposable output as a reason") +grep -qiF 'share substantial context' <<<"$boundary" || missing+=("the keep-it-in-the-active-session rule") +grep -qiF 'iterative refinement' <<<"$boundary" || missing+=("the iterative-refinement exception") +(( ${#missing[@]} == 0 )) || fail "worker boundary omits: ${missing[*]}" + +worker_block="$(awk '/^Worker configuration:/{f=1} f{print} f && /^[[:space:]]*$/{exit}' "$SKILL")" +keyres_block="$(awk '/^### Key Resources/{f=1; next} f && /^(### |## )/{exit} f{print}' "$SKILL")" +region="$(printf '%s\n%s\n' "$worker_block" "$keyres_block")" +[[ -n "${region//[[:space:]]/}" ]] || fail "neither the Worker configuration block nor Key Resources could be located" + +missing=() +grep -qi 'read-only' <<<"$region" || missing+=("the read-only-worker warning") +grep -qi 'general-purpose' <<<"$region" || missing+=("the general-purpose recommendation for write workers") +grep -qi 'built-in' <<<"$region" || missing+=("the provider built-in worker vocabulary") +(( ${#missing[@]} == 0 )) || fail "the worker-configuration region omits: ${missing[*]}" + +stale="$(grep -nE '\.(oh|claude|codex|pi)/agents/[A-Za-z0-9_-]+\.md' "$SKILL" || true)" +[[ -z "$stale" ]] || fail "/delegate still cites project-agent definition files: $stale" + +roles="$(grep -nEi 'subagent_type: *(implementer|critic|pm|council)|`(implementer|critic|pm|council)`' "$SKILL" || true)" +[[ -z "$roles" ]] || fail "/delegate still names retired project-agent roles as worker types: $roles" + +echo "PASS: /delegate bounds workers to isolated/parallel work, keeps the read-only warning, and names no project agents" >&2 diff --git a/.oh/evals/probes/no-project-agent-catalog.sh b/.oh/evals/probes/no-project-agent-catalog.sh new file mode 100755 index 00000000..5ca0e490 --- /dev/null +++ b/.oh/evals/probes/no-project-agent-catalog.sh @@ -0,0 +1,44 @@ +#!/usr/bin/env bash +# tier: A +# source: ADR #929 — .oh/agents/ is retired; provider-link and update logic must not recreate it +# desc: no project-agent catalog exists in the tree, the index, the oh payload manifest, or the +# provider wiring, and link-providers.sh --init does not recreate one +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" +cd "$ROOT" + +fail() { echo "REGRESSION: $*" >&2; exit 1; } + +catalogs=(.oh/agents .claude/agents .codex/agents .pi/agents) +for path in "${catalogs[@]}"; do + [ ! -e "$path" ] && [ ! -L "$path" ] || fail "project-agent catalog is back in the tree: $path" +done +[ -z "$(git ls-files .oh/agents .claude/agents .codex/agents .pi/agents)" ] \ + || fail "a project-agent catalog path is still tracked in the git index" + +grep -qF 'agents' .oh/manifest.json && fail "the oh payload manifest still ships an agents/** pack" + +LINKER=".oh/scripts/link-providers.sh" +[ -x "$LINKER" ] || fail "$LINKER is missing or not executable" +wiring="$(awk '/^provider_links=\(/{f=1; next} f && /^\)/{exit} f{print}' "$LINKER")" +[ -n "$wiring" ] || fail "could not read provider_links from $LINKER" +grep -qF 'agents' <<<"$wiring" && fail "link-providers.sh still wires a project-agent provider symlink" + +INIT_TS=".oh/cli/src/commands/init.ts" +[ -f "$INIT_TS" ] || fail "$INIT_TS is missing" +ts_links="$(awk '/^const PROVIDER_LINKS/{f=1; next} f && /^\];/{exit} f{print}' "$INIT_TS")" +[ -n "$ts_links" ] || fail "could not read PROVIDER_LINKS from $INIT_TS" +grep -qF 'agents' <<<"$ts_links" && fail "oh init still creates a project-agent provider symlink" + +created=() +bash "$LINKER" --init >/dev/null 2>&1 || true +for path in "${catalogs[@]}"; do + if [ -e "$path" ] || [ -L "$path" ]; then + created+=("$path") + rm -rf "$path" + fi +done +(( ${#created[@]} == 0 )) || fail "link-providers.sh --init recreated the project-agent catalog: ${created[*]}" + +echo "PASS: no project-agent catalog in the tree, index, manifest, or provider wiring; --init recreates none" >&2 diff --git a/.oh/evals/probes/rl-delegation-write-worker.sh b/.oh/evals/probes/rl-delegation-write-worker.sh deleted file mode 100755 index da86976e..00000000 --- a/.oh/evals/probes/rl-delegation-write-worker.sh +++ /dev/null @@ -1,38 +0,0 @@ -#!/usr/bin/env bash -# tier: A -# source: retro lesson 2026-06-10 (rl-delegation) #57 -# desc: /delegate SKILL.md must warn that the implementer/pm/critic sub-agent types are -# read-only and recommend subagent_type general-purpose for any write/edit worker -set -euo pipefail - -ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" -SKILL="$ROOT/.claude/skills/delegate/SKILL.md" - -if [[ ! -f "$SKILL" ]]; then - echo "SKIPPED: delegate skill absent: $SKILL" >&2 - exit 2 -fi - -worker_block="$(awk '/^Worker configuration:/{f=1} f{print} f && /^[[:space:]]*$/{exit}' "$SKILL")" -keyres_block="$(awk '/^### Key Resources/{f=1; next} f && /^(### |## )/{exit} f{print}' "$SKILL")" - -region="$(printf '%s\n%s\n' "$worker_block" "$keyres_block")" - -if [[ -z "${region//[[:space:]]/}" ]]; then - echo "REGRESSION: warning region not found in $SKILL (neither the 'Worker configuration:' block nor the '### Key Resources' section could be located)" >&2 - exit 1 -fi - -missing=() -grep -qi 'read-only' <<<"$region" || missing+=("'read-only' warning") -grep -qiE 'implementer|critic|(^|[^a-z])pm([^a-z]|$)' <<<"$region" \ - || missing+=("an agent name (implementer/pm/critic)") -grep -qi 'general-purpose' <<<"$region" || missing+=("'general-purpose' recommendation") - -if (( ${#missing[@]} > 0 )); then - echo "REGRESSION: required phrase missing from the /delegate read-only warning region: ${missing[*]}" >&2 - exit 1 -fi - -echo "PASS: /delegate warns implementer/pm/critic are read-only and recommends general-purpose for write workers" >&2 -exit 0 diff --git a/.oh/evals/probes/roles-are-skills.sh b/.oh/evals/probes/roles-are-skills.sh new file mode 100755 index 00000000..64721db5 --- /dev/null +++ b/.oh/evals/probes/roles-are-skills.sh @@ -0,0 +1,51 @@ +#!/usr/bin/env bash +# tier: A +# source: ADR #929 — roles are behavior, skills encode behavior, agents execute behavior +# desc: active core docs and tooling encode durable roles as skills; no role is defined as a +# required project-agent identity, and no active surface cites a project-agent file +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" +cd "$ROOT" + +fail() { echo "REGRESSION: $*" >&2; exit 1; } + +for skill in architect spec audit retro delegate builder; do + [ -f ".oh/skills/$skill/SKILL.md" ] || fail "role-owning skill missing: .oh/skills/$skill/SKILL.md" +done + +for role in architect advisor auditor implementer critic pm council first-mate prime; do + for dir in .oh/agents .claude/agents .codex/agents .pi/agents; do + [ ! -e "$dir/$role.md" ] || fail "role reintroduced as a project agent: $dir/$role.md" + done +done + +agent_file_refs="$(grep -rnE '\.(oh|claude|codex|pi)/agents/[A-Za-z0-9_-]+\.md' \ + .oh/skills docs AGENTS.md README.md .oh/README.md 2>/dev/null \ + | grep -v '^\.oh/skills/wiki/corpus/' \ + | grep -v '^docs/rfcs/preserved-changelog-rationale\.md:' || true)" +if [ -n "$agent_file_refs" ]; then + echo "REGRESSION: active surfaces still cite project-agent definition files:" >&2 + printf '%s\n' "$agent_file_refs" >&2 + exit 1 +fi + +retired_role_uses="$(grep -rnE '\b(Advisor|First Mate)\b' .oh/skills 2>/dev/null \ + | grep -v '^\.oh/skills/wiki/corpus/' \ + | grep -vE '\b([Nn]o|[Nn]ot|[Nn]ever|[Nn]either)\b' || true)" +if [ -n "$retired_role_uses" ]; then + echo "REGRESSION: active skills still invoke a retired role identity (a retired role may only appear in a negation):" >&2 + printf '%s\n' "$retired_role_uses" >&2 + exit 1 +fi + +grep -qF 'canonical primitive for a reusable role' docs/glossary.md \ + || fail "docs/glossary.md does not name skills as the canonical reusable-role primitive" +grep -qF 'the **runtime and the owner of the' docs/glossary.md \ + || fail "docs/glossary.md does not name the active coding agent as the runtime/owner" +grep -qF -e '- **worker / subagent**' docs/glossary.md \ + || fail "docs/glossary.md does not define worker/subagent as a bounded execution context" +grep -qF -e '- **rule**' docs/glossary.md || fail "docs/glossary.md does not define rule" +grep -qF -e '- **rfc / adr**' docs/glossary.md || fail "docs/glossary.md does not define rfc/adr" + +echo "PASS: durable roles are skills; no active surface defines or cites a project-agent identity" >&2 diff --git a/.oh/evals/probes/skills-vendored.sh b/.oh/evals/probes/skills-vendored.sh index 85084cb9..40735171 100755 --- a/.oh/evals/probes/skills-vendored.sh +++ b/.oh/evals/probes/skills-vendored.sh @@ -1,8 +1,8 @@ #!/usr/bin/env bash # tier: A -# source: absorb .mifune submodule into .oh — the skills/agents/hooks pack is vendored +# source: absorb .mifune submodule into .oh — the skills/hooks pack is vendored # directly under .oh/ (no submodule); provider symlinks resolve into it from a clean clone -# desc: there is NO .mifune submodule; .oh/skills|agents|hooks are tracked in-repo and the +# desc: there is NO .mifune submodule; .oh/skills|hooks are tracked in-repo and the # provider symlinks resolve into .oh/ with no init/network step; the Hermes link is # created when the hermes binary is on PATH and not otherwise (#920 replaced the # INSTALL_HERMES flag with that presence check, so it works in both sandbox flavors) @@ -31,7 +31,7 @@ for path in \ git ls-files --error-unmatch "$path" >/dev/null 2>&1 || fail "pack file not tracked in-repo: $path" done -for link in .pi/skills .claude/skills .codex/skills .claude/agents .claude/hooks .codex/agents; do +for link in .pi/skills .claude/skills .codex/skills .claude/hooks; do [ -L "$link" ] || fail "$link is not a symlink" [ -e "$link" ] || fail "$link target does not resolve" done @@ -64,5 +64,5 @@ if [ "${SKILLS_VENDORED_SKIP_CLEAN_CLONE:-0}" != "1" ]; then cd "$ROOT" fi -echo "PASS: skills/agents/hooks are vendored under .oh/ (no submodule) and provider symlinks resolve from a clean clone" >&2 +echo "PASS: skills/hooks are vendored under .oh/ (no submodule) and provider symlinks resolve from a clean clone" >&2 exit 0 diff --git a/.oh/manifest.json b/.oh/manifest.json index 59e4bc40..cb6d63d5 100644 --- a/.oh/manifest.json +++ b/.oh/manifest.json @@ -6,7 +6,6 @@ "templates/**", "evals/**", "skills/**", - "agents/**", "hooks/**", "skills.lock", "README.md", diff --git a/.oh/scripts/link-providers.sh b/.oh/scripts/link-providers.sh index 35648fe6..7384902a 100755 --- a/.oh/scripts/link-providers.sh +++ b/.oh/scripts/link-providers.sh @@ -27,9 +27,7 @@ provider_links=( ".pi/skills|../.oh/skills" ".claude/skills|../.oh/skills" ".codex/skills|../.oh/skills" - ".claude/agents|../.oh/agents" ".claude/hooks|../.oh/hooks" - ".codex/agents|../.claude/agents" ) HERMES_LINK=".hermes/skills/openharness" @@ -147,7 +145,7 @@ check_protected_paths() { entry="$(printf '%s' "$entry" | xargs)" [ -n "$entry" ] || continue case "$entry" in - .oh/skills/*|.oh/agents/*|.oh/hooks/*) + .oh/skills/*|.oh/hooks/*) [ -e "$entry" ] || fail "protected pack path missing: $entry" ;; esac diff --git a/.oh/skills/architect/SKILL.md b/.oh/skills/architect/SKILL.md new file mode 100644 index 00000000..a7640afd --- /dev/null +++ b/.oh/skills/architect/SKILL.md @@ -0,0 +1,183 @@ +--- +name: architect +description: | + Decide what the system should become before /spec commits an implementation + plan. Runs inline in the active coding-agent session, grounds every claim in + current repository sources, and returns one Architecture Brief. + TRIGGER when: a change alters module/system boundaries, the execution or + ownership model, persistent state, a security or isolation boundary, a public + API or compatibility contract, a lifecycle/state machine, provider + portability, shared vocabulary, cross-skill control-plane behavior, a new + reusable abstraction, the retirement of an existing one, or any structural + decision that is expensive to reverse; asked to "design this", "what should + the architecture be", "is this architecturally significant", "ADR for X", + before /spec plan on a structural change. + Do NOT trigger for ordinary local changes — a bug fix, a copy edit, a new + test, a contained refactor inside one module. +argument-hint: "<problem | issue number | plan path>" +allowed-tools: Read, Glob, Grep, Bash +--- + +# Architect + +Determine what the system should become before `/spec` commits the +implementation plan. This skill decides architecture; it does not implement. + +Arguments received: `$ARGUMENTS` + +## Execution model + +Run **inline in the active coding-agent session**. The session that invoked +`/architect` keeps ownership of the work and of everything that follows it. + +- Do not fork the context, launch another coding-agent process, open a tmux or + Herdr session, or hand the decision to a separate persistent identity. +- Do not create an `architect` agent definition. The role is this skill; the + runtime is the session already reading it. +- Use `/delegate` only for bounded, self-contained fan-out — parallel + independent source reads whose verbose output should stay out of this + context. Reconcile every delegated result here; the brief is written by this + session. + +Roles are behavior, skills encode behavior, and agents execute behavior. A +second agent context is justified only when isolation or parallelism is itself +the thing being bought. + +## 1. Classify + +Architecture review is selective, not ceremony. Treat the change as +**ARCHITECTURAL** when it materially affects one or more of: + +- system or module boundaries; +- the execution or ownership model; +- persistent state or the data model; +- a security or isolation boundary; +- a public API or compatibility contract; +- a lifecycle or state machine; +- provider portability; +- shared conceptual vocabulary; +- cross-skill or control-plane behavior; +- a new reusable abstraction; +- the replacement or retirement of an existing abstraction; +- a structural decision that is difficult or expensive to reverse. + +Otherwise the change is **NOT-ARCHITECTURAL**. Say so in two or three +sentences, name the reason, route the work straight to `/spec plan`, and stop. +Do not produce the full brief for a small local change. + +## 2. Ground the decision + +Before reasoning, read real sources. Synthesized recall is advisory; current +source and accepted decision records are authority. + +1. Read every applicable `AGENTS.md` from the repository root down to the + directories the change touches. Local instructions win; in one directory + `AGENTS.md` is canonical. +2. Read the RFC/ADR index at `docs/rfcs/README.md` and any listed proposal that + already constrains this decision. An accepted decision is a constraint until + a new proposal supersedes it. +3. Query tracked repository knowledge when it is available: + `/wiki query <subsystem> --patterns` returns failure modes this harness has + already paid for. Cite the `[[pattern-...]]` slugs that changed the + recommendation. +4. Inspect the authoritative code, tests, probes, and docs for the surfaces in + scope. Read them; do not infer their shape. +5. Name the actual decision or decisions. A feature request restated is not a + decision. +6. Label every claim as fact, constraint, assumption, or judgment. An + unverified assumption must say so. + +## 3. Decide + +For each material decision, work through: + +- **Current state** — what exists now, cited by path. +- **Drivers** — the outcome wanted and why now. +- **Invariants** — what must survive the change untouched. +- **Options** — including reuse, extend, retire, and do-nothing where those are + live. A single-option brief is not an analysis. +- **Tradeoffs** — complexity, reversibility, portability, security, + operability, context cost, migration risk. +- **Interaction** — how the option meets existing accepted decisions. +- **Failure modes** — how it breaks, and how it rolls back. +- **Retirement** — what this consolidates or deletes rather than adds beside. +- **Falsification** — the evidence that would show the recommendation wrong. + +Favor the smallest architecture that satisfies the actual requirement. Do not +add an orchestration or process layer without evidence it improves outcomes. +Prefer deleting an abstraction to growing a second one next to it. + +## 4. Record durable decisions + +Durable architecture decisions reuse the existing convention in +[`docs/rfcs/README.md`](../../../docs/rfcs/README.md): a GitHub issue titled +`ADR: <title>` (or `RFC: <title>`), moving through `Draft` → `Accepted` → +`Superseded`, indexed on that page. + +Do not invent an architecture database, service, document taxonomy, decision +directory, or per-skill decision store. Do not require a record for every +change — most decisions are captured well enough by the code and its probes. +Propose a record only when the decision is architecturally significant and +expensive to rediscover. + +## 5. Output + +Return exactly one compact brief. When the classification is +`NOT-ARCHITECTURAL`, return only the classification, the reason, and the +routing sentence. + +```markdown +## Architecture Brief + +### Classification +ARCHITECTURAL | NOT-ARCHITECTURAL + +### Current State +... + +### Decision Drivers +- ... + +### Invariants +- ... + +### Options Considered +1. ... +2. ... +3. ... + +### Recommendation +... + +### Tradeoffs / Consequences +- ... + +### Retirement / Consolidation +- ... + +### Migration / Sequencing +- ... + +### Validation +- evidence or PoC needed before/after implementation + +### Non-Goals +- ... + +### Decision Record +NONE | UPDATE <issue/RFC/ADR> | PROPOSE ADR: <title> +``` + +## Boundaries + +| In scope | Out of scope | +|---|---| +| Naming the decision and the options | Writing the implementation | +| Recommending one option with tradeoffs | Opening branches, worktrees, or PRs | +| Pointing a durable decision at an RFC/ADR issue | Creating a new decision store | +| Handing the accepted direction to `/spec plan` | Owning the build | + +`/architect` decides architecture. `/spec` turns the accepted direction into +executable work. Neither is a mandatory phase for the other: a +`NOT-ARCHITECTURAL` change goes straight to `/spec plan`, and a brief that ends +in a rejected option produces no plan at all. diff --git a/.oh/skills/audit/references/context.md b/.oh/skills/audit/references/context.md index cbb4f945..2fced3e8 100644 --- a/.oh/skills/audit/references/context.md +++ b/.oh/skills/audit/references/context.md @@ -67,13 +67,12 @@ TOKENS=$(($(wc -c < "$f") / 4)) #### Dimension B — Load-bearing (0-2) -Citation count: how many skills, agents, tracked docs, and orchestrator files reference this file by name (a measurable proxy for "this content is actively consumed"). +Citation count: how many skills, tracked docs, and orchestrator files reference this file by name (a measurable proxy for "this content is actively consumed"). ```bash FILE_BASE=$(basename "$f") REFS=$(grep -rl "$FILE_BASE" \ "$HARNESS/.oh/skills" \ - "$HARNESS/.oh/agents" \ "$HARNESS/AGENTS.md" \ "$HARNESS/docs" 2>/dev/null \ | grep -v "^${f}$" | wc -l) diff --git a/.oh/skills/audit/references/harness.md b/.oh/skills/audit/references/harness.md index f1c10c7e..76c492f4 100644 --- a/.oh/skills/audit/references/harness.md +++ b/.oh/skills/audit/references/harness.md @@ -53,7 +53,6 @@ Read the following before spawning agents. Pass the assembled snapshot to every # Harness structure ls "$AUDIT_ROOT/.claude/skills/" -ls "$AUDIT_ROOT/.claude/agents/" 2>/dev/null || echo "no agents dir" ls "$AUDIT_ROOT/crons/" 2>/dev/null || echo "no crons" tail -20 "$AUDIT_ROOT/crons/.cron.log" 2>/dev/null ls "$AUDIT_ROOT/.oh/skills/wiki/corpus/" 2>/dev/null | head -20 @@ -108,7 +107,7 @@ Assemble a **Context Snapshot** (compact markdown, ~300 words): ### 3. Spawn 4 auditors in ONE message (parallel) -Launch 4 Agent tool calls **in a single message**. Each receives the Context Snapshot and its specific audit mandate below. All agents use **sonnet** model and **Ultra compression** for their output (consumed by the synthesis step, not humans). +Launch 4 Agent tool calls **in a single message**. Each receives the Context Snapshot and its specific audit mandate below. All four are bounded provider-native workers driven by the prompts in this file — there is no repository agent definition behind any of them, so pass a provider built-in `subagent_type` and carry the perspective in the prompt. All use **sonnet** model and **Ultra compression** for their output (consumed by the synthesis step, not humans). --- diff --git a/.oh/skills/blog/SKILL.md b/.oh/skills/blog/SKILL.md index b62d8372..5a138997 100644 --- a/.oh/skills/blog/SKILL.md +++ b/.oh/skills/blog/SKILL.md @@ -6,7 +6,7 @@ allowed-tools: Read, Write, Edit, Glob, Grep, Bash, Agent description: | Turn a source scenario into a publishable blog post. First supported playbook: Loom/demo-to-blog, where a raw notes file plus Loom screenshots become a - polished site post after Advisor briefing, photo/text alignment audit, site + polished site post after a /delegate briefing, photo/text alignment audit, site convention audit, narrative/fact-check audit, source-faithful image handling (exact URLs or local files downloaded from them), optional social/channel-promotion artifact generation, and build verification. Manual-invoke only because it writes content and may spawn @@ -70,4 +70,4 @@ Result: CREATED | DRY-RUN | BLOCKED /blog Create blog from @.claude/specs/openharness-demo --target /worktrees openharness-web ``` -Expected behavior: read `demo.md`, audit all Loom images, use Advisor plus specialized delegates, write a Docusaurus post in the target blog repo, embed only selected safe screenshots using exact `demo.md` URLs or local files downloaded from those URLs, run validation, and report changed paths. +Expected behavior: read `demo.md`, audit all Loom images, obtain a `/delegate` briefing, spawn specialized delegates, write a Docusaurus post in the target blog repo, embed only selected safe screenshots using exact `demo.md` URLs or local files downloaded from those URLs, run validation, and report changed paths. diff --git a/.oh/skills/blog/references/loom-to-blog.md b/.oh/skills/blog/references/loom-to-blog.md index 46cc8c83..3b0a1560 100644 --- a/.oh/skills/blog/references/loom-to-blog.md +++ b/.oh/skills/blog/references/loom-to-blog.md @@ -263,7 +263,7 @@ Expected actions: 1. Read `.claude/specs/openharness-demo/demo.md`. 2. Extract/download every Loom screenshot. 3. Audit all source sections and images. -4. Use Advisor plus the three default delegates. +4. Obtain the `/delegate` briefing, then spawn the three default delegates. 5. Write the Docusaurus post under `openharness-web/blog/`. 6. Embed selected safe screenshots with exact `demo.md` URLs or local files downloaded from those exact URLs if Loom hotlinks break. 7. Generate `promos/<date-slug>.md` if social promotion was requested. diff --git a/.oh/skills/builder/SKILL.md b/.oh/skills/builder/SKILL.md index fb4546f6..b1adcde8 100644 --- a/.oh/skills/builder/SKILL.md +++ b/.oh/skills/builder/SKILL.md @@ -1,11 +1,13 @@ --- name: builder description: | - Author and refine provider-portable agents, reference skills, task-style command - skills, and path-scoped rules using one repository-grounded workflow. TRIGGER - when: asked to create, build, scaffold, convert, review, or update an agent, - skill, command, workflow, rule, coding standard, or contextual instruction. -argument-hint: "agent|skill|command|rule <name-or-request>" + Author and refine reference skills, task-style command skills, and path-scoped + rules using one repository-grounded workflow. TRIGGER when: asked to create, + build, scaffold, convert, review, or update a skill, command, workflow, rule, + coding standard, or contextual instruction. Skills are the canonical primitive + for a reusable role, procedure, or body of judgment — there is no project-agent + artifact type. +argument-hint: "skill|command|rule <name-or-request>" allowed-tools: Read, Write, Edit, Glob, Grep, Bash --- @@ -25,7 +27,6 @@ Arguments received: `$ARGUMENTS` | Type | Read and follow | |------|-----------------| - | `agent` | `references/agent.md` | | `skill` | `references/skill.md` | | `command` | `references/command.md` | | `rule` | `references/rule.md` | @@ -35,9 +36,13 @@ Arguments received: `$ARGUMENTS` modifying files: ```text - Usage: /builder <agent|skill|command|rule> <name-or-request> + Usage: /builder <skill|command|rule> <name-or-request> ``` + `agent` is not an artifact type. A reusable role, procedure, or specialist + judgment is authored as a skill; a bounded isolated worker context is an + execution choice made by `/delegate`, not a repository artifact. + 4. Read the selected reference completely, then execute its protocol against the remainder of `$ARGUMENTS`. The selected reference is authoritative for artifact shape and type-specific validation. @@ -53,8 +58,8 @@ steps. through the target directory. More local instructions win; in one directory, `AGENTS.md` is canonical. - Identify the source-of-truth artifact directory. In Open Harness and equipped - projects, edit `.oh/agents/` or `.oh/skills/`; provider directories such as - `.claude/`, `.codex/`, and `.pi/` are generated or symlinked exposure surfaces. + projects, edit `.oh/skills/`; provider directories such as `.claude/`, + `.codex/`, and `.pi/` are generated or symlinked exposure surfaces. - Outside an Open Harness layout, follow the target project's documented canonical path rather than creating `.oh/` speculatively. - Inspect two or three nearby artifacts of the same type. Reuse their naming, @@ -106,7 +111,7 @@ or create unsafe side effects. Otherwise use the request and repository evidence behavior; key design choices; and validation evidence. Never claim a check ran if it did not. - Append a `PROPOSED` record to `.oh/skills/wiki/corpus/skill-impact.md` when a skill - or agent edit lands: the next `SI-nnnn` id, the one-sentence proposal, the single + edit lands: the next `SI-nnnn` id, the one-sentence proposal, the single target artifact, the motivating pattern slugs, and the unified diff scoped to that target path. `motivating patterns: none (direct request)` is a legitimate value — record it rather than inventing a pattern to cite. Force-add the ledger diff --git a/.oh/skills/builder/references/agent.md b/.oh/skills/builder/references/agent.md deleted file mode 100644 index 0a51c2ed..00000000 --- a/.oh/skills/builder/references/agent.md +++ /dev/null @@ -1,156 +0,0 @@ -# Agent Builder Reference - -Author a reusable specialist that runs in an isolated context with an explicit -role, tool boundary, and output contract. In Open Harness, write the canonical -artifact to `.oh/agents/<name>.md`; provider surfaces expose that directory through -symlinks. - -## Contents - -1. [Choose an agent](#choose-an-agent) -2. [Discover the role](#discover-the-role) -3. [Design the definition](#design-the-definition) -4. [Frontmatter](#frontmatter) -5. [Write the body](#write-the-body) -6. [Validate](#validate) -7. [Report](#report) - -## Choose an agent - -Use an agent when the work benefits from a durable specialist identity, isolated -context, reusable judgment, and a clear delegation boundary. Do not create one for: - -- a one-off task that the current assistant can complete directly; -- a repeatable user-invoked procedure, which belongs in a task-style skill; -- background knowledge or file-specific guidance, which belongs in a reference - skill, optionally with `paths:`; -- generic exploration, planning, or execution already covered by a built-in or - existing project agent. - -Before authoring, compare the proposed role with nearby agents and built-ins. Extend -an existing role when the new responsibilities share the same evidence, tools, and -success criteria. Split only when the specialist has a distinct delegation trigger -and can return a coherent result independently. - -## Discover the role - -1. Read applicable project instructions and the closest agent definitions. -2. Inspect the code, docs, tests, artifacts, and workflows the specialist must - understand. Do not invent project architecture or domain rules. -3. Define: - - primary user and delegation triggers; - - in-scope decisions and explicit exclusions; - - evidence the agent must inspect before advising; - - collaboration seams with adjacent agents; - - output shape and measurable completion criteria. -4. Select the minimum tool set. A read-only advisor normally needs `Read, Glob, - Grep` and sometimes `Bash` for known non-mutating inspection. Add `Edit` or - `Write` only when mutation is part of the role. -5. Choose a model only when the role consistently warrants a cost or latency - override. Omit `model` to inherit by default. - -## Design the definition - -A strong agent definition has one center of gravity. Include only sections that -change its behavior: - -- **Role**: one paragraph naming the specialist, responsibility, and value. -- **Scope and boundaries**: concrete in-scope and out-of-scope decisions. -- **Tool policy**: how each allowed tool may be used; note that allowing `Bash` - does not make it read-only. -- **Evidence discipline**: sources to inspect and how to label evidence, - inference, assumptions, or uncertainty when relevant. -- **Workflow**: an adaptive sequence for discovery, analysis, and completion. -- **Collaboration seams**: when to hand work to existing specialists rather than - duplicating them. -- **Output contract**: concise templates suited to actual requests. -- **Quality bar**: observable success and failure conditions. - -Avoid encyclopedic tutorials, generic claims of excellence, hard-coded stack -assumptions, and speculative file maps. Agents start with fresh context; tell them -how to discover facts rather than embedding facts likely to drift. - -## Frontmatter - -Use the repository's demonstrated portable subset unless a target provider is -explicitly selected: - -```yaml ---- -name: design-reviewer -description: | - Focused description of what the agent does and when to delegate to it. - Use proactively when <specific observable triggers>. -tools: Read, Glob, Grep, Bash -model: sonnet ---- -``` - -Rules: - -- `name`: lowercase kebab-case and equal to the filename stem. -- `description`: front-load role and positive triggers; add meaningful negative - triggers when neighboring roles could overlap. -- `tools`: comma-separated least-privilege allowlist. Omission may inherit all - parent tools, including provider integrations, so omit only deliberately. -- `model`: optional. Prefer inheritance; justify `opus`, `sonnet`, or `haiku` - from stable role complexity rather than current-session preference. -- Provider-only fields may be used only after inspecting that provider's current - docs and local examples. Do not imply unsupported fields are portable. - -For a behaviorally read-only agent that includes `Bash`, state explicit -non-mutating command limits in the body. Frontmatter cannot express a read-only -Bash subset. - -## Write the body - -1. Start with the role and outcome, not a repeat of frontmatter. -2. State boundaries before the workflow so the agent cannot expand its remit while - gathering context. -3. Make discovery repository-grounded: applicable instructions first, then the - narrowest relevant sources. -4. Make the workflow proportional. Small reviews should not be forced through - irrelevant exhaustive phases. -5. Define how to handle missing evidence, blocked decisions, and unsafe requests. -6. Provide output shapes with required evidence and decisions, but avoid a single - rigid template for unrelated request types. -7. Name adjacent agents and handoff seams only when those agents actually exist. -8. Keep the definition concise enough to leave working context for the delegated - task. Move stable long-tail material to a skill if multiple agents need it. - -## Validate - -- [ ] File is `.oh/agents/<name>.md` in an Open Harness layout, not a provider - mirror. -- [ ] Frontmatter has exactly one opening and closing delimiter. -- [ ] `name` matches the filename and is lowercase kebab-case. -- [ ] Description states positive triggers and avoids neighboring-role overlap. -- [ ] Tool access is least privilege; body restrictions agree with frontmatter. -- [ ] Model is omitted or explicitly justified. -- [ ] Referenced agents, paths, commands, and tools exist. -- [ ] Role, exclusions, workflow, collaboration seams, and outputs agree. -- [ ] No invented project facts or claims that behavioral limits are enforced by - frontmatter. -- [ ] Provider-link check passes when available. -- [ ] File remains below 500 lines. - -## Report - -Return: - -```markdown -## Agent Created: <name> - -**File**: `.oh/agents/<name>.md` -**Role**: <one sentence> -**Tools**: <allowlist and why> -**Model**: <inherit or override and why> -**Boundaries**: <key exclusions> -**Validation**: <checks and results> -``` - -Use `Created` for a new artifact and `Updated` for a focused revision of an -existing one; the update path is the common case once an artifact exists. Add a -**Motivated by** line naming the `[[pattern-...]]` slugs read in shared protocol -step 1, or `none (direct request)`, and a **Ledger** line naming the `SI-nnnn` id -appended in step 4. diff --git a/.oh/skills/builder/references/command.md b/.oh/skills/builder/references/command.md index 468aabbf..fdac0bca 100644 --- a/.oh/skills/builder/references/command.md +++ b/.oh/skills/builder/references/command.md @@ -20,7 +20,8 @@ Author a deliberate, user-invoked workflow as a skill. The public type remains Use this type for an on-demand procedure that performs a recognizable job from input to reported result: deploy, release, publish, sync, migrate, scaffold, sweep, or triage. Use `/builder skill` for knowledge Claude should apply inline while -working. Use an agent for a reusable specialist identity with isolated context. +working. A specialist role is also a skill; `/delegate` decides when its work +runs in a bounded isolated worker context. A task-style skill should have: @@ -73,8 +74,7 @@ Guidance: explicit slash invocation is safe. - Omit `model` to inherit unless the workflow has a stable model requirement. - Use `context: fork` only when intermediate work would pollute the parent and the - body is a self-contained task prompt. Select an existing agent explicitly only - when its role fits. + body is a self-contained task prompt. Otherwise run inline in the active session. - Add `paths:` rarely; file-triggered loading is usually a reference-skill concern. ## Authoring protocol diff --git a/.oh/skills/builder/references/skill.md b/.oh/skills/builder/references/skill.md index 64805a03..4fff758c 100644 --- a/.oh/skills/builder/references/skill.md +++ b/.oh/skills/builder/references/skill.md @@ -20,8 +20,8 @@ Harness, write `.oh/skills/<name>/SKILL.md`; `.claude/skills`, `.codex/skills`, Use this type for conventions, domain knowledge, decision guidance, or contextual instructions Claude should consult while doing another task. If the request is a deliberate end-to-end procedure such as deploy, release, publish, sync, or sweep, -use `/builder command` instead. Use an agent only when the work needs a durable -specialist identity and isolated context. +use `/builder command` instead. A durable specialist role is a skill too — the +active session adopts it; there is no project-agent artifact to author. Reference skills should usually remain model-invocable. They may use `paths:` when specific files provide a reliable loading signal. Do not use `context: fork` for @@ -82,7 +82,7 @@ Common fields: | `user-invocable` | Set `false` only when hiding the slash-menu entry is intentional. | | `disable-model-invocation` | Usually false or omitted for reference skills. `true` makes it manual-only and prevents subagent preload. | | `model` / `effort` | Omit to inherit unless a stable task requirement justifies an override. | -| `context: fork` / `agent` | Avoid for passive reference content; use only when the body is an actionable isolated task. | +| `context: fork` | Avoid for passive reference content; use only when the body is an actionable isolated task. | | `shell` / `hooks` | Add only after verifying local runtime support and a concrete need. | Place matching information in frontmatter. The body is invisible until the skill @@ -136,8 +136,7 @@ must not require another chain of references to become usable. - [ ] `name` matches the directory and uses lowercase kebab-case. - [ ] Frontmatter delimiters and YAML structure are valid. - [ ] Description front-loads triggers and stays within the listing limit. -- [ ] Positive and negative trigger examples distinguish it from agents and task - skills. +- [ ] Positive and negative trigger examples distinguish it from task skills. - [ ] `paths:` globs are narrow and match real files when present. - [ ] Every declared argument, tool, script, reference, and asset is used and exists. - [ ] SKILL.md is below 500 lines; references over 100 lines have a contents list. diff --git a/.oh/skills/delegate/SKILL.md b/.oh/skills/delegate/SKILL.md index 48121cfc..344e5adf 100644 --- a/.oh/skills/delegate/SKILL.md +++ b/.oh/skills/delegate/SKILL.md @@ -3,8 +3,9 @@ name: delegate description: | TRIGGER when: asked to delegate work, execute or parallelize a plan, "run this plan", "delegate this", or after /prd or plan creation. Decomposes work by - dependency, launches worker sub-agents in parallel waves, validates completion, - and reports results while preserving failure isolation and recursion limits. + dependency, launches bounded provider-native workers in parallel waves, validates + completion, and reports results while preserving failure isolation and recursion + limits. Keeps work in the active session when phases share substantial context. argument-hint: "[--plan <path>] [--dry-run]" --- @@ -16,6 +17,28 @@ completes before the next begins. Results are collected, validated, and reported **Core principle: maximize parallelism while respecting dependencies absolutely.** +## When a worker is justified + +> Use a worker only when the task is self-contained and gains from parallelism, +> isolated context, restricted tools, or containment of verbose disposable output. +> Keep work in the active session when phases share substantial context or require +> iterative refinement. + +A worker is a **bounded execution context**, not a project role. The active coding +agent is the runtime and stays the owner of the work; skills — `/architect`, +`/spec`, `/audit`, `/retro` — are how it adopts a role. Delegation buys isolation +or parallelism and nothing else. + +| Delegate it | Keep it in the active session | +|---|---| +| Independent parallel research or source sweeps | Planning, implementation, and testing that share one evolving picture | +| Verbose disposable output — logs, search dumps, test runs | Work that needs iterative refinement against operator feedback | +| Disjoint file ownership with no shared mutable state | Two tasks that touch the same file | +| A deliberate tool or permission restriction | Anything whose result you would have to re-derive to use | + +Do not invent named architectural roles for workers. `/delegate` owns fan-out +policy; other skills must not grow a competing worker hierarchy beside it. + ## Worker model and thinking policy Apply this policy to every worker: @@ -179,7 +202,7 @@ Worker configuration: or `xhigh`). Never pass `max`. If unsupported, use the nearest supported thinking level while keeping the inherited or explicitly overridden model unchanged. - **run_in_background**: true (for waves with 2+ tasks) -- **subagent_type** (read-only trap): the `implementer`, `pm`, and `critic` sub-agent types are **read-only** (`tools: Read, Glob, Grep, Bash` — no `Write`/`Edit`) and will **silently make zero file changes** if a worker is told to create or edit files. For any worker that must `Write`/`Edit` files, set `subagent_type: general-purpose` (or `claude`) in the `Agent` tool call; reserve `implementer`/`pm`/`critic` for analysis-only workers. +- **subagent_type**: use a **provider-native built-in** type only. This repository defines no project agents, so no `subagent_type` resolves to a repository file. For a worker that must `Write`/`Edit`, use `general-purpose` (or `claude`); for a read-only sweep whose verbose output should stay out of this context, use a read-only built-in such as `Explore`. Verify a type is offered by the running provider before naming it — an unrecognized `subagent_type` either errors or silently degrades. Never name a type on the assumption that a repository agent definition backs it. **a.1) Recursion-authorization gate** @@ -287,9 +310,13 @@ level without changing models. | Resource | Path | |----------|------| -| Agent: Implementer | `.claude/agents/implementer.md` — read-only (no Write/Edit) | -| Agent: Critic | `.claude/agents/critic.md` — read-only (no Write/Edit) | -| Agent: PM | `.claude/agents/pm.md` — read-only (no Write/Edit) | -| Agent: Council | `.claude/agents/council.md` | - -The `implementer`/`pm`/`critic` agent types above are read-only and will silently make zero file changes. For any worker that must `Write`/`Edit` files, set `subagent_type: general-purpose` (or `claude`) — both are built-in agent types with no agent-definition file, so there is no `.claude/agents/` path to reference. +| Write-capable worker | `subagent_type: general-purpose` (or `claude`) — provider built-in | +| Read-only sweep worker | a read-only provider built-in such as `Explore` | +| Worker boundary rule | **When a worker is justified** above | +| Architecture decisions | `/architect` — runs inline, never as a worker identity | + +There is **no** repository-authored agent catalog: no `.oh/agents/` pack and no +`.claude/agents/` or `.codex/agents/` provider mirror. Every `subagent_type` above +is a provider built-in with no definition file behind it, so there is no repository +path to cite. A read-only built-in makes zero file changes by design — never assign +one a task that must `Write` or `Edit`. diff --git a/.oh/skills/spec/references/plan.md b/.oh/skills/spec/references/plan.md index 55076d7d..1b344e78 100644 --- a/.oh/skills/spec/references/plan.md +++ b/.oh/skills/spec/references/plan.md @@ -38,6 +38,26 @@ folder for `/spec execute` to consume. --- +## Architecture-significance check + +Before deriving the slug, judge once whether the topic is architecture-significant +— it materially changes system or module boundaries, the execution or ownership +model, persistent state, a security or isolation boundary, a public API or +compatibility contract, a lifecycle, provider portability, shared vocabulary, +cross-skill control-plane behavior, or introduces or retires a reusable +abstraction. + +- **Not significant** (the common case): continue straight into the pipeline. Most + changes are ordinary and need no architecture pass. +- **Significant**: run `/architect <topic>` inline in this session first, then plan + against its accepted recommendation. `/architect` decides what the system should + become; `plan` turns that direction into the task folder. + +This is a one-line judgment, not a phase. `/architect` is never mandatory, never +spawns a session, and never owns the build. + +--- + ## The pipeline Run these in order; each is an existing primitive — compose, don't re-derive. diff --git a/.oh/skills/strategic-proposal/SKILL.md b/.oh/skills/strategic-proposal/SKILL.md index 319cd6b2..4e0dd9cb 100644 --- a/.oh/skills/strategic-proposal/SKILL.md +++ b/.oh/skills/strategic-proposal/SKILL.md @@ -121,21 +121,25 @@ Assemble a structured markdown briefing to pass to ALL 5 experts: Launch 5 Agent tool calls **in a single message** for parallel execution: -| Expert | Agent file | Perspective | Model | -|--------|-----------|-------------|-------| -| **Product** | `.claude/agents/expert-product.md` | Data models, APIs, features | sonnet | -| **Docs** | `.claude/agents/expert-docs.md` | Documentation, fork showcase UX | sonnet | -| **Security** | `.claude/agents/expert-security.md` | Auth, headers, access control | sonnet | -| **Registry** | `.claude/agents/expert-registry.md` | Docker registry, licensing | sonnet | -| **Agent Systems** | `.claude/agents/expert-agent-systems.md` | Agent autonomy, Ralph loop | sonnet | - -Pass each expert the Current State Briefing + instruction to read their agent definition file and follow its output format. +| Expert | Perspective | Model | +|--------|-------------|-------| +| **Product** | Data models, APIs, features | sonnet | +| **Docs** | Documentation, fork showcase UX | sonnet | +| **Security** | Auth, headers, access control | sonnet | +| **Registry** | Docker registry, licensing | sonnet | +| **Agent Systems** | Agent autonomy, Ralph loop | sonnet | + +Each expert is a **prompt for a bounded provider-native worker**, not a repository +agent definition. Use `subagent_type: general-purpose` (or a read-only built-in when +the expert only reads) and put the perspective, the Current State Briefing, and the +required output format in the prompt itself. There is no `.claude/agents/` file to +read — this repository authors no project agents. Experts operate **independently** — they do NOT see each other's proposals. ### 5. Strategic Council DRAFT -Launch a single Agent tool call using the council agent (`.claude/agents/strategic-council.md`): +Launch a single Agent tool call for the council worker — a provider-native worker whose prompt carries the council role: Pass the council: - All 5 expert proposals @@ -147,7 +151,7 @@ Save the council's draft output for the next step. ### 6. Strategic Critic review -Launch a single Agent tool call using the strategic critic (`.claude/agents/strategic-critic.md`): +Launch a single Agent tool call for the strategic critic — a provider-native worker whose prompt carries the adversarial role: Pass the critic: - The council's DRAFT roadmap @@ -159,7 +163,7 @@ The critic provides **adversarial backpressure** — its job is to find what's w ### 7. Strategic Council FINAL -Launch a second Agent tool call using the council agent (`.claude/agents/strategic-council.md`): +Launch a second Agent tool call for the council worker, reusing the same council role prompt: Pass the council: - Its own DRAFT roadmap from step 5 @@ -204,12 +208,10 @@ gh issue edit <NUMBER> --repo mifunedev/openharness --body "<council output>" ### Key Resources -| Resource | Path | -|----------|------| -| Expert: Product | `.claude/agents/expert-product.md` | -| Expert: Docs | `.claude/agents/expert-docs.md` | -| Expert: Security | `.claude/agents/expert-security.md` | -| Expert: Registry | `.claude/agents/expert-registry.md` | -| Expert: Agent Systems | `.claude/agents/expert-agent-systems.md` | -| Strategic Council | `.claude/agents/strategic-council.md` | -| Strategic Critic | `.claude/agents/strategic-critic.md` | +| Resource | Where it lives | +|----------|----------------| +| Expert roles (Product, Docs, Security, Registry, Agent Systems) | Prompts written inline in step 4 of this skill | +| Strategic Council role | Prompt written inline in steps 5 and 7 of this skill | +| Strategic Critic role | Prompt written inline in step 6 of this skill | +| Worker type for every role above | A provider built-in (`general-purpose`, or a read-only built-in) — no repository agent file backs any of them | +| Worker boundary policy | `/delegate` — **When a worker is justified** | diff --git a/.oh/skills/worktrees/SKILL.md b/.oh/skills/worktrees/SKILL.md index be53f441..1d922ba9 100644 --- a/.oh/skills/worktrees/SKILL.md +++ b/.oh/skills/worktrees/SKILL.md @@ -31,7 +31,7 @@ under `.worktrees/`. **Track/preserve** deliberate, durable harness changes, including: -- `.oh/skills/`, `.oh/agents/`, `.oh/hooks/`, `.oh/scripts/`, `.devcontainer/`, +- `.oh/skills/`, `.oh/hooks/`, `.oh/scripts/`, `.devcontainer/`, `.github/`, docs, templates, and supported configuration defaults. - `.worktrees/AGENTS.md`, `projects/AGENTS.md`, and other lifecycle diff --git a/AGENTS.md b/AGENTS.md index 99c6f363..fd2c1858 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -36,7 +36,7 @@ change agent-owned files after initial scaffolding. ### 2. Coding-harness choice does not change the workspace Claude Code, Codex, Pi, and other coding harnesses use the same project state and -shared primitives. Canonical agents, task procedures, and hooks live under `.oh/`. +shared primitives. Canonical skills, task procedures, and hooks live under `.oh/`. Compatibility directories expose those primitives through symlinks. Change the canonical `.oh/` source. Do not patch a generated mirror. @@ -174,7 +174,7 @@ The repository has one sandbox definition and four control-plane areas: This directory stays outside the `.oh/` control plane. - `.oh/scripts/`, `.oh/install/`, and `.oh/cli/` implement lifecycle and runtime behavior. -- `.oh/skills/`, `.oh/agents/`, and `.oh/hooks/` hold portable agent primitives. +- `.oh/skills/` and `.oh/hooks/` hold portable primitives; skills encode roles. - `.oh/tasks/` holds task-specific plans, graphs, progress, and evidence. - `.oh/evals/` holds regression probes and capability benchmarks. diff --git a/CHANGELOG.md b/CHANGELOG.md index ba4b8c29..682b137e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -27,11 +27,19 @@ Update policy and release automation live in [`/git`](.claude/skills/git/SKILL.m - **BREAKING:** Retire the DeepAgents harness — `deepagents-cli` is deprecated upstream. `install.deepagents` is no longer a settable oh.json field ([#910](https://github.com/mifunedev/openharness/issues/910)). - **BREAKING:** Retire Prime Agent and its `.prime/agent/` provider surface; `oh harness install prime-agent` is no longer available ([#918](https://github.com/mifunedev/openharness/issues/918)). - **BREAKING:** Retire the `projectRoot` / `OH_PROJECT_ROOT` config knob — the checkout is fixed at `/home/sandbox/harness`, nested inside the home mount ([#898](https://github.com/mifunedev/openharness/issues/898)). +- **BREAKING:** Retire `.oh/agents/` with its `.claude/agents` and `.codex/agents` provider symlinks; the `oh` payload manifest no longer ships an `agents/**` pack ([#929](https://github.com/mifunedev/openharness/issues/929)). +- **BREAKING:** Retire `/builder agent` and its authoring reference; `/builder` now dispatches `skill`, `command`, and `rule` only ([#929](https://github.com/mifunedev/openharness/issues/929)). +- Remove the stale `implementer`/`critic`/`pm`/`council` worker types from `/delegate` and the dangling expert, council, and critic agent paths from `/strategic-proposal` ([#929](https://github.com/mifunedev/openharness/issues/929)). +- Retire `rl-delegation-write-worker.sh`; `delegate-worker-boundary.sh` carries its read-only-worker lesson forward ([#929](https://github.com/mifunedev/openharness/issues/929)). +- Retire the last `Advisor` role noun from `/blog`, which told the reader to "use Advisor" for a role that no longer exists; the briefing is `/delegate`'s ([#929](https://github.com/mifunedev/openharness/issues/929)). ### Changed - `.oh/logs/` carries an `AGENTS.md` with a `CLAUDE.md` symlink instead of a `README.md`, matching the directories whose contents are produced apart from the root context. ([#924](https://github.com/mifunedev/openharness/issues/924)) - **BREAKING:** `/spec execute` no longer launches a coding agent; the agent that runs it is the single implementation owner through the final PR gates ([#928](https://github.com/mifunedev/openharness/issues/928)). - Task identity and `RUNNING` state depend on `.oh/tasks/<slug>/` alone, never on a terminal session, tab, or pane ([#928](https://github.com/mifunedev/openharness/issues/928)). +- `/delegate` keeps work in the active session when phases share substantial context and spawns provider-native workers only for self-contained, isolated, or parallel work ([#929](https://github.com/mifunedev/openharness/issues/929)). +- `/spec plan` judges architecture significance once and routes a significant topic through `/architect` before planning ([#929](https://github.com/mifunedev/openharness/issues/929)). +- `docs/glossary.md` defines coding agent, skill, worker/subagent, rule, and RFC/ADR as five distinct terms ([#929](https://github.com/mifunedev/openharness/issues/929)). ### Removed - **BREAKING:** Retire the automated `/spec` Advisor handoff — detached tmux launch, `agent-spec-*` sessions and their sweep kill, pane logging, runner fallbacks ([#928](https://github.com/mifunedev/openharness/issues/928)). @@ -49,6 +57,8 @@ Update policy and release automation live in [`/git`](.claude/skills/git/SKILL.m - Add a `--patterns` mode to `/wiki query` that filters on `kind:`, reads up to five pattern entries, and ranks them by term-hit count before recency. ([#916](https://github.com/mifunedev/openharness/pull/916)) - Add a fault-injection requirement to the probe contract: a probe is not green until its REGRESSION branch has been driven against a broken input. ([#916](https://github.com/mifunedev/openharness/pull/916)) - Add capability task `CB-005` scoring whether a lesson reaches a validated skill change; two runs score 0.67 then 1.33, moving the suite mean to 1.44 over a changed task set. ([#916](https://github.com/mifunedev/openharness/pull/916)) +- Add `/architect`, an inline architecture-decision skill that classifies significance, grounds analysis in repository sources, and returns one Architecture Brief ([#929](https://github.com/mifunedev/openharness/issues/929)). +- Add six tier-A probes covering the `/architect` contract, roles-as-skills, the retired builder agent type, the `/delegate` worker boundary, the absent agent catalog, and RFC/ADR reuse ([#929](https://github.com/mifunedev/openharness/issues/929)). ### Fixed - Fix `/wiki lint` generating the corpus index from the working tree instead of the git-tracked set, which made any untracked scratch entry a `wiki-readme-index.sh` regression. ([#916](https://github.com/mifunedev/openharness/pull/916)) diff --git a/README.md b/README.md index a5d650ec..5065dd2f 100644 --- a/README.md +++ b/README.md @@ -159,9 +159,9 @@ defaults. Details: [lifecycle commands](docs/lifecycle-commands.md#vs-code-reope ## 🧩 How the primitive pack ships -Open Harness vendors the shared skills/agents/hooks primitive pack directly into the `.oh/` control plane: `.oh/skills/`, `.oh/agents/`, `.oh/hooks/`, and `.oh/skills.lock` are tracked as ordinary files in this repo. The `oh` CLI lays them down during `oh init`/`oh update`, so a fresh checkout has the skills immediately — no submodule, no recursive clone, no network step. +Open Harness vendors the shared skills/hooks primitive pack directly into the `.oh/` control plane: `.oh/skills/`, `.oh/hooks/`, and `.oh/skills.lock` are tracked as ordinary files in this repo. Skills are the reusable-behavior primitive; the harness ships no repository-authored agent definitions, and provider-native sub-agents remain available as a bounded execution primitive through `/delegate`. The `oh` CLI lays them down during `oh init`/`oh update`, so a fresh checkout has the skills immediately — no submodule, no recursive clone, no network step. -Provider surfaces are symlinks into `.oh/`: `.pi/skills`, `.claude/skills`, and `.codex/skills` point at `.oh/skills`; `.claude/agents` → `.oh/agents`; `.claude/hooks` → `.oh/hooks`. `.pi/` itself remains the Pi provider surface in v1. +Provider surfaces are symlinks into `.oh/`: `.pi/skills`, `.claude/skills`, and `.codex/skills` point at `.oh/skills`; `.claude/hooks` → `.oh/hooks`. `.pi/` itself remains the Pi provider surface in v1. ## 🚀 Use it diff --git a/docs/README.md b/docs/README.md index 29877e29..7ea36995 100644 --- a/docs/README.md +++ b/docs/README.md @@ -28,7 +28,7 @@ run `oh harness install hermes`, then `hermes setup`. ## How the primitive pack ships -Open Harness vendors the shared skills/agents/hooks primitive pack directly into the `.oh/` control plane (`.oh/skills/`, `.oh/agents/`, `.oh/hooks/`, `.oh/skills.lock`), tracked as ordinary files — the `oh` CLI lays them down during `oh init`/`oh update`, so a fresh checkout has them with no submodule or network step. Provider paths such as `.pi/skills`, `.claude/skills`, and `.codex/skills` are symlinks into `.oh/skills`; `.pi/` remains a provider surface for v1. +Open Harness vendors the shared skills/hooks primitive pack directly into the `.oh/` control plane (`.oh/skills/`, `.oh/hooks/`, `.oh/skills.lock`), tracked as ordinary files — the `oh` CLI lays them down during `oh init`/`oh update`, so a fresh checkout has them with no submodule or network step. Provider paths such as `.pi/skills`, `.claude/skills`, and `.codex/skills` are symlinks into `.oh/skills`; `.pi/` remains a provider surface for v1. ## Setup & first steps diff --git a/docs/glossary.md b/docs/glossary.md index 3d0a52a5..ad7d949e 100644 --- a/docs/glossary.md +++ b/docs/glossary.md @@ -12,7 +12,7 @@ Terms are listed alphabetically below. These names describe separate layers, not interchangeable jobs: - **Model** — The LLM selected by a provider; it proposes text and tool calls, while the surrounding agent, harness, and policy decide where those requests run and what is allowed. See the **model** glossary entry. -- **Agent / CLI** — The process that wraps a model with tools, instructions, and session state, such as Claude Code, Codex, Pi, or a scoped worker defined under `.oh/agents/`. See the **agent** glossary entry. +- **Agent / CLI** — The process that wraps a model with tools, instructions, and session state, such as Claude Code, Codex, or Pi. It is the runtime: the active session owns the work. See the **agent** glossary entry. - **Harness** — The repo, Docker sandbox, and `.oh/` control plane that give agents a reproducible workspace and lifecycle. See the **harness** glossary entry. - **Loop** — A repeated workflow that the harness drives until a terminal state, such as the `/spec execute` implementation cycle ending after every story passes. See the **loop** and **terminal state** glossary entries. - **Policy** — The provider-portable rules, skills, and hooks that constrain agent behavior and tool use. See the **policy** and **tool** glossary entries. @@ -20,12 +20,12 @@ These names describe separate layers, not interchangeable jobs: ## Terms -- **agent** — A model-driven worker that reads the workspace and does the task: - an agent CLI (Claude Code, Codex, Pi, …) running inside the sandbox, or a - scoped sub-agent defined under `.oh/agents/` that the orchestrator spawns for - a bounded job. No sub-agent ships with the harness today; `.oh/agents/` is the - canonical place to define one. - Source: [`.oh/agents/README.md`](../.oh/agents/README.md). +- **agent / coding agent** — The running model-plus-tools process that reads the + workspace and does the task: Claude Code, Codex, Pi, or another coding harness + running inside the sandbox. The agent is the **runtime and the owner of the + work**; a role never implies a separate session or process. The repository + authors no agent definition files — a durable role is a **skill** the active + session adopts. Source: [`AGENTS.md`](../AGENTS.md). - **artifact** — Any inspectable file a workflow stage produces and a later stage or a human then consumes. The canonical example is the `.oh/tasks/<slug>/` task @@ -78,11 +78,23 @@ These names describe separate layers, not interchangeable jobs: hook-enforced security rules. Source: [`.oh/skills/git/SKILL.md`](../.oh/skills/git/SKILL.md). -- **primitive** — A reusable unit from the shared pack — skills, agents, and - hooks — vendored directly into the `.oh/` control plane and exposed to each - provider (`.claude/`, `.codex/`, `.pi/`) via symlinks into `.oh/`. +- **primitive** — A reusable unit from the shared pack — skills and hooks — + vendored directly into the `.oh/` control plane and exposed to each provider + (`.claude/`, `.codex/`, `.pi/`) via symlinks into `.oh/`. Source: [`README.md`](README.md) (the primitive pack under `.oh/skills/`, - `.oh/agents/`, `.oh/hooks/`). + `.oh/hooks/`). + +- **rfc / adr** — A durable architecture decision, recorded as a GitHub issue + titled `RFC:` or `ADR:` and indexed on the RFC/ADR page. Three states — + `Draft`, `Accepted`, `Superseded` — and no further taxonomy. This is the only + decision store; `/architect` points durable decisions here rather than + creating another one. Source: [`docs/rfcs/README.md`](rfcs/README.md). + +- **rule** — Ambient repository policy an agent carries without invoking + anything: an `AGENTS.md` (aliased `CLAUDE.md` for provider compatibility) that + applies to every task under its directory, or a path-scoped reference skill. + Distinct from a skill, which is invoked for a job. + Source: [`AGENTS.md`](../AGENTS.md). - **runtime** — The always-on machinery that wakes the agent on a schedule: a tiny croner that reads scheduled-agent definitions from `crons/` and fires @@ -104,8 +116,11 @@ These names describe separate layers, not interchangeable jobs: - **skill** — A packaged, invocable workflow (a `SKILL.md` plus optional references and scripts) that an agent runs via the Skill tool or a `/name` - slash command; the shared set lives under `.oh/skills/`. - Source: [`.oh/skills/`](../.oh/skills/). + slash command; the shared set lives under `.oh/skills/`. **Skills are the + canonical primitive for a reusable role, procedure, checklist, constraint set, + or body of domain judgment** — `/architect`, `/spec`, `/audit`, `/retro`, and + `/delegate` are roles encoded this way, loaded into the active session rather + than spawned as separate identities. Source: [`.oh/skills/`](../.oh/skills/). - **terminal state** — The end state that closes a workflow cycle. `/spec execute` completes implementation only after every story passes; the operative path @@ -121,6 +136,14 @@ These names describe separate layers, not interchangeable jobs: over Claude and Pi session traces to score prompts by outcome. Source: [`mine-traces.mjs`](../.oh/skills/prompt-miner/scripts/mine-traces.mjs). +- **worker / subagent** — An optional bounded, isolated execution context the + active agent spawns for one self-contained job — parallelism, context + isolation, verbose disposable output, or a deliberate tool restriction. It is + an execution primitive, not a project role: workers are provider built-ins + with no repository definition file, and `/delegate` owns when one is + justified. Source: + [`.oh/skills/delegate/SKILL.md`](../.oh/skills/delegate/SKILL.md). + - **worktree** — A separate git working directory under `.worktrees/` that isolates a branch so parallel work doesn't collide; the `/worktrees` skill manages their lifecycle and `/spec execute` builds each task in one. diff --git a/docs/harness-manifest.md b/docs/harness-manifest.md index 8879ba2f..ac42f572 100644 --- a/docs/harness-manifest.md +++ b/docs/harness-manifest.md @@ -20,8 +20,7 @@ control-plane surfaces described in the [`.oh/` directory layout](oh-directory-l name: openharness version: 1 -agents: - definitions: .oh/agents/ +primitives: skills: .oh/skills/ hooks: .oh/hooks/ @@ -40,8 +39,10 @@ policies: - `name` and `version` are plain labels for humans. They do not imply a manifest version registry. -- `agents` points at the real provider-portable primitive pack: agent - definitions, skills, and hooks already live under `.oh/`. +- `primitives` points at the real provider-portable primitive pack: skills and + hooks already live under `.oh/`. There is no agent-definitions entry — skills + are the reusable-role primitive, and provider-native sub-agents are a bounded + execution choice made by `/delegate`, not a repository artifact. - `loops` points at today's scheduled cron prompts and task artifact directory. `/spec execute` owns implementation directly; it does not delegate to a separate implementation process. diff --git a/docs/oh-directory-layout.md b/docs/oh-directory-layout.md index 2d85dac8..47fa5407 100644 --- a/docs/oh-directory-layout.md +++ b/docs/oh-directory-layout.md @@ -22,7 +22,6 @@ Every entry below is present in a fresh clone unless noted otherwise. | `README.md` | file | Namespace anchor (keeps `.oh/` in a fresh clone) and the governing-principle doc for the control plane. | Humans; shipped forward via `manifest.json`. | | `manifest.json` | file | `oh update` payload allowlist — an `{ include, rootInclude, exclude }` document. `include` globs are relative to `.oh/` and land in `<target>/.oh/`; `rootInclude` globs are relative to the repo root and land in `<target>/` (today: `crons/**`). | `oh update` (`.oh/cli`). | | `skills.lock` | file | Pinned lockfile for the vendored skill pack (`skills.v1` schema). | `.oh/scripts/link-providers.sh` (vendored-pack validation). | -| `agents/` | dir | Provider-portable sub-agent definitions, one `<name>.md` per agent. Currently empty — no sub-agent ships with the harness. Audit routing belongs to `/audit`, not an agent registry. | Agent providers via symlinks (`.claude/agents` → `.oh/agents`); the Agent tool. | | `cli/` | dir | The in-tree `oh` CLI — a standalone npm package built into the image as `/opt/oh`. | `npm --prefix .oh/cli`; the `oh` binary (`oh init` / `oh update`). | | `evals/` | dir | The fitness-function suite — regression `probes/` (incl. `cc-safety-net-wiring.sh`, the destructive-command guard wiring probe), the `capability/` benchmark, trajectory `datasets/`, and the `RESULTS.md` scoreboard. | `/eval` and the `.oh/scripts` eval runner. | | `hooks/` | dir | Provider-portable **secret-exposure** hook scripts (`deny-env-dump.sh`, `deny-secret-paths.sh`, `notify_slack.sh`, `warn-devtcp.sh`). The complementary **destructive-command** guard (cc-safety-net) is not a script here — it is a global binary baked into the image plus guard-wrapped entries in the provider configs (`.claude/settings.json`, `.codex/hooks.json`, the `npm:cc-safety-net` package in `.pi/settings.json`); see [security-considerations.md §3](security-considerations.md). | Agent providers via symlinks (`.claude/hooks` → `.oh/hooks`). | diff --git a/docs/rfcs/README.md b/docs/rfcs/README.md index cc7875da..fbb81209 100644 --- a/docs/rfcs/README.md +++ b/docs/rfcs/README.md @@ -30,6 +30,7 @@ No stage gates, no editors, no numbering scheme beyond the GitHub issue number. | [RSI survey mapping](rfc-rsi-survey-mapping.md) | Draft | #525 companion. Maps the recursive-self-improvement survey ([arXiv 2607.07663](https://arxiv.org/html/2607.07663v1)) onto this repository. Holds the taxonomy placement, the verification-hierarchy rung assignment for the harness's own signals, five findings the repository already evidences, and two proposed roadmap children. Decision artifact — no runtime change. | | [#532](https://github.com/mifunedev/openharness/issues/532) | [Accepted — resolved lightweight; heavy scope deferred](adr-0001-standards-scope.md) | Standards process — keep the lightweight RFC / ADR convention; defer the full taxonomy, registries, lifecycle, and conformance profiles until a concrete future issue needs them. | | [#592](https://github.com/mifunedev/openharness/issues/592) | Draft | Runtime support — the A1/A2/A3 axis taxonomy and the "supported runtime" contract; the [companion spec](rfc-runtime-support.md) holds the fit matrix, the Cloudflare fit, and the Crabbox control-plane comparison. Implementation epic [#591](https://github.com/mifunedev/openharness/issues/591). | +| [#929](https://github.com/mifunedev/openharness/issues/929) | Accepted | Skills are the canonical role/procedure primitive and the active coding agent is the runtime; adds `/architect` as an inline skill, keeps provider-native sub-agents as a bounded execution primitive behind `/delegate`, and retires repository-authored project agents. | | [#733](https://github.com/mifunedev/openharness/issues/733) | Draft | Brain/hands boundary — the [Phase-0 decisions](rfc-brain-hands-boundary.md) behind the `ExecutionTarget` seam: the brain/hands split, the eval capability rule, the four-class state taxonomy (with the Hermes known-violation), the identical-path workspace stance, and synchronous `attach()` in `contractVersion: 1`. Sole authority for those decisions — cite, do not restate. Epic [#731](https://github.com/mifunedev/openharness/issues/731). | ## Decision records From fb681d1c7068843efc716070392654c5ccbddf84 Mon Sep 17 00:00:00 2001 From: Ryan Eggleston <kre8mymedia@gmail.com> Date: Mon, 31 Aug 2026 20:27:55 -0600 Subject: [PATCH 20/23] FROM task/933-retire-cron-worktree-isolation TO development (#934) * task: retire cron worktree isolation Remove the worktree: frontmatter key, .worktrees/cron/ per-fire worktrees, the CRON_WORKTREE export, the fallback-worktree prune/inspect/reap machinery, and their log states. Every cron now fires in the shared root checkout under the id-scoped overlap lock. Tracks #933. Submitted-by: Claude * task: drop the CRON_WORKTREE root seam from audit-run Nothing sets CRON_WORKTREE now that cron worktree isolation is retired. audit-run resolves its root from the script path alone; the root-contract probe runs a fixture-local copy of the script instead of overriding the root through the environment. Tracks #933. Submitted-by: Claude --- .oh/README.md | 3 +- .oh/evals/probes/audit-run-root-contract.sh | 42 +-- .oh/evals/probes/worktrees-layout.sh | 4 +- .oh/scripts/__tests__/cron-runtime.test.ts | 278 +------------------- .oh/scripts/cron-runtime.ts | 217 +-------------- .oh/skills/audit/scripts/audit-run.sh | 6 +- .oh/skills/spec/references/execute.md | 14 +- .worktrees/AGENTS.md | 1 - CHANGELOG.md | 1 + crons/AGENTS.md | 17 +- crons/prompt-miner.md | 15 +- 11 files changed, 58 insertions(+), 540 deletions(-) diff --git a/.oh/README.md b/.oh/README.md index df5ba288..ccff2096 100644 --- a/.oh/README.md +++ b/.oh/README.md @@ -74,8 +74,7 @@ a symlink and nothing reads the bare `tasks/` path anymore. The ignored worktree root briefly lived at `.oh/worktrees/` and moved back **out** to the repo root as `.worktrees/`, with no back-compat symlink in either -direction. The location is a fixed convention rather than a setting, and cron -worktree isolation uses `.worktrees/cron/`. +direction. The location is a fixed convention rather than a setting. Clones of non-harness repositories, formerly `.oh/worktrees/project/<owner>/<repo>/`, now live at `projects/<owner>/<repo>/`, and each keeps its own worktrees at `projects/<owner>/<repo>/.worktrees/`. Both roots diff --git a/.oh/evals/probes/audit-run-root-contract.sh b/.oh/evals/probes/audit-run-root-contract.sh index 1c8bffa4..9cc95bf6 100755 --- a/.oh/evals/probes/audit-run-root-contract.sh +++ b/.oh/evals/probes/audit-run-root-contract.sh @@ -5,7 +5,7 @@ set -euo pipefail unset AUDIT_RUN_ID AUDIT_ROOT AUDIT_TMP_ROOT AUDIT_EVIDENCE_PATH \ AUDIT_ROUTE AUDIT_TARGET AUDIT_TARGET_ARGS_JSON AUDIT_AGENT_COMMAND_JSON -REPO="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)"; RUN="$REPO/.oh/skills/audit/scripts/audit-run.sh" +REPO="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" tmp=$(mktemp -d); tmpdir=$(mktemp -d); trap 'rm -rf "$tmp" "$tmpdir"' EXIT mkdir -p "$tmp/.oh/skills/audit/references" "$tmp/.oh/scripts" for route in implementation pr prs harness context skills eval-quality drift full; do @@ -16,11 +16,13 @@ printf '# private external route\n' >"$tmp/.oh/skills/audit/references/external- cp "$REPO/.oh/scripts/locked-append.sh" "$tmp/.oh/scripts/locked-append.sh" mkdir -p "$tmp/.oh/skills/audit/scripts" cp "$REPO/.oh/skills/audit/scripts/audit-evidence.sh" "$tmp/.oh/skills/audit/scripts/audit-evidence.sh" +cp "$REPO/.oh/skills/audit/scripts/audit-run.sh" "$tmp/.oh/skills/audit/scripts/audit-run.sh" +RUN="$tmp/.oh/skills/audit/scripts/audit-run.sh" cat >"$tmp/complete-driver" <<'DRIVER' #!/usr/bin/env bash "$AUDIT_ROOT/.oh/skills/audit/scripts/audit-evidence.sh" complete TEST-COMPLETE DRIVER -chmod +x "$tmp/complete-driver" "$tmp/.oh/skills/audit/scripts/audit-evidence.sh" +chmod +x "$tmp/complete-driver" "$tmp/.oh/skills/audit/scripts/audit-evidence.sh" "$RUN" git -C "$tmp" init -q; git -C "$tmp" config user.email test@example.invalid; git -C "$tmp" config user.name test git -C "$tmp" add .; git -C "$tmp" commit -qm init fail(){ echo "REGRESSION: $*" >&2; exit 1; } @@ -30,17 +32,17 @@ still_running(){ [[ -n $st && ${st#Z} == "$st" ]] } export TMPDIR="$tmpdir" -set +e; usage_out=$(CRON_WORKTREE="$tmp" CRON_LOG_ROOT="$tmp" bash "$RUN" nope 2>&1); usage_rc=$?; set -e +set +e; usage_out=$(bash "$RUN" nope 2>&1); usage_rc=$?; set -e [[ $usage_rc -eq 64 ]] || fail 'unknown target accepted/wrong usage rc' [[ ${usage_out%%$'\n'*} == 'usage: /audit <implementation|pr|prs|harness|context|skills|eval-quality|drift|full> [target options]' ]] || fail 'usage first line is not exact' for route in implementation pr prs harness context skills eval-quality drift full; do grep -q "^| $route |" <<<"$usage_out" || fail "usage table missing $route"; done [[ -z $(find "$tmpdir" -mindepth 1 -print -quit) && ! -e "$tmp/.oh/logs" ]] || fail 'invalid usage created lifecycle state' -if CRON_WORKTREE="$tmp" CRON_LOG_ROOT="$tmp" bash "$RUN" harness --external source --focus x -- true >/dev/null 2>&1; then fail 'external/focus conflict accepted'; fi -if CRON_WORKTREE="$tmp" CRON_LOG_ROOT="$tmp" bash "$RUN" harness --wiki-ingest -- true >/dev/null 2>&1; then fail 'external-only option reached survey mode'; fi -if CRON_WORKTREE="$tmp" CRON_LOG_ROOT="$tmp" bash "$RUN" implementation -- true >/dev/null 2>&1; then fail 'missing implementation slug accepted'; fi -if CRON_WORKTREE="$tmp" CRON_LOG_ROOT="$tmp" bash "$RUN" pr 7 --repo bad -- true >/dev/null 2>&1; then fail 'invalid focused repo accepted'; fi -if CRON_WORKTREE="$tmp" CRON_LOG_ROOT="$tmp" bash "$RUN" drift >/dev/null 2>&1; then fail 'missing route driver accepted'; fi -if CRON_WORKTREE="$tmp" CRON_LOG_ROOT="$tmp" bash "$RUN" drift -- true >/dev/null 2>&1; then fail 'true callback certified completion'; fi +if bash "$RUN" harness --external source --focus x -- true >/dev/null 2>&1; then fail 'external/focus conflict accepted'; fi +if bash "$RUN" harness --wiki-ingest -- true >/dev/null 2>&1; then fail 'external-only option reached survey mode'; fi +if bash "$RUN" implementation -- true >/dev/null 2>&1; then fail 'missing implementation slug accepted'; fi +if bash "$RUN" pr 7 --repo bad -- true >/dev/null 2>&1; then fail 'invalid focused repo accepted'; fi +if bash "$RUN" drift >/dev/null 2>&1; then fail 'missing route driver accepted'; fi +if bash "$RUN" drift -- true >/dev/null 2>&1; then fail 'true callback certified completion'; fi cat >"$tmp/fake-agent" <<'AGENT' #!/usr/bin/env bash prompt=${!#} @@ -55,14 +57,14 @@ leaked=$(printenv | grep -c '^AUDIT_' || true) printf 'route report\nAUDIT-EVIDENCE: DRIFT-OK\n' AGENT chmod +x "$tmp/fake-agent" -AUDIT_AGENT_COMMAND_JSON="[\"$tmp/fake-agent\"]" CRON_WORKTREE="$tmp" CRON_LOG_ROOT="$tmp" \ +AUDIT_AGENT_COMMAND_JSON="[\"$tmp/fake-agent\"]" \ bash "$RUN" drift -- "$REPO/.oh/skills/audit/scripts/route-driver.sh" >/dev/null \ || fail 'canonical production route driver did not publish correlated evidence (rc 8 = it leaked AUDIT_* into the agent)' -CRON_WORKTREE="$tmp" CRON_LOG_ROOT="$tmp" bash "$RUN" pr 7 --base stack-parent -- "$tmp/complete-driver" >/dev/null -CRON_WORKTREE="$tmp" CRON_LOG_ROOT="$tmp" bash "$RUN" prs --mine -- "$tmp/complete-driver" >/dev/null -CRON_WORKTREE="$tmp" CRON_LOG_ROOT="$tmp" bash "$RUN" full --repo owner/name -- "$tmp/complete-driver" >/dev/null +bash "$RUN" pr 7 --base stack-parent -- "$tmp/complete-driver" >/dev/null +bash "$RUN" prs --mine -- "$tmp/complete-driver" >/dev/null +bash "$RUN" full --repo owner/name -- "$tmp/complete-driver" >/dev/null [[ ! -e "$tmp/.oh/logs" ]] || fail 'a run wrote the deleted .oh/logs tier' -CRON_WORKTREE="$tmp" CRON_LOG_ROOT="$tmp" bash "$RUN" drift -- bash -c ' +bash "$RUN" drift -- bash -c ' [[ $AUDIT_ROUTE == "$AUDIT_ROOT/.oh/skills/audit/references/drift.md" ]] [[ ! -e "$AUDIT_ROOT/.oh/logs" ]] [[ $PWD == "$AUDIT_ROOT" ]] @@ -72,11 +74,11 @@ CRON_WORKTREE="$tmp" CRON_LOG_ROOT="$tmp" bash "$RUN" drift -- bash -c ' ' [[ $(<"$tmp/driver-marker") == route-ran ]] || fail 'selected route driver did not run/chdir or receive bindings' rm "$tmp/driver-marker" -rec=$(CRON_WORKTREE="$tmp" CRON_LOG_ROOT="$tmp" bash "$RUN" drift -- "$tmp/complete-driver" 2>&1 >/dev/null) +rec=$(bash "$RUN" drift -- "$tmp/complete-driver" 2>&1 >/dev/null) [[ $(grep -c '^audit -- run-id=' <<<"$rec") -eq 1 ]] || fail 'terminal run record did not follow driver' [[ ! -e "$tmp/.oh/logs" ]] || fail 'run record was written to the deleted .oh/logs tier' for n in 1 2; do - CRON_WORKTREE="$tmp" CRON_LOG_ROOT="$tmp" bash "$RUN" drift -- \ + bash "$RUN" drift -- \ bash -c 'printf "%s|%s" "$AUDIT_RUN_ID" "$AUDIT_ROOT" >"$AUDIT_TMP_ROOT/seen"; "$AUDIT_ROOT/.oh/skills/audit/scripts/audit-evidence.sh" complete DRIFT-OK' 2>"$tmp/rec.$n" & pids[n]=$! done wait "${pids[1]}"; wait "${pids[2]}" @@ -96,12 +98,12 @@ printf '%s\n' "$PWD" "$AUDIT_TARGET" "$AUDIT_TARGET_ARGS_JSON" "$@" >"$AUDIT_ROO "$AUDIT_ROOT/.oh/skills/audit/scripts/audit-evidence.sh" complete PRS-AUDIT-COMPLETE DRIVER chmod +x "$tmp/args-driver" -CRON_WORKTREE="$tmp" CRON_LOG_ROOT="$tmp" bash "$RUN" prs --label 'needs review' --base development -- "$tmp/args-driver" +bash "$RUN" prs --label 'needs review' --base development -- "$tmp/args-driver" mapfile -t seen <"$tmp/args-seen" [[ ${seen[0]} == "$tmp" && ${seen[1]} == prs && ${seen[2]} == '["--label","needs review","--base","development"]' ]] || fail 'named argument bindings differ' [[ ${seen[3]} == prs && ${seen[4]} == --label && ${seen[5]} == 'needs review' && ${seen[6]} == --base && ${seen[7]} == development ]] || fail 'driver argv not exact' set +e -failed_rec=$(CRON_WORKTREE="$tmp" CRON_LOG_ROOT="$tmp" bash "$RUN" drift -- bash -c 'exit 23' 2>&1 >/dev/null) +failed_rec=$(bash "$RUN" drift -- bash -c 'exit 23' 2>&1 >/dev/null) failed_rc=$? set -e [[ $failed_rc -eq 23 ]] || fail 'driver failure rc was not propagated' @@ -122,7 +124,7 @@ DRIVER chmod +x "$tmp/signal-driver" for sig in INT TERM HUP; do rm -f "$tmp/pids-seen" "$tmp/${sig,,}-seen" - SIGNAL_NAME=$sig CRON_WORKTREE="$tmp" CRON_LOG_ROOT="$tmp" bash "$RUN" drift -- "$tmp/signal-driver" 2>"$tmp/sig-rec" & wrapper=$! + SIGNAL_NAME=$sig bash "$RUN" drift -- "$tmp/signal-driver" 2>"$tmp/sig-rec" & wrapper=$! for _ in {1..50}; do [[ -s "$tmp/pids-seen" ]] && break; sleep .05; done [[ -s "$tmp/pids-seen" ]] || fail "$sig signal fixture did not start" read -r driver_pid grandchild_pid <"$tmp/pids-seen" @@ -142,7 +144,7 @@ while :; do sleep 1; done DRIVER chmod +x "$tmp/direct-driver" rm -f "$tmp/pids-seen" "$tmp/int-seen" -AUDIT_FORCE_DIRECT=1 CRON_WORKTREE="$tmp" CRON_LOG_ROOT="$tmp" bash "$RUN" drift -- "$tmp/direct-driver" & wrapper=$! +AUDIT_FORCE_DIRECT=1 bash "$RUN" drift -- "$tmp/direct-driver" & wrapper=$! for _ in {1..50}; do [[ -s "$tmp/pids-seen" ]] && break; sleep .05; done driver_pid=$(<"$tmp/pids-seen") kill -INT "$wrapper" diff --git a/.oh/evals/probes/worktrees-layout.sh b/.oh/evals/probes/worktrees-layout.sh index 05fb65e8..5b99939b 100755 --- a/.oh/evals/probes/worktrees-layout.sh +++ b/.oh/evals/probes/worktrees-layout.sh @@ -55,8 +55,8 @@ for sample in ".worktrees/feat/1-probe" "projects/an-owner/a-repo"; do fi done -if ! grep -Fq 'const WORKTREES_DIR = ".worktrees";' .oh/scripts/cron-runtime.ts; then - echo "REGRESSION: cron-runtime.ts does not pin WORKTREES_DIR to the .worktrees constant" >&2 +if grep -Fq 'WORKTREES_DIR' .oh/scripts/cron-runtime.ts; then + echo "REGRESSION: cron-runtime.ts references a worktree root it no longer owns" >&2 exit 1 fi if ! grep -Fq 'const CRONS_DIR = path.resolve("crons");' .oh/scripts/cron-runtime.ts; then diff --git a/.oh/scripts/__tests__/cron-runtime.test.ts b/.oh/scripts/__tests__/cron-runtime.test.ts index 0cc9ec04..84ef062d 100644 --- a/.oh/scripts/__tests__/cron-runtime.test.ts +++ b/.oh/scripts/__tests__/cron-runtime.test.ts @@ -22,7 +22,6 @@ import { buildTmuxWrapper, decideOverlap, fire, - inspectFallbackWorktree, isValidAgentBin, isValidCronId, isValidRemote, @@ -31,7 +30,6 @@ import { loadCrons, onJobError, parseCronFile, - pruneAndCountFallbackWorktrees, readFailureTail, reloadEntryForFire, remoteForRepo, @@ -43,7 +41,6 @@ import { scheduleAll, sighupHandler, tmuxSessionName, - worktreeInUse, } from "../cron-runtime"; vi.mock("node:fs", async (importOriginal) => { @@ -164,52 +161,27 @@ Heartbeat body. expect(parseCronFile(`---\nschedule: "* * * * *"\n---\nbody\n`, "c.md")?.repo).toBeUndefined(); }); - it("parses worktree: true and defaults to false otherwise", () => { - expect( - parseCronFile(`---\nschedule: "* * * * *"\nworktree: true\n---\nbody\n`, "a.md") - ?.worktree, - ).toBe(true); - expect( - parseCronFile(`---\nschedule: "* * * * *"\nworktree: false\n---\nbody\n`, "b.md") - ?.worktree, - ).toBe(false); - expect( - parseCronFile(`---\nschedule: "* * * * *"\n---\nbody\n`, "c.md")?.worktree, - ).toBe(false); - }); }); describe("decideOverlap", () => { - it("returns 'worktree' for a worktree cron regardless of lock state", () => { - for (const pidfileExists of [false, true]) { - for (const holderAlive of [false, true]) { - for (const overlap of [false, true]) { - expect( - decideOverlap({ overlap, worktree: true, pidfileExists, holderAlive }), - ).toBe("worktree"); - } - } - } - }); - it("runs (never skips) when overlap is allowed", () => { expect( - decideOverlap({ overlap: true, worktree: false, pidfileExists: true, holderAlive: true }), + decideOverlap({ overlap: true, pidfileExists: true, holderAlive: true }), ).toBe("run"); }); it("reclaims (runs) when there is no live holder of the id lock", () => { expect( - decideOverlap({ overlap: false, worktree: false, pidfileExists: false, holderAlive: false }), + decideOverlap({ overlap: false, pidfileExists: false, holderAlive: false }), ).toBe("run"); expect( - decideOverlap({ overlap: false, worktree: false, pidfileExists: true, holderAlive: false }), + decideOverlap({ overlap: false, pidfileExists: true, holderAlive: false }), ).toBe("run"); }); - it("skips a non-worktree cron only when a live holder owns the id lock", () => { + it("skips only when a live holder owns the id lock", () => { expect( - decideOverlap({ overlap: false, worktree: false, pidfileExists: true, holderAlive: true }), + decideOverlap({ overlap: false, pidfileExists: true, holderAlive: true }), ).toBe("skip"); }); }); @@ -461,24 +433,7 @@ describe("buildTmuxWrapper", () => { ); }); - it("uses a session-scoped pidfile and exports CRON_WORKTREE for an isolated worktree fire", () => { - const wt = buildTmuxWrapper({ - session: "cron-autopilot-0610-1805", - id: "autopilot", - agentBin: "pi", - promptFile: "/tmp/cron-autopilot-0610-1805.prompt", - pidFile: "/tmp/cron-autopilot-0610-1805.pid", - worktree: "/home/sandbox/harness/.worktrees/cron/cron-autopilot-0610-1805", - }); - expect(wt).toContain("echo $$ > '/tmp/cron-autopilot-0610-1805.pid';"); - expect(wt).toContain("rm -f '/tmp/cron-autopilot-0610-1805.pid';"); - expect(wt).not.toContain("/tmp/cron-autopilot.pid"); - expect(wt).toContain( - "CRON_OVERLAP_PIDFILE='/tmp/cron-autopilot-0610-1805.pid' CRON_WORKTREE='/home/sandbox/harness/.worktrees/cron/cron-autopilot-0610-1805';", - ); - }); - - it("omits CRON_WORKTREE and defaults to the id-scoped pidfile for a primary fire", () => { + it("defaults to the id-scoped pidfile and never exports CRON_WORKTREE", () => { expect(wrapper).toContain("echo $$ > '/tmp/cron-autopilot.pid';"); expect(wrapper).not.toContain("CRON_WORKTREE="); }); @@ -1131,7 +1086,6 @@ describe("reloadBody", () => { overlap: false, catchup: false, tmux: false, - worktree: false, body: "cached body\n", filePath: missingPath, }; @@ -1147,219 +1101,6 @@ describe("reloadBody", () => { }); }); -describe("fallback worktree pruning", () => { - const git = (cwd: string, args: string[]): void => { - const result = spawnSync("git", args, { cwd, encoding: "utf-8" }); - expect(result.status, `${args.join(" ")}\n${result.stderr}`).toBe(0); - }; - - const initRepoWithFallbackWorktrees = (): { repo: string; dirtyWt: string; cleanWt: string } => { - const repo = path.join(tmp, "repo"); - mkdirSync(repo, { recursive: true }); - git(repo, ["init", "-b", "development"]); - writeFileSync(path.join(repo, "README.md"), "fixture\n"); - git(repo, ["add", "README.md"]); - git(repo, ["-c", "user.name=Test", "-c", "user.email=test@example.com", "commit", "-m", "init"]); - - const dirtyWt = path.join(repo, ".worktrees", "cron", "cron-autopilot-dirty"); - const cleanWt = path.join(repo, ".worktrees", "cron", "cron-autopilot-clean"); - git(repo, ["worktree", "add", "--detach", dirtyWt, "HEAD"]); - git(repo, ["worktree", "add", "--detach", cleanWt, "HEAD"]); - writeFileSync(path.join(dirtyWt, "uncommitted.txt"), "salvage me\n"); - return { repo, dirtyWt, cleanWt }; - }; - - it("resolves the cron worktree root at the fixed .worktrees/ root", async () => { - const repo = path.join(tmp, "envrepo"); - mkdirSync(path.join(repo, ".worktrees", "cron"), { recursive: true }); - - const priorCwd = process.cwd(); - try { - process.chdir(repo); - - vi.resetModules(); - const runtime = await import("../cron-runtime.ts"); - expect(runtime.pruneAndCountFallbackWorktrees("autopilot")).toBe(0); - expect(existsSync(path.join(repo, ".worktrees", "cron"))).toBe(true); - } finally { - process.chdir(priorCwd); - vi.resetModules(); - } - }); - - it("reports dirty fallback worktree state including untracked files", () => { - const { dirtyWt } = initRepoWithFallbackWorktrees(); - - const state = inspectFallbackWorktree(dirtyWt); - - expect(state.dirty).toBe(true); - expect(state.changes).toContain("?? uncommitted.txt"); - expect(state.ref).not.toBe("unknown"); - }); - - it("treats a matching live tmux session name as fallback worktree liveness", () => { - const wt = path.join(tmp, "repo", ".worktrees", "cron", "cron-autopilot-0618-1505"); - - expect(worktreeInUse(wt, [], ["cron-autopilot-0618-1505"])).toBe(true); - expect(worktreeInUse(wt, [path.join(wt, "nested")], [])).toBe(true); - expect(worktreeInUse(wt, [path.join(tmp, "elsewhere")], ["autopilot-feat-445-example"])).toBe(false); - }); - - it("does not prune a clean fallback worktree while a matching tmux session is live", () => { - const { repo, cleanWt } = initRepoWithFallbackWorktrees(); - const priorCwd = process.cwd(); - const priorPath = process.env.PATH; - const binDir = path.join(tmp, "bin"); - const fakeTmux = path.join(binDir, "tmux"); - mkdirSync(binDir, { recursive: true }); - writeFileSync( - fakeTmux, - `#!/usr/bin/env bash -set -euo pipefail -case "$1" in - list-panes) printf '%s\\n' '${path.join(tmp, "elsewhere").replace(/'/g, "'\\''")}' ;; - ls) printf '%s\\n' 'cron-autopilot-clean' ;; - *) exit 1 ;; -esac -`, - ); - chmodSync(fakeTmux, 0o755); - - try { - process.chdir(repo); - process.env.PATH = `${binDir}:${priorPath ?? ""}`; - - expect(pruneAndCountFallbackWorktrees("autopilot")).toBe(1); - } finally { - process.env.PATH = priorPath; - process.chdir(priorCwd); - } - - expect(existsSync(cleanWt)).toBe(true); - }); - - it("preserves dirty dead fallback worktrees but removes clean dead ones", () => { - const { repo, dirtyWt, cleanWt } = initRepoWithFallbackWorktrees(); - const priorCwd = process.cwd(); - const appendSpy = vi.mocked(fsModule.appendFileSync); - appendSpy.mockClear(); - - try { - process.chdir(repo); - expect(pruneAndCountFallbackWorktrees("autopilot")).toBe(0); - } finally { - process.chdir(priorCwd); - } - - expect(existsSync(dirtyWt)).toBe(true); - expect(readFileSync(path.join(dirtyWt, "uncommitted.txt"), "utf-8")).toBe("salvage me\n"); - expect(existsSync(cleanWt)).toBe(false); - expect( - appendSpy.mock.calls.some((c) => - String(c[1]).includes("\tautopilot\tWORKTREE_DIRTY\t") && - String(c[1]).includes("?? uncommitted.txt"), - ), - ).toBe(true); - }); - - - it("reports an orphaned fallback worktree as orphaned, not dirty", () => { - const { repo } = initRepoWithFallbackWorktrees(); - const orphanWt = orphanWorktree(repo, "cron-autopilot-orphan"); - - const state = inspectFallbackWorktree(orphanWt); - - expect(state.orphaned).toBe(true); - expect(state.dirty).toBe(false); - expect(state.changes).toEqual([]); - expect(state.reason ?? "").not.toBe(""); - }); - - const orphanWorktree = (repo: string, name: string): string => { - const wt = path.join(repo, ".worktrees", "cron", name); - git(repo, ["worktree", "add", "--detach", wt, "HEAD"]); - rmSync(path.join(repo, ".git", "worktrees", name), { recursive: true, force: true }); - return wt; - }; - - it("does NOT classify a worktree as orphaned when its .git file is unreadable", () => { - const { repo } = initRepoWithFallbackWorktrees(); - const wt = path.join(repo, ".worktrees", "cron", "cron-autopilot-unreadable"); - git(repo, ["worktree", "add", "--detach", wt, "HEAD"]); - const dotGit = path.join(wt, ".git"); - chmodSync(dotGit, 0o000); - try { - const state = inspectFallbackWorktree(wt); - expect(state.orphaned ?? false).toBe(false); - } finally { - chmodSync(dotGit, 0o644); - } - }); - - it("does NOT classify a worktree as orphaned when its .git file is corrupt", () => { - const { repo } = initRepoWithFallbackWorktrees(); - const wt = path.join(repo, ".worktrees", "cron", "cron-autopilot-corrupt"); - git(repo, ["worktree", "add", "--detach", wt, "HEAD"]); - writeFileSync(path.join(wt, ".git"), "this is not a gitdir pointer\n"); - - const state = inspectFallbackWorktree(wt); - - expect(state.orphaned ?? false).toBe(false); - }); - - it("does not count an orphaned worktree toward the live count", () => { - const { repo } = initRepoWithFallbackWorktrees(); - orphanWorktree(repo, "cron-autopilot-orphan-count"); - const priorCwd = process.cwd(); - let live: number; - try { - process.chdir(repo); - live = pruneAndCountFallbackWorktrees("autopilot"); - } finally { - process.chdir(priorCwd); - } - expect(live).toBe(0); - }); - - it("removes an orphaned fallback worktree while still preserving a dirty one", () => { - const { repo, dirtyWt } = initRepoWithFallbackWorktrees(); - const orphanWt = orphanWorktree(repo, "cron-autopilot-orphan"); - const priorCwd = process.cwd(); - const appendSpy = vi.mocked(fsModule.appendFileSync); - appendSpy.mockClear(); - - try { - process.chdir(repo); - pruneAndCountFallbackWorktrees("autopilot"); - } finally { - process.chdir(priorCwd); - } - - expect(existsSync(orphanWt)).toBe(false); - expect( - appendSpy.mock.calls.some((c) => - String(c[1]).includes("\tautopilot\tWORKTREE_ORPHANED\t"), - ), - ).toBe(true); - - expect( - appendSpy.mock.calls.some((c) => - String(c[1]).includes("\tautopilot\tWORKTREE_DIRTY\t") && - String(c[1]).includes("cron-autopilot-orphan"), - ), - ).toBe(false); - - expect(existsSync(dirtyWt)).toBe(true); - expect(readFileSync(path.join(dirtyWt, "uncommitted.txt"), "utf-8")).toBe("salvage me\n"); - expect( - appendSpy.mock.calls.some((c) => - String(c[1]).includes("\tautopilot\tWORKTREE_DIRTY\t") && - String(c[1]).includes("?? uncommitted.txt"), - ), - ).toBe(true); - }); -}); - describe("onJobError", () => { it("logs an ERR_JOB line through the injected logger", () => { const spy = vi.fn(); @@ -1520,7 +1261,6 @@ describe("runPreflight + the fire() preflight gate", () => { overlap: false, catchup: false, tmux: true, - worktree: true, preflight, repo: undefined as string | undefined, body: "body\n", @@ -1578,7 +1318,7 @@ describe("runPreflight + the fire() preflight gate", () => { const script = preflightScript({ exit: 10, stdout: "SKIPPED-CAP-DAILY" }); writeFileSync( cronFile, - `---\nid: autopilot\nschedule: "* * * * *"\nenabled: true\ntmux: true\nworktree: true\npreflight: ${script}\n---\nbody\n`, + `---\nid: autopilot\nschedule: "* * * * *"\nenabled: true\ntmux: true\npreflight: ${script}\n---\nbody\n`, ); fire({ ...entry(script), filePath: cronFile }); const lines = loggedLines(); @@ -1592,7 +1332,7 @@ describe("runPreflight + the fire() preflight gate", () => { const script = preflightScript({ exit: 10, stdout: "SKIPPED-CAP-DAILY" }); writeFileSync( cronFile, - `---\nid: autopilot\nschedule: "* * * * *"\nenabled: true\ntmux: true\nworktree: true\npreflight: ${script}\nrepo: mifunedev/openharness\n---\nbody\n`, + `---\nid: autopilot\nschedule: "* * * * *"\nenabled: true\ntmux: true\npreflight: ${script}\nrepo: mifunedev/openharness\n---\nbody\n`, ); fire({ ...entry(undefined), filePath: cronFile }); @@ -1625,7 +1365,7 @@ describe("runPreflight + the fire() preflight gate", () => { const cronFile = path.join(tmp, "autopilot.md"); writeFileSync( cronFile, - `---\nid: autopilot\nschedule: "* * * * *"\nenabled: true\ntmux: true\nworktree: true\npreflight: scripts/definitely-missing-preflight.sh\n---\nbody\n`, + `---\nid: autopilot\nschedule: "* * * * *"\nenabled: true\ntmux: true\npreflight: scripts/definitely-missing-preflight.sh\n---\nbody\n`, ); fire({ ...entry("scripts/definitely-missing-preflight.sh"), filePath: cronFile }); const lines = loggedLines(); diff --git a/.oh/scripts/cron-runtime.ts b/.oh/scripts/cron-runtime.ts index bfc45c10..796d78bf 100644 --- a/.oh/scripts/cron-runtime.ts +++ b/.oh/scripts/cron-runtime.ts @@ -11,7 +11,6 @@ export interface CronEntry { overlap: boolean; catchup: boolean; tmux: boolean; - worktree: boolean; agentBin?: string; preflight?: string; repo?: string; @@ -20,7 +19,6 @@ export interface CronEntry { } const CRONS_DIR = path.resolve("crons"); -const WORKTREES_DIR = ".worktrees"; const PID_FILE = path.join(CRONS_DIR, ".pid"); const LOG_FILE = path.join(CRONS_DIR, ".cron.log"); const AGENT_BIN_FALLBACK = "claude"; @@ -108,7 +106,6 @@ export function parseCronFile(content: string, file: string): CronEntry | null { overlap: fm.overlap === "true", catchup: fm.catchup === "true", tmux: fm.tmux === "true", - worktree: fm.worktree === "true", agentBin: fm.agent || undefined, preflight: fm.preflight || undefined, repo: fm.repo || undefined, @@ -221,7 +218,6 @@ const FIRE_RELOAD_FIELDS: (keyof CronEntry)[] = [ "overlap", "catchup", "tmux", - "worktree", "agentBin", "preflight", "repo", @@ -357,7 +353,6 @@ export function buildTmuxWrapper(opts: { agentBin: string; promptFile: string; pidFile?: string; - worktree?: string; repo?: string; remote?: string; }): string { @@ -369,12 +364,11 @@ export function buildTmuxWrapper(opts: { const pidFile = opts.pidFile ?? `/tmp/cron-${id}.pid`; const quotedAgent = shellQuote(agentBin); const quotedPidFile = shellQuote(pidFile); - const worktreeExport = opts.worktree ? ` CRON_WORKTREE=${shellQuote(opts.worktree)}` : ""; const repoExport = opts.repo ? ` CRON_REPO=${shellQuote(opts.repo)}` : ""; const remoteExport = opts.remote ? ` CRON_REMOTE=${shellQuote(opts.remote)}` : ""; return ( `echo $$ > ${quotedPidFile}; ` + - `export CRON_TMUX_SESSION=${shellQuote(session)} CRON_KEEP_MARKER=${shellQuote(`/tmp/${session}.keep`)} CRON_OVERLAP_PIDFILE=${quotedPidFile}${worktreeExport}${repoExport}${remoteExport}; ` + + `export CRON_TMUX_SESSION=${shellQuote(session)} CRON_KEEP_MARKER=${shellQuote(`/tmp/${session}.keep`)} CRON_OVERLAP_PIDFILE=${quotedPidFile}${repoExport}${remoteExport}; ` + buildCronAgentCommand({ id, agentBin, @@ -487,8 +481,6 @@ export function buildCronAgentCommand(opts: { ); } -const WORKTREE_MAX_CONCURRENT = 6; - function isProcessAlive(pid: number): boolean { try { process.kill(pid, 0); @@ -498,209 +490,23 @@ function isProcessAlive(pid: number): boolean { } } -export type OverlapDecision = "run" | "skip" | "worktree"; +export type OverlapDecision = "run" | "skip"; export function decideOverlap(opts: { overlap: boolean; - worktree: boolean; pidfileExists: boolean; holderAlive: boolean; }): OverlapDecision { - if (opts.worktree) return "worktree"; if (opts.overlap) return "run"; if (!opts.pidfileExists || !opts.holderAlive) return "run"; return "skip"; } -const FALLBACK_WORKTREE_DIR = path.join(WORKTREES_DIR, "cron"); - -function detectBaseRef(remote = "origin"): string | null { - if (!isValidRemote(remote)) return null; - for (const ref of ["development", "main", "master"]) { - if ( - spawnSync("git", ["show-ref", "--verify", "--quiet", `refs/remotes/${remote}/${ref}`]) - .status === 0 - ) { - return `${remote}/${ref}`; - } - } - for (const ref of ["development", "main", "master"]) { - if (spawnSync("git", ["show-ref", "--verify", "--quiet", `refs/heads/${ref}`]).status === 0) { - return ref; - } - } - return null; -} - -function livePaneCwds(): string[] { - const r = spawnSync("tmux", ["list-panes", "-a", "-F", "#{pane_current_path}"], { - encoding: "utf-8", - }); - if (r.status !== 0 || !r.stdout) return []; - return r.stdout.split("\n").map((s) => s.trim()).filter(Boolean); -} - -function liveTmuxSessionNames(): string[] { - const r = spawnSync("tmux", ["ls", "-F", "#{session_name}"], { encoding: "utf-8" }); - if (r.status !== 0 || !r.stdout) return []; - return r.stdout.split("\n").map((s) => s.trim()).filter(Boolean); -} - -export function worktreeInUse( - wtPath: string, - cwds: string[], - sessionNames: string[] = liveTmuxSessionNames(), -): boolean { - const abs = path.resolve(wtPath); - return ( - cwds.some((p) => p === abs || p.startsWith(abs + path.sep)) || - sessionNames.includes(path.basename(abs)) - ); -} - -export interface FallbackWorktreeState { - dirty: boolean; - ref: string; - changes: string[]; - reason?: string; - orphaned?: boolean; -} - -function isOrphanedWorktree(wtPath: string): boolean { - const dotGit = path.join(wtPath, ".git"); - let raw: string; - try { - const st = fs.statSync(dotGit); - if (!st.isFile()) return false; - raw = fs.readFileSync(dotGit, "utf-8"); - } catch { - return false; - } - const match = /^gitdir:\s*(.+?)\s*$/m.exec(raw); - if (!match) return false; - const admin = path.resolve(path.dirname(dotGit), match[1]); - return !fs.existsSync(admin); -} - -function worktreeRef(wtPath: string): string { - const branch = spawnSync("git", ["-C", wtPath, "rev-parse", "--abbrev-ref", "HEAD"], { - encoding: "utf-8", - }); - const name = (branch.stdout || "").trim(); - if (branch.status === 0 && name && name !== "HEAD") return name; - - const head = spawnSync("git", ["-C", wtPath, "rev-parse", "--short", "HEAD"], { - encoding: "utf-8", - }); - return head.status === 0 && head.stdout.trim() ? head.stdout.trim() : "unknown"; -} - -export function inspectFallbackWorktree(wtPath: string): FallbackWorktreeState { - const status = spawnSync("git", ["-C", wtPath, "status", "--porcelain"], { - encoding: "utf-8", - }); - const ref = worktreeRef(wtPath); - if (status.status !== 0) { - const reason = (status.stderr || status.error?.message || "git status failed").trim(); - if (isOrphanedWorktree(wtPath)) { - return { dirty: false, orphaned: true, ref, changes: [], reason }; - } - return { dirty: true, ref, changes: [], reason }; - } - - const changes = (status.stdout || "").split("\n").map((s) => s.trim()).filter(Boolean); - return { dirty: changes.length > 0, ref, changes }; -} - -function formatWorktreeChanges(state: FallbackWorktreeState): string { - if (state.reason) return `status-unavailable=${state.reason.slice(0, 160)}`; - const shown = state.changes.slice(0, 20).join("; "); - const suffix = state.changes.length > 20 ? `; +${state.changes.length - 20} more` : ""; - return shown + suffix; -} - -export function pruneAndCountFallbackWorktrees(id: string): number { - const dir = path.resolve(FALLBACK_WORKTREE_DIR); - let names: string[]; - try { - names = fs.readdirSync(dir); - } catch { - return 0; - } - const cwds = livePaneCwds(); - const sessionNames = liveTmuxSessionNames(); - let live = 0; - for (const name of names) { - if (!name.startsWith(`cron-${id}-`)) continue; - const wt = path.join(dir, name); - if (worktreeInUse(wt, cwds, sessionNames)) { - live++; - continue; - } - - const state = inspectFallbackWorktree(wt); - if (state.orphaned) { - log(id, "WORKTREE_ORPHANED", `${wt} ref=${state.ref} reason=${(state.reason ?? "").slice(0, 160)}`); - try { - fs.rmSync(wt, { recursive: true, force: true }); - } catch (err) { - log(id, "WORKTREE_ORPHAN_RM_FAILED", `${wt} ${String(err).slice(0, 160)}`); - } - continue; - } - if (state.dirty) { - log(id, "WORKTREE_DIRTY", `${wt} ref=${state.ref} changes=${formatWorktreeChanges(state)}`); - continue; - } - - spawnSync("git", ["worktree", "remove", "--force", wt], { stdio: "ignore" }); - } - spawnSync("git", ["worktree", "prune"], { stdio: "ignore" }); - return live; -} - -function createFallbackWorktree(entry: CronEntry, session: string): string | null { - const live = pruneAndCountFallbackWorktrees(entry.id); - if (live >= WORKTREE_MAX_CONCURRENT) { - log( - entry.id, - "ERR_WORKTREE_CAP", - `${live} live worktree runs >= cap ${WORKTREE_MAX_CONCURRENT}`, - ); - return null; - } - const remote = entry.repo ? remoteForRepo(entry.repo) : undefined; - if (entry.repo && !remote) { - log(entry.id, "REPO_REMOTE_MISSING", `no local remote for ${entry.repo}`); - return null; - } - const base = detectBaseRef(remote || "origin"); - if (!base) { - log(entry.id, "ERR_WORKTREE", "no base ref (development/main/master) found"); - return null; - } - const dir = path.resolve(FALLBACK_WORKTREE_DIR); - try { - fs.mkdirSync(dir, { recursive: true }); - } catch { - } - const wtPath = path.join(dir, session); - const add = spawnSync("git", ["worktree", "add", "--detach", wtPath, base], { - encoding: "utf-8", - }); - if (add.status !== 0) { - log(entry.id, "ERR_WORKTREE", `git worktree add failed: ${(add.stderr || "").trim().slice(0, 150)}`); - return null; - } - return wtPath; -} - function fireTmux(entry: CronEntry): void { const session = tmuxSessionName(entry.id, new Date()); const idPidFile = `/tmp/cron-${entry.id}.pid`; - let cwd = process.cwd(); - let pidFile = idPidFile; - let worktree: string | undefined; + const cwd = process.cwd(); + const pidFile = idPidFile; const agentBin = entry.agentBin || resolveAgentBin(); if (!isValidAgentBin(agentBin)) { log(entry.id, "AGENT_INVALID", `invalid agent: ${agentBin}`); @@ -720,7 +526,6 @@ function fireTmux(entry: CronEntry): void { } const decision = decideOverlap({ overlap: entry.overlap, - worktree: entry.worktree, pidfileExists, holderAlive, }); @@ -728,13 +533,6 @@ function fireTmux(entry: CronEntry): void { log(entry.id, "SKIPPED_OVERLAP"); return; } - if (decision === "worktree") { - const wt = createFallbackWorktree(entry, session); - if (wt === null) return; - cwd = wt; - worktree = wt; - pidFile = `/tmp/${session}.pid`; - } const promptFile = `/tmp/${session}.prompt`; const body = reloadBody(entry); @@ -754,7 +552,6 @@ function fireTmux(entry: CronEntry): void { agentBin, promptFile, pidFile, - worktree, repo: entry.repo, remote: repoRemote, }), @@ -762,11 +559,7 @@ function fireTmux(entry: CronEntry): void { { stdio: "ignore" }, ); child.on("error", (e: Error) => log(entry.id, "ERR", String(e))); - if (worktree) { - log(entry.id, "SPAWNED_WORKTREE", `${session} ${worktree}`); - } else { - log(entry.id, "SPAWNED", session); - } + log(entry.id, "SPAWNED", session); } const PREFLIGHT_TIMEOUT_MS = 60_000; diff --git a/.oh/skills/audit/scripts/audit-run.sh b/.oh/skills/audit/scripts/audit-run.sh index 473c2836..e2be1448 100755 --- a/.oh/skills/audit/scripts/audit-run.sh +++ b/.oh/skills/audit/scripts/audit-run.sh @@ -120,11 +120,7 @@ if [[ -n ${AUDIT_RUN_ID:-} ]]; then [[ $resolved_root == "$AUDIT_ROOT" ]] || { echo 'audit: inherited root must be canonical' >&2; exit 64; } outer=false else - if [[ -n ${CRON_WORKTREE:-} ]] && git -C "$CRON_WORKTREE" rev-parse --show-toplevel >/dev/null 2>&1; then - resolved_root=$(git -C "$CRON_WORKTREE" rev-parse --show-toplevel) - else - resolved_root=$(git -C "$script_root" rev-parse --show-toplevel) - fi + resolved_root=$(git -C "$script_root" rev-parse --show-toplevel) resolved_root=$(cd "$resolved_root" && pwd -P) AUDIT_RUN_ID="audit-$(date -u +%Y%m%dT%H%M%SZ)-$BASHPID" AUDIT_ROOT=$resolved_root diff --git a/.oh/skills/spec/references/execute.md b/.oh/skills/spec/references/execute.md index cc9feb13..36a0bf32 100644 --- a/.oh/skills/spec/references/execute.md +++ b/.oh/skills/spec/references/execute.md @@ -198,18 +198,14 @@ background-shell launch, and no runner selection. There is **no fallback runner is no handoff step**. The agent that reached this line implements the task itself and carries it through every gate below. -**Build worktree — reuse vs. create.** Isolation stays. When `$CRON_WORKTREE` is set (a -`worktree: true` cron's default), this run is ALREADY inside an isolated worktree that step 2 -put on the feature branch, so **reuse it** — do NOT create a second worktree (a second -`git worktree add` for the same branch would nest under the cron worktree via the relative -path, or fail with `branch already checked out`). Standalone (no `$CRON_WORKTREE`), create +**Build worktree — reuse vs. create.** Isolation stays. When this run is ALREADY inside an +isolated worktree that step 2 put on the feature branch, **reuse it** — do NOT create a second +worktree (a second `git worktree add` for the same branch would nest under the current worktree +via the relative path, or fail with `branch already checked out`). Otherwise create `.worktrees/<prefix>/<N>-<slug>` via `/worktrees` and work there: ```bash -WT="${CRON_WORKTREE:-}" # set by the cron runtime in worktree mode -if [ -n "$WT" ]; then - cd "$WT" # already on <prefix>/<N>-<slug> -else +if [ "$(git rev-parse --abbrev-ref HEAD)" != "<prefix>/<N>-<slug>" ]; then git worktree add ".worktrees/<prefix>/<N>-<slug>" "<prefix>/<N>-<slug>" cd ".worktrees/<prefix>/<N>-<slug>" fi diff --git a/.worktrees/AGENTS.md b/.worktrees/AGENTS.md index 2276ed14..fd55ccff 100644 --- a/.worktrees/AGENTS.md +++ b/.worktrees/AGENTS.md @@ -10,7 +10,6 @@ its worktrees at its own root, so a project clone under `projects/` has a | --------- | --------------- | | `agent/` | Per-agent checkouts — either a `git worktree` of an `agent/<name>` branch in this repo, or a standalone clone of a repo that adopts the Open Harness shape (including a fork of an orchestrator). | | `feat/` `bug/` `task/` `audit/` `skill/` | Branch worktrees named after the branch prefix in `.oh/skills/git/SKILL.md`. | -| `cron/` | `cron/<session>` — per-fire isolation worktrees created by `worktree: true` crons. The runtime prunes and reaps these; do not manage them by hand. | | `archive/` | `archive/<YYYY-MM-DD>` — weekly cleanup-tasks archive sweeps. | Lifecycle is `git worktree add` / `git worktree remove`. The root is always diff --git a/CHANGELOG.md b/CHANGELOG.md index 682b137e..f82a1b30 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,6 +18,7 @@ Update policy and release automation live in [`/git`](.claude/skills/git/SKILL.m - **BREAKING:** Stop baking OpenCode, Hermes, and Grok Build into the image; `oh harness install <id>` installs them into `~/.local` as the sandbox user ([#908](https://github.com/mifunedev/openharness/issues/908)). ### Removed +- **BREAKING:** Retire cron worktree isolation — the `worktree:` frontmatter key, `.worktrees/cron/` per-fire worktrees, the `CRON_WORKTREE` export, and every `*_WORKTREE*` log state are gone; crons fire in the shared root under the id lock. - **BREAKING:** Retire the `OH_IMAGE_ONLY` flag; `entrypoint.sh` detects the sandbox flavor from whether `/home/sandbox/harness` is a bind mount holding `.oh/`, and logs the detected mode ([#920](https://github.com/mifunedev/openharness/issues/920)). - **BREAKING:** Retire `docker-compose.hermes-dashboard.yml` and its published `127.0.0.1:9119`; the dashboard now binds container loopback, reachable over cloudflared or Tailscale ([#920](https://github.com/mifunedev/openharness/issues/920)). - Remove the duplicate agent-browser and Tailscale installers from `entrypoint.sh`; the tool catalog is the sole owner of both pins and Tailscale's two checksums ([#920](https://github.com/mifunedev/openharness/issues/920)). diff --git a/crons/AGENTS.md b/crons/AGENTS.md index 254ff100..b471a265 100644 --- a/crons/AGENTS.md +++ b/crons/AGENTS.md @@ -29,7 +29,7 @@ See `.oh/scripts/cron-runtime.ts` for the runtime implementation | Change | Takes effect | | --- | --- | | Body (the agent prompt) | Next fire, automatically — the runtime re-reads the file. Logs `BODY_RELOADED`. | -| Frontmatter (`schedule`, `enabled`, `timezone`, `overlap`, `agent`, `tmux`, `worktree`, `preflight`) | Only after a `SIGHUP` reschedule or a runtime restart. Until then the **old** schedule is live. | +| Frontmatter (`schedule`, `enabled`, `timezone`, `overlap`, `agent`, `tmux`, `preflight`) | Only after a `SIGHUP` reschedule or a runtime restart. Until then the **old** schedule is live. | | Adding or removing a `<id>.md` file | Only after a `SIGHUP` reschedule or a runtime restart. | | `.oh/scripts/cron-runtime.ts` itself | Only after a full runtime restart. | @@ -48,7 +48,6 @@ enabled: true overlap: false # skip new fire if previous still running catchup: false # don't replay missed fires after downtime tmux: false # optional — run each fire in its own detached tmux session -worktree: false # optional — run each fire in a fresh .worktrees/cron/<session> (root stays clean; never SKIPPED_OVERLAP) agent: pi # optional — override CRON_AGENT_BIN for this job only description: <one-line> --- @@ -90,7 +89,6 @@ status column is one of: | `SCHED_INVALID` | A cron was skipped because its `schedule:` is not a valid cron expression (`msg` contains the offending schedule string). | | `AGENT_INVALID` | A cron was skipped because its `agent:` override or effective `CRON_AGENT_BIN` is not a safe executable token/path. | | `SPAWNED` | A `tmux: true` fire launched its detached session (`msg` is the session name). | -| `SPAWNED_WORKTREE` | A `worktree: true` fire launched its detached session inside a fresh isolated `.worktrees/cron/<session>` worktree (`msg` is `<session> <worktree-path>`); the shared root checkout is untouched. | | `FIRE` | A scheduled job fired and began running its body. | | `AGENT_START` | The shell wrapper started an agent for the task (`msg` is `agent=<name>`). A Claude→Codex fallback emits a second `AGENT_START` for Codex. | | `AGENT_FALLBACK` | Default Claude execution hit a usage/session-limit pattern and retried through Codex (`msg` is `from=claude to=codex`). | @@ -99,9 +97,7 @@ status column is one of: | `EXIT_n` | The fired child process exited non-zero with code `n`. When the job's log file is populated, a bounded tail of the job's trailing output is appended as the `msg` (4th column) — the optional field already used by `ERR_JOB`/`ERR`/`BODY_RELOADED` — so the failure is diagnosable from `.cron.log` alone (see example below). | | `ERR` | The child process failed to spawn (process-level error, not a job throw). | | `ERR_JOB` | A synchronous cron job-callback threw; recorded instead of being swallowed. Format: `<id>\tERR_JOB\t<error-string>`. | -| `SKIPPED_OVERLAP` | A fire was skipped because the previous run was still in flight (`overlap: false`). Only possible for **non-`worktree`** crons — a `worktree: true` cron isolates instead of skipping (see `SPAWNED_WORKTREE`). | -| `ERR_WORKTREE` | A `worktree: true` fire could not create its isolated worktree (no base ref, or `git worktree add` failed); the fire did not run. A surfaced failure, never a silent skip. | -| `ERR_WORKTREE_CAP` | A `worktree: true` fire hit the live-worktree concurrency cap for its id; the fire did not run (retried next fire once a stuck session + its worktree are reaped). A surfaced failure, never a silent skip. | +| `SKIPPED_OVERLAP` | A fire was skipped because the previous run was still in flight (`overlap: false`). | An enriched `EXIT_n` line (tab-separated, tail whitespace-collapsed and bounded to 200 chars): @@ -143,7 +139,6 @@ A job with `tmux: true` in its frontmatter runs each fire in its own detached tm - **Env exported into the agent**: `CRON_TMUX_SESSION=<session>`, `CRON_KEEP_MARKER=/tmp/<session>.keep`, and `CRON_OVERLAP_PIDFILE=/tmp/cron-<id>.pid`. - **Keep-marker contract**: if the agent `touch`es `$CRON_KEEP_MARKER` before exiting, the session persists by resuming the run's own conversation as a live, attachable agent (`claude --continue` for a Claude run, `pi --continue` for an `agent: pi` run, or `codex` after a Claude→Codex fallback), falling back to a shell if that exits; otherwise it auto-closes when the agent finishes. Claude/Codex tmux runs are headless (`claude -p`, or `codex exec --sandbox danger-full-access` after a Claude usage/session-limit fallback). Pi tmux runs intentionally use the positional TUI shape (`pi "$(cat prompt)"`), matching `tmux new -s <name> pi "<prompt>"`, so attaching mid-run shows the live Pi pane instead of a blank piped/headless screen. By convention a job keeps its session only when the run produced something worth revisiting (e.g. a PR was opened). - **Overlap guard**: a per-id pidfile `/tmp/cron-<id>.pid` blocks a new fire while a previous one is still running when `overlap: false`; the skipped fire logs `SKIPPED_OVERLAP`. Kept interactive sessions that reach a terminal state can remove `$CRON_OVERLAP_PIDFILE` themselves before staying alive for manual review, so an intentionally retained pane does not suppress future fires. -- **Worktree isolation (`worktree: true`)**: instead of serializing on the shared root checkout, a `worktree: true` `tmux` cron runs **every** fire in a fresh detached `.worktrees/cron/<session>` worktree cut from the base branch (`development`→`main`→`master`), exported to the run as `$CRON_WORKTREE`. The root checkout is never touched for source/branch work (no dirty-env stalls) and a fire is **never silently skipped** — it isolates (`SPAWNED_WORKTREE`) or surfaces a failure (`ERR_WORKTREE`, or `ERR_WORKTREE_CAP` at the live-worktree concurrency cap). Isolated fires use a session-scoped lock (`/tmp/<session>.pid`) so they never clobber the id-scoped overlap lock; the runtime prunes dead-session worktrees before counting the cap, and the heartbeat reaps stuck sessions + their worktrees. Dead-session pruning distinguishes three outcomes, because a `git status` **failure** is not evidence of uncommitted work: (1) **clean** — `git status --porcelain` succeeds and is empty → the worktree is removed; (2) **dirty** — `git status --porcelain` succeeds and reports modified or untracked files → the runtime preserves the worktree and logs `WORKTREE_DIRTY` with its path, ref, and changed files for manual salvage; (3) **orphaned** — the directory survives but its `.git/worktrees/<name>` admin entry is gone, so `git status` cannot report at all → the runtime logs `WORKTREE_ORPHANED` and removes the directory outright (`git worktree remove` would itself fail, and `git worktree prune` cannot reach it — prune clears entries whose *directory* is missing, which is the inverse case). Only a provable orphan is reaped this way: any other `git status` failure (permissions, a corrupt repo) still falls through to the preserve-and-log-`WORKTREE_DIRTY` branch, because it cannot be shown that there is nothing to salvage. `SKIPPED_OVERLAP` remains the behaviour for non-`worktree` crons (heartbeat/cleanup/eval). A `worktree: true` cron treats runtime observability as the narrow exception: its source work stays in `$CRON_WORKTREE`, but it resolves `$CRON_LOG_ROOT` to the shared root checkout and appends `crons/.cron.log` there so humans can still inspect liveness after the ephemeral worktree is reaped. Jobs with `tmux` absent or `false` keep the default in-process spawn. @@ -169,7 +164,7 @@ The bare `kill -HUP "$(cat crons/.pid)"` form works only from *inside* the conta ## Layout -The runtime always reads `crons/` at the repository root, and isolated cron -worktrees and `/worktrees` scratch roots always live under `.worktrees/`. These -locations are a convention, not a setting; `.oh/scripts/oh-path crons` and -`.oh/scripts/oh-path worktrees` resolve them. +The runtime always reads `crons/` at the repository root, and `/worktrees` +scratch roots always live under `.worktrees/`. These locations are a convention, +not a setting; `.oh/scripts/oh-path crons` and `.oh/scripts/oh-path worktrees` +resolve them. diff --git a/crons/prompt-miner.md b/crons/prompt-miner.md index a917412d..d50f179f 100644 --- a/crons/prompt-miner.md +++ b/crons/prompt-miner.md @@ -6,7 +6,6 @@ enabled: false overlap: false catchup: false tmux: true -worktree: true repo: mifunedev/openharness description: Daily prompt-miner — mine 24h of session traces for prompt-quality markers and ship a top finding to the origin fork via /spec (opt-in, cap-gated) --- @@ -14,9 +13,8 @@ description: Daily prompt-miner — mine 24h of session traces for prompt-qualit # prompt-miner You are running on a daily prompt-miner cycle, inside your own detached tmux -session **in an isolated git worktree** (`$CRON_WORKTREE`, set by the cron runtime -because this cron declares `worktree: true`). The shared root checkout is never -touched for source/branch work. Your job is to mine the last 24h of session +session. Create an isolated `/worktrees` checkout for any source or branch work so +the shared root checkout stays clean. Your job is to mine the last 24h of session traces for a high-confidence prompt-quality marker and, when one clears the bar, ship it to the **origin fork** through `/spec` — never upstream, never auto-merged. @@ -110,13 +108,12 @@ gh pr edit <PR> --repo mifunedev/openharness --add-label prompt-miner ### 4. Append the liveness line -Append a `crons/.cron.log` liveness line, resolving the **shared root** under -worktree mode (the worktree is reaped after the run; humans + heartbeat read the -root checkout). Mirror the autopilot convention: honor `$CRON_LOG_ROOT` if -set, else map `$CRON_WORKTREE` back to its shared root, else the current toplevel. +Append a `crons/.cron.log` liveness line against the **shared root** (humans + +heartbeat read the root checkout, never a build worktree). Honor `$CRON_LOG_ROOT` +if set, else map the current checkout back to its shared root. ```bash -ROOT="${CRON_LOG_ROOT:-$(git -C "${CRON_WORKTREE:-.}" worktree list --porcelain 2>/dev/null | awk 'NR==1 && $1 == "worktree" { sub(/^worktree /,""); print; exit }' || true)}" +ROOT="${CRON_LOG_ROOT:-$(git worktree list --porcelain 2>/dev/null | awk 'NR==1 && $1 == "worktree" { sub(/^worktree /,""); print; exit }' || true)}" ROOT="${ROOT:-$(git rev-parse --show-toplevel)}" printf '[%s]\tprompt-miner\t%s\t%s\n' "$(date -Iseconds)" "<STATUS>" "<msg>" \ | "$ROOT/.oh/scripts/locked-append.sh" "$ROOT/crons/.cron.log" From 11f21a2e842d4f6dcc2686fdea4fb81f2ef76750 Mon Sep 17 00:00:00 2001 From: Ryan Eggleston <kre8mymedia@gmail.com> Date: Mon, 31 Aug 2026 21:23:50 -0600 Subject: [PATCH 21/23] FROM feat/926-repo-knowledge-loop TO development (#927) Closes #926. Make /spec a closed repo-knowledge learning loop: planning recalls tracked knowledge and re-grounds it against current sources before the PRD exists, execution re-grounds against HEAD and derives knowledge invalidation from the actual diff, freshness becomes source-change aware via verified_at, and durable knowledge moves to a tracked .oh/knowledge/ surface with per-machine scratch physically separated. Retired atomically with every consumer: /spec ship, the generated prompt.md, the STATUS: COMPLETE sentinel, mandatory /compact as a semantic stage, the .oh/memory vocabulary, and /wiki lint's orphan and 90-day checks. Twelve probes added (the ten #926 requires plus two minted from the run's retro), each fault-injected. Reconciled with #930, #931, and #934, which merged during the build. Submitted-by: Claude --- .claude/protected-paths.txt | 1 + .github/workflows/ci-harness.yml | 2 + .gitignore | 29 +- .oh/README.md | 14 +- .oh/evals/RESULTS.md | 264 +++--- .../tasks/CB-005-compile-a-lesson.md | 4 +- .../decisions}/skill-impact.md | 0 .oh/evals/probes/audit-stale-references.sh | 2 +- .oh/evals/probes/builder-wiki-proposer.sh | 6 +- .../cleanup-no-agent-session-coupling.sh | 16 +- .../docs-20260901-followup-artifact-cited.sh | 62 ++ .oh/evals/probes/docs-build-fast-path.sh | 2 +- .../probes/evals-20260901-suite-tree-clean.sh | 67 ++ .../probes/knowledge-path-single-owner.sh | 63 ++ .../probes/knowledge-source-freshness.sh | 171 ++++ .../knowledge-tracked-query-boundary.sh | 68 ++ .oh/evals/probes/retired-memory-vocabulary.sh | 66 ++ .../probes/retro-deterministic-contract.sh | 2 +- .oh/evals/probes/roles-are-skills.sh | 2 - .../probes/spec-execute-knowledge-impact.sh | 67 ++ .../probes/spec-execute-running-contract.sh | 77 ++ .oh/evals/probes/spec-family-contract.sh | 39 +- .../spec-no-generated-prompt-contract.sh | 57 ++ .../probes/spec-plan-knowledge-context.sh | 74 ++ .../probes/spec-plan-reconciliation-gate.sh | 61 ++ .oh/evals/probes/spec-ready-finalization.sh | 19 +- .../task-completion-structured-state.sh | 89 ++ .oh/evals/probes/wiki-compile-contract.sh | 2 +- .oh/evals/probes/wiki-kind-schema-contract.sh | 113 ++- .oh/evals/probes/wiki-pattern-persistence.sh | 28 +- .../probes/wiki-query-pattern-isolation.sh | 62 +- .oh/evals/probes/wiki-readme-index.sh | 23 +- .oh/evals/probes/wiki-related-slugs.sh | 57 +- .../probes/wiki-skill-impact-append-only.sh | 19 +- .oh/knowledge/README.md | 79 ++ .oh/knowledge/local/README.md | 27 + .../pattern-docs-prohibition-by-example.md | 18 +- .../pattern-evals-pipefail-early-exit.md | 66 ++ .../pattern-evals-prose-literal-pinning.md | 14 +- .../pattern-evals-unexercised-oracle.md | 0 .../pattern-spec-self-staling-reuse-record.md | 62 ++ ...attern-wiki-external-model-over-mapping.md | 24 +- .../pattern-wiki-ungated-check-drift.md | 20 +- .../2026-06-27-recursive-language-models.md | 0 ...07-04-crabbox-remote-exec-control-plane.md | 0 .../2026-07-04-runtime-isolation-landscape.md | 0 .../raw/2026-07-17-audit-architecture.md | 0 .../raw/2026-07-18-markitdown.md | 0 .../raw/2026-08-12-build-executor-ladder.md | 0 .../raw/2026-08-23-release-versioning.md | 0 ...2026-08-24-plan-vs-built-reconciliation.md | 0 .../raw/2026-08-26-prime-agent-harness.md | 0 ...-08-31-wikiskill-experience-compilation.md | 0 .oh/knowledge/raw/README.md | 25 + .../source}/audit-architecture.md | 7 + .../source}/compose-env-boundary.md | 2 + .../crabbox-remote-exec-control-plane.md | 1 + .../source}/document-ingestion.md | 8 +- .../source}/fresh-machine-setup.md | 2 + .../source}/managed-agents.md | 10 +- .../molt-agentic-reinforcement-learning.md | 5 +- .../source}/oh-cli-portable-lifecycle.md | 6 +- .../source}/plan-vs-built-reconciliation.md | 38 +- .../source}/recursive-language-models.md | 1 + .../recursive-self-improvement-survey.md | 3 +- .../source}/release-versioning.md | 7 + .../source}/runtime-isolation-landscape.md | 1 + .../source}/sandbox-dependency-installs.md | 2 + .../wikiskill-experience-compilation.md | 17 +- .oh/manifest.json | 1 + .oh/skills/audit/references/harness.md | 8 +- .../audit/references/reviewer-evidence-doc.md | 7 + .oh/skills/benchmark/SKILL.md | 4 +- .oh/skills/builder/SKILL.md | 9 +- .oh/skills/imagine/SKILL.md | 2 +- .oh/skills/retro/SKILL.md | 25 +- .oh/skills/spec/SKILL.md | 262 ++++-- .oh/skills/spec/references/execute.md | 778 +++++++++++------- .oh/skills/spec/references/plan.md | 292 +++++-- .oh/skills/spec/references/retro.md | 97 +-- .oh/skills/spec/references/ship.md | 121 --- .oh/skills/spec/templates/task-prompt.md | 32 +- .oh/skills/strategic-proposal/SKILL.md | 2 +- .../references/open-harness-v2mom-council.md | 2 +- .oh/skills/sync/references/catchup.md | 2 +- .oh/skills/sync/references/publish.md | 5 +- .oh/skills/weigh/references/workflow-shape.md | 2 +- .oh/skills/wiki/SKILL.md | 115 +-- .oh/skills/wiki/corpus/README.md | 52 -- .oh/skills/wiki/corpus/raw/README.md | 15 - .oh/skills/wiki/references/compile.md | 64 +- .../references/concurrent-ingest-worktrees.md | 8 +- .../references/github-repo-research-wiki.md | 4 +- .oh/skills/wiki/references/ingest.md | 90 +- .oh/skills/wiki/references/lint.md | 654 ++++++--------- .../references/official-docs-research-wiki.md | 2 +- .oh/skills/wiki/references/query.md | 326 ++++---- .oh/skills/wiki/references/schema.md | 501 ++++++----- .../references/social-image-wiki-ingest.md | 8 +- .oh/skills/wiki/scripts/knowledge-impact.sh | 172 ++++ .oh/tasks/README.md | 28 +- .oh/tasks/compose-env-boundary/prd.json | 2 +- .oh/tasks/compose-env-boundary/prd.md | 8 +- .oh/tasks/compose-env-boundary/progress.txt | 2 +- .oh/tasks/compose-env-boundary/prompt.md | 31 - .../repo-knowledge-loop/eval-result.json | 15 + .oh/tasks/repo-knowledge-loop/evidence.md | 599 ++++++++++++++ .oh/tasks/repo-knowledge-loop/prd.json | 120 +++ .oh/tasks/repo-knowledge-loop/prd.md | 293 +++++++ .oh/tasks/repo-knowledge-loop/progress.txt | 84 ++ .../repo-knowledge-loop/simplify-rounds.json | 8 + CHANGELOG.md | 17 + crons/cleanup-tasks.md | 21 +- docs/glossary.md | 15 +- docs/oh-directory-layout.md | 3 +- docs/rfcs/rfc-brain-hands-boundary.md | 2 +- docs/rfcs/rfc-rsi-survey-mapping.md | 6 +- 117 files changed, 4996 insertions(+), 1993 deletions(-) rename .oh/{skills/wiki/corpus => evals/decisions}/skill-impact.md (100%) create mode 100755 .oh/evals/probes/docs-20260901-followup-artifact-cited.sh create mode 100755 .oh/evals/probes/evals-20260901-suite-tree-clean.sh create mode 100755 .oh/evals/probes/knowledge-path-single-owner.sh create mode 100755 .oh/evals/probes/knowledge-source-freshness.sh create mode 100755 .oh/evals/probes/knowledge-tracked-query-boundary.sh create mode 100755 .oh/evals/probes/retired-memory-vocabulary.sh create mode 100755 .oh/evals/probes/spec-execute-knowledge-impact.sh create mode 100755 .oh/evals/probes/spec-execute-running-contract.sh create mode 100755 .oh/evals/probes/spec-no-generated-prompt-contract.sh create mode 100755 .oh/evals/probes/spec-plan-knowledge-context.sh create mode 100755 .oh/evals/probes/spec-plan-reconciliation-gate.sh create mode 100755 .oh/evals/probes/task-completion-structured-state.sh create mode 100644 .oh/knowledge/README.md create mode 100644 .oh/knowledge/local/README.md rename .oh/{skills/wiki/corpus => knowledge/patterns}/pattern-docs-prohibition-by-example.md (73%) create mode 100644 .oh/knowledge/patterns/pattern-evals-pipefail-early-exit.md rename .oh/{skills/wiki/corpus => knowledge/patterns}/pattern-evals-prose-literal-pinning.md (77%) rename .oh/{skills/wiki/corpus => knowledge/patterns}/pattern-evals-unexercised-oracle.md (100%) create mode 100644 .oh/knowledge/patterns/pattern-spec-self-staling-reuse-record.md rename .oh/{skills/wiki/corpus => knowledge/patterns}/pattern-wiki-external-model-over-mapping.md (70%) rename .oh/{skills/wiki/corpus => knowledge/patterns}/pattern-wiki-ungated-check-drift.md (73%) rename .oh/{skills/wiki/corpus => knowledge}/raw/2026-06-27-recursive-language-models.md (100%) rename .oh/{skills/wiki/corpus => knowledge}/raw/2026-07-04-crabbox-remote-exec-control-plane.md (100%) rename .oh/{skills/wiki/corpus => knowledge}/raw/2026-07-04-runtime-isolation-landscape.md (100%) rename .oh/{skills/wiki/corpus => knowledge}/raw/2026-07-17-audit-architecture.md (100%) rename .oh/{skills/wiki/corpus => knowledge}/raw/2026-07-18-markitdown.md (100%) rename .oh/{skills/wiki/corpus => knowledge}/raw/2026-08-12-build-executor-ladder.md (100%) rename .oh/{skills/wiki/corpus => knowledge}/raw/2026-08-23-release-versioning.md (100%) rename .oh/{skills/wiki/corpus => knowledge}/raw/2026-08-24-plan-vs-built-reconciliation.md (100%) rename .oh/{skills/wiki/corpus => knowledge}/raw/2026-08-26-prime-agent-harness.md (100%) rename .oh/{skills/wiki/corpus => knowledge}/raw/2026-08-31-wikiskill-experience-compilation.md (100%) create mode 100644 .oh/knowledge/raw/README.md rename .oh/{skills/wiki/corpus => knowledge/source}/audit-architecture.md (93%) rename .oh/{skills/wiki/corpus => knowledge/source}/compose-env-boundary.md (98%) rename .oh/{skills/wiki/corpus => knowledge/source}/crabbox-remote-exec-control-plane.md (99%) rename .oh/{skills/wiki/corpus => knowledge/source}/document-ingestion.md (91%) rename .oh/{skills/wiki/corpus => knowledge/source}/fresh-machine-setup.md (98%) rename .oh/{skills/wiki/corpus => knowledge/source}/managed-agents.md (90%) rename .oh/{skills/wiki/corpus => knowledge/source}/molt-agentic-reinforcement-learning.md (95%) rename .oh/{skills/wiki/corpus => knowledge/source}/oh-cli-portable-lifecycle.md (93%) rename .oh/{skills/wiki/corpus => knowledge/source}/plan-vs-built-reconciliation.md (66%) rename .oh/{skills/wiki/corpus => knowledge/source}/recursive-language-models.md (99%) rename .oh/{skills/wiki/corpus => knowledge/source}/recursive-self-improvement-survey.md (98%) rename .oh/{skills/wiki/corpus => knowledge/source}/release-versioning.md (95%) rename .oh/{skills/wiki/corpus => knowledge/source}/runtime-isolation-landscape.md (99%) rename .oh/{skills/wiki/corpus => knowledge/source}/sandbox-dependency-installs.md (98%) rename .oh/{skills/wiki/corpus => knowledge/source}/wikiskill-experience-compilation.md (83%) delete mode 100644 .oh/skills/spec/references/ship.md delete mode 100644 .oh/skills/wiki/corpus/README.md delete mode 100644 .oh/skills/wiki/corpus/raw/README.md create mode 100755 .oh/skills/wiki/scripts/knowledge-impact.sh delete mode 100644 .oh/tasks/compose-env-boundary/prompt.md create mode 100644 .oh/tasks/repo-knowledge-loop/eval-result.json create mode 100644 .oh/tasks/repo-knowledge-loop/evidence.md create mode 100644 .oh/tasks/repo-knowledge-loop/prd.json create mode 100644 .oh/tasks/repo-knowledge-loop/prd.md create mode 100644 .oh/tasks/repo-knowledge-loop/progress.txt create mode 100644 .oh/tasks/repo-knowledge-loop/simplify-rounds.json diff --git a/.claude/protected-paths.txt b/.claude/protected-paths.txt index 575fed3d..81394445 100644 --- a/.claude/protected-paths.txt +++ b/.claude/protected-paths.txt @@ -33,6 +33,7 @@ health-check retro t3 # --- Orchestrator scripts (.oh/scripts/) --- +.oh/skills/wiki/scripts/knowledge-impact.sh .oh/scripts/cron-runtime.ts .oh/scripts/sandbox-healthcheck.sh .oh/scripts/link-providers.sh diff --git a/.github/workflows/ci-harness.yml b/.github/workflows/ci-harness.yml index 0f818e97..68630193 100644 --- a/.github/workflows/ci-harness.yml +++ b/.github/workflows/ci-harness.yml @@ -25,6 +25,7 @@ on: - "crons/**" - ".devcontainer/**" - ".oh/evals/**" + - ".oh/knowledge/**" - "AGENTS.md" - "CLAUDE.md" pull_request: @@ -47,6 +48,7 @@ on: - "crons/**" - ".devcontainer/**" - ".oh/evals/**" + - ".oh/knowledge/**" - "AGENTS.md" - "CLAUDE.md" diff --git a/.gitignore b/.gitignore index 9fc2899f..11fff15b 100644 --- a/.gitignore +++ b/.gitignore @@ -69,20 +69,21 @@ crons/*.log # Legacy/future workspace package build artifacts packages/*/build/ packages/*/node_modules/ -# Wiki corpus owned by the /wiki skill — entries are local scratch by default; -# the operator whitelists curated entries into git with `git add -f`. Raw -# snapshots are immutable provenance, fully ignored except the directory anchor. -.oh/skills/wiki/corpus/* -!.oh/skills/wiki/corpus/README.md -!.oh/skills/wiki/corpus/raw/ -.oh/skills/wiki/corpus/raw/* -!.oh/skills/wiki/corpus/raw/README.md +# Durable repository knowledge (.oh/knowledge/) is TRACKED: source/ and +# patterns/ hold shared synthesis, raw/ holds the immutable external evidence +# those pages cite. Only local/ is ignored — per-machine scratch that /wiki query +# and every /spec flow refuse to read, so a page on one machine can never +# silently inform a plan another machine cannot reproduce. +.oh/knowledge/local/* +!.oh/knowledge/local/README.md -# Per-session agent memory and cron runtime state — local scratch, like the -# wiki corpus above. Nothing here is tracked on any branch; the operator -# whitelists an entry with `git add -f` if one is ever worth keeping. -# Written by entrypoint.sh's seed_workspace_volume when there is no checkout bind. -# It must never land in a real checkout, so it is ignored on every branch. +# Cron runtime state written by entrypoint.sh's seed_workspace_volume when there +# is no checkout bind. It must never land in a real checkout, so it is ignored on +# every branch. .oh/.image-seeded -.oh/memory/ .oh/crons/ + +# RETIRED COMPATIBILITY TOMBSTONE — .oh/memory/ was deleted as a concept; nothing +# writes or reads it. This rule exists only so a stale directory left by a +# pre-0.4.0 checkout can never be committed. Remove it in 0.7.0. +.oh/memory/ diff --git a/.oh/README.md b/.oh/README.md index ccff2096..f7356b90 100644 --- a/.oh/README.md +++ b/.oh/README.md @@ -23,8 +23,8 @@ is obsolete): — exported to the four agent providers via symlinks (`.claude/`, `.codex/`, `.pi/`, `.hermes/`): the `oh` CLI (`cli/`), installer + lifecycle scripts (`scripts/`), container-install inputs (`install/`), the - regression/capability eval suite (`evals/`), the long-term memory + session - logs (`memory/`), user-local deploy + regression/capability eval suite (`evals/`), the durable repository-knowledge + surface (`knowledge/`), user-local deploy config (`config.json`), and the Ralph/spec task workdirs (`tasks/` — ephemeral build scratch, now at `.oh/tasks/`). The former top-level `packages/` folder @@ -36,8 +36,9 @@ is obsolete): devcontainer spec + Docker COPY, `package.json`, `pnpm-*.yaml`, `.github/`, `.husky/`). The scheduled-agent cron definitions live at the repo root in `crons/` — operator schedule content, not shipped machinery. The - eval suite stays under `.oh/evals/`, and the Ralph/spec task workdirs under - `.oh/tasks/`. The worktree root (`.worktrees/`) and the project-clone root + eval suite stays under `.oh/evals/`, durable repository knowledge under + `.oh/knowledge/`, and the Ralph/spec task workdirs under `.oh/tasks/`. The + worktree root (`.worktrees/`) and the project-clone root (`projects/`) sit at the repo root, because a repository keeps its worktrees at its own root and a project clone is a peer repo, not control-plane machinery; the rendered docs site and the `blog/` archive @@ -45,7 +46,7 @@ is obsolete): ### Relocated into `.oh/` (no back-compat symlinks) -The runtime-machinery directories (`scripts/`, `install/`, `evals/`, `memory/`) moved into `.oh/` +The runtime-machinery directories (`scripts/`, `install/`, `evals/`) moved into `.oh/` **without** back-compat symlinks at the old root paths — every consumer was repointed to the real `.oh/…` location: @@ -116,6 +117,7 @@ The shared skills and hooks are vendored directly under `.oh/` (`.oh/skills`, `. | `install/` | Container-install inputs (`.zshrc`, `.tmux.conf`, `banner.sh`, `install.sh` prerequisites) consumed by the Dockerfile + entrypoint. Old path: `install/` (no symlink — repointed). | | `scripts/` | Installer, lifecycle, cron-runtime, and eval-support scripts (`docker-compose.sh`, `cron-runtime.ts`, `locked-append.sh`, `harness-config.sh`, …). Old path: `scripts/` (no symlink — repointed). | | `evals/` | The fitness-function suite — regression probes (`probes/`), capability benchmark (`capability/`), trajectory datasets (`datasets/`), and the `RESULTS.md` scoreboard. Old path: `evals/` (no symlink — repointed). | +| `knowledge/` | Durable repository knowledge — `source/` and `patterns/` entity pages, `raw/` immutable external snapshots, gitignored `local/` scratch, and the generated `README.md` index. The `/wiki` skill owns the procedure; this directory owns the data. | | `patches/` | Vendored pnpm dependency patches (applied at install via `package.json` `patchedDependencies`). | | `config.json` | User-local, gitignored `composeOverrides[]` source. Read here first; legacy repo-root `config.json` is honored as a fallback. | @@ -146,7 +148,7 @@ source instead of the bundled `.oh/templates/`. | Belongs in `.oh/` | Stays at root | |------|------| -| OpenHarness's own machinery addressed as a unit: the `oh` CLI, installer/lifecycle scripts, container-install inputs, compose config, the fitness-function eval suite (`.oh/evals/`), and the Ralph/spec task workdirs (`.oh/tasks/`) | Human-facing Markdown docs (`docs/`) plus the scheduled-agent cron definitions (`crons/`), and surfaces **forced to root by external tooling** (`.devcontainer/`, `package.json`, `pnpm-*.yaml`, `.github/`, `.husky/`) | +| OpenHarness's own machinery addressed as a unit: the `oh` CLI, installer/lifecycle scripts, container-install inputs, compose config, the fitness-function eval suite (`.oh/evals/`), the durable repository-knowledge surface (`.oh/knowledge/`), and the Ralph/spec task workdirs (`.oh/tasks/`) | Human-facing Markdown docs (`docs/`) plus the scheduled-agent cron definitions (`crons/`), and surfaces **forced to root by external tooling** (`.devcontainer/`, `package.json`, `pnpm-*.yaml`, `.github/`, `.husky/`) | ### Why these specifically stay at root diff --git a/.oh/evals/RESULTS.md b/.oh/evals/RESULTS.md index 96eaa4a1..25ddcc2c 100644 --- a/.oh/evals/RESULTS.md +++ b/.oh/evals/RESULTS.md @@ -6,131 +6,143 @@ probe id; git history is the time series.** Schema and exit-code semantics are i | probe | tier | last-run (UTC) | status | source | |-------|------|----------------|--------|--------| -| agent-browser-cli | A | 2026-09-01 02:15 | PASS | retro lesson 2026-06-07 (agent-browser 0.8.5 CLI) | -| agents-identity-contract | A | 2026-09-01 02:15 | PASS | issue #854 — T3-style root identity, glossary, and skill-owned procedures | -| architect-skill-contract | A | 2026-09-01 02:15 | PASS | ADR #929 — skills are the role primitive; /architect is a skill, not an agent | -| architecture-record-reuse | A | 2026-09-01 02:15 | PASS | ADR #929 — durable decisions reuse the existing RFC/ADR issue convention | -| artifact-contract-audit | A | 2026-09-01 02:15 | PASS | issue #583/#645 — production /audit implementation Gate 1 behavior | -| audit-dispatcher-contract | A | 2026-09-01 02:15 | PASS | issue #645 — audit consolidation public taxonomy | -| audit-implementation-behavior | A | 2026-09-01 02:15 | PASS | issue #645 — implementation root/repo/browser behavior | -| audit-pr-acquire | A | 2026-09-01 02:15 | PASS | issue #645 — production PR acquisition behavior | -| audit-pr-classifier | A | 2026-09-01 02:15 | PASS | issue #645 — deterministic focused and queue PR classifier | -| audit-run-root-contract | A | 2026-09-01 02:15 | PASS | issue #645 — executable immutable audit root/run correlation | -| audit-shellcheck-coverage | A | 2026-09-01 02:15 | PASS | issue #645 — private audit scripts require release and CI lint coverage | -| audit-slop-gate | A | 2026-09-01 02:15 | PASS | .claude/specs/images/reduce-slop.png — the four correctness gates all pass a change | -| audit-stale-references | A | 2026-09-01 02:15 | PASS | issue #645 — clean-breaking audit migration | -| boot-lint-glob | A | 2026-09-01 02:15 | PASS | issue #90, issue #120 | -| builder-no-agent-artifact | A | 2026-09-01 02:15 | PASS | ADR #929 — /builder agent is retired; a reusable role is authored as a skill | -| builder-skill-consolidation | A | 2026-09-01 02:15 | PASS | issue #643 — consolidate artifact builders behind one /builder dispatcher | -| builder-wiki-proposer | A | 2026-09-01 02:15 | PASS | wikiskill arXiv:2608.27454 — the skill proposer reads accumulated knowledge first | -| capability-benchmark-schema | A | 2026-09-01 02:15 | PASS | issue #167 — capability benchmark instrument | -| cc-safety-net-wiring | A | 2026-09-01 02:15 | SKIPPED | .oh/tasks/cc-safety-net/prd.json US-007 2026-07-19 | -| changelog-entry-length | A | 2026-09-01 02:15 | PASS | conversation 2026-08-24 — CHANGELOG.md grew to 259KB of bullet prose because "one line" was unquantified | -| cleanup-no-agent-session-coupling | A | 2026-09-01 02:15 | PASS | issue #928 — retire automated /spec agent handoff | -| cleanup-tasks-scoped-guard | A | 2026-09-01 02:15 | PASS | issue #85 | -| cleanup-tasks-worktree-grooming | A | 2026-09-01 02:15 | PASS | issue #168; issue #327 | -| cli-publish-typecheck-scope | A | 2026-09-01 02:15 | PASS | release run 33271077312 — v0.5.0 pushed its GHCR image, then failed to publish | -| close-issues-on-development | A | 2026-09-01 02:15 | PASS | issue #841 (closing keywords never fire because the default branch is main) 2026-08-26 | -| codex-stale-response-retry | A | 2026-09-01 02:15 | PASS | issue #506 — Codex previous_response_not_found RCA | -| compose-config-path-parity | A | 2026-09-01 02:15 | PASS | PR #833 (remove harness.yaml — the wrapper and VS Code "Reopen in Container" paths must resolve the same service) 2026-08-26 | -| compose-env-boundary | A | 2026-09-01 02:15 | PASS | #920 — the epic #903→#911 made the CLI provision harnesses and tools from | -| config-schema-parity | A | 2026-09-01 02:15 | PASS | PR #833 (one schema file — DOCKER_SOCKET, SANDBOX_SSH, OH_SANDBOX_IMAGE, OH_PULL_POLICY, SKIP_PNPM_INSTALL were consumed but undocumented); rewritten for the oh.json/secrets split by PR #887 | -| context-tier-size-budget | A | 2026-09-01 02:15 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-007) — the always-on tier was 85,256 B | -| continual-learning-20260831 | A | 2026-09-01 02:15 | PASS | retro lesson 2026-08-31 (unexercised oracle) — a probe green in a 112-probe run carried three parser defects | -| cron-claude-codex-fallback | A | 2026-09-01 02:15 | PASS | conversation 2026-06-12 (default Codex fallback for crons) | -| cron-watchdog | A | 2026-09-01 02:15 | PASS | issues #130/#453 (cron runtime watchdog + legacy system-cron reaping) 2026-06-19 | -| crons-directory-guide | A | 2026-09-01 02:15 | PASS | issue #874 | -| curl-bash-safe-alternatives | A | 2026-09-01 02:15 | PASS | vet-run/vet integration — public curl|bash examples need review-first alternatives | -| datasets-schema | A | 2026-09-01 02:15 | PASS | issue #196 — .oh/evals/datasets verifiable trajectory corpus (Repo2RLEnv-inspired) | -| debugmcp-availability | A | 2026-09-01 02:15 | SKIPPED | issue #297 — DebugMCP MCP debug-server availability | -| default-provisioning | A | 2026-09-01 02:15 | PASS | #902 — `oh harness install` must work from inside the sandbox, where | -| delegate-model-effort-policy | A | 2026-09-01 02:15 | PASS | conversation 2026-07-11 (delegate model inheritance and thinking policy) | -| delegate-worker-boundary | A | 2026-09-01 02:15 | PASS | ADR #929 — subagents are a bounded execution primitive, not a project-role ontology; | -| devtcp-hook | A | 2026-09-01 02:15 | PASS | retro lesson 2026-06-10 (zsh /dev/tcp) | -| docker-inspect-env-guard | A | 2026-09-01 02:15 | PASS | operator directive 2026-08-08 (agents keep the docker socket, but must | -| docs-build-fast-path | A | 2026-09-01 02:15 | PASS | #455 — docs builds must stay out of fast harness/eval/release gates; #536 — docs site externalized to openharness-web; docs markdown relocated to docs/ | -| drift-check-cron-staleness-glob | A | 2026-09-01 02:15 | PASS | issue #98; issue #225 (restart-required cron frontmatter/config drift) | -| entrypoint-pnpm-manifest-fingerprint | A | 2026-09-01 02:15 | PASS | issue #521 (manifest-aware sandbox installs) 2026-07-01 | -| escalate-contract | A | 2026-09-01 02:15 | PASS | issue #799 — seven comments on a GitHub thread produced zero notifications and nobody | -| eval-ci-gate | A | 2026-09-01 02:15 | PASS | #103 — eval probe suite gated in CI | -| eval-contract-text-20260831 | A | 2026-09-01 02:15 | PASS | retro lesson 2026-08-31 (prose-literal pinning) — two assertions failed on first run purely from source line wrapping | -| eval-gate | A | 2026-09-01 02:15 | PASS | retro lesson 2026-06-11 (eval-gate) | -| eval-results-atomic | A | 2026-09-01 02:15 | PASS | issue #83 (eval-results-atomic-write) | -| eval-runner-exit | A | 2026-09-01 02:15 | PASS | retro lesson 2026-06-11 (eval-runner-exit) #29 | -| eval-runs-once-per-cycle | A | 2026-09-01 02:15 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-006) — /eval ran 3x per cycle on the | -| execution-target-contract | A | 2026-09-01 02:15 | PASS | issue #733 (ExecutionTarget contract + Docker Compose adapter) 2026-08-10 | -| get-oh-bootstrap | A | 2026-09-01 02:15 | PASS | get-oh.sh bootstrap — the Node-bootstrapping host-side path to the standalone `oh` CLI (also on npm as @mifune/openharness; see oh-npm-package.sh) | -| git-skill | A | 2026-09-01 02:15 | PASS | conversation 2026-06-15 — rules are not always supported; git workflow must be the /git skill | -| harness-audit-empty-output-gate | A | 2026-09-01 02:15 | PASS | issue #246 — /audit harness must fail closed on empty auditor outputs | -| harness-ci-core-paths | A | 2026-09-01 02:15 | PASS | #165 — core sandbox config files must trigger harness CI | -| harness-ci-hooks-paths | A | 2026-09-01 02:15 | PASS | issue #202 — credential/security hook changes must trigger harness CI | -| harness-yaml-migration | A | 2026-09-01 02:15 | PASS | PR #833 (migrate-harness-yaml.sh — append / uncomment-in-place / preserve / overwrite, plus a silent no-op second run) 2026-08-26 | -| headless-tmux-preserved | A | 2026-09-01 02:15 | PASS | issue #928 — retire automated /spec agent handoff | -| health-check-docker-stats | A | 2026-09-01 02:15 | PASS | retro lesson 2026-06-10 (docker stats vs ps Size) | -| health-check-socket-degrade | A | 2026-09-01 02:15 | PASS | issue #762 (refs #756) — /health-check degrades to one statement, not nine failures | -| heartbeat-logging-contract | A | 2026-09-01 02:15 | PASS | issue #447 (heartbeat log append hardening) 2026-06-18 | -| image-seed-hygiene | A | 2026-09-01 02:15 | PASS | issue #900 (slim the sandbox image) 2026-08-30 | -| markitdown-wiki-ingest | A | 2026-09-01 02:15 | PASS | issue #649 — pinned local-document normalization contract for /wiki ingest | -| next-dev-prod | A | 2026-09-01 02:15 | SKIPPED | retro lesson 2026-06-04 | -| no-project-agent-catalog | A | 2026-09-01 02:15 | PASS | ADR #929 — .oh/agents/ is retired; provider-link and update logic must not recreate it | -| oh-compose-env-wiring | A | 2026-09-01 02:15 | PASS | issue #880 (oh as the only front door — oh.json is the non-secret config surface) | -| oh-config-surfaces | A | 2026-09-01 02:15 | PASS | PR #887 (config split across two authored surfaces — a tracked oh.json and a secrets-only root dotenv — with nothing left under $HOME) | -| oh-destroy-guard | A | 2026-09-01 02:15 | PASS | issue #879 — `oh` becomes the only front door, so `make destroy` must | -| oh-devcontainer-restructure | A | 2026-09-01 02:15 | PASS | consolidate devcontainer — .oh/devcontainer/ folded back into .devcontainer/ | -| oh-home-mount | A | 2026-09-01 02:15 | PASS | issue #898 (single $HOME mount) 2026-08-30 | -| oh-image-only-deploy | A | 2026-09-01 02:15 | PASS | .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only | -| oh-init-headless-config | A | 2026-09-01 02:15 | PASS | PR #827 (installer answers landed in the losing config file); retargeted to the .example.env template by PR #833, then to oh.json by PR #887 | -| oh-init-scaffold | A | 2026-09-01 02:15 | PASS | issue #531 Phase 2 | -| oh-lifecycle-surface | A | 2026-09-01 02:15 | PASS | issue #881 — the Makefile is retired and `oh` is the only front door | -| oh-npm-package | A | 2026-09-01 02:15 | PASS | npm publish path for the standalone `oh` CLI (@mifune/openharness) — alternative to get-oh.sh | -| oh-payload-manifest | A | 2026-09-01 02:15 | PASS | issue #531 follow-on (.oh payload manifest — oh update ships a declared allowlist) | -| oh-sandbox-image-mode | A | 2026-09-01 02:15 | PASS | conversation 2026-07-05 (basic Docker deployment — prebuilt-image mode) | -| oh-shipped-repo-overridable | A | 2026-09-01 02:15 | PASS | issue #531 follow-on (de-hardcode residual — shipped .oh shell scripts keep the upstream repo overridable) | -| oh-standalone-lifecycle | A | 2026-09-01 02:15 | PASS | issue #564 | -| oh-update | A | 2026-09-01 02:15 | PASS | issue #531 Phase 3 (oh update — upgrade only the .oh control plane) | -| operator-config-guard | A | 2026-09-01 02:15 | PASS | operator directives 2026-08-06 (.config/ and settings.local.json are operator-only) | -| pnpm-audit-ci-gate | A | 2026-09-01 02:15 | PASS | issue #171 — pnpm security audits must run in CI | -| post-bridge-publish-confirmation | A | 2026-09-01 02:15 | PASS | #523 — post-bridge live publishing requires an explicit final confirmation gate | -| prd-output-path-contract | A | 2026-09-01 02:15 | PASS | retro lesson 2026-06-19 | -| prompt-miner-schema-compat | A | 2026-09-01 02:15 | PASS | issue #253 — prompt-miner JSONL schema-drift guard | -| prompt-miner-symlink-entrypoint | A | 2026-09-01 02:15 | PASS | issue #663 — prompt-miner engine no-ops via the documented .claude/skills symlink | -| prompt-miner-weakness-record | A | 2026-09-01 02:15 | PASS | issue #580 — prompt-miner weakness-record (WH-xxx) cluster output | -| protected-path-deletion | A | 2026-09-01 02:15 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-001) — the critique gate was deleted, | -| protected-paths-resolve | A | 2026-09-01 02:15 | PASS | issue #753 — .claude/protected-paths.txt named 7 paths that did not exist. | -| registry-portability-gate | A | 2026-09-01 02:15 | PASS | issue #758 | -| registry-portability | A | 2026-09-01 02:15 | SKIPPED | issue #758 | -| retro-deterministic-contract | A | 2026-09-01 02:15 | PASS | issue #443 — /retro deterministic output and self-contained helper contract | -| rlm-context-budget | A | 2026-09-01 02:15 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-006 | -| roles-are-skills | A | 2026-09-01 02:15 | PASS | ADR #929 — roles are behavior, skills encode behavior, agents execute behavior | -| runtime-preflight-gate | A | 2026-09-01 02:15 | PASS | issue #806 § B1 (open sandbox.substrate vs sandbox.runtime selector); | -| sandbox-boot-guard-ci | A | 2026-09-01 02:15 | PASS | issue #449 (sandbox image build CI guard) 2026-06-19; | -| sandbox-node-base | A | 2026-09-01 02:15 | PASS | openharness#878 — oh as the only front door, T0 sandbox base image | -| skill-paths | A | 2026-09-01 02:15 | PASS | issue #43 — stale path references; extended by issue #69 — apps/->packages/ rename guard; extended by issue #870 — deleted .oh/agents/advisor.md | -| skills-dir-clean | A | 2026-09-01 02:15 | PASS | conversation 2026-06-29 — Pi parses every top-level `.md` in the skills | -| skills-task-tool-coupling | A | 2026-09-01 02:15 | PASS | council review 2026-08-29 (issue #886) — /delegate instructed Claude-Code-only | -| skills-vendored | A | 2026-09-01 02:15 | PASS | absorb .mifune submodule into .oh — the skills/hooks pack is vendored | -| slack-admin-command-surface | A | 2026-09-01 02:15 | PASS | issue #354 — Slack bridge docs must distinguish Pi /msg-bridge commands from Slack DM admin text handlers | -| spec-family-contract | A | 2026-09-01 02:15 | PASS | issue #265; spec-simplification issue #816; workflow authority issue #854; | -| spec-no-advisor-session-coupling | A | 2026-09-01 02:15 | PASS | issue #928 — retire automated /spec agent handoff | -| spec-no-agent-handoff | A | 2026-09-01 02:15 | PASS | issue #928 — retire automated /spec agent handoff | -| spec-ready-finalization | A | 2026-09-01 02:15 | PASS | issue #134; spec-simplification issue #816; workflow authority issue #854 | -| spec-single-owner | A | 2026-09-01 02:15 | PASS | conversation 2026-06-19 (single-owner implementation workflow, issue #257); | -| ste-checker-contract | A | 2026-09-01 02:15 | PASS | issue #750 PR audit — the /ste checker had four fail-open paths (unclosed | -| submitted-by-trailers | A | 2026-09-01 02:15 | PASS | conversation 2026-06-12 (commit attribution trailers); the single-owner | -| sync-skill-contract | A | 2026-09-01 02:15 | PASS | issue #331 — /sync dispatcher skill (bidirectional origin↔upstream sync) | -| t3-headless-launch | A | 2026-09-01 02:15 | PASS | issue #858 — /t3 launched a bare `npx --yes t3`, which is the local GUI and | -| tailscale-tool-boundary | A | 2026-09-01 02:15 | PASS | issue #858 — Tailscale mobile access for T3 Code. There is no tailnet, no | -| tool-catalog-boundary | A | 2026-09-01 02:15 | PASS | agent-browser's exclusion from the harness catalog (#821), the three-catalog | -| version-parity | A | 2026-09-01 02:15 | PASS | conversation 2026-08-29 — the oh CLI became the only lifecycle door, so its | -| weigh-scorer-contract | A | 2026-09-01 02:15 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-003 (2026-06-27) | -| wiki-compile-contract | A | 2026-09-01 02:15 | PASS | wikiskill arXiv:2608.27454 — Wiki Maintainer role added as /wiki compile | -| wiki-kind-schema-contract | A | 2026-09-01 02:15 | PASS | wikiskill arXiv:2608.27454 — pattern layer added to the wiki corpus | -| wiki-pattern-persistence | A | 2026-09-01 02:15 | PASS | wikiskill arXiv:2608.27454 — pattern pages are never rolled back | -| wiki-query-pattern-isolation | A | 2026-09-01 02:15 | PASS | wikiskill arXiv:2608.27454 — proposer-only pattern access | -| wiki-readme-index | A | 2026-09-01 02:15 | PASS | issue #132 — wiki README index drift guard | -| wiki-related-slugs | A | 2026-09-01 02:15 | PASS | wikiskill arXiv:2608.27454 — wiki lint related-slug check | -| wiki-skill-impact-append-only | A | 2026-09-01 02:15 | PASS | wikiskill arXiv:2608.27454 — skill-change ledger, never rolled back | -| workflow-boundaries | A | 2026-09-01 02:15 | PASS | conversation 2026-06-19 (workflow consolidation, issue #259); authority moved to /spec in issue #854 | -| worktrees-layout | A | 2026-09-01 02:15 | PASS | issue #872 | +| agent-browser-cli | A | 2026-09-01 03:14 | PASS | retro lesson 2026-06-07 (agent-browser 0.8.5 CLI) | +| agents-identity-contract | A | 2026-09-01 03:14 | PASS | issue #854 — T3-style root identity, glossary, and skill-owned procedures | +| architect-skill-contract | A | 2026-09-01 03:14 | PASS | ADR #929 — skills are the role primitive; /architect is a skill, not an agent | +| architecture-record-reuse | A | 2026-09-01 03:14 | PASS | ADR #929 — durable decisions reuse the existing RFC/ADR issue convention | +| artifact-contract-audit | A | 2026-09-01 03:14 | PASS | issue #583/#645 — production /audit implementation Gate 1 behavior | +| audit-dispatcher-contract | A | 2026-09-01 03:14 | PASS | issue #645 — audit consolidation public taxonomy | +| audit-implementation-behavior | A | 2026-09-01 03:14 | PASS | issue #645 — implementation root/repo/browser behavior | +| audit-pr-acquire | A | 2026-09-01 03:14 | PASS | issue #645 — production PR acquisition behavior | +| audit-pr-classifier | A | 2026-09-01 03:14 | PASS | issue #645 — deterministic focused and queue PR classifier | +| audit-run-root-contract | A | 2026-09-01 03:14 | PASS | issue #645 — executable immutable audit root/run correlation | +| audit-shellcheck-coverage | A | 2026-09-01 03:14 | PASS | issue #645 — private audit scripts require release and CI lint coverage | +| audit-slop-gate | A | 2026-09-01 03:14 | PASS | .claude/specs/images/reduce-slop.png — the four correctness gates all pass a change | +| audit-stale-references | A | 2026-09-01 03:14 | PASS | issue #645 — clean-breaking audit migration | +| boot-lint-glob | A | 2026-09-01 03:14 | PASS | issue #90, issue #120 | +| builder-no-agent-artifact | A | 2026-09-01 03:14 | PASS | ADR #929 — /builder agent is retired; a reusable role is authored as a skill | +| builder-skill-consolidation | A | 2026-09-01 03:14 | PASS | issue #643 — consolidate artifact builders behind one /builder dispatcher | +| builder-wiki-proposer | A | 2026-09-01 03:14 | PASS | wikiskill arXiv:2608.27454 — the skill proposer reads accumulated knowledge first | +| capability-benchmark-schema | A | 2026-09-01 03:14 | PASS | issue #167 — capability benchmark instrument | +| cc-safety-net-wiring | A | 2026-09-01 03:14 | SKIPPED | .oh/tasks/cc-safety-net/prd.json US-007 2026-07-19 | +| changelog-entry-length | A | 2026-09-01 03:14 | PASS | conversation 2026-08-24 — CHANGELOG.md grew to 259KB of bullet prose because "one line" was unquantified | +| cleanup-no-agent-session-coupling | A | 2026-09-01 03:14 | PASS | issue #928 — retire automated /spec agent handoff; | +| cleanup-tasks-scoped-guard | A | 2026-09-01 03:14 | PASS | issue #85 | +| cleanup-tasks-worktree-grooming | A | 2026-09-01 03:14 | PASS | issue #168; issue #327 | +| cli-publish-typecheck-scope | A | 2026-09-01 03:14 | PASS | release run 33271077312 — v0.5.0 pushed its GHCR image, then failed to publish | +| close-issues-on-development | A | 2026-09-01 03:14 | PASS | issue #841 (closing keywords never fire because the default branch is main) 2026-08-26 | +| codex-stale-response-retry | A | 2026-09-01 03:14 | PASS | issue #506 — Codex previous_response_not_found RCA | +| compose-config-path-parity | A | 2026-09-01 03:14 | PASS | PR #833 (remove harness.yaml — the wrapper and VS Code "Reopen in Container" paths must resolve the same service) 2026-08-26 | +| compose-env-boundary | A | 2026-09-01 03:14 | PASS | #920 — the epic #903→#911 made the CLI provision harnesses and tools from | +| config-schema-parity | A | 2026-09-01 03:14 | PASS | PR #833 (one schema file — DOCKER_SOCKET, SANDBOX_SSH, OH_SANDBOX_IMAGE, OH_PULL_POLICY, SKIP_PNPM_INSTALL were consumed but undocumented); rewritten for the oh.json/secrets split by PR #887 | +| context-tier-size-budget | A | 2026-09-01 03:14 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-007) — the always-on tier was 85,256 B | +| continual-learning-20260831 | A | 2026-09-01 03:14 | PASS | retro lesson 2026-08-31 (unexercised oracle) — a probe green in a 112-probe run carried three parser defects | +| cron-claude-codex-fallback | A | 2026-09-01 03:14 | PASS | conversation 2026-06-12 (default Codex fallback for crons) | +| cron-watchdog | A | 2026-09-01 03:14 | PASS | issues #130/#453 (cron runtime watchdog + legacy system-cron reaping) 2026-06-19 | +| crons-directory-guide | A | 2026-09-01 03:14 | PASS | issue #874 | +| curl-bash-safe-alternatives | A | 2026-09-01 03:14 | PASS | vet-run/vet integration — public curl|bash examples need review-first alternatives | +| datasets-schema | A | 2026-09-01 03:14 | PASS | issue #196 — .oh/evals/datasets verifiable trajectory corpus (Repo2RLEnv-inspired) | +| debugmcp-availability | A | 2026-09-01 03:14 | SKIPPED | issue #297 — DebugMCP MCP debug-server availability | +| default-provisioning | A | 2026-09-01 03:14 | PASS | #902 — `oh harness install` must work from inside the sandbox, where | +| delegate-model-effort-policy | A | 2026-09-01 03:14 | PASS | conversation 2026-07-11 (delegate model inheritance and thinking policy) | +| delegate-worker-boundary | A | 2026-09-01 03:14 | PASS | ADR #929 — subagents are a bounded execution primitive, not a project-role ontology; | +| devtcp-hook | A | 2026-09-01 03:14 | PASS | retro lesson 2026-06-10 (zsh /dev/tcp) | +| docker-inspect-env-guard | A | 2026-09-01 03:14 | PASS | operator directive 2026-08-08 (agents keep the docker socket, but must | +| docs-20260901-followup-artifact-cited | A | 2026-09-01 03:14 | PASS | retro lesson 2026-09-01 (issue #926) — evidence.md recorded an acceptance criterion | +| docs-build-fast-path | A | 2026-09-01 03:14 | PASS | #455 — docs builds must stay out of fast harness/eval/release gates; #536 — docs site externalized to openharness-web; docs markdown relocated to docs/ | +| drift-check-cron-staleness-glob | A | 2026-09-01 03:14 | PASS | issue #98; issue #225 (restart-required cron frontmatter/config drift) | +| entrypoint-pnpm-manifest-fingerprint | A | 2026-09-01 03:14 | PASS | issue #521 (manifest-aware sandbox installs) 2026-07-01 | +| escalate-contract | A | 2026-09-01 03:14 | PASS | issue #799 — seven comments on a GitHub thread produced zero notifications and nobody | +| eval-ci-gate | A | 2026-09-01 03:14 | PASS | #103 — eval probe suite gated in CI | +| eval-contract-text-20260831 | A | 2026-09-01 03:14 | PASS | retro lesson 2026-08-31 (prose-literal pinning) — two assertions failed on first run purely from source line wrapping | +| eval-gate | A | 2026-09-01 03:14 | PASS | retro lesson 2026-06-11 (eval-gate) | +| eval-results-atomic | A | 2026-09-01 03:14 | PASS | issue #83 (eval-results-atomic-write) | +| eval-runner-exit | A | 2026-09-01 03:14 | PASS | retro lesson 2026-06-11 (eval-runner-exit) #29 | +| eval-runs-once-per-cycle | A | 2026-09-01 03:14 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-006) — /eval ran 3x per cycle on the | +| evals-20260901-suite-tree-clean | A | 2026-09-01 03:14 | PASS | retro lesson 2026-09-01 (issue #926) — a probe carrying `&>2` wrote a file named 2 | +| execution-target-contract | A | 2026-09-01 03:14 | PASS | issue #733 (ExecutionTarget contract + Docker Compose adapter) 2026-08-10 | +| get-oh-bootstrap | A | 2026-09-01 03:14 | PASS | get-oh.sh bootstrap — the Node-bootstrapping host-side path to the standalone `oh` CLI (also on npm as @mifune/openharness; see oh-npm-package.sh) | +| git-skill | A | 2026-09-01 03:14 | PASS | conversation 2026-06-15 — rules are not always supported; git workflow must be the /git skill | +| harness-audit-empty-output-gate | A | 2026-09-01 03:14 | PASS | issue #246 — /audit harness must fail closed on empty auditor outputs | +| harness-ci-core-paths | A | 2026-09-01 03:14 | PASS | #165 — core sandbox config files must trigger harness CI | +| harness-ci-hooks-paths | A | 2026-09-01 03:14 | PASS | issue #202 — credential/security hook changes must trigger harness CI | +| harness-yaml-migration | A | 2026-09-01 03:14 | PASS | PR #833 (migrate-harness-yaml.sh — append / uncomment-in-place / preserve / overwrite, plus a silent no-op second run) 2026-08-26 | +| headless-tmux-preserved | A | 2026-09-01 03:14 | PASS | issue #928 — retire automated /spec agent handoff | +| health-check-docker-stats | A | 2026-09-01 03:14 | PASS | retro lesson 2026-06-10 (docker stats vs ps Size) | +| health-check-socket-degrade | A | 2026-09-01 03:14 | PASS | issue #762 (refs #756) — /health-check degrades to one statement, not nine failures | +| heartbeat-logging-contract | A | 2026-09-01 03:14 | PASS | issue #447 (heartbeat log append hardening) 2026-06-18 | +| image-seed-hygiene | A | 2026-09-01 03:14 | PASS | issue #900 (slim the sandbox image) 2026-08-30 | +| knowledge-path-single-owner | A | 2026-09-01 03:14 | PASS | issue #926 — durable knowledge moved to .oh/knowledge/ with no compatibility alias | +| knowledge-source-freshness | A | 2026-09-01 03:14 | PASS | issue #926 — age is telemetry; a page is stale when a declared source moved | +| knowledge-tracked-query-boundary | A | 2026-09-01 03:14 | PASS | issue #926 — ignored local scratch must never be an implicit input | +| markitdown-wiki-ingest | A | 2026-09-01 03:14 | PASS | issue #649 — pinned local-document normalization contract for /wiki ingest | +| next-dev-prod | A | 2026-09-01 03:14 | SKIPPED | retro lesson 2026-06-04 | +| no-project-agent-catalog | A | 2026-09-01 03:14 | PASS | ADR #929 — .oh/agents/ is retired; provider-link and update logic must not recreate it | +| oh-compose-env-wiring | A | 2026-09-01 03:14 | PASS | issue #880 (oh as the only front door — oh.json is the non-secret config surface) | +| oh-config-surfaces | A | 2026-09-01 03:14 | PASS | PR #887 (config split across two authored surfaces — a tracked oh.json and a secrets-only root dotenv — with nothing left under $HOME) | +| oh-destroy-guard | A | 2026-09-01 03:14 | PASS | issue #879 — `oh` becomes the only front door, so `make destroy` must | +| oh-devcontainer-restructure | A | 2026-09-01 03:14 | PASS | consolidate devcontainer — .oh/devcontainer/ folded back into .devcontainer/ | +| oh-home-mount | A | 2026-09-01 03:14 | PASS | issue #898 (single $HOME mount) 2026-08-30 | +| oh-image-only-deploy | A | 2026-09-01 03:14 | PASS | .oh/tasks/image-only-deploy/prd.json US-004 (issue #609, Flavor B image-only | +| oh-init-headless-config | A | 2026-09-01 03:14 | PASS | PR #827 (installer answers landed in the losing config file); retargeted to the .example.env template by PR #833, then to oh.json by PR #887 | +| oh-init-scaffold | A | 2026-09-01 03:14 | PASS | issue #531 Phase 2 | +| oh-lifecycle-surface | A | 2026-09-01 03:14 | PASS | issue #881 — the Makefile is retired and `oh` is the only front door | +| oh-npm-package | A | 2026-09-01 03:14 | PASS | npm publish path for the standalone `oh` CLI (@mifune/openharness) — alternative to get-oh.sh | +| oh-payload-manifest | A | 2026-09-01 03:14 | PASS | issue #531 follow-on (.oh payload manifest — oh update ships a declared allowlist) | +| oh-sandbox-image-mode | A | 2026-09-01 03:14 | PASS | conversation 2026-07-05 (basic Docker deployment — prebuilt-image mode) | +| oh-shipped-repo-overridable | A | 2026-09-01 03:14 | PASS | issue #531 follow-on (de-hardcode residual — shipped .oh shell scripts keep the upstream repo overridable) | +| oh-standalone-lifecycle | A | 2026-09-01 03:14 | PASS | issue #564 | +| oh-update | A | 2026-09-01 03:14 | PASS | issue #531 Phase 3 (oh update — upgrade only the .oh control plane) | +| operator-config-guard | A | 2026-09-01 03:14 | PASS | operator directives 2026-08-06 (.config/ and settings.local.json are operator-only) | +| pnpm-audit-ci-gate | A | 2026-09-01 03:14 | PASS | issue #171 — pnpm security audits must run in CI | +| post-bridge-publish-confirmation | A | 2026-09-01 03:14 | PASS | #523 — post-bridge live publishing requires an explicit final confirmation gate | +| prd-output-path-contract | A | 2026-09-01 03:14 | PASS | retro lesson 2026-06-19 | +| prompt-miner-schema-compat | A | 2026-09-01 03:14 | PASS | issue #253 — prompt-miner JSONL schema-drift guard | +| prompt-miner-symlink-entrypoint | A | 2026-09-01 03:14 | PASS | issue #663 — prompt-miner engine no-ops via the documented .claude/skills symlink | +| prompt-miner-weakness-record | A | 2026-09-01 03:14 | PASS | issue #580 — prompt-miner weakness-record (WH-xxx) cluster output | +| protected-path-deletion | A | 2026-09-01 03:14 | PASS | .oh/tasks/spec-simplification/ (issue #816, US-001) — the critique gate was deleted, | +| protected-paths-resolve | A | 2026-09-01 03:14 | PASS | issue #753 — .claude/protected-paths.txt named 7 paths that did not exist. | +| registry-portability-gate | A | 2026-09-01 03:14 | PASS | issue #758 | +| registry-portability | A | 2026-09-01 03:14 | SKIPPED | issue #758 | +| retired-memory-vocabulary | A | 2026-09-01 03:14 | PASS | issue #926 — .oh/README.md and .gitignore still described a deleted subsystem | +| retro-deterministic-contract | A | 2026-09-01 03:14 | PASS | issue #443 — /retro deterministic output and self-contained helper contract | +| rlm-context-budget | A | 2026-09-01 03:14 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-006 | +| roles-are-skills | A | 2026-09-01 03:14 | PASS | ADR #929 — roles are behavior, skills encode behavior, agents execute behavior | +| runtime-preflight-gate | A | 2026-09-01 03:14 | PASS | issue #806 § B1 (open sandbox.substrate vs sandbox.runtime selector); | +| sandbox-boot-guard-ci | A | 2026-09-01 03:14 | PASS | issue #449 (sandbox image build CI guard) 2026-06-19; | +| sandbox-node-base | A | 2026-09-01 03:14 | PASS | openharness#878 — oh as the only front door, T0 sandbox base image | +| skill-paths | A | 2026-09-01 03:14 | PASS | issue #43 — stale path references; extended by issue #69 — apps/->packages/ rename guard; extended by issue #870 — deleted .oh/agents/advisor.md | +| skills-dir-clean | A | 2026-09-01 03:14 | PASS | conversation 2026-06-29 — Pi parses every top-level `.md` in the skills | +| skills-task-tool-coupling | A | 2026-09-01 03:14 | PASS | council review 2026-08-29 (issue #886) — /delegate instructed Claude-Code-only | +| skills-vendored | A | 2026-09-01 03:14 | PASS | absorb .mifune submodule into .oh — the skills/hooks pack is vendored | +| slack-admin-command-surface | A | 2026-09-01 03:14 | PASS | issue #354 — Slack bridge docs must distinguish Pi /msg-bridge commands from Slack DM admin text handlers | +| spec-execute-knowledge-impact | A | 2026-09-01 03:14 | PASS | issue #926 — the planner predicts, the diff decides | +| spec-execute-running-contract | A | 2026-09-01 03:14 | PASS | issue #926 — execute returned before the build finished while promising a ready PR; | +| spec-family-contract | A | 2026-09-01 03:14 | PASS | issue #265; spec-simplification issue #816; workflow authority issue #854; | +| spec-no-advisor-session-coupling | A | 2026-09-01 03:14 | PASS | issue #928 — retire automated /spec agent handoff | +| spec-no-agent-handoff | A | 2026-09-01 03:14 | PASS | issue #928 — retire automated /spec agent handoff | +| spec-no-generated-prompt-contract | A | 2026-09-01 03:14 | PASS | issue #926 — a persisted copy of a template drifts from the template | +| spec-plan-knowledge-context | A | 2026-09-01 03:14 | PASS | issue #926 — /spec wrote knowledge more reliably than it read it | +| spec-plan-reconciliation-gate | A | 2026-09-01 03:14 | PASS | issue #926 — an approved plan may not become a materially different PRD silently | +| spec-ready-finalization | A | 2026-09-01 03:14 | PASS | issue #134; spec-simplification issue #816; workflow authority issue #854 | +| spec-single-owner | A | 2026-09-01 03:14 | PASS | conversation 2026-06-19 (single-owner implementation workflow, issue #257); | +| ste-checker-contract | A | 2026-09-01 03:14 | PASS | issue #750 PR audit — the /ste checker had four fail-open paths (unclosed | +| submitted-by-trailers | A | 2026-09-01 03:14 | PASS | conversation 2026-06-12 (commit attribution trailers); the single-owner | +| sync-skill-contract | A | 2026-09-01 03:14 | PASS | issue #331 — /sync dispatcher skill (bidirectional origin↔upstream sync) | +| t3-headless-launch | A | 2026-09-01 03:14 | PASS | issue #858 — /t3 launched a bare `npx --yes t3`, which is the local GUI and | +| tailscale-tool-boundary | A | 2026-09-01 03:14 | PASS | issue #858 — Tailscale mobile access for T3 Code. There is no tailnet, no | +| task-completion-structured-state | A | 2026-09-01 03:14 | PASS | issue #926 — a prose sentinel duplicated structured task-graph state | +| tool-catalog-boundary | A | 2026-09-01 03:14 | PASS | agent-browser's exclusion from the harness catalog (#821), the three-catalog | +| version-parity | A | 2026-09-01 03:14 | PASS | conversation 2026-08-29 — the oh CLI became the only lifecycle door, so its | +| weigh-scorer-contract | A | 2026-09-01 03:14 | PASS | .oh/tasks/rlm-weighted-trajectories/prd.json US-003 (2026-06-27) | +| wiki-compile-contract | A | 2026-09-01 03:14 | PASS | wikiskill arXiv:2608.27454 — Wiki Maintainer role added as /wiki compile | +| wiki-kind-schema-contract | A | 2026-09-01 03:14 | PASS | wikiskill arXiv:2608.27454 (pattern layer); issue #926 (repo|external|pattern kinds) | +| wiki-pattern-persistence | A | 2026-09-01 03:14 | PASS | wikiskill arXiv:2608.27454 — pattern pages are never rolled back | +| wiki-query-pattern-isolation | A | 2026-09-01 03:14 | PASS | wikiskill arXiv:2608.27454 — proposer-only pattern access | +| wiki-readme-index | A | 2026-09-01 03:14 | PASS | issue #132 — knowledge README index drift guard | +| wiki-related-slugs | A | 2026-09-01 03:14 | PASS | wikiskill arXiv:2608.27454 — wiki lint link checks | +| wiki-skill-impact-append-only | A | 2026-09-01 03:14 | PASS | wikiskill arXiv:2608.27454 — skill-change ledger, never rolled back | +| workflow-boundaries | A | 2026-09-01 03:14 | PASS | conversation 2026-06-19 (workflow consolidation, issue #259); authority moved to /spec in issue #854 | +| worktrees-layout | A | 2026-09-01 03:14 | PASS | issue #872 | <!-- benchmark: pass-rate = PASS / (PASS + REGRESSION + TIMEOUT); SKIPPED excluded --> diff --git a/.oh/evals/capability/tasks/CB-005-compile-a-lesson.md b/.oh/evals/capability/tasks/CB-005-compile-a-lesson.md index ba0de56e..dd921f8b 100644 --- a/.oh/evals/capability/tasks/CB-005-compile-a-lesson.md +++ b/.oh/evals/capability/tasks/CB-005-compile-a-lesson.md @@ -14,7 +14,7 @@ Carry one supported lesson end-to-end, from observation to a recorded verdict: `/retro` produces a `supported` hypothesis at `medium`+ confidence → `/wiki compile` creates or patches exactly one `kind: pattern` page → `/builder` proposes one atomic edit to one artifact, **citing the motivating pattern slug it read** → `/benchmark` -emits a verdict → `.oh/skills/wiki/corpus/skill-impact.md` holds a `PROPOSED` record +emits a verdict → `.oh/evals/decisions/skill-impact.md` holds a `PROPOSED` record and its matching `-V` verdict record. The capability under test is whether the harness can turn an observation into a @@ -28,7 +28,7 @@ that gap. `CB-003` scored the nearest capability and was retired when `/retro` b report-only, leaving the ceiling with no view of knowledge persistence at all. ## Success signal -- A tracked `.oh/skills/wiki/corpus/pattern-*.md` whose `## Detail` names an +- A tracked `.oh/knowledge/patterns/pattern-*.md` whose `## Detail` names an observable symptom and a `path:line` root cause, and whose `sources:` uses the pinned `<path>@<short-sha>` evidence form. - `/builder`'s report names the `[[pattern-...]]` slug it read, or records diff --git a/.oh/skills/wiki/corpus/skill-impact.md b/.oh/evals/decisions/skill-impact.md similarity index 100% rename from .oh/skills/wiki/corpus/skill-impact.md rename to .oh/evals/decisions/skill-impact.md diff --git a/.oh/evals/probes/audit-stale-references.sh b/.oh/evals/probes/audit-stale-references.sh index fe223365..6369fc18 100755 --- a/.oh/evals/probes/audit-stale-references.sh +++ b/.oh/evals/probes/audit-stale-references.sh @@ -26,7 +26,7 @@ if ((${#bad[@]})); then printf '%s\n' "${bad[@]}" >&2; echo 'REGRESSION: active # shellcheck disable=SC2016 # literal Markdown route token bare_audit='`/audit`' for caller in \ - .oh/skills/wiki/corpus/recursive-language-models.md \ + .oh/knowledge/source/recursive-language-models.md \ .oh/skills/weigh \ .oh/skills/benchmark/SKILL.md \ .oh/skills/spec/SKILL.md \ diff --git a/.oh/evals/probes/builder-wiki-proposer.sh b/.oh/evals/probes/builder-wiki-proposer.sh index 2b4e0baf..8bb56a93 100755 --- a/.oh/evals/probes/builder-wiki-proposer.sh +++ b/.oh/evals/probes/builder-wiki-proposer.sh @@ -16,15 +16,15 @@ failures=() need() { grep -qF -- "$1" "$SKILL" || failures+=("builder/SKILL.md missing contract text: $1"); } need '/wiki query <artifact-name-or-subsystem> --patterns' -need '.oh/skills/wiki/corpus/skill-impact.md' +need '.oh/evals/decisions/skill-impact.md' need 'Do not re-propose a change recorded there as `REJECTED`' need 'Append a `PROPOSED` record' need 'none (direct request)' need 'Never edit an existing record' # The ledger this skill is told to write must actually exist. -[[ -f "$ROOT/.oh/skills/wiki/corpus/skill-impact.md" ]] \ - || failures+=("builder cites .oh/skills/wiki/corpus/skill-impact.md but the ledger does not exist") +[[ -f "$ROOT/.oh/evals/decisions/skill-impact.md" ]] \ + || failures+=("builder cites .oh/evals/decisions/skill-impact.md but the ledger does not exist") # The reads builder is told to perform must be within its declared tool allowlist. grep -q '^allowed-tools: Read, Write, Edit, Glob, Grep, Bash$' "$SKILL" \ diff --git a/.oh/evals/probes/cleanup-no-agent-session-coupling.sh b/.oh/evals/probes/cleanup-no-agent-session-coupling.sh index 38bae13a..8c4f9207 100755 --- a/.oh/evals/probes/cleanup-no-agent-session-coupling.sh +++ b/.oh/evals/probes/cleanup-no-agent-session-coupling.sh @@ -1,8 +1,9 @@ #!/usr/bin/env bash # tier: A -# source: issue #928 — retire automated /spec agent handoff -# desc: the weekly task sweep archives on the progress.txt STATUS marker alone; it never -# detects or kills a separately launched implementation-agent session +# source: issue #928 — retire automated /spec agent handoff; +# issue #926 — completion derives from structured task-graph state +# desc: the weekly task sweep archives on the task graph alone; it never detects or kills a +# separately launched implementation-agent session set -euo pipefail ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" @@ -22,7 +23,12 @@ kill_hits=$(grep -nF -- 'tmux kill-session' "$CLEANUP" || true) [[ -n "$kill_hits" ]] && found+=("sweep kills an agent session: ${kill_hits//$'\n'/ ; }") missing=() -grep -qF 'STATUS: COMPLETE' "$CLEANUP" || missing+=("sweep no longer keys archival on the progress.txt STATUS: COMPLETE marker") +# Archival keys on structured task-graph state. Issue #928 decoupled the sweep from an +# agent session while the prose sentinel was still authoritative; issue #926 retired that +# sentinel, so the decoupling this probe guards is now expressed against prd.json. The +# invariant is unchanged: completion is read from the task folder, never from a session. +grep -qF 'all(.userStories[]; .passes == true)' "$CLEANUP" \ + || missing+=("sweep no longer keys archival on prd.json structured story state") grep -qiF 'never' "$CLEANUP" && grep -qiF 'tied to a terminal session' "$CLEANUP" \ || missing+=("sweep no longer states that task state is untied from a terminal session") @@ -37,4 +43,4 @@ if (( ${#found[@]} + ${#missing[@]} )); then exit 1 fi -echo "PASS: task cleanup archives on STATUS: COMPLETE alone, kills no agent session, and keeps its live-pane grooming guard" >&2 +echo "PASS: task cleanup archives on structured task-graph state alone, kills no agent session, and keeps its live-pane grooming guard" >&2 diff --git a/.oh/evals/probes/docs-20260901-followup-artifact-cited.sh b/.oh/evals/probes/docs-20260901-followup-artifact-cited.sh new file mode 100755 index 00000000..11a2131f --- /dev/null +++ b/.oh/evals/probes/docs-20260901-followup-artifact-cited.sh @@ -0,0 +1,62 @@ +#!/usr/bin/env bash +# tier: A +# source: retro lesson 2026-09-01 (issue #926) — evidence.md recorded an acceptance criterion +# as satisfied by an openharness-web follow-up that had not been filed; every internal +# gate passed and external verification caught it +# desc: the reviewer evidence contract requires a follow-up to be CITED, not named, and every +# tracked evidence.md that leans on a follow-up carries a resolvable issue/PR URL in the +# same bullet. No gate inside this repository can see an artifact in another one, so the +# citation is the only thing separating "deferred" from "done". +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" +CONTRACT="$ROOT/.oh/skills/audit/references/reviewer-evidence-doc.md" + +[[ -f "$CONTRACT" ]] || { echo "SKIPPED: reviewer evidence contract absent: $CONTRACT" >&2; exit 2; } + +failures=() + +# --- the rule is stated where an author will meet it ---------------------- +grep -qF 'Follow-ups are cited, not named' "$CONTRACT" \ + || failures+=("reviewer-evidence-doc.md no longer carries the follow-up citation rule") +grep -qF 'Naming a follow-up in prose is a plan' "$CONTRACT" \ + || failures+=("reviewer-evidence-doc.md no longer says why naming a follow-up is not satisfying it") + +# --- every tracked evidence.md that leans on a follow-up cites one -------- +# Trigger only on a bullet that ties a follow-up to a criterion being satisfied; +# "filed as a follow-up" prose elsewhere in a doc is not a claim of completion. +url='https://github\.com/[A-Za-z0-9._-]+/[A-Za-z0-9._-]+/(issues|pull)/[0-9]+' +checked=0 +while IFS= read -r rel; do + [[ -n "$rel" ]] || continue + abs="$ROOT/$rel" + [[ -f "$abs" ]] || continue + checked=$((checked + 1)) + # Read bullet-wise: a claim and its citation belong to the same bullet. + while IFS= read -r bullet; do + [[ -n "$bullet" ]] || continue + grep -qiE 'follow-up|follow up|separate (issue|pr)|mirrored to' <<<"$bullet" || continue + grep -qiE 'criterion|acceptance|\bmet\b|satisfie' <<<"$bullet" || continue + grep -qE "$url" <<<"$bullet" \ + || failures+=("$rel: a bullet ties an acceptance criterion to a follow-up but cites no issue/PR URL: $(cut -c1-90 <<<"$bullet")") + done < <(awk ' + /^[[:space:]]*[-*][[:space:]]/ { if (b != "") print b; b = $0; next } + /^[[:space:]]*$/ { if (b != "") { print b; b = "" } next } + b != "" { b = b " " $0 } + END { if (b != "") print b } + ' "$abs") +done < <(git -C "$ROOT" ls-files -- '.oh/tasks/*/evidence.md') + +if (( checked == 0 )); then + echo "SKIPPED: no tracked evidence.md to check" >&2 + exit 2 +fi + +if ((${#failures[@]})); then + printf 'REGRESSION: a follow-up is named without being cited:\n' >&2 + printf ' - %s\n' "${failures[@]}" >&2 + exit 1 +fi + +echo "PASS: the evidence contract requires follow-ups to be cited, and every tracked evidence.md that leans on one carries its URL" >&2 +exit 0 diff --git a/.oh/evals/probes/docs-build-fast-path.sh b/.oh/evals/probes/docs-build-fast-path.sh index d29d37d4..b798547b 100755 --- a/.oh/evals/probes/docs-build-fast-path.sh +++ b/.oh/evals/probes/docs-build-fast-path.sh @@ -37,7 +37,7 @@ legacy_docs_hits="$(git -C "$ROOT" grep -nI -F '.oh/docs' -- \ ':!docs/rfcs/preserved-changelog-rationale.md' \ ':!docs/rfcs/rfc-trace-ledger.md' \ ':!.oh/tasks/**' \ - ':!.oh/skills/wiki/corpus/raw/**' \ + ':!.oh/knowledge/raw/**' \ ':!.oh/evals/RESULTS.md' \ ':!.oh/evals/probes/docs-build-fast-path.sh')" legacy_docs_rc=$? diff --git a/.oh/evals/probes/evals-20260901-suite-tree-clean.sh b/.oh/evals/probes/evals-20260901-suite-tree-clean.sh new file mode 100755 index 00000000..702ffd46 --- /dev/null +++ b/.oh/evals/probes/evals-20260901-suite-tree-clean.sh @@ -0,0 +1,67 @@ +#!/usr/bin/env bash +# tier: A +# source: retro lesson 2026-09-01 (issue #926) — a probe carrying `&>2` wrote a file named 2 +# into the repository root on every suite run, for hours, while the suite reported +# all-green; it reached git through a merge commit +# desc: no probe writes into the repository — no `&>N` file redirect, no redirect targeting a +# path under $ROOT — and no redirect residue is tracked at the repository root. +# This is a STATIC guard plus a residue check, deliberately narrow: it cannot run the +# suite to observe writes without recursing into itself, and a broad "any relative +# redirect" scan is unusable noise (shell comparisons, prose arrows, and heredocs +# written inside mktemp directories all match it). +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" +PROBES="$ROOT/.oh/evals/probes" +SELF="$(basename "${BASH_SOURCE[0]}")" + +[[ -d "$PROBES" ]] || { echo "SKIPPED: probe directory absent: $PROBES" >&2; exit 2; } + +failures=() + +# --- 1. `&>N` redirects to a FILE named N, not to descriptor N ------------- +# The exact defect this lesson comes from. `>&N` duplicates a descriptor; `&>N` +# creates a file. Both are valid shell, one character apart, and the suite stays +# green either way because the probe's own exit status is unaffected. +while IFS= read -r hit; do + [[ -n "$hit" ]] || continue + file="${hit%%:*}" + [[ "$(basename "$file")" == "$SELF" ]] && continue + failures+=("$(basename "$file"): '&>N' redirects to a FILE named N, not to a descriptor — use '>&N'") +done < <(grep -rnE '&>[0-9]' "$PROBES" 2>/dev/null || true) + +# --- 2. no redirect writes into the repository by path -------------------- +# Anchored on $ROOT / $HARNESS rather than on "any relative path": the loose form +# also matches `(( n > CAP ))`, `a -> b` in prose, and heredocs a probe writes +# inside its own mktemp directory, which makes it noise rather than a check. +# $AUDIT_ROOT is deliberately NOT anchored: it is invocation-scoped by contract, and +# probes legitimately point it at a mktemp fixture they created. +while IFS= read -r hit; do + [[ -n "$hit" ]] || continue + file="${hit%%:*}" + [[ "$(basename "$file")" == "$SELF" ]] && continue + rest="${hit#*:}"; line="${rest#*:}" + code="${line%%#*}" + grep -qE '>>?[[:space:]]*"?\$\{?(ROOT|HARNESS)\b' <<<"$code" || continue + failures+=("$(basename "$file"): redirect writes into the repository: ${code## }") +done < <(grep -rnE '>>?[[:space:]]*"?\$\{?(ROOT|HARNESS)' "$PROBES" 2>/dev/null || true) + +# --- 3. no redirect residue tracked at the repository root ---------------- +# The file that motivated this probe entered the tree in a merge commit and passed +# every gate. A tracked root entry named for a bare descriptor number is residue, +# never source. +while IFS= read -r rel; do + [[ -n "$rel" ]] || continue + case "$rel" in */*) continue ;; esac + grep -qE '^([0-9]+|&[0-9]+)$' <<<"$rel" \ + && failures+=("$rel: tracked repository-root file is redirect residue, not source") +done < <(git -C "$ROOT" ls-files) + +if ((${#failures[@]})); then + printf 'REGRESSION: a probe writes into the repository, or redirect residue is tracked:\n' >&2 + printf ' - %s\n' "${failures[@]}" >&2 + exit 1 +fi + +echo "PASS: no probe redirects into the repository and no redirect residue is tracked at the root" >&2 +exit 0 diff --git a/.oh/evals/probes/knowledge-path-single-owner.sh b/.oh/evals/probes/knowledge-path-single-owner.sh new file mode 100755 index 00000000..97bf932e --- /dev/null +++ b/.oh/evals/probes/knowledge-path-single-owner.sh @@ -0,0 +1,63 @@ +#!/usr/bin/env bash +# tier: A +# source: issue #926 — durable knowledge moved to .oh/knowledge/ with no compatibility alias +# desc: exactly one writable knowledge location survives the migration — nothing is tracked +# under the retired corpus path, the directory does not exist, and no active tracked +# surface still names it +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" +cd "$ROOT" + +# Assembled from fragments so this probe is not itself a hit for the pattern it +# enforces, and therefore needs no self-exemption in its own grep. +RETIRED=".oh/skills/wiki/""corpus" + +failures=() + +[[ -e "$RETIRED" ]] && failures+=("the retired corpus directory still exists on disk: $RETIRED") + +tracked="$(git ls-files -- "$RETIRED" || true)" +[[ -n "$tracked" ]] && failures+=("files are still tracked under the retired corpus path: $(tr '\n' ' ' <<<"$tracked")") + +# No active tracked surface may reference it. CHANGELOG.md, the preserved +# changelog rationale, and .oh/knowledge/raw/ are historical records, excluded +# here for the same reason .oh/evals/probes/audit-stale-references.sh excludes +# them — an immutable capture is never rewritten to match a later rename. +set +e +refs="$(git grep -n -F -- "$RETIRED" -- \ + ':!CHANGELOG.md' \ + ':!docs/rfcs/preserved-changelog-rationale.md' \ + ':!.oh/knowledge/raw/**' \ + ':!.oh/evals/RESULTS.md' \ + ":!${BASH_SOURCE[0]#"$ROOT"/}")" +rc=$? +set -e +[[ $rc -eq 0 || $rc -eq 1 ]] || failures+=("reference scan failed") +[[ -n "$refs" ]] && failures+=("active surface still references the retired corpus path: $(tr '\n' ' ' <<<"$refs")") + +# The replacement surface must actually be there, and be the writable one. +for d in .oh/knowledge/source .oh/knowledge/patterns .oh/knowledge/raw .oh/knowledge/local; do + [[ -d "$d" ]] || failures+=("knowledge surface missing: $d") +done +[[ -f .oh/knowledge/README.md ]] || failures+=("knowledge index missing: .oh/knowledge/README.md") + +# .oh/knowledge/ must ship with the payload, or a consumer repo silently loses it. +grep -qF '"knowledge/**"' .oh/manifest.json \ + || failures+=(".oh/manifest.json does not ship knowledge/** — consumer repos would lose the surface") + +# CI must react to knowledge changes on both event types. +push_paths="$(awk '/^ push:/{p=1} /^ pull_request:/{p=0} p' .github/workflows/ci-harness.yml)" +pr_paths="$(awk '/^ pull_request:/{p=1} /^concurrency:/{p=0} p' .github/workflows/ci-harness.yml)" +grep -qF '.oh/knowledge/**' <<<"$push_paths" \ + || failures+=("ci-harness.yml push paths do not include .oh/knowledge/**") +grep -qF '.oh/knowledge/**' <<<"$pr_paths" \ + || failures+=("ci-harness.yml pull_request paths do not include .oh/knowledge/**") + +if ((${#failures[@]})); then + printf 'REGRESSION: %s\n' "${failures[@]}" >&2 + exit 1 +fi + +echo "PASS: one writable knowledge surface at .oh/knowledge/ — the retired corpus path is gone from disk, from git, and from every active reference, and the new surface ships and gates in CI" >&2 +exit 0 diff --git a/.oh/evals/probes/knowledge-source-freshness.sh b/.oh/evals/probes/knowledge-source-freshness.sh new file mode 100755 index 00000000..863cbd4f --- /dev/null +++ b/.oh/evals/probes/knowledge-source-freshness.sh @@ -0,0 +1,171 @@ +#!/usr/bin/env bash +# tier: A +# source: issue #926 — age is telemetry; a page is stale when a declared source moved +# desc: knowledge freshness is source-change and commit-aware, not age-based — every tracked +# sources: entry resolves, and knowledge-impact.sh really marks a page needs-review when +# a declared dependency changes after verified_at (exercised against a scratch repo) +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" +SCRIPT="$ROOT/.oh/skills/wiki/scripts/knowledge-impact.sh" +SCHEMA="$ROOT/.oh/skills/wiki/references/schema.md" +LINT="$ROOT/.oh/skills/wiki/references/lint.md" + +for f in "$SCRIPT" "$SCHEMA" "$LINT"; do + [[ -f "$f" ]] || { echo "SKIPPED: required file absent: $f" >&2; exit 2; } +done + +failures=() +unverifiable=0 + +# --- contract ------------------------------------------------------------- +grep -qF '## 5. Freshness is a source-change fact, not an age' "$SCHEMA" \ + || failures+=("schema.md no longer declares source-change freshness") +grep -qF 'telemetry, not a validity test' "$SCHEMA" \ + || failures+=("schema.md no longer demotes age to telemetry") +grep -qF 'knowledge-impact.sh --verified' "$LINT" \ + || failures+=("lint.md's freshness check no longer calls the one implementation") +grep -qF 'Age does not decide validity' "$LINT" \ + || failures+=("lint.md no longer records that the age rule was retired as a validity test") + +# --- every declared source must resolve in the form it declares ----------- +resolve_sources() { + local rel="$1" abs="$ROOT/$1" fm dep sha path + fm="$(awk '/^---$/{f=!f; next} f{print}' "$abs")" + while IFS= read -r dep; do + [[ -n "$dep" ]] || continue + case "$dep" in + raw/*) + [[ -f "$ROOT/.oh/knowledge/$dep" ]] \ + || failures+=("$rel: snapshot source does not resolve: $dep") + ;; + http://*|https://*) + # A bare upstream reference is the weakest provenance form (schema.md § 4). + # It cannot be resolved locally; it is accepted, not verified. + ;; + *@*) + sha="${dep##*@}"; path="${dep%@*}" + # A shallow clone (CI checks out depth 1) simply does not have the pinned + # commit. That is a clone-depth fact, not a provenance defect, so the pin is + # counted UNVERIFIABLE here rather than failed — the check still runs, and + # still fails, for every pin whose commit IS present. + if ! git -C "$ROOT" cat-file -e "${sha}^{commit}" 2>/dev/null; then + unverifiable=$((unverifiable + 1)) + continue + fi + # A pin names a revision of the file's CONTENT. If the path has since moved, + # look the basename up in that commit's tree rather than calling it broken. + if ! git -C "$ROOT" cat-file -e "$sha:$path" 2>/dev/null; then + # `git ... | grep -q` would SIGPIPE git and trip pipefail on a MATCH, + # so capture the tree first and match against the captured text. + tree="$(git -C "$ROOT" ls-tree -r --name-only "$sha" 2>/dev/null || true)" + # A basename that matches more than one path proves nothing about WHICH + # file the pin meant, so an ambiguous fallback is a failure, not a hit. + hits="$(grep -cE "(^|/)$(basename "$path" | sed 's/[].[^$*\\]/\\&/g')\$" <<<"$tree" || true)" + [[ "$hits" == "1" ]] \ + || failures+=("$rel: pinned source does not resolve at $sha (basename hits: $hits): $dep") + fi + ;; + *) + # shellcheck disable=SC2086 # deliberate glob expansion + compgen -G "$ROOT/$dep" >/dev/null 2>&1 \ + || failures+=("$rel: repository source does not resolve: $dep") + ;; + esac + done < <(awk ' + /^sources:/ {s=1; next} + s && /^[[:space:]]*-[[:space:]]/ { sub(/^[[:space:]]*-[[:space:]]*/, ""); print; next } + s { exit } + ' <<<"$fm") +} + +while IFS= read -r rel; do + [[ -n "$rel" ]] || continue + [[ "$(basename "$rel")" == "README.md" ]] && continue + resolve_sources "$rel" +done < <(git -C "$ROOT" ls-files -- '.oh/knowledge/source/*.md' '.oh/knowledge/patterns/*.md') + +# --- behavioral: drive the NEEDS-REVIEW branch, not just the PASS branch --- +# A probe whose failing branch has never been seen has an unverified oracle, so +# this builds a scratch repository where the answer is known in both directions. +tmp="$(mktemp -d)" +trap 'rm -rf "$tmp"' EXIT + +( + cd "$tmp" + git init -q . + git config user.email probe@example.com + git config user.name probe + mkdir -p .oh/knowledge/source src + printf 'v1\n' > src/watched.txt + printf 'v1\n' > src/ignored.txt + git add -A && git commit -qm base + BASE="$(git rev-parse HEAD)" + + cat > .oh/knowledge/source/watched.md <<PAGE +--- +title: "Watched" +slug: watched +kind: repo +tags: [probe] +created: 2026-01-01 +updated: 2026-01-01 +sources: + - src/watched.txt +verified_at: $BASE +confidence: provisional +--- + +# Watched + +## Relevant Source Files +- \`src/watched.txt\` — the dependency under test + +## Summary +Probe fixture. + +## Detail +Probe fixture. + +## See Also +PAGE + git add -A && git commit -qm page +) >/dev/null 2>&1 + +before="$(bash "$SCRIPT" --root "$tmp" --format slugs || true)" +if [[ -n "$before" ]]; then + failures+=("a page whose declared source has not changed was reported needs-review: $before") +fi + +( + cd "$tmp" + printf 'v2\n' > src/watched.txt + git add -A && git commit -qm "touch the declared dependency" +) >/dev/null 2>&1 + +after="$(bash "$SCRIPT" --root "$tmp" --format slugs || true)" +if [[ "$after" != "watched" ]]; then + failures+=("changing a declared dependency did not mark the page needs-review (got: '${after:-none}')") +fi + +# A page must not be invalidated by churn it never declared. Reset the watched +# dependency to its verified content and move an UNdeclared path instead. +( + cd "$tmp" + git checkout -q "$(git rev-list --max-parents=0 HEAD)" -- src/watched.txt + printf 'v2\n' > src/ignored.txt + git add -A && git commit -qm "restore the declared dependency; touch an undeclared path" +) >/dev/null 2>&1 + +undeclared="$(bash "$SCRIPT" --root "$tmp" --format slugs || true)" +if [[ -n "$undeclared" ]]; then + failures+=("a change to an undeclared path invalidated a page: $undeclared") +fi + +if ((${#failures[@]})); then + printf 'REGRESSION: %s\n' "${failures[@]}" >&2 + exit 1 +fi + +echo "PASS: knowledge freshness is source-change aware, every resolvable declared source resolves (${unverifiable} pin(s) unverifiable in this clone depth), and both the needs-review and the not-affected branches were exercised against a scratch repository" >&2 +exit 0 diff --git a/.oh/evals/probes/knowledge-tracked-query-boundary.sh b/.oh/evals/probes/knowledge-tracked-query-boundary.sh new file mode 100755 index 00000000..019a2356 --- /dev/null +++ b/.oh/evals/probes/knowledge-tracked-query-boundary.sh @@ -0,0 +1,68 @@ +#!/usr/bin/env bash +# tier: A +# source: issue #926 — ignored local scratch must never be an implicit input +# desc: /wiki query and every /spec flow read tracked knowledge only; .oh/knowledge/local/ is +# gitignored, holds nothing tracked but its README anchor, and no read path enumerates it +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" +cd "$ROOT" + +QUERY=".oh/skills/wiki/references/query.md" +PLAN=".oh/skills/spec/references/plan.md" +EXECUTE=".oh/skills/spec/references/execute.md" + +for f in "$QUERY" "$PLAN" "$EXECUTE"; do + [[ -f "$f" ]] || { echo "SKIPPED: required file absent: $f" >&2; exit 2; } +done + +failures=() + +# 1. The ignore rule must actually ignore it. +probe_file=".oh/knowledge/local/.boundary-probe-$$.md" +mkdir -p .oh/knowledge/local +printf 'scratch\n' > "$probe_file" +if ! git check-ignore -q "$probe_file"; then + failures+=(".oh/knowledge/local/ is not gitignored — a scratch page would enter the shared set") +fi +rm -f "$probe_file" + +# 2. Nothing but the README anchor may be tracked there. +while IFS= read -r rel; do + [[ -n "$rel" ]] || continue + [[ "$rel" == ".oh/knowledge/local/README.md" ]] && continue + failures+=("tracked file under the ignored scratch tier: $rel") +done < <(git ls-files -- '.oh/knowledge/local') + +# 3. The query enumeration must name only the two shared directories. +grep -qF 'DIR="$KNOWLEDGE/source"' "$QUERY" \ + || failures+=("query.md does not enumerate .oh/knowledge/source/") +grep -qF 'DIR="$KNOWLEDGE/patterns"' "$QUERY" \ + || failures+=("query.md does not enumerate .oh/knowledge/patterns/") + +# 4. Every mention of the scratch tier in a read path must be a prohibition, never +# an enumeration. A read path that globs it is the failure this probe exists for. +for f in "$QUERY" "$PLAN" "$EXECUTE"; do + if grep -nE 'knowledge/local/\*|knowledge/local/\*\.md|for .* in .*knowledge/local' "$f"; then + failures+=("$f enumerates the ignored scratch tier") + fi +done + +# 5. The prohibition must be stated where a reader of each path will meet it. +grep -qF 'does **not** read `.oh/knowledge/local/`' "$QUERY" \ + || failures+=("query.md does not state that it refuses the scratch tier") +grep -qF 'query path reads it' "$PLAN" \ + || failures+=("plan.md does not state that recall refuses the scratch tier") + +# 6. The scratch tier must document its explicit promotion path, or it becomes a +# dead end people work around by hand-moving files past the schema. +grep -qF '/wiki ingest' .oh/knowledge/local/README.md \ + || failures+=(".oh/knowledge/local/README.md documents no explicit promotion path") + +if ((${#failures[@]})); then + printf 'REGRESSION: %s\n' "${failures[@]}" >&2 + exit 1 +fi + +echo "PASS: .oh/knowledge/local/ is ignored, holds nothing tracked but its anchor, is enumerated by no read path, and has an explicit promotion path" >&2 +exit 0 diff --git a/.oh/evals/probes/retired-memory-vocabulary.sh b/.oh/evals/probes/retired-memory-vocabulary.sh new file mode 100755 index 00000000..2fd78d71 --- /dev/null +++ b/.oh/evals/probes/retired-memory-vocabulary.sh @@ -0,0 +1,66 @@ +#!/usr/bin/env bash +# tier: A +# source: issue #926 — .oh/README.md and .gitignore still described a deleted subsystem +# desc: no current architecture surface describes the retired per-session memory tier as +# active, and the surviving ignore rule is explicitly labelled a compatibility tombstone +# with a removal horizon +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" +cd "$ROOT" + +# Assembled from fragments so this probe is not itself a hit for the path it +# retires, and therefore needs no self-exemption. +RETIRED=".oh/""memory" + +failures=() + +[[ -e "$RETIRED" ]] && failures+=("the retired memory directory exists in the checkout: $RETIRED") + +tracked="$(git ls-files -- "$RETIRED" || true)" +[[ -n "$tracked" ]] && failures+=("files are tracked under the retired memory path: $tracked") + +# 1. Current architecture docs must not present it as a live subsystem. +# CHANGELOG.md, the preserved changelog rationale, and the RFCs are historical +# records, excluded exactly as .oh/evals/probes/audit-stale-references.sh +# excludes them. +set +e +hits="$(git grep -n -F -- "$RETIRED" -- \ + '.oh/README.md' 'docs' 'AGENTS.md' '.oh/templates' \ + ':!docs/rfcs/**')" +rc=$? +set -e +[[ $rc -eq 0 || $rc -eq 1 ]] || failures+=("memory-vocabulary scan failed") +[[ -n "$hits" ]] && failures+=("a current architecture doc still names the retired memory tier: $(tr '\n' ' ' <<<"$hits")") + +# 2. The .oh/ contents table must not list it. +grep -nE '^\|[[:space:]]*`memory/`' .oh/README.md \ + && failures+=(".oh/README.md still lists memory/ in its contents table") + +# 3. The ignore rule may remain ONLY as a labelled tombstone with a removal horizon. +if grep -qF "$RETIRED/" .gitignore; then + block="$(grep -B8 -F "$RETIRED/" .gitignore)" + grep -qi 'tombstone' <<<"$block" \ + || failures+=(".gitignore keeps the retired memory rule without labelling it a tombstone") + grep -qiE 'remove it in [0-9]+\.[0-9]+' <<<"$block" \ + || failures+=(".gitignore's retired memory rule names no removal horizon") + grep -qi 'local scratch, like' <<<"$block" \ + && failures+=(".gitignore still describes the retired memory tier as current scratch") +fi + +# 4. A skill that mentions it must be speaking in the past tense about a deletion, +# never describing a surface that exists. +while IFS= read -r line; do + [[ -n "$line" ]] || continue + file="${line%%:*}"; rest="${line#*:}"; text="${rest#*:}" + grep -qiE 'deleted|removed|retired|used to' <<<"$text" \ + || failures+=("$file names the retired memory tier in the present tense: $text") +done < <(git grep -n -F -- "$RETIRED" -- '.oh/skills' || true) + +if ((${#failures[@]})); then + printf 'REGRESSION: %s\n' "${failures[@]}" >&2 + exit 1 +fi + +echo "PASS: the retired memory tier appears in no current architecture doc, and its ignore rule is a labelled tombstone with a removal horizon" >&2 +exit 0 diff --git a/.oh/evals/probes/retro-deterministic-contract.sh b/.oh/evals/probes/retro-deterministic-contract.sh index cec643c5..9a7be3c8 100755 --- a/.oh/evals/probes/retro-deterministic-contract.sh +++ b/.oh/evals/probes/retro-deterministic-contract.sh @@ -31,7 +31,7 @@ for literal in \ '| ID | Subsystem | Hypothesis | Evidence for | Evidence against | Verdict | Confidence | Promotion |' \ '[<subsystem> · <confidence> · harden|proceduralize|eval] — probe: <id> | basis:' \ 'Bypassing the schema/scripts' \ - 'argument-hint: "[--dry-run] [--focus <subsystem>] [auto-approve]"' \ + 'argument-hint: "[--task <slug>] [--dry-run] [--focus <subsystem>] [auto-approve]"' \ 'STATUS: RETRO-DONE' do if ! grep -Fq "$literal" "$PI_DIR/SKILL.md"; then diff --git a/.oh/evals/probes/roles-are-skills.sh b/.oh/evals/probes/roles-are-skills.sh index 64721db5..da47e267 100755 --- a/.oh/evals/probes/roles-are-skills.sh +++ b/.oh/evals/probes/roles-are-skills.sh @@ -22,7 +22,6 @@ done agent_file_refs="$(grep -rnE '\.(oh|claude|codex|pi)/agents/[A-Za-z0-9_-]+\.md' \ .oh/skills docs AGENTS.md README.md .oh/README.md 2>/dev/null \ - | grep -v '^\.oh/skills/wiki/corpus/' \ | grep -v '^docs/rfcs/preserved-changelog-rationale\.md:' || true)" if [ -n "$agent_file_refs" ]; then echo "REGRESSION: active surfaces still cite project-agent definition files:" >&2 @@ -31,7 +30,6 @@ if [ -n "$agent_file_refs" ]; then fi retired_role_uses="$(grep -rnE '\b(Advisor|First Mate)\b' .oh/skills 2>/dev/null \ - | grep -v '^\.oh/skills/wiki/corpus/' \ | grep -vE '\b([Nn]o|[Nn]ot|[Nn]ever|[Nn]either)\b' || true)" if [ -n "$retired_role_uses" ]; then echo "REGRESSION: active skills still invoke a retired role identity (a retired role may only appear in a negation):" >&2 diff --git a/.oh/evals/probes/spec-execute-knowledge-impact.sh b/.oh/evals/probes/spec-execute-knowledge-impact.sh new file mode 100755 index 00000000..b55b7c63 --- /dev/null +++ b/.oh/evals/probes/spec-execute-knowledge-impact.sh @@ -0,0 +1,67 @@ +#!/usr/bin/env bash +# tier: A +# source: issue #926 — the planner predicts, the diff decides +# desc: /spec execute derives final knowledge impact from the ACTUAL changed paths plus page +# dependency metadata via the one shared primitive, and resolves every impacted page to +# exactly one of UPDATED / REVERIFIED / NOT-AFFECTED +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" +EXECUTE="$ROOT/.oh/skills/spec/references/execute.md" +SPEC="$ROOT/.oh/skills/spec/SKILL.md" +IMPACT="$ROOT/.oh/skills/wiki/scripts/knowledge-impact.sh" + +for f in "$EXECUTE" "$SPEC" "$IMPACT"; do + [[ -f "$f" ]] || { echo "SKIPPED: required file absent: $f" >&2; exit 2; } +done + +failures=() + +# The gate exists and is fed by the real diff. +grep -qF '### 6. Actual Knowledge Impact' "$EXECUTE" \ + || failures+=("execute.md has no Actual Knowledge Impact gate") +grep -qF 'git diff --name-only' "$EXECUTE" \ + || failures+=("execute.md's knowledge gate does not read the actual changed paths") +grep -qF 'knowledge-impact.sh \' "$EXECUTE" \ + || grep -qF 'knowledge-impact.sh --changed' "$EXECUTE" \ + || failures+=("execute.md does not call knowledge-impact.sh with the changed-path set") + +# Three explicit terminal states, and no silent fourth. +for state in '`UPDATED`' '`REVERIFIED`' '`NOT-AFFECTED (<reason>)`'; do + grep -qF -- "$state" "$EXECUTE" || failures+=("execute.md does not define the $state page state") +done +grep -qF 'is not a state, it is a skipped page' "$EXECUTE" \ + || failures+=("execute.md allows a reasonless NOT-AFFECTED") + +# The prediction must be explicitly demoted. +grep -qF 'stops being authoritative' "$EXECUTE" \ + || failures+=("execute.md still treats the planner's prediction as the oracle") +grep -qF 'the planner predicts, the diff decides' "$SPEC" \ + || grep -qF 'The planner predicts, the diff decides' "$SPEC" \ + || failures+=("the /spec dispatcher does not state that the diff decides") + +# Invalidation logic lives in the knowledge primitive, not duplicated in /spec. +grep -qF 'the one implementation of dependency-aware' "$EXECUTE" \ + || failures+=("execute.md does not defer invalidation to the knowledge primitive") + +# The primitive really accepts the changed-path mode. +grep -qF -- '--changed' "$IMPACT" \ + || failures+=("knowledge-impact.sh has no --changed mode for the actual diff") + +# Behavioral: --changed must select on declared dependencies, not on everything. +hit="$(bash "$IMPACT" --root "$ROOT" --format slugs --changed .oh/skills/spec/references/execute.md || true)" +grep -qx 'plan-vs-built-reconciliation' <<<"$hit" \ + || failures+=("--changed did not flag the page that declares the changed path as a source (got: '${hit:-none}')") + +miss="$(bash "$IMPACT" --root "$ROOT" --format slugs --changed .oh/evals/probes/spec-execute-knowledge-impact.sh || true)" +if [[ -n "$miss" ]]; then + failures+=("--changed flagged pages for a path no page declares: $miss") +fi + +if ((${#failures[@]})); then + printf 'REGRESSION: %s\n' "${failures[@]}" >&2 + exit 1 +fi + +echo "PASS: /spec execute derives knowledge impact from the actual diff through the shared primitive and resolves every page to one explicit state" >&2 +exit 0 diff --git a/.oh/evals/probes/spec-execute-running-contract.sh b/.oh/evals/probes/spec-execute-running-contract.sh new file mode 100755 index 00000000..cfd5a9e7 --- /dev/null +++ b/.oh/evals/probes/spec-execute-running-contract.sh @@ -0,0 +1,77 @@ +#!/usr/bin/env bash +# tier: A +# source: issue #926 — execute returned before the build finished while promising a ready PR; +# reconciled with issue #928, which made RUNNING task state rather than a process +# desc: execution is modelled as PLANNED -> RUNNING -> READY | DRAFT-BLOCKED(<gate>); RUNNING +# is a real state of the TASK, mirrored into a status file that names no session, and a +# run that returns mid-build is never reported as a synchronous READY +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" +EXECUTE="$ROOT/.oh/skills/spec/references/execute.md" +SPEC="$ROOT/.oh/skills/spec/SKILL.md" +PROMPT="$ROOT/.oh/skills/spec/templates/task-prompt.md" + +for f in "$EXECUTE" "$SPEC" "$PROMPT"; do + [[ -f "$f" ]] || { echo "SKIPPED: required file absent: $f" >&2; exit 2; } +done + +failures=() + +# The lifecycle, in both the dispatcher and the procedure. +for f in "$EXECUTE" "$SPEC"; do + name="$(basename "$f")" + grep -qF 'PLANNED' "$f" || failures+=("$name does not name the PLANNED state") + grep -qF 'RUNNING' "$f" || failures+=("$name does not name the RUNNING state") + grep -qF 'DRAFT-BLOCKED(<gate>)' "$f" \ + || failures+=("$name does not name DRAFT-BLOCKED(<gate>) with the gate parameterized") +done + +# RUNNING must be observable, not narrated. +grep -qF '/tmp/spec-<slug>.state' "$EXECUTE" \ + || failures+=("execute.md defines no status file, so RUNNING is not observable") +grep -qF '/tmp/spec-<slug>.state' "$PROMPT" \ + || failures+=("the task prompt does not tell the owner to keep the status file current") + +# RUNNING is a fact about the task graph, not about a running process (issue #928). +grep -qF "jq -e 'all(.userStories[]; .passes == true)'" "$EXECUTE" \ + || failures+=("execute.md does not derive RUNNING from the task graph") +grep -qF 'never the existence of a named process, session, tab, or pane' "$EXECUTE" \ + || failures+=("execute.md no longer decouples RUNNING from a named process/session/tab/pane") + +# The status file is a mirror; the task graph is the authority. +grep -qF 'the task graph wins' "$EXECUTE" \ + || failures+=("execute.md does not subordinate the status file to the task graph") +grep -qF 'RUNNING %s' "$EXECUTE" \ + || failures+=("execute.md never writes the RUNNING state") +grep -qF "printf 'READY %s" "$EXECUTE" \ + || failures+=("execute.md never writes the READY terminal state") +grep -qF "printf 'DRAFT-BLOCKED(%s) %s" "$EXECUTE" \ + || failures+=("execute.md never writes a parameterized DRAFT-BLOCKED terminal state") + +# The honesty rule: launching is RUNNING, not READY. +grep -qF 'not ceremony' "$SPEC" || grep -qF 'not decoration' "$EXECUTE" \ + || failures+=("neither surface states that RUNNING is a real state rather than ceremony") +grep -qF 'reports the state it actually reached' "$EXECUTE" \ + || failures+=("execute.md does not say that it reports the state it actually reached") +grep -qF 'Promise a PR it has not seen' "$EXECUTE" \ + || failures+=("execute.md no longer forbids promising a PR the node has not seen") + +# A silent stop is not a terminal state. +grep -qF 'a silent stop is not' "$EXECUTE" \ + || failures+=("execute.md no longer rejects a silent stop as a terminal state") + +# The single-owner model survives the lifecycle change (issue #926's pinned comment, +# as reconciled by issue #928: ownership is a role, not a terminal topology). +grep -qF 'the agent that is running it' "$EXECUTE" \ + || failures+=("the single-owner executor model was retired") +grep -qF 'Ownership is a **role**, not a terminal' "$EXECUTE" \ + || failures+=("execute.md no longer states that ownership is a role rather than a topology") + +if ((${#failures[@]})); then + printf 'REGRESSION: %s\n' "${failures[@]}" >&2 + exit 1 +fi + +echo "PASS: RUNNING is task state mirrored into a session-free status file, and a run that returns mid-build never promises a synchronous READY" >&2 +exit 0 diff --git a/.oh/evals/probes/spec-family-contract.sh b/.oh/evals/probes/spec-family-contract.sh index 51737899..69f792a4 100755 --- a/.oh/evals/probes/spec-family-contract.sh +++ b/.oh/evals/probes/spec-family-contract.sh @@ -1,9 +1,9 @@ #!/usr/bin/env bash # tier: A # source: issue #265; spec-simplification issue #816; workflow authority issue #854; -# ship-by-default issue #914 -# desc: /spec owns the four-node workflow — ship (the default, composing plan then -# execute), plan, execute, retro — and execute.md carries the complete build. +# repo-knowledge-loop issue #926 +# desc: /spec owns the workflow — plan, execute, and a retro wrapper, with an approved +# plan path as the default entry — and execute.md carries the complete build. set -u ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" @@ -11,8 +11,9 @@ SKILLS="$ROOT/.claude/skills" SPEC="$SKILLS/spec" AGENTS="$ROOT/AGENTS.md" -subs=(ship plan execute retro) -retired_subs=(critique) +subs=(plan execute retro) +# `ship` owned no mechanics of its own; the approved-plan path replaced it (#926). +retired_subs=(critique ship) if [ ! -f "$SPEC/SKILL.md" ]; then echo "SKIPPED: /spec dispatcher absent (no .claude/skills/spec/SKILL.md)" >&2 @@ -36,7 +37,7 @@ for s in "${retired_subs[@]}"; do done [ -e "$SKILLS/approve" ] && missing+=("approve: retired skill directory still present (.claude/skills/approve)") -for f in "$SPEC/SKILL.md" "$SPEC/references"/ship.md "$SPEC/references"/plan.md \ +for f in "$SPEC/SKILL.md" "$SPEC/references"/plan.md \ "$SPEC/references"/execute.md "$SPEC/references"/retro.md; do [ -f "$f" ] || continue rel="${f#"$ROOT"/}" @@ -48,19 +49,19 @@ for f in "$SPEC/SKILL.md" "$SPEC/references"/ship.md "$SPEC/references"/plan.md done [ -e "$SKILLS/ship-spec" ] && missing+=("ship-spec: the all-in-one composer must be absorbed and deleted, not left beside /spec") -SHIP="$SPEC/references/ship.md" -if [ -f "$SHIP" ]; then - # ship composes plan and execute; owning a build literal would fork execute.md. - for literal in 'gh pr create' 'gh pr ready' 'gh issue create' 'git push'; do - grep -qF "$literal" "$SHIP" && missing+=("references/ship.md carries the build literal '$literal' — ship composes plan and execute and must not fork execute.md") - done - grep -qF 'commitment gate' "$SHIP" || missing+=("references/ship.md does not state how it treats the commitment gate") +# retro is a wrapper, not a second engine: it must own no retro mechanics. +RETRO="$SPEC/references/retro.md" +if [ -f "$RETRO" ]; then + grep -qF '/retro --task' "$RETRO" \ + || missing+=("references/retro.md does not delegate to the task-scoped /retro form") + grep -qE '^## Hypotheses|^\| ID \| Subsystem' "$RETRO" \ + && missing+=("references/retro.md carries a second retro ontology instead of delegating") fi -# The dispatcher must route an unrecognized first token to ship, not to usage. -grep -qF 'ship|plan|execute|retro)' "$SPEC/SKILL.md" \ - || missing+=("SKILL.md dispatch case does not name all four nodes") -grep -qE 'DEFAULT: not a node name' "$SPEC/SKILL.md" \ - || missing+=("SKILL.md no longer routes an unrecognized first token to ship (a plan path would print usage)") +# The dispatcher must route an unrecognized first token to the approved-plan path. +grep -qF 'plan|execute|retro)' "$SPEC/SKILL.md" \ + || missing+=("SKILL.md dispatch case does not name all three nodes") +grep -qE 'DEFAULT: an approved plan path' "$SPEC/SKILL.md" \ + || missing+=("SKILL.md no longer routes an unrecognized first token to the approved-plan path (a plan path would print usage)") EXEC="$SPEC/references/execute.md" if [ -f "$EXEC" ]; then grep -qF 'reuses those by reference' "$EXEC" && missing+=("execute.md still defers its build mechanics by reference instead of holding them") @@ -89,5 +90,5 @@ if [ "${#missing[@]}" -gt 0 ]; then exit 1 fi -echo "PASS: /spec owns the workflow, dispatches four procedures with ship as the default composing plan and execute, carries no loop ## Handoff, keeps retired surfaces absent, and holds the build literals" >&2 +echo "PASS: /spec owns the workflow, dispatches plan/execute/retro with an approved plan path as the default, carries no loop ## Handoff, keeps retired surfaces absent, and holds the build literals" >&2 exit 0 diff --git a/.oh/evals/probes/spec-no-generated-prompt-contract.sh b/.oh/evals/probes/spec-no-generated-prompt-contract.sh new file mode 100755 index 00000000..db92c3ec --- /dev/null +++ b/.oh/evals/probes/spec-no-generated-prompt-contract.sh @@ -0,0 +1,57 @@ +#!/usr/bin/env bash +# tier: A +# source: issue #926 — a persisted copy of a template drifts from the template +# desc: the durable task contract is prd.md + prd.json + progress.txt; no generated prompt.md +# is written, verified, or required, and the launch prompt is rendered at execution time +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" +cd "$ROOT" + +PLAN=".oh/skills/spec/references/plan.md" +EXECUTE=".oh/skills/spec/references/execute.md" +SPEC=".oh/skills/spec/SKILL.md" +TEMPLATE=".oh/skills/spec/templates/task-prompt.md" +TASKS_README=".oh/tasks/README.md" + +for f in "$PLAN" "$EXECUTE" "$SPEC" "$TEMPLATE" "$TASKS_README"; do + [[ -f "$f" ]] || { echo "SKIPPED: required file absent: $f" >&2; exit 2; } +done + +failures=() + +# The template survives — it is the render source, not an artifact. +grep -qiF 'template' "$TEMPLATE" \ + || failures+=("the task prompt no longer identifies itself as a render-time template") +grep -qF 'Render the launch prompt now; do not persist it' "$EXECUTE" \ + || grep -qF 'rendered at execution time' "$EXECUTE" \ + || failures+=("execute.md does not render the launch prompt at execution time") + +# No surface may require, verify, or write the generated artifact. +generated="prompt""\\.md" +for f in "$PLAN" "$EXECUTE" "$SPEC" "$TASKS_README"; do + hits="$(grep -nE "\\.oh/tasks/[^ \`]*/$generated|\\\$f\" .*$generated" "$f" || true)" + [[ -n "$hits" ]] && failures+=("$f still names a task-folder prompt artifact: $hits") +done + +# The three-file contract is what gets verified. +grep -qF 'for f in prd.md prd.json progress.txt; do' "$PLAN" \ + || failures+=("plan.md does not verify the three-file contract") +grep -qF 'three-file contract' "$EXECUTE" \ + || failures+=("execute.md does not name the three-file contract as its precondition") + +# And the reason is written down, so it is not reintroduced as a convenience. +grep -qF 'drift' "$PLAN" \ + || failures+=("plan.md does not record why the generated copy was retired") + +# Repository-wide: nothing under the task surface may carry the retired artifact. +tracked="$(git ls-files -- '.oh/tasks/*/'"prompt"'.md' || true)" +[[ -n "$tracked" ]] && failures+=("a generated task prompt is still tracked: $tracked") + +if ((${#failures[@]})); then + printf 'REGRESSION: %s\n' "${failures[@]}" >&2 + exit 1 +fi + +echo "PASS: the durable task contract is prd.md + prd.json + progress.txt; the launch prompt is rendered, never persisted" >&2 +exit 0 diff --git a/.oh/evals/probes/spec-plan-knowledge-context.sh b/.oh/evals/probes/spec-plan-knowledge-context.sh new file mode 100755 index 00000000..9d92c05b --- /dev/null +++ b/.oh/evals/probes/spec-plan-knowledge-context.sh @@ -0,0 +1,74 @@ +#!/usr/bin/env bash +# tier: A +# source: issue #926 — /spec wrote knowledge more reliably than it read it +# desc: /spec plan recalls TRACKED knowledge and re-grounds it against current sources before +# the PRD exists, and records Knowledge Context in prd.md; the retired Wiki Alignment +# block is gone from the planning contract +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" +PLAN="$ROOT/.oh/skills/spec/references/plan.md" +SPEC="$ROOT/.oh/skills/spec/SKILL.md" + +for f in "$PLAN" "$SPEC"; do + [[ -f "$f" ]] || { echo "SKIPPED: required file absent: $f" >&2; exit 2; } +done + +failures=() +need() { grep -qF -- "$1" "$PLAN" || failures+=("plan.md missing contract text: $1"); } +# An EXACT-LINE pin: a heading that merely names the block must not satisfy the +# assertion that the block itself is still specified. +need_line() { grep -qxF -- "$1" "$PLAN" || failures+=("plan.md no longer specifies the block: $1"); } + +# The three-step recall contract, and the section that records it. +need '### 2. Recall tracked knowledge' +need '### 3. Ground the recalled claims against current sources' +need '/wiki query <terms>' +need '/wiki query <terms> --patterns' +need_line '## Knowledge Context' +need_line '## Expected Knowledge Impact' +need '- **Base commit**: `<sha>`' +need '- **Queries**: `<queries used>`' +need '- **Knowledge used**: `[[slug]]`, ... or `none`' +need '- **Grounded against**: `<repo-relative paths>`' +need '- **Conflicts discovered**: `none` or concise reconciliation' + +# Order matters: recall and grounding must precede the PRD, or the loop is open. +recall_line=$(grep -n '^### 2\. Recall tracked knowledge' "$PLAN" | cut -d: -f1) +ground_line=$(grep -n '^### 3\. Ground the recalled claims' "$PLAN" | cut -d: -f1) +prd_line=$(grep -n '^### 4\. `/prd` ' "$PLAN" | cut -d: -f1) +if [[ -z "$recall_line" || -z "$ground_line" || -z "$prd_line" ]]; then + failures+=("plan.md no longer has the numbered recall / ground / prd steps this probe reads") +elif (( recall_line >= ground_line || ground_line >= prd_line )); then + failures+=("plan.md orders the pipeline wrong — recall and grounding must precede /prd") +fi + +# Knowledge is a cache, not an authority. +grep -qF 'the source wins and the page is wrong' "$PLAN" \ + || failures+=("plan.md no longer subordinates recalled knowledge to the repository") + +# Recall reads the SHARED set only. +grep -qF '.oh/knowledge/source/' "$PLAN" \ + || failures+=("plan.md does not name the tracked entity-page directory") +grep -qF '.oh/knowledge/patterns/' "$PLAN" \ + || failures+=("plan.md does not name the tracked pattern directory") + +# The retired planning-time oracle must not come back AS A BLOCK. A code-spanned +# mention is how the supersession is documented; a heading is the reappearance. +if grep -nE '^## Wiki Alignment[[:space:]]*$' "$PLAN" "$SPEC"; then + failures+=("the retired Wiki Alignment planning block reappeared as a section heading") +fi +grep -qF '`## Wiki Alignment` is superseded' "$PLAN" \ + || failures+=("plan.md does not record that Wiki Alignment was superseded") + +# The dispatcher must describe the loop, not just the nodes. +grep -qF 'recall tracked knowledge' "$SPEC" \ + || failures+=("the /spec dispatcher no longer shows recall in the loop") + +if ((${#failures[@]})); then + printf 'REGRESSION: %s\n' "${failures[@]}" >&2 + exit 1 +fi + +echo "PASS: /spec plan recalls tracked knowledge and re-grounds it before the PRD, and records Knowledge Context" >&2 +exit 0 diff --git a/.oh/evals/probes/spec-plan-reconciliation-gate.sh b/.oh/evals/probes/spec-plan-reconciliation-gate.sh new file mode 100755 index 00000000..e154eadb --- /dev/null +++ b/.oh/evals/probes/spec-plan-reconciliation-gate.sh @@ -0,0 +1,61 @@ +#!/usr/bin/env bash +# tier: A +# source: issue #926 — an approved plan may not become a materially different PRD silently +# desc: /spec plan records Plan Reconciliation and STOPS for operator re-approval when +# grounding materially changes the approved intent; /spec execute refuses a folder +# whose reconciliation says the intent was not preserved +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" +PLAN="$ROOT/.oh/skills/spec/references/plan.md" +EXECUTE="$ROOT/.oh/skills/spec/references/execute.md" +SPEC="$ROOT/.oh/skills/spec/SKILL.md" + +for f in "$PLAN" "$EXECUTE" "$SPEC"; do + [[ -f "$f" ]] || { echo "SKIPPED: required file absent: $f" >&2; exit 2; } +done + +failures=() + +# EXACT-LINE: a heading that names the block must not satisfy the assertion that +# the block itself is still specified. +grep -qxF '## Plan Reconciliation' "$PLAN" \ + || failures+=("plan.md no longer specifies the ## Plan Reconciliation block") + +# Every field a downstream gate reads. +for lit in \ + '- **Source plan**: `<path>`' \ + '- **Intent preserved**: YES | NO' \ + '- **Material deviations**: `none` or list' \ + '- **Constraints discovered during grounding**: `none` or list' +do + grep -qF -- "$lit" "$PLAN" || failures+=("plan.md missing reconciliation field: $lit") +done + +# The gate must STOP, not warn. +grep -qF 'stop before execution' "$PLAN" \ + || failures+=("plan.md's reconciliation gate does not stop before execution") +grep -qF 'require operator re-approval' "$PLAN" \ + || failures+=("plan.md does not require operator re-approval on a material deviation") +grep -qF 'material' "$PLAN" \ + || failures+=("plan.md does not distinguish a material deviation from a discovered constraint") + +# The dispatcher must state that approval covers the approved intent only. +grep -qF 'not whatever grounding turns it' "$SPEC" \ + || failures+=("the /spec dispatcher no longer bounds the approval to the approved intent") + +# Execute is the enforcement point: a NO must not flow through. +grep -qF 'Intent preserved: YES' "$EXECUTE" \ + || failures+=("execute.md does not check the reconciliation verdict as a precondition") +grep -qF 'DRAFT-BLOCKED(reconciliation)' "$EXECUTE" \ + || failures+=("execute.md has no reconciliation blocked state") +grep -qF 'route back to' "$EXECUTE" \ + || failures+=("execute.md does not route a failed precondition back to /spec plan") + +if ((${#failures[@]})); then + printf 'REGRESSION: %s\n' "${failures[@]}" >&2 + exit 1 +fi + +echo "PASS: a materially changed approved intent stops for re-approval and cannot flow into /spec execute" >&2 +exit 0 diff --git a/.oh/evals/probes/spec-ready-finalization.sh b/.oh/evals/probes/spec-ready-finalization.sh index a62f119a..d1fa608c 100755 --- a/.oh/evals/probes/spec-ready-finalization.sh +++ b/.oh/evals/probes/spec-ready-finalization.sh @@ -63,6 +63,23 @@ for section in 'diverged' 'unverified'; do fi done +# A promotable verdict describes ONE head. Undrafting at a commit and then pushing +# past it leaves a ready PR standing on a classification that no longer describes +# what a reviewer sees, so the procedure must (a) confirm the PR head is the commit +# being promoted and (b) re-open the gate on any later push. +if ! grep -qF 'headRefOid' <<<"$final_section"; then + echo "REGRESSION: /spec execute does not confirm the PR head is the commit it is promoting" >&2 + exit 1 +fi +if ! grep -qF 'gate re-opens on every push after the undraft' <<<"$final_section"; then + echo "REGRESSION: /spec execute no longer re-opens the promotable gate on a post-undraft push" >&2 + exit 1 +fi +if ! grep -qF 'gh pr ready --undo' <<<"$final_section"; then + echo "REGRESSION: /spec execute names no way back to draft when a pushed head stops being promotable" >&2 + exit 1 +fi + execute_line=$(grep -E '^\| `execute` \|' "$SPEC" || true) if [[ -z "$execute_line" ]]; then echo "REGRESSION: /spec dispatcher missing execute row" >&2 @@ -84,5 +101,5 @@ if [[ -e "$PI_EXEC" ]] && ! grep -qF 'Finalization contract' "$PI_EXEC"; then exit 1 fi -echo "PASS: /spec execute treats the draft PR as a checkpoint, refuses the undraft without a tracked evidence.md, surfaces divergence + unverified in the PR body, and gates ready-for-review on the promotable classification" >&2 +echo "PASS: /spec execute treats the draft PR as a checkpoint, refuses the undraft without a tracked evidence.md, surfaces divergence + unverified in the PR body, gates ready-for-review on the promotable classification, and re-opens that gate when the head moves past it" >&2 exit 0 diff --git a/.oh/evals/probes/task-completion-structured-state.sh b/.oh/evals/probes/task-completion-structured-state.sh new file mode 100755 index 00000000..58221949 --- /dev/null +++ b/.oh/evals/probes/task-completion-structured-state.sh @@ -0,0 +1,89 @@ +#!/usr/bin/env bash +# tier: A +# source: issue #926 — a prose sentinel duplicated structured task-graph state +# desc: task completion derives from prd.json (every required story passes); the retired +# progress.txt sentinel survives in no active consumer, doc, template, or cron +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" +cd "$ROOT" + +CRON="crons/cleanup-tasks.md" +TASKS_README=".oh/tasks/README.md" +EXECUTE=".oh/skills/spec/references/execute.md" +SPEC=".oh/skills/spec/SKILL.md" +PROMPT=".oh/skills/spec/templates/task-prompt.md" + +for f in "$CRON" "$TASKS_README" "$EXECUTE" "$SPEC" "$PROMPT"; do + [[ -f "$f" ]] || { echo "SKIPPED: required file absent: $f" >&2; exit 2; } +done + +failures=() + +# Assembled from fragments so this probe is not itself a hit for the token it +# retires, and therefore needs no self-exemption. +SENTINEL="STATUS: ""COMPLETE" + +# 1. No ACTIVE surface may key on the prose sentinel. Historical records are +# excluded for the same reason .oh/evals/probes/audit-stale-references.sh +# excludes them: the changelog and the RFCs cite it as a past exhibit, and +# .oh/knowledge/raw/ holds immutable captures that are never rewritten. +set +e +hits="$(git grep -n -F -- "$SENTINEL" -- \ + ':!CHANGELOG.md' \ + ':!docs/rfcs/**' \ + ':!.oh/knowledge/raw/**' \ + ':!.oh/evals/RESULTS.md' \ + ':!.oh/tasks/**' \ + ":!${BASH_SOURCE[0]#"$ROOT"/}")" +rc=$? +set -e +[[ $rc -eq 0 || $rc -eq 1 ]] || failures+=("sentinel scan failed") +[[ -n "$hits" ]] && failures+=("active surface still keys on the retired completion sentinel: $(tr '\n' ' ' <<<"$hits")") + +# 2. Any RFC that still cites it must mark it retired, so no reader takes it for +# current architecture. +while IFS= read -r line; do + [[ -n "$line" ]] || continue + file="${line%%:*}"; rest="${line#*:}"; text="${rest#*:}" + grep -qi 'retired\|no longer\|since been' <<<"$text" \ + || failures+=("$file cites the retired sentinel without marking it retired: $text") +done < <(git grep -n -F -- "$SENTINEL" -- 'docs/rfcs' || true) + +# 3. The structured check must be the one that runs. +JQ_CHECK="all(.userStories[]; .passes == true)" +grep -qF -- "$JQ_CHECK" "$CRON" \ + || failures+=("the cleanup cron does not derive completion from prd.json story state") +grep -qF -- "$JQ_CHECK" "$TASKS_README" \ + || failures+=(".oh/tasks/README.md does not document the structured completion check") +grep -qF -- "$JQ_CHECK" "$EXECUTE" \ + || failures+=("execute.md does not derive completion from prd.json story state") +grep -qF -- "$JQ_CHECK" "$PROMPT" \ + || failures+=("the task prompt does not tell the owner how completion is decided") +grep -qF 'Completion is structured state' "$SPEC" \ + || failures+=("the /spec dispatcher does not state that completion is structured state") + +# 4. An unreadable task graph must not read as complete. +grep -qiF 'no readable `prd.json`' "$CRON" \ + || grep -qiF 'unreadable' "$CRON" \ + || failures+=("the cleanup cron does not say what happens when prd.json is unreadable") + +# 5. Behavioral: the documented jq expression must actually answer both ways. +tmp="$(mktemp -d)"; trap 'rm -rf "$tmp"' EXIT +printf '%s\n' '{"userStories":[{"passes":true},{"passes":true}]}' > "$tmp/done.json" +printf '%s\n' '{"userStories":[{"passes":true},{"passes":false}]}' > "$tmp/open.json" +printf '%s\n' '{"userStories":[]}' > "$tmp/empty.json" +jq -e "$JQ_CHECK" "$tmp/done.json" >/dev/null 2>&1 \ + || failures+=("the documented completion check rejects an all-passing task graph") +jq -e "$JQ_CHECK" "$tmp/open.json" >/dev/null 2>&1 \ + && failures+=("the documented completion check accepts a graph with an unpassed story") +jq -e "$JQ_CHECK" "$tmp/empty.json" >/dev/null 2>&1 \ + || failures+=("the documented completion check rejects an empty task graph unexpectedly") + +if ((${#failures[@]})); then + printf 'REGRESSION: %s\n' "${failures[@]}" >&2 + exit 1 +fi + +echo "PASS: task completion derives from prd.json structured state; the prose sentinel survives only in marked historical records" >&2 +exit 0 diff --git a/.oh/evals/probes/wiki-compile-contract.sh b/.oh/evals/probes/wiki-compile-contract.sh index 1ec72120..37c46602 100755 --- a/.oh/evals/probes/wiki-compile-contract.sh +++ b/.oh/evals/probes/wiki-compile-contract.sh @@ -29,7 +29,7 @@ need "$SKILL" "\`compile\`'s pattern-page writes" need "$COMPILE" '/wiki compile [--from <path>] [--task <slug>] [--dry-run]' need "$COMPILE" 'MUST NOT write a `raw/` snapshot of a `/retro` report' -need "$COMPILE" '§ 7 as amended by § 7a' +need "$COMPILE" '§ 11 as amended by § 11a' need "$COMPILE" 'orchestrator-only' need "$COMPILE" 'One page per failure mode' need "$COMPILE" '## Contents' diff --git a/.oh/evals/probes/wiki-kind-schema-contract.sh b/.oh/evals/probes/wiki-kind-schema-contract.sh index d185a3f0..07b89350 100755 --- a/.oh/evals/probes/wiki-kind-schema-contract.sh +++ b/.oh/evals/probes/wiki-kind-schema-contract.sh @@ -1,78 +1,125 @@ #!/usr/bin/env bash # tier: A -# source: wikiskill arXiv:2608.27454 — pattern layer added to the wiki corpus -# desc: the wiki schema declares kind: source|pattern with an absent-means-source default, the pattern merge amendment, and the persistence invariant; every tracked entry obeys the kind/filename/placement rules +# source: wikiskill arXiv:2608.27454 (pattern layer); issue #926 (repo|external|pattern kinds) +# desc: the knowledge schema declares kind repo|external|pattern with sources: as the single +# dependency declaration and verified_at: as the freshness pin; every tracked entry obeys +# the kind, directory, filename, and required-section rules set -euo pipefail ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" SCHEMA="$ROOT/.oh/skills/wiki/references/schema.md" -CORPUS_REL=".oh/skills/wiki/corpus" +KNOWLEDGE_REL=".oh/knowledge" if [[ ! -f "$SCHEMA" ]]; then - echo "SKIPPED: wiki schema absent: $SCHEMA" >&2 + echo "SKIPPED: knowledge schema absent: $SCHEMA" >&2 exit 2 fi failures=() +# Short, wrap-safe fragments only: headings, table cells, and code tokens cannot +# straddle a hard-wrap boundary the way a pinned sentence can. need() { grep -qF -- "$1" "$SCHEMA" || failures+=("schema.md missing contract text: $1") } -need '| `kind` | enum | no |' -need 'An absent `kind:` field means `kind: source`.' -need 'Consumers that filter on `kind` MUST apply the' -need 'Pattern placement.' -need 'a pattern in a subdirectory would be invisible to both' -need '## 7a. Pattern amendment to the body-merge strategy' +need '| `kind` | enum | yes |' +need 'Must agree with the directory' +need '| `verified_at` | commit sha | `kind: repo` only |' +need '## 4. `sources:` is the dependency declaration' +need '## 5. Freshness is a source-change fact, not an age' +need 'never snapshots this repository' +need '| Bare upstream reference |' +need '### 11a. Pattern amendment' need 'is **append-only**' -need '## 8. Pattern persistence invariant' +need '## 12. Pattern persistence invariant' need 'is never rolled back' -need 'Reverting a `corpus/` path as collateral of a skill revert is forbidden.' +need 'as collateral of a skill revert is' -# Structural checks over tracked corpus entries. +# Structural checks over tracked entries. +entries=() while IFS= read -r rel; do + [[ -n "$rel" ]] || continue + entries+=("$rel") +done < <(git -C "$ROOT" ls-files -- \ + "$KNOWLEDGE_REL/source/*.md" \ + "$KNOWLEDGE_REL/patterns/*.md") + +if ((${#entries[@]} == 0)); then + echo "SKIPPED: no tracked knowledge entries" >&2 + exit 2 +fi + +for rel in "${entries[@]}"; do base="$(basename "$rel")" [[ "$base" == "README.md" ]] && continue - [[ "$base" == "skill-impact.md" ]] && continue abs="$ROOT/$rel" [[ -f "$abs" ]] || continue fm="$(awk '/^---$/{f=!f; next} f{print}' "$abs")" slug="$(grep '^slug:' <<<"$fm" | awk '{print $2}' | head -1 || true)" - [[ -z "$slug" ]] && continue - kind="$(grep '^kind:' <<<"$fm" | awk '{print $2}' | head -1 || true)" - kind="${kind:-source}" + if [[ -z "$slug" ]]; then + failures+=("$rel: no slug: in frontmatter") + continue + fi + [[ "$slug" == "${base%.md}" ]] || failures+=("$rel: slug '$slug' does not match the filename") + kind="$(grep '^kind:' <<<"$fm" | awk '{print $2}' | head -1 || true)" case "$kind" in - source|pattern) ;; - *) failures+=("$rel: kind must be source or pattern, got '$kind'");; + repo|external|pattern) ;; + "") failures+=("$rel: kind: is required (repo, external, or pattern)"); continue ;; + *) failures+=("$rel: kind must be repo, external, or pattern, got '$kind'"); continue ;; esac - if [[ "$base" == pattern-* && "$kind" != "pattern" ]]; then - failures+=("$rel: filename says pattern but kind is '$kind'") + # The directory IS the kind boundary; frontmatter must agree with it. + case "$rel" in + "$KNOWLEDGE_REL"/patterns/*) + [[ "$kind" == "pattern" ]] \ + || failures+=("$rel: lives in patterns/ but kind is '$kind'") + [[ "$base" == pattern-* ]] \ + || failures+=("$rel: a patterns/ filename must carry the pattern- prefix") + ;; + "$KNOWLEDGE_REL"/source/*) + [[ "$kind" == "repo" || "$kind" == "external" ]] \ + || failures+=("$rel: lives in source/ but kind is '$kind'") + [[ "$base" == pattern-* ]] \ + && failures+=("$rel: a source/ filename must not carry the pattern- prefix") + ;; + esac + + grep -q '^sources:' <<<"$fm" || failures+=("$rel: sources: is required") + + if [[ "$kind" == "repo" || "$kind" == "pattern" ]]; then + grep -q '^## Relevant Source Files$' "$abs" \ + || failures+=("$rel: kind: $kind requires a '## Relevant Source Files' section") fi - if [[ "$kind" == "pattern" && "$base" != pattern-* ]]; then - failures+=("$rel: kind is pattern but filename lacks the pattern- prefix") + + if [[ "$kind" == "repo" ]]; then + grep -q '^verified_at:' <<<"$fm" \ + || failures+=("$rel: kind: repo requires verified_at: (the freshness pin)") fi - if [[ "$kind" == "pattern" ]]; then - grep -q '^## Relevant Source Files$' "$abs" \ - || failures+=("$rel: kind: pattern requires a '## Relevant Source Files' section") - grep -q '^sources:' <<<"$fm" \ - || failures+=("$rel: kind: pattern requires at least one sources: entry") + + if [[ "$kind" == "external" ]]; then + grep -qE '^[[:space:]]*-[[:space:]]+(raw/|https?://)' <<<"$fm" \ + || failures+=("$rel: kind: external requires a raw/ snapshot or a bare upstream URL in sources:") fi -done < <(git -C "$ROOT" ls-files -- "$CORPUS_REL/*.md" ":!:$CORPUS_REL/raw/*") +done -# Placement: no tracked entry may live in a corpus subdirectory other than raw/. +# Placement: nothing tracked under .oh/knowledge/ outside the three real directories. while IFS= read -r rel; do [[ -n "$rel" ]] || continue - failures+=("$rel: corpus entries are flat; only raw/ may be a subdirectory") -done < <(git -C "$ROOT" ls-files -- "$CORPUS_REL/*/*" ":!:$CORPUS_REL/raw/*") + case "$rel" in + "$KNOWLEDGE_REL"/README.md) ;; + "$KNOWLEDGE_REL"/source/*.md|"$KNOWLEDGE_REL"/patterns/*.md) ;; + "$KNOWLEDGE_REL"/raw/*|"$KNOWLEDGE_REL"/local/README.md) ;; + *) failures+=("$rel: unexpected tracked path under the knowledge surface") ;; + esac +done < <(git -C "$ROOT" ls-files -- "$KNOWLEDGE_REL") if ((${#failures[@]})); then printf 'REGRESSION: %s\n' "${failures[@]}" >&2 exit 1 fi -echo "PASS: wiki kind schema declared and every tracked corpus entry obeys the kind, filename, and placement rules" >&2 +echo "PASS: the knowledge schema declares repo/external/pattern kinds with verified_at freshness, and every tracked entry obeys the kind, directory, filename, and section rules" >&2 exit 0 diff --git a/.oh/evals/probes/wiki-pattern-persistence.sh b/.oh/evals/probes/wiki-pattern-persistence.sh index 8d3ba307..820b0e9b 100755 --- a/.oh/evals/probes/wiki-pattern-persistence.sh +++ b/.oh/evals/probes/wiki-pattern-persistence.sh @@ -5,7 +5,7 @@ set -euo pipefail ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" -CORPUS_REL=".oh/skills/wiki/corpus" +CORPUS_REL=".oh/knowledge" # WIKI_PERSISTENCE_BASE overrides the comparison point. It exists so the invariant # can actually be exercised against a real revert instead of only asserted: cut a @@ -25,12 +25,14 @@ if [[ -z "$base" ]]; then exit 2 fi +# Pattern pages are keyed by BASENAME, not by path: the knowledge surface has +# been relocated before, and a page that merely moved must not read as deleted. base_patterns=() while IFS= read -r rel; do [[ -n "$rel" ]] || continue - [[ "$(basename "$rel")" == pattern-* ]] || continue base_patterns+=("$rel") -done < <(git -C "$ROOT" ls-tree -r --name-only "$base" -- "$CORPUS_REL" | grep '\.md$' || true) +done < <(git -C "$ROOT" ls-tree -r --name-only "$base" -- '.oh' \ + | grep -E '(^|/)pattern-[a-z0-9-]+\.md$' || true) if ((${#base_patterns[@]} == 0)); then echo "SKIPPED: no kind: pattern entries tracked at the merge-base — nothing to protect yet" >&2 @@ -38,23 +40,27 @@ if ((${#base_patterns[@]} == 0)); then fi +count_sources() { + awk '/^---$/{f=!f; next} f{print}' \ + | awk '/^sources:/{s=1; next} s && /^[a-z_-]+:/{s=0} s && /^[[:space:]]*- /{n++} END{print n+0}' +} + deleted=(); shrunk=() for rel in "${base_patterns[@]}"; do - if ! git -C "$ROOT" ls-files --error-unmatch "$rel" >/dev/null 2>&1; then - deleted+=("$rel") + head_rel="$CORPUS_REL/patterns/$(basename "$rel")" + if ! git -C "$ROOT" ls-files --error-unmatch "$head_rel" >/dev/null 2>&1; then + deleted+=("$(basename "$rel")") continue fi - before="$(git -C "$ROOT" show "$base:$rel" | awk '/^---$/{f=!f; next} f{print}' \ - | awk '/^sources:/{s=1; next} s && /^[a-z_-]+:/{s=0} s && /^[[:space:]]*- /{n++} END{print n+0}')" - after="$(awk '/^---$/{f=!f; next} f{print}' "$ROOT/$rel" \ - | awk '/^sources:/{s=1; next} s && /^[a-z_-]+:/{s=0} s && /^[[:space:]]*- /{n++} END{print n+0}')" + before="$(git -C "$ROOT" show "$base:$rel" | count_sources)" + after="$(count_sources < "$ROOT/$head_rel")" if (( after < before )); then - shrunk+=("$rel ($before -> $after)") + shrunk+=("$head_rel ($before -> $after)") fi done if ((${#deleted[@]})); then - printf 'REGRESSION: pattern page removed since the merge-base — see schema.md § 8, patterns are never rolled back: %s\n' "${deleted[@]}" >&2 + printf 'REGRESSION: pattern page removed since the merge-base — see schema.md § 12, patterns are never rolled back: %s\n' "${deleted[@]}" >&2 exit 1 fi if ((${#shrunk[@]})); then diff --git a/.oh/evals/probes/wiki-query-pattern-isolation.sh b/.oh/evals/probes/wiki-query-pattern-isolation.sh index 446f6c0a..58c62bf5 100755 --- a/.oh/evals/probes/wiki-query-pattern-isolation.sh +++ b/.oh/evals/probes/wiki-query-pattern-isolation.sh @@ -1,12 +1,14 @@ #!/usr/bin/env bash # tier: A # source: wikiskill arXiv:2608.27454 — proposer-only pattern access -# desc: /wiki query declares two disjoint kind modes with per-mode caps and term-hit ranking on the locked awk, and no kind: pattern entry appears in a default-mode result set +# desc: /wiki query declares two disjoint directory-scoped modes with per-mode caps and +# term-hit ranking on the locked awk, and no kind: pattern entry can appear in a +# default-mode result set set -euo pipefail ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" QUERY="$ROOT/.oh/skills/wiki/references/query.md" -CORPUS_REL=".oh/skills/wiki/corpus" +KNOWLEDGE_REL=".oh/knowledge" if [[ ! -f "$QUERY" ]]; then echo "SKIPPED: wiki query reference absent: $QUERY" >&2 @@ -17,14 +19,12 @@ failures=() need() { grep -qF -- "$1" "$QUERY" || failures+=("query.md missing contract text: $1"); } need '/wiki query <topic> [--patterns]' -need 'an absent `kind:` counts as `source`' need 'There is deliberately **no `--all` mode**' need 'This is a **default, not a boundary**' -need 'WANT_KIND=source' -need 'WANT_KIND=pattern' +need 'DIR="$KNOWLEDGE/source"' +need 'DIR="$KNOWLEDGE/patterns"' need 'CAP=3' need 'CAP=5' -need 'kind="${kind:-source}"' need "awk '/^---\$/{f=!f; next} f{print}'" if ((${#failures[@]})); then @@ -32,41 +32,51 @@ if ((${#failures[@]})); then exit 1 fi -# Behavioral: default mode must never surface a kind: pattern entry. +# Behavioral: the directory IS the mode, so default mode enumerates source/ only. +# A kind: pattern page tracked under source/ would leak into every default result. patterns=() +leaked=() while IFS= read -r rel; do [[ -n "$rel" ]] || continue abs="$ROOT/$rel" [[ -f "$abs" ]] || continue kind="$(awk '/^---$/{f=!f; next} f{print}' "$abs" | grep '^kind:' | awk '{print $2}' | head -1 || true)" - [[ "${kind:-source}" == "pattern" ]] && patterns+=("$rel") -done < <(git -C "$ROOT" ls-files -- "$CORPUS_REL/*.md" ":!:$CORPUS_REL/raw/*") + case "$rel" in + "$KNOWLEDGE_REL"/patterns/*) + [[ "$kind" == "pattern" ]] && patterns+=("$rel") + ;; + "$KNOWLEDGE_REL"/source/*) + [[ "$kind" == "pattern" ]] && leaked+=("$rel") + ;; + esac +done < <(git -C "$ROOT" ls-files -- \ + "$KNOWLEDGE_REL/source/*.md" \ + "$KNOWLEDGE_REL/patterns/*.md") + +if ((${#leaked[@]})); then + printf 'REGRESSION: kind: pattern entry tracked in source/, where default-mode query reads it: %s\n' "${leaked[@]}" >&2 + exit 1 +fi if ((${#patterns[@]} == 0)); then - echo "SKIPPED: no tracked kind: pattern entries yet — contract text verified, isolation untestable" >&2 + echo "SKIPPED: no tracked kind: pattern entries yet — contract text verified, split untestable" >&2 exit 2 fi -# Run the documented default-mode filter (query.md § 3) over the whole tracked -# corpus, using a term drawn from a pattern page's own frontmatter. No pattern -# entry may survive it. +# Run the documented default-mode enumeration (query.md § 2) with a term drawn from +# a pattern page's own frontmatter. No pattern page may survive it. probe_term="$(awk '/^---$/{f=!f; next} f{print}' "$ROOT/${patterns[0]}" \ | grep '^slug:' | awk '{print $2}' | head -1)" -WANT_KIND=source matched=() -while IFS= read -r rel; do - abs="$ROOT/$rel" - [[ -f "$abs" ]] || continue - fm="$(awk '/^---$/{f=!f; next} f{print}' "$abs")" - kind="$(grep '^kind:' <<<"$fm" | awk '{print $2}' | head -1 || true)" - kind="${kind:-source}" - [[ "$kind" == "$WANT_KIND" ]] || continue +for entry in "$ROOT/$KNOWLEDGE_REL"/source/*.md; do + [[ -f "$entry" ]] || continue + [[ "$(basename "$entry")" == "README.md" ]] && continue + fm="$(awk '/^---$/{f=!f; next} f{print}' "$entry")" grep -qi -- "$probe_term" <<<"$fm" || continue - matched+=("$rel") -done < <(git -C "$ROOT" ls-files -- "$CORPUS_REL/*.md" ":!:$CORPUS_REL/raw/*") + matched+=("${entry#"$ROOT"/}") +done -leaked=() for m in "${matched[@]:-}"; do [[ -n "$m" ]] || continue for p in "${patterns[@]}"; do @@ -75,9 +85,9 @@ for m in "${matched[@]:-}"; do done if ((${#leaked[@]})); then - printf 'REGRESSION: pattern entry survived the default-mode kind filter: %s\n' "${leaked[@]}" >&2 + printf 'REGRESSION: pattern entry survived the default-mode enumeration: %s\n' "${leaked[@]}" >&2 exit 1 fi -echo "PASS: /wiki query declares two disjoint kind modes and no pattern entry passes the default filter" >&2 +echo "PASS: /wiki query declares two disjoint directory-scoped modes and no pattern entry can reach a default-mode result set" >&2 exit 0 diff --git a/.oh/evals/probes/wiki-readme-index.sh b/.oh/evals/probes/wiki-readme-index.sh index 5958cf2e..5af2aee4 100755 --- a/.oh/evals/probes/wiki-readme-index.sh +++ b/.oh/evals/probes/wiki-readme-index.sh @@ -1,20 +1,20 @@ #!/usr/bin/env bash # tier: A -# source: issue #132 — wiki README index drift guard -# desc: .oh/skills/wiki/corpus/README.md Index must match the git-tracked corpus/*.md frontmatter +# source: issue #132 — knowledge README index drift guard +# desc: .oh/knowledge/README.md Index must match the tracked source/*.md and patterns/*.md frontmatter set -euo pipefail ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" -WIKI="$ROOT/.oh/skills/wiki/corpus" +WIKI="$ROOT/.oh/knowledge" README="$WIKI/README.md" if [[ ! -d "$WIKI" ]]; then - echo "SKIPPED: wiki dir absent: $WIKI" >&2 + echo "SKIPPED: knowledge surface absent: $WIKI" >&2 exit 2 fi if [[ ! -f "$README" ]]; then - echo "REGRESSION: .oh/skills/wiki/corpus/README.md is missing" >&2 + echo "REGRESSION: .oh/knowledge/README.md is missing" >&2 exit 1 fi @@ -24,7 +24,9 @@ rows_tmp="$(mktemp)" trap 'rm -f "$expected_tmp" "$actual_tmp" "$rows_tmp"' EXIT tracked_wiki_files() { - git -C "$ROOT" ls-files -- '.oh/skills/wiki/corpus/*.md' ':!:.oh/skills/wiki/corpus/raw/*' + git -C "$ROOT" ls-files -- \ + '.oh/knowledge/source/*.md' \ + '.oh/knowledge/patterns/*.md' } while IFS= read -r relpath; do @@ -42,17 +44,20 @@ done < <(tracked_wiki_files) sort -r "$rows_tmp" | cut -f2- > "$expected_tmp" +# Anchor on the ## Index heading first: the README preamble may legitimately +# contain other four-column tables, and the separator line alone is ambiguous. awk ' - /^\| --- \| --- \| --- \| --- \|$/ { in_index=1; next } + /^## Index$/ { at_index=1; next } + at_index && /^\| --- \| --- \| --- \| --- \|$/ { in_index=1; next } in_index && /^\| / { print; next } in_index && !/^\| / { exit } ' "$README" > "$actual_tmp" if ! diff_output="$(diff -u "$expected_tmp" "$actual_tmp")"; then - echo "REGRESSION: .oh/skills/wiki/corpus/README.md Index is out of sync with the tracked corpus/*.md frontmatter" >&2 + echo "REGRESSION: .oh/knowledge/README.md Index is out of sync with the tracked entry frontmatter" >&2 echo "$diff_output" >&2 exit 1 fi -echo "PASS: .oh/skills/wiki/corpus/README.md Index matches the git-tracked corpus/*.md frontmatter" >&2 +echo "PASS: .oh/knowledge/README.md Index matches the tracked source/ and patterns/ frontmatter" >&2 exit 0 diff --git a/.oh/evals/probes/wiki-related-slugs.sh b/.oh/evals/probes/wiki-related-slugs.sh index 94f35d9d..4cd6bb80 100755 --- a/.oh/evals/probes/wiki-related-slugs.sh +++ b/.oh/evals/probes/wiki-related-slugs.sh @@ -1,12 +1,13 @@ #!/usr/bin/env bash # tier: A -# source: wikiskill arXiv:2608.27454 — wiki lint related-slug check -# desc: every related: frontmatter slug in a tracked wiki entry resolves to an existing tracked entry, and /wiki lint declares the check +# source: wikiskill arXiv:2608.27454 — wiki lint link checks +# desc: every related: frontmatter slug and every [[slug]] body link in a tracked knowledge +# entry resolves to an existing tracked entry, and /wiki lint declares both checks set -euo pipefail ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" LINT="$ROOT/.oh/skills/wiki/references/lint.md" -CORPUS_REL=".oh/skills/wiki/corpus" +KNOWLEDGE_REL=".oh/knowledge" if [[ ! -f "$LINT" ]]; then echo "SKIPPED: wiki lint reference absent" >&2 @@ -14,7 +15,14 @@ if [[ ! -f "$LINT" ]]; then fi failures=() -for lit in '### 7a. Broken related-slug check' 'RELATED_BROKEN' 'Broken related-slug findings'; do +for lit in \ + '### 7. Check 4 — broken outbound' \ + '### 8. Check 5 — broken `related:` slugs' \ + 'RELATED_BROKEN' \ + 'BROKEN_LINKS' \ + 'Broken related-slug findings' \ + 'Broken outbound link findings' +do grep -qF -- "$lit" "$LINT" || failures+=("lint.md missing contract text: $lit") done if ((${#failures[@]})); then @@ -27,35 +35,48 @@ entries=() while IFS= read -r rel; do base="$(basename "$rel")" [[ "$base" == "README.md" ]] && continue - [[ "$base" == "skill-impact.md" ]] && continue abs="$ROOT/$rel" [[ -f "$abs" ]] || continue slug="$(awk '/^---$/{f=!f; next} f{print}' "$abs" | grep '^slug:' | awk '{print $2}' | head -1 || true)" [[ -z "$slug" ]] && continue KNOWN["$slug"]=1 entries+=("$rel") -done < <(git -C "$ROOT" ls-files -- "$CORPUS_REL/*.md" ":!:$CORPUS_REL/raw/*") +done < <(git -C "$ROOT" ls-files -- \ + "$KNOWLEDGE_REL/source/*.md" \ + "$KNOWLEDGE_REL/patterns/*.md") if ((${#entries[@]} == 0)); then - echo "SKIPPED: no tracked corpus entries" >&2 + echo "SKIPPED: no tracked knowledge entries" >&2 exit 2 fi -broken=(); seen_related=0 +broken=(); seen_related=0; seen_link=0 for rel in "${entries[@]}"; do fm="$(awk '/^---$/{f=!f; next} f{print}' "$ROOT/$rel")" line="$(grep '^related:' <<<"$fm" | head -1 || true)" - [[ -z "$line" ]] && continue - seen_related=1 - rel_slugs="$(sed 's/^related: *//; s/[][]//g; s/,/ /g' <<<"$line")" - for r in $rel_slugs; do - [[ -z "$r" ]] && continue - [[ -n "${KNOWN[$r]:-}" ]] || broken+=("$rel -> related: $r (no such entry)") - done + if [[ -n "$line" ]]; then + seen_related=1 + rel_slugs="$(sed 's/^related: *//; s/[][]//g; s/,/ /g' <<<"$line")" + for r in $rel_slugs; do + [[ -z "$r" ]] && continue + [[ -n "${KNOWN[$r]:-}" ]] || broken+=("$rel -> related: $r (no such entry)") + done + fi + + # A [[slug]] inside a code span or fence is a MENTION (schema prose showing the + # syntax), not a navigational link. Strip both before extracting. + body="$(awk '/^---$/{n++; if(n==2){p=1; next}} p{print}' "$ROOT/$rel" \ + | awk '/^```/{f=!f; next} !f' \ + | sed 's/`[^`]*`//g')" + while IFS= read -r link; do + [[ -z "$link" ]] && continue + seen_link=1 + [[ -n "${KNOWN[$link]:-}" ]] || broken+=("$rel -> [[$link]] (no such entry)") + done < <(grep -oE '\[\[[a-z0-9-]+\]\]' <<<"$body" | sed 's/\[\[\(.*\)\]\]/\1/') done -if (( seen_related == 0 )); then - echo "SKIPPED: no tracked entry declares a related: field" >&2 +if (( seen_related == 0 && seen_link == 0 )); then + echo "SKIPPED: no tracked entry declares a related: field or a [[slug]] link" >&2 exit 2 fi if ((${#broken[@]})); then @@ -63,5 +84,5 @@ if ((${#broken[@]})); then exit 1 fi -echo "PASS: every related: slug in the tracked corpus resolves to an existing entry" >&2 +echo "PASS: every related: slug and every [[slug]] body link in the tracked knowledge base resolves to an existing entry" >&2 exit 0 diff --git a/.oh/evals/probes/wiki-skill-impact-append-only.sh b/.oh/evals/probes/wiki-skill-impact-append-only.sh index 4107dfd6..4a8107d7 100755 --- a/.oh/evals/probes/wiki-skill-impact-append-only.sh +++ b/.oh/evals/probes/wiki-skill-impact-append-only.sh @@ -1,11 +1,11 @@ #!/usr/bin/env bash # tier: A # source: wikiskill arXiv:2608.27454 — skill-change ledger, never rolled back -# desc: skill-impact.md is tracked, carries no slug (so it never enters the corpus index), and every SI record present at the merge-base is present and byte-identical at HEAD +# desc: the skill-impact ledger lives on the decisions surface, is tracked, carries no slug (so it never enters the knowledge index), and every SI record present at the merge-base is present and byte-identical at HEAD set -euo pipefail ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" -LEDGER_REL=".oh/skills/wiki/corpus/skill-impact.md" +LEDGER_REL=".oh/evals/decisions/skill-impact.md" LEDGER="$ROOT/$LEDGER_REL" if [[ ! -f "$LEDGER" ]]; then @@ -17,7 +17,7 @@ if ! git -C "$ROOT" ls-files --error-unmatch "$LEDGER_REL" >/dev/null 2>&1; then exit 1 fi if grep -q '^slug:' "$LEDGER"; then - echo "REGRESSION: $LEDGER_REL carries a slug: field — it would become a corpus index row" >&2 + echo "REGRESSION: $LEDGER_REL carries a slug: field — it would become a knowledge index row" >&2 exit 1 fi @@ -47,7 +47,14 @@ if [[ -z "$base" ]]; then exit 2 fi -if ! git -C "$ROOT" cat-file -e "$base:$LEDGER_REL" 2>/dev/null; then +# The ledger has moved between surfaces before, so locate it at the base by +# filename rather than by today's path — a relocation must not read as "new". +base_rel="$LEDGER_REL" +if ! git -C "$ROOT" cat-file -e "$base:$base_rel" 2>/dev/null; then + base_rel="$(git -C "$ROOT" ls-tree -r --name-only "$base" \ + | grep -E '(^|/)skill-impact\.md$' | head -1 || true)" +fi +if [[ -z "$base_rel" ]]; then echo "PASS: $LEDGER_REL is new on this branch; nothing to compare against the merge-base" >&2 exit 0 fi @@ -76,7 +83,7 @@ records() { done } -base_recs="$(git -C "$ROOT" show "$base:$LEDGER_REL" | records | sort)" +base_recs="$(git -C "$ROOT" show "$base:$base_rel" | records | sort)" head_recs="$(records < "$LEDGER" | sort)" missing=(); mutated=() @@ -99,5 +106,5 @@ if ((${#mutated[@]})); then exit 1 fi -echo "PASS: skill-impact.md is tracked, index-invisible, and append-only against the merge-base" >&2 +echo "PASS: the skill-impact ledger is on the decisions surface, tracked, index-invisible, and append-only against the merge-base" >&2 exit 0 diff --git a/.oh/knowledge/README.md b/.oh/knowledge/README.md new file mode 100644 index 00000000..b7b6b19f --- /dev/null +++ b/.oh/knowledge/README.md @@ -0,0 +1,79 @@ +# `.oh/knowledge/` — durable repository knowledge + +The knowledge surface: a personal-scale, LLM-readable cache of what this +repository has been understood to be. Pages are read whole into context on +demand (`/wiki query`), never retrieved through vector search. + +**`.oh/knowledge/` owns the data. `.oh/skills/wiki/` owns the procedure.** The +`/wiki` skill is how pages are written, queried, linted, and compiled; nothing +about a page's schema, provenance, or lifecycle lives inside that skill's +implementation tree. + +**The repository is the source of truth.** Knowledge is a derived cache of +understanding. A page is orientation; code and tests are implementation truth and +canonical docs/RFCs are intended-design truth. When a page and the source +disagree, the source wins and the page is wrong. + +## Layout + +| Directory | Holds | Tracked | Read by a normal `/wiki query` | +| --- | --- | --- | --- | +| `source/` | Entity pages — `kind: repo` synthesis of this repository, `kind: external` synthesis of an outside source | yes | yes (default mode) | +| `patterns/` | `kind: pattern` pages — failure modes and working strategies this harness observed in its own runs | yes | yes (`--patterns` mode only) | +| `raw/` | Immutable external snapshots, `<yyyy-mm-dd>-<slug>.md`, cited by `kind: external` pages | yes | no | +| `local/` | Per-machine scratch | **no — gitignored** | **never** | + +`local/` is the boundary that keeps knowledge shareable. A page only one machine +can see must never inform a plan another machine cannot reproduce, so `/wiki +query` and every `/spec` flow enumerate `source/` and `patterns/` and nothing +else. Promotion out of `local/` is an explicit `/wiki ingest --from-draft` call. + +## Conventions + +- **Entry files**: `source/<slug>.md` and `patterns/pattern-<subsystem>-<mode>.md` + — one topic per file, frontmatter-typed, `[[slug]]`-linked. The directory is the + `kind` boundary; `kind:` in frontmatter must agree with it. +- **Slug charset**: `[a-z0-9-]+` (lowercased kebab-case), matching the filename. +- **Word cap**: ≤ 600 words per entry (≤ 900 for architecture pages carrying + source evidence and diagrams). +- **Cross-links**: `[[slug]]` (Obsidian-style, grep-friendly). +- **Provenance**: `sources:` is the single dependency declaration. A `raw/` path + is resolved relative to **this directory**, not to the page's own directory. +- **Freshness**: `kind: repo` pages carry `verified_at: <commit>` and go + `needs-review` when a declared repository source changed after it. Age is + telemetry, not a validity test. +- **Index**: the table below is generated by `/wiki lint` from the git-tracked + entry set and guarded by `.oh/evals/probes/wiki-readme-index.sh` — never edit + it by hand. + +## Canonical docs + +Schema, frontmatter spec, provenance kinds, and every authoring rule: +[`.oh/skills/wiki/references/schema.md`](../skills/wiki/references/schema.md). + +## Index + +| Slug | Title | Tags | Updated | +| --- | --- | --- | --- | +| wikiskill-experience-compilation | WikiSkill: Compiling Agent Experience into Persistent Knowledge (arXiv 2608.27454) | [skill-evolution, persistent-knowledge, continual-learning, harness-evolution, self-improvement, wiki, ablation, skill-transfer] | 2026-09-01 | +| plan-vs-built-reconciliation | Plan-vs-Built Reconciliation | [spec-execute, evidence, merge-gate, comprehension, audit, task-folder, review] | 2026-09-01 | +| pattern-wiki-ungated-check-drift | Report-only checks nothing gates on stop being run | [wiki, lint, evals, probes, report-only, drift, gating] | 2026-09-01 | +| pattern-wiki-external-model-over-mapping | Mapping an external model onto the harness reimports a tier it deleted | [wiki, ingest, architecture, external-sources, scope-creep, design-review] | 2026-09-01 | +| pattern-spec-self-staling-reuse-record | A commit-keyed reuse record goes stale the moment it is committed | [spec, evals, caching, provenance, freshness, build-cycle] | 2026-09-01 | +| pattern-evals-prose-literal-pinning | Contract probes that pin multi-word prose break on reflow, not on drift | [evals, probes, contract-text, grep, false-failure, documentation] | 2026-09-01 | +| pattern-evals-pipefail-early-exit | A short-circuiting reader turns a successful match into a failed pipeline | [evals, probes, bash, pipefail, sigpipe, false-failure, shell] | 2026-09-01 | +| pattern-docs-prohibition-by-example | Documenting a forbidden literal by quoting it violates the rule | [docs, evals, probes, vocabulary, guards, self-reference] | 2026-09-01 | +| oh-cli-portable-lifecycle | oh CLI Portable Lifecycle | [cli, oh, lifecycle, standalone, init, sandbox, remote-fetch, execution-target] | 2026-09-01 | +| document-ingestion | Local Document Ingestion | [wiki, ingestion, markitdown, documents, provenance, security] | 2026-09-01 | +| sandbox-dependency-installs | Sandbox Dependency Installs | [sandbox, devcontainer, pnpm, dependencies, boot] | 2026-08-31 | +| recursive-self-improvement-survey | Recursive Self-Improvement in AI (survey, arXiv 2607.07663) | [rsi, self-improvement, verification-hierarchy, skill-libraries, harness-evolution, self-evaluation, model-collapse, capability-benchmark] | 2026-08-31 | +| recursive-language-models | Recursive Language Models | [rlm, context-as-environment, weighted-trajectories, agent-harness, llm-agents, self-consistency] | 2026-08-31 | +| pattern-evals-unexercised-oracle | A probe that has never failed has an unverified oracle | [evals, probes, oracles, skipped, fault-injection, continual-learning] | 2026-08-31 | +| compose-env-boundary | Compose Environment Boundary | [compose, devcontainer, oh-json, cli, entrypoint, boundary, provisioning, sandbox] | 2026-08-31 | +| runtime-isolation-landscape | Runtime Isolation Landscape (2026) | [runtime, isolation, sandbox, gvisor, firecracker, kata, microvm, cloudflare, e2b, daytona, fly, modal] | 2026-08-27 | +| molt-agentic-reinforcement-learning | Molt: A Scalable PyTorch-Native Training Framework for Agentic Reinforcement Learning | [agentic-rl, training, readability, observability, trajectories, async, correctness, agent-harness, nvidia] | 2026-08-27 | +| managed-agents | Scaling Managed Agents: Decoupling the brain from the hands | [agents, meta-harness, sessions, sandbox, resilience, security, scaling, anthropic, model-evolution] | 2026-08-27 | +| fresh-machine-setup | Fresh-Machine Setup Flow | [setup, onboarding, installation, gateway, ssh, github, slack] | 2026-08-27 | +| crabbox-remote-exec-control-plane | Crabbox — Remote-Exec Control Plane | [runtime, sandbox, remote-execution, fan-out, control-plane, crabbox, cloudflare-workers, ssh, rsync] | 2026-08-27 | +| release-versioning | Release Versioning | [release, versioning, semver, calver, github-actions, ghcr, tags, workflow, package-json] | 2026-08-23 | +| audit-architecture | Audit Architecture | [audit, pr, workflow, safety, observability] | 2026-08-12 | diff --git a/.oh/knowledge/local/README.md b/.oh/knowledge/local/README.md new file mode 100644 index 00000000..ef30e48f --- /dev/null +++ b/.oh/knowledge/local/README.md @@ -0,0 +1,27 @@ +# `.oh/knowledge/local/` — per-machine scratch, never an input + +Gitignored working space for knowledge that is not ready to be shared: a draft +page, a half-verified note, an experiment. Everything here is invisible to every +other machine and to CI. + +**Nothing reads this directory.** `/wiki query` enumerates `../source/` and +`../patterns/`. Every `/spec` flow consumes tracked knowledge only. There is no +flag that folds a local page into a normal result set, and that is the point: a +plan grounded in a page one machine can see is a plan nobody else can reproduce. + +## Promotion + +Promotion is explicit and goes through the one authorized write path: + +```bash +/wiki ingest --from-draft <slug> # a draft at $TMPDIR/oh-wiki-drafts/<slug>.md +/wiki ingest .oh/knowledge/local/<slug>.md --slug <slug> +``` + +Either lands a schema-valid, tracked page under `.oh/knowledge/source/`. Moving +the file by hand is not promotion — it skips the schema, the provenance, and the +index. + +## Canonical docs + +[`.oh/skills/wiki/references/schema.md`](../../skills/wiki/references/schema.md). diff --git a/.oh/skills/wiki/corpus/pattern-docs-prohibition-by-example.md b/.oh/knowledge/patterns/pattern-docs-prohibition-by-example.md similarity index 73% rename from .oh/skills/wiki/corpus/pattern-docs-prohibition-by-example.md rename to .oh/knowledge/patterns/pattern-docs-prohibition-by-example.md index df47d237..8be6f34b 100644 --- a/.oh/skills/wiki/corpus/pattern-docs-prohibition-by-example.md +++ b/.oh/knowledge/patterns/pattern-docs-prohibition-by-example.md @@ -4,9 +4,11 @@ slug: pattern-docs-prohibition-by-example kind: pattern tags: [docs, evals, probes, vocabulary, guards, self-reference] created: 2026-08-31 -updated: 2026-08-31 +updated: 2026-09-01 sources: - .oh/evals/probes/audit-stale-references.sh@ce7b7db2 + - .oh/evals/probes/knowledge-path-single-owner.sh@fcbeedea + - .oh/tasks/repo-knowledge-loop/evidence.md@fcbeedea - .oh/skills/wiki/references/schema.md@c841e567 - .oh/skills/wiki/references/compile.md@c841e567 confidence: provisional @@ -52,5 +54,19 @@ forbidden literal, add the specific `path:line` to the guard's exemption list ra than broadening its exclusion pathspec — the narrow exemption stays reviewable and cannot silently cover a real violation elsewhere in the same file. +Issue #926 found the sharpest instance: the guard fired on the **evidence +document written to prove the migration was complete**, whose shell transcripts +quoted the retired path as literal command text. `/audit implementation` gate 2 +returned `AUDIT-FAIL` naming two lines of `evidence.md`. Two further rules follow +from that. First, a guard that scans every tracked file also scans the proof, so +transcripts must resolve a retired path programmatically — from the base tree, or +from the guard's own output — rather than pasting it. Second, do **not** exempt +the document that describes the migration: an exemption there is a hole in the +guard placed exactly where the retired vocabulary is most likely to be reused. +The guard authored for this migration +(`.oh/evals/probes/knowledge-path-single-owner.sh`) also follows the other half of +the rule — it assembles the forbidden path from fragments, so it is not a hit for +itself and needs no self-exemption at all. + ## See Also - [[pattern-evals-prose-literal-pinning]] diff --git a/.oh/knowledge/patterns/pattern-evals-pipefail-early-exit.md b/.oh/knowledge/patterns/pattern-evals-pipefail-early-exit.md new file mode 100644 index 00000000..105656f4 --- /dev/null +++ b/.oh/knowledge/patterns/pattern-evals-pipefail-early-exit.md @@ -0,0 +1,66 @@ +--- +title: "A short-circuiting reader turns a successful match into a failed pipeline" +slug: pattern-evals-pipefail-early-exit +kind: pattern +tags: [evals, probes, bash, pipefail, sigpipe, false-failure, shell] +created: 2026-09-01 +updated: 2026-09-01 +sources: + - .oh/evals/probes/knowledge-source-freshness.sh@fcbeedea + - .oh/skills/wiki/references/lint.md@fcbeedea + - .oh/tasks/repo-knowledge-loop/evidence.md@fcbeedea +confidence: provisional +--- + +# A short-circuiting reader turns a successful match into a failed pipeline + +## Relevant Source Files +- `.oh/evals/probes/knowledge-source-freshness.sh` — the probe where the defect + appeared, and the capture-then-match form that fixes it. +- `.oh/skills/wiki/references/lint.md` — the same check written as procedure, + carrying the warning so the next author meets it before writing the pipeline. +- `.oh/tasks/repo-knowledge-loop/evidence.md@fcbeedea` — the run that produced the + observation. + +## Summary +Under `set -o pipefail`, a reader that exits as soon as it is satisfied kills the +writer with SIGPIPE, and the pipeline's status becomes the writer's 141 rather +than the reader's 0. The assertion then reports **not found** at exactly the +moment it found what it was looking for, and it does so intermittently, because a +writer that finishes before the reader exits produces the correct answer. + +## Detail +**Symptom.** A shell assertion of the form `<producer> | grep -q <pattern>` +reports failure while the same pattern demonstrably matches the producer's output +when run by hand. In this harness it appeared as a pinned-provenance check +insisting that a path was absent from a commit tree that plainly contained it; the +same two commands, run separately, matched immediately. Roughly two debug cycles +went into the pattern and the escaping before the pipeline itself was suspected. + +**Root cause.** `grep -q` exits on its first match by design. That closes the read +end while the producer is still writing, the producer takes SIGPIPE and exits 141, +and `pipefail` propagates the highest non-zero status — so the pipeline is false +*because* the match succeeded. The failure is load-dependent: a producer whose +output fits the pipe buffer finishes first and the pipeline returns 0, which is +why the shape survives review and passes on small fixtures. Every ingredient is +individually correct — `pipefail` catches real producer failures, `-q` avoids +buffering a large tree, `set -e` stops on error — and the defect exists only in +their combination. + +**Workaround.** Capture the producer's output into a variable, then match against +the capture: + +```bash +tree="$(git ls-tree -r --name-only "$sha" 2>/dev/null || true)" +grep -qxF "$path" <<<"$tree" +``` + +The producer runs to completion, its own failure is still handled explicitly, and +the match is a plain exit status. Where capture is genuinely too large, terminate +the pipeline with `|| true` and test a captured count instead of relying on the +pipeline's status. Do not reach for `set +o pipefail` around the line: that +disarms the guard for real producer failures in the same statement. + +## See Also +- [[pattern-evals-unexercised-oracle]] +- [[pattern-evals-prose-literal-pinning]] diff --git a/.oh/skills/wiki/corpus/pattern-evals-prose-literal-pinning.md b/.oh/knowledge/patterns/pattern-evals-prose-literal-pinning.md similarity index 77% rename from .oh/skills/wiki/corpus/pattern-evals-prose-literal-pinning.md rename to .oh/knowledge/patterns/pattern-evals-prose-literal-pinning.md index b9489270..833bd22b 100644 --- a/.oh/skills/wiki/corpus/pattern-evals-prose-literal-pinning.md +++ b/.oh/knowledge/patterns/pattern-evals-prose-literal-pinning.md @@ -4,9 +4,11 @@ slug: pattern-evals-prose-literal-pinning kind: pattern tags: [evals, probes, contract-text, grep, false-failure, documentation] created: 2026-08-31 -updated: 2026-08-31 +updated: 2026-09-01 sources: - .oh/evals/probes/wiki-kind-schema-contract.sh@bfe22487 + - .oh/evals/probes/spec-plan-knowledge-context.sh@fcbeedea + - .oh/tasks/repo-knowledge-loop/evidence.md@fcbeedea - .oh/skills/wiki/references/schema.md@c841e567 confidence: provisional --- @@ -50,6 +52,16 @@ it. Where a whole-sentence assertion is genuinely required, normalize before matching — fold the document's whitespace to single spaces and match against the normalized text — instead of pinning the stored bytes. +A second failure mode of the same matcher, found by issue #926: a **short** pin +can be too weak rather than too brittle. `grep -qF '## Knowledge Context'` +survived deleting the block it guards, because the section heading that *names* +the block contains the same substring. Both planning probes reported PASS against +a document with the contract removed. Where the pinned text is a whole line — a +heading, a template block, a table row — assert it with `grep -qxF` so a mention +cannot satisfy an assertion about the thing itself +(`.oh/evals/probes/spec-plan-knowledge-context.sh:29-32`). Fault injection is what +surfaced it; neither probe had ever been run against a broken input. + ## See Also - [[pattern-evals-unexercised-oracle]] - [[pattern-docs-prohibition-by-example]] diff --git a/.oh/skills/wiki/corpus/pattern-evals-unexercised-oracle.md b/.oh/knowledge/patterns/pattern-evals-unexercised-oracle.md similarity index 100% rename from .oh/skills/wiki/corpus/pattern-evals-unexercised-oracle.md rename to .oh/knowledge/patterns/pattern-evals-unexercised-oracle.md diff --git a/.oh/knowledge/patterns/pattern-spec-self-staling-reuse-record.md b/.oh/knowledge/patterns/pattern-spec-self-staling-reuse-record.md new file mode 100644 index 00000000..ccbc2dfc --- /dev/null +++ b/.oh/knowledge/patterns/pattern-spec-self-staling-reuse-record.md @@ -0,0 +1,62 @@ +--- +title: "A commit-keyed reuse record goes stale the moment it is committed" +slug: pattern-spec-self-staling-reuse-record +kind: pattern +tags: [spec, evals, caching, provenance, freshness, build-cycle] +created: 2026-09-01 +updated: 2026-09-01 +sources: + - .oh/skills/spec/references/execute.md@fcbeedea + - .oh/evals/probes/eval-runs-once-per-cycle.sh@fcbeedea + - .oh/tasks/repo-knowledge-loop/eval-result.json@fcbeedea +confidence: provisional +--- + +# A commit-keyed reuse record goes stale the moment it is committed + +## Relevant Source Files +- `.oh/skills/spec/references/execute.md` — publishes `eval-result.json` keyed to + `git rev-parse HEAD` and `git add -f`s it in the same step. +- `.oh/evals/probes/eval-runs-once-per-cycle.sh` — the oracle that requires every + reader to compare the record's `commit` against HEAD before reusing it. +- `.oh/tasks/repo-knowledge-loop/eval-result.json@fcbeedea` — the record three + consecutive audits declined to reuse. + +## Summary +A cache record that stores the commit it was measured against, and is then +committed into that same repository, can never satisfy `commit == HEAD`: writing +it moves HEAD past the commit it names. Readers that honor the freshness key +therefore always take the fallback, and the optimization the record exists for +never fires — while every artifact still reports the design as working. + +## Detail +**Symptom.** A downstream reader that is documented to reuse a published result +re-derives it every time, and nothing reports a problem. In this harness +`/audit implementation` gate 2 read `.oh/tasks/<slug>/eval-result.json` on three +consecutive runs of one build, found `commit` behind HEAD each time, and ran the +127-probe suite itself — which is the contract's correct behavior on a stale +record and also its only behavior. + +**Root cause.** The record is produced by `bash .oh/skills/eval/run.sh` at some +commit `C`, written with `"commit": "$(git rev-parse HEAD)"` = `C`, and then +staged and committed as `C+1`. From `C+1` onward the file asserts `C`, so the +equality the reader checks is false at every subsequent commit including the one +that introduced the file. The freshness rule itself is right — a record from an +earlier HEAD describes code no longer under test — and the guard that enforces it +is green. Nothing is broken; the reuse path is simply unreachable, and no artifact +distinguishes "the fallback fired because the branch moved" from "the fallback +always fires". + +**Workaround.** State the lifetime in the contract rather than implying reuse is +the normal path: a committed reuse record is valid only for readers running +*before* it is committed, so a reader that finds it stale is behaving normally and +should say so rather than reporting an anomaly. Where reuse across commits is +actually wanted, key the record on the content it measured — the tree hash of the +inputs, or the probe-set digest — rather than on the commit that contains the +record, so writing the record does not invalidate it. Do not answer this by +relaxing the equality check to "recent enough": that reintroduces the age +heuristic the freshness rule replaced. + +## See Also +- [[plan-vs-built-reconciliation]] +- [[pattern-evals-unexercised-oracle]] diff --git a/.oh/skills/wiki/corpus/pattern-wiki-external-model-over-mapping.md b/.oh/knowledge/patterns/pattern-wiki-external-model-over-mapping.md similarity index 70% rename from .oh/skills/wiki/corpus/pattern-wiki-external-model-over-mapping.md rename to .oh/knowledge/patterns/pattern-wiki-external-model-over-mapping.md index 8e27350e..e6537d64 100644 --- a/.oh/skills/wiki/corpus/pattern-wiki-external-model-over-mapping.md +++ b/.oh/knowledge/patterns/pattern-wiki-external-model-over-mapping.md @@ -4,9 +4,11 @@ slug: pattern-wiki-external-model-over-mapping kind: pattern tags: [wiki, ingest, architecture, external-sources, scope-creep, design-review] created: 2026-08-31 -updated: 2026-08-31 +updated: 2026-09-01 sources: - - .oh/skills/wiki/corpus/wikiskill-experience-compilation.md@933f6741 + - .oh/knowledge/source/wikiskill-experience-compilation.md@786920fd + - .oh/skills/wiki/references/schema.md@786920fd + - .oh/tasks/repo-knowledge-loop/progress.txt@786920fd - .oh/skills/wiki/references/compile.md@c841e567 - .oh/skills/wiki/references/schema.md@c841e567 confidence: provisional @@ -15,7 +17,7 @@ confidence: provisional # Mapping an external model onto the harness reimports a tier it deleted ## Relevant Source Files -- `.oh/skills/wiki/corpus/wikiskill-experience-compilation.md@933f6741` — the ingested paper whose layer model was mapped onto the local corpus. +- `.oh/knowledge/source/wikiskill-experience-compilation.md@933f6741` — the ingested paper whose layer model was mapped onto the local corpus. - `.oh/skills/wiki/references/compile.md@c841e567` — the resulting subcommand, and the explicit prohibition the mapping needed. - `.oh/skills/wiki/references/schema.md@c841e567` — the same prohibition stated in the schema. @@ -30,7 +32,7 @@ where the original deletion rationale no longer matches the name. **Symptom.** A design derived from an ingested paper proposes a structure that a prior deliberate deletion forbids, and it does not read as a regression because it arrives under the source's terminology rather than the local one. Mapping the paper -at `.oh/skills/wiki/corpus/wikiskill-experience-compilation.md@933f6741` onto the +at `.oh/knowledge/source/wikiskill-experience-compilation.md@933f6741` onto the corpus proposed snapshotting session retrospective reports into the corpus's raw tier. Six of seven mapped structures transferred without objection; the seventh would have rebuilt the removed per-session journal tier — one dated entry per run, @@ -48,12 +50,22 @@ artifact carried the mapping's exclusion. *not* transfer and write the refusal into the local contract in the local vocabulary, at the same time as the structures that do. The mapping is complete only when its exclusions are written down. Both -`.oh/skills/wiki/references/compile.md:113-116` and -`.oh/skills/wiki/references/schema.md:167-171` now state the prohibition explicitly, +`.oh/skills/wiki/references/compile.md:130-134` and +`.oh/skills/wiki/references/schema.md:220-224` now state the prohibition explicitly, each naming the removed tier and the report-only contract, so the next reader deriving a design from the same paper meets the exclusion inside the local documents rather than having to remember it. +Applied again in issue #926, where the same paper's layer model drove a knowledge +surface. Two exclusions were written down at the same time as the structures that +transferred: a repository-derived page never snapshots this repository's own +source into `raw/`, which the external model would have implied +(`.oh/skills/wiki/references/schema.md:157-160`), and a bare upstream URL is +recorded as a legacy provenance form rather than an option, so the weaker shape +cannot spread (`.oh/skills/wiki/references/schema.md:311-316`). The mapping is +complete only when its exclusions are written down — this run treated that as the +exit condition rather than as review feedback. + ## See Also - [[wikiskill-experience-compilation]] - [[pattern-wiki-ungated-check-drift]] diff --git a/.oh/skills/wiki/corpus/pattern-wiki-ungated-check-drift.md b/.oh/knowledge/patterns/pattern-wiki-ungated-check-drift.md similarity index 73% rename from .oh/skills/wiki/corpus/pattern-wiki-ungated-check-drift.md rename to .oh/knowledge/patterns/pattern-wiki-ungated-check-drift.md index fd521383..11284db7 100644 --- a/.oh/skills/wiki/corpus/pattern-wiki-ungated-check-drift.md +++ b/.oh/knowledge/patterns/pattern-wiki-ungated-check-drift.md @@ -4,9 +4,11 @@ slug: pattern-wiki-ungated-check-drift kind: pattern tags: [wiki, lint, evals, probes, report-only, drift, gating] created: 2026-08-31 -updated: 2026-08-31 +updated: 2026-09-01 sources: - - .oh/skills/wiki/corpus/recursive-language-models.md@8fab04ab + - .oh/knowledge/source/recursive-language-models.md@786920fd + - .oh/skills/wiki/references/lint.md@786920fd + - .oh/tasks/repo-knowledge-loop/progress.txt@786920fd - .oh/skills/wiki/references/lint.md@8fab04ab - .oh/evals/probes/wiki-readme-index.sh@8fab04ab confidence: provisional @@ -16,7 +18,7 @@ confidence: provisional ## Relevant Source Files - `.oh/skills/wiki/references/lint.md` — the six health checks, all report-only. -- `.oh/skills/wiki/corpus/recursive-language-models.md@8fab04ab` — carried three unresolvable links for two months. +- `.oh/knowledge/source/recursive-language-models.md@8fab04ab` — carried three unresolvable links for two months. - `.oh/evals/probes/wiki-related-slugs.sh` — the probe minted to close this instance. - `.oh/skills/spec/references/execute.md` — where the groom triad was deliberately cut from the cycle. @@ -26,7 +28,7 @@ being run at all. Its findings do not accumulate as visible debt; they accumulat silence, and the check's own green-looking absence is mistaken for health. ## Detail -**Symptom.** `.oh/skills/wiki/corpus/recursive-language-models.md@8fab04ab` shipped +**Symptom.** `.oh/knowledge/source/recursive-language-models.md@8fab04ab` shipped `related: [inspectable-agent-harness, prompt-miner, repo2rlenv]` and three matching `[[slug]]` body links. None of the three slugs has ever existed in the corpus. The `related:` list was genuinely unchecked, but the body links were covered by `/wiki @@ -54,5 +56,15 @@ answer this pattern by re-adding the check to a per-cycle tail — that restores cost the removal was right to avoid, and still depends on someone reading advisory output. +Corroborated at scale by issue #926, which applied the workaround to the whole +check list rather than to one finding. `/wiki lint` was reduced to six checks and +each was given a named oracle in a table the reference now carries +(`.oh/skills/wiki/references/lint.md:23-33`), so no surviving check depends on +being run. The same pass retired the two checks that had no oracle and no +consequence — orphan detection and the 90-day age rule +(`.oh/skills/wiki/references/lint.md:40-52`) — which is the other half of the +lesson: a report-only check that nobody gates on and that nothing can enforce is +not underused, it is not a check. + ## See Also - [[wikiskill-experience-compilation]] diff --git a/.oh/skills/wiki/corpus/raw/2026-06-27-recursive-language-models.md b/.oh/knowledge/raw/2026-06-27-recursive-language-models.md similarity index 100% rename from .oh/skills/wiki/corpus/raw/2026-06-27-recursive-language-models.md rename to .oh/knowledge/raw/2026-06-27-recursive-language-models.md diff --git a/.oh/skills/wiki/corpus/raw/2026-07-04-crabbox-remote-exec-control-plane.md b/.oh/knowledge/raw/2026-07-04-crabbox-remote-exec-control-plane.md similarity index 100% rename from .oh/skills/wiki/corpus/raw/2026-07-04-crabbox-remote-exec-control-plane.md rename to .oh/knowledge/raw/2026-07-04-crabbox-remote-exec-control-plane.md diff --git a/.oh/skills/wiki/corpus/raw/2026-07-04-runtime-isolation-landscape.md b/.oh/knowledge/raw/2026-07-04-runtime-isolation-landscape.md similarity index 100% rename from .oh/skills/wiki/corpus/raw/2026-07-04-runtime-isolation-landscape.md rename to .oh/knowledge/raw/2026-07-04-runtime-isolation-landscape.md diff --git a/.oh/skills/wiki/corpus/raw/2026-07-17-audit-architecture.md b/.oh/knowledge/raw/2026-07-17-audit-architecture.md similarity index 100% rename from .oh/skills/wiki/corpus/raw/2026-07-17-audit-architecture.md rename to .oh/knowledge/raw/2026-07-17-audit-architecture.md diff --git a/.oh/skills/wiki/corpus/raw/2026-07-18-markitdown.md b/.oh/knowledge/raw/2026-07-18-markitdown.md similarity index 100% rename from .oh/skills/wiki/corpus/raw/2026-07-18-markitdown.md rename to .oh/knowledge/raw/2026-07-18-markitdown.md diff --git a/.oh/skills/wiki/corpus/raw/2026-08-12-build-executor-ladder.md b/.oh/knowledge/raw/2026-08-12-build-executor-ladder.md similarity index 100% rename from .oh/skills/wiki/corpus/raw/2026-08-12-build-executor-ladder.md rename to .oh/knowledge/raw/2026-08-12-build-executor-ladder.md diff --git a/.oh/skills/wiki/corpus/raw/2026-08-23-release-versioning.md b/.oh/knowledge/raw/2026-08-23-release-versioning.md similarity index 100% rename from .oh/skills/wiki/corpus/raw/2026-08-23-release-versioning.md rename to .oh/knowledge/raw/2026-08-23-release-versioning.md diff --git a/.oh/skills/wiki/corpus/raw/2026-08-24-plan-vs-built-reconciliation.md b/.oh/knowledge/raw/2026-08-24-plan-vs-built-reconciliation.md similarity index 100% rename from .oh/skills/wiki/corpus/raw/2026-08-24-plan-vs-built-reconciliation.md rename to .oh/knowledge/raw/2026-08-24-plan-vs-built-reconciliation.md diff --git a/.oh/skills/wiki/corpus/raw/2026-08-26-prime-agent-harness.md b/.oh/knowledge/raw/2026-08-26-prime-agent-harness.md similarity index 100% rename from .oh/skills/wiki/corpus/raw/2026-08-26-prime-agent-harness.md rename to .oh/knowledge/raw/2026-08-26-prime-agent-harness.md diff --git a/.oh/skills/wiki/corpus/raw/2026-08-31-wikiskill-experience-compilation.md b/.oh/knowledge/raw/2026-08-31-wikiskill-experience-compilation.md similarity index 100% rename from .oh/skills/wiki/corpus/raw/2026-08-31-wikiskill-experience-compilation.md rename to .oh/knowledge/raw/2026-08-31-wikiskill-experience-compilation.md diff --git a/.oh/knowledge/raw/README.md b/.oh/knowledge/raw/README.md new file mode 100644 index 00000000..44b0ee92 --- /dev/null +++ b/.oh/knowledge/raw/README.md @@ -0,0 +1,25 @@ +# `.oh/knowledge/raw/` — immutable external snapshots + +One file per fetch, never overwritten. Each records the content of an external +URL or document at the moment `/wiki ingest` captured it. `kind: external` entity +pages under `.oh/knowledge/source/` cite these snapshots through `sources:`, +giving every outside claim a concrete provenance trail. + +## Conventions + +- **Naming**: `<yyyy-mm-dd>-<slug>.md` — UTC date of the fetch plus the topic slug. +- **Format**: the file opens with `# Source: <url>` followed by the fetched body + verbatim. +- **Immutability**: never edited after creation. Re-ingesting a URL writes a new + dated snapshot; the prior one stays. +- **Tracked**: these files are committed. An untracked snapshot is provenance a + fresh clone cannot verify, which is the same as no provenance. +- **External only**: a `kind: repo` page cites the repository paths it depends on + directly. It never snapshots this repository's own source into here. +- **Not queried**: `/wiki query` reads entity pages. Snapshots exist for audit + and provenance. + +## Canonical docs + +Full schema and authoring conventions: +[`.oh/skills/wiki/references/schema.md`](../../skills/wiki/references/schema.md). diff --git a/.oh/skills/wiki/corpus/audit-architecture.md b/.oh/knowledge/source/audit-architecture.md similarity index 93% rename from .oh/skills/wiki/corpus/audit-architecture.md rename to .oh/knowledge/source/audit-architecture.md index 816ec138..3b2c3e54 100644 --- a/.oh/skills/wiki/corpus/audit-architecture.md +++ b/.oh/knowledge/source/audit-architecture.md @@ -1,11 +1,18 @@ --- title: "Audit Architecture" slug: audit-architecture +kind: repo tags: [audit, pr, workflow, safety, observability] created: 2026-07-17 updated: 2026-08-12 sources: - raw/2026-07-17-audit-architecture.md + - .oh/skills/audit/SKILL.md + - .oh/skills/audit/scripts/audit-run.sh + - .oh/skills/audit/scripts/pr-classify.sh + - .oh/skills/audit/scripts/implementation-gates.sh + - .oh/skills/audit/references/full.md +verified_at: 5b426f977b0e892786998638ba526796bfcd2b99 related: [oh-cli-portable-lifecycle] confidence: confirmed --- diff --git a/.oh/skills/wiki/corpus/compose-env-boundary.md b/.oh/knowledge/source/compose-env-boundary.md similarity index 98% rename from .oh/skills/wiki/corpus/compose-env-boundary.md rename to .oh/knowledge/source/compose-env-boundary.md index a4e17153..59747b6f 100644 --- a/.oh/skills/wiki/corpus/compose-env-boundary.md +++ b/.oh/knowledge/source/compose-env-boundary.md @@ -1,6 +1,7 @@ --- title: "Compose Environment Boundary" slug: compose-env-boundary +kind: repo tags: [compose, devcontainer, oh-json, cli, entrypoint, boundary, provisioning, sandbox] created: 2026-08-31 updated: 2026-08-31 @@ -11,6 +12,7 @@ sources: - .oh/cli/src/lib/config-render.ts - .oh/scripts/provision-defaults.sh - .oh/evals/probes/compose-env-boundary.sh +verified_at: 1c5f37230822ec2bbc5ed316be92ad295722b693 related: [sandbox-dependency-installs, oh-cli-portable-lifecycle] confidence: confirmed --- diff --git a/.oh/skills/wiki/corpus/crabbox-remote-exec-control-plane.md b/.oh/knowledge/source/crabbox-remote-exec-control-plane.md similarity index 99% rename from .oh/skills/wiki/corpus/crabbox-remote-exec-control-plane.md rename to .oh/knowledge/source/crabbox-remote-exec-control-plane.md index 5617c9c8..53b03e34 100644 --- a/.oh/skills/wiki/corpus/crabbox-remote-exec-control-plane.md +++ b/.oh/knowledge/source/crabbox-remote-exec-control-plane.md @@ -1,6 +1,7 @@ --- title: "Crabbox — Remote-Exec Control Plane" slug: crabbox-remote-exec-control-plane +kind: external tags: [runtime, sandbox, remote-execution, fan-out, control-plane, crabbox, cloudflare-workers, ssh, rsync] created: 2026-07-04 updated: 2026-08-27 diff --git a/.oh/skills/wiki/corpus/document-ingestion.md b/.oh/knowledge/source/document-ingestion.md similarity index 91% rename from .oh/skills/wiki/corpus/document-ingestion.md rename to .oh/knowledge/source/document-ingestion.md index 21e6a596..f5072766 100644 --- a/.oh/skills/wiki/corpus/document-ingestion.md +++ b/.oh/knowledge/source/document-ingestion.md @@ -1,14 +1,16 @@ --- title: "Local Document Ingestion" slug: document-ingestion +kind: repo tags: [wiki, ingestion, markitdown, documents, provenance, security] created: 2026-07-18 -updated: 2026-07-18 +updated: 2026-09-01 sources: - raw/2026-07-18-markitdown.md - .oh/skills/wiki/references/ingest.md - .oh/evals/probes/markitdown-wiki-ingest.sh - - .oh/tasks/markitdown-wiki-pilot/smoke-evidence.md + - .oh/tasks/markitdown-wiki-pilot/smoke-evidence.md@0fd2efcb +verified_at: 786920fd10cc8317e0328047b2ca12599c7f3d7b related: [sandbox-dependency-installs] confidence: provisional --- @@ -16,7 +18,7 @@ confidence: provisional # Local Document Ingestion ## Relevant Source Files -- `.oh/skills/wiki/corpus/raw/2026-07-18-markitdown.md` — immutable upstream README snapshot documenting supported formats, direct CLI usage, optional dependencies, and security guidance. +- `.oh/knowledge/raw/2026-07-18-markitdown.md` — immutable upstream README snapshot documenting supported formats, direct CLI usage, optional dependencies, and security guidance. - `.oh/skills/wiki/references/ingest.md` — owns source classification, validation, conversion, review, provenance publication, and synthesis handoff. - `.oh/evals/probes/markitdown-wiki-ingest.sh` — guards the pinned/local-only contract and prevents wrapper or runtime-package drift. - `.oh/tasks/markitdown-wiki-pilot/smoke-evidence.md` — records real conversion, resource-limit, cleanup, and network-boundary evidence. diff --git a/.oh/skills/wiki/corpus/fresh-machine-setup.md b/.oh/knowledge/source/fresh-machine-setup.md similarity index 98% rename from .oh/skills/wiki/corpus/fresh-machine-setup.md rename to .oh/knowledge/source/fresh-machine-setup.md index fcaf668a..db7cc72a 100644 --- a/.oh/skills/wiki/corpus/fresh-machine-setup.md +++ b/.oh/knowledge/source/fresh-machine-setup.md @@ -1,6 +1,7 @@ --- title: "Fresh-Machine Setup Flow" slug: fresh-machine-setup +kind: repo tags: [setup, onboarding, installation, gateway, ssh, github, slack] created: 2026-07-02 updated: 2026-08-27 @@ -13,6 +14,7 @@ sources: - docs/harnesses/hermes.md - .devcontainer/entrypoint.sh - .oh/scripts/gateway.sh +verified_at: 0b292bf8662d11179132ad034b68181a29b0fe43 related: [sandbox-dependency-installs, oh-cli-portable-lifecycle] confidence: provisional --- diff --git a/.oh/skills/wiki/corpus/managed-agents.md b/.oh/knowledge/source/managed-agents.md similarity index 90% rename from .oh/skills/wiki/corpus/managed-agents.md rename to .oh/knowledge/source/managed-agents.md index a0171967..70a1afbb 100644 --- a/.oh/skills/wiki/corpus/managed-agents.md +++ b/.oh/knowledge/source/managed-agents.md @@ -1,11 +1,17 @@ --- title: "Scaling Managed Agents: Decoupling the brain from the hands" slug: managed-agents +kind: repo tags: [agents, meta-harness, sessions, sandbox, resilience, security, scaling, anthropic, model-evolution] created: 2026-08-08 updated: 2026-08-27 sources: - - raw/2026-08-08-managed-agents.md + - docs/rfcs/rfc-trace-ledger.md + - docs/rfcs/rfc-runtime-support.md + - docs/security-considerations.md + - docs/harnesses/overview.md + - .devcontainer/docker-compose.yml +verified_at: af16fc85be4468a5681ba87b0d27b371cbd54db2 related: [audit-architecture, runtime-isolation-landscape, recursive-language-models] confidence: provisional --- @@ -13,7 +19,7 @@ confidence: provisional # Scaling Managed Agents: Decoupling the brain from the hands ## Relevant Source Files -- `raw/2026-08-08-managed-agents.md` — immutable capture of Anthropic's article, published 2026-04-08. +- *(external seed)* — the Anthropic article this page was seeded from (published 2026-04-08) was never snapshotted and its URL was never recorded, so it is unrecoverable from this repository and is not a declared source. Everything below is verifiable against the repository documents this page compares that article to. - `docs/rfcs/rfc-trace-ledger.md:33-101,125-155` — Open Harness's proposed append-only event ledger, session index, privacy rules, and replay/diagnosis minimums. - `.devcontainer/docker-compose.yml:30-104` — the current single sandbox service, bind-mounted checkout, persistent auth/tool volumes, optional credential environment, and restart policy. - `docs/security-considerations.md:34-43,121-134` — current secret-path hooks, named-volume state, and the Docker-socket isolation caveat. diff --git a/.oh/skills/wiki/corpus/molt-agentic-reinforcement-learning.md b/.oh/knowledge/source/molt-agentic-reinforcement-learning.md similarity index 95% rename from .oh/skills/wiki/corpus/molt-agentic-reinforcement-learning.md rename to .oh/knowledge/source/molt-agentic-reinforcement-learning.md index 4d1efee6..2c4dc845 100644 --- a/.oh/skills/wiki/corpus/molt-agentic-reinforcement-learning.md +++ b/.oh/knowledge/source/molt-agentic-reinforcement-learning.md @@ -1,11 +1,12 @@ --- title: "Molt: A Scalable PyTorch-Native Training Framework for Agentic Reinforcement Learning" slug: molt-agentic-reinforcement-learning +kind: external tags: [agentic-rl, training, readability, observability, trajectories, async, correctness, agent-harness, nvidia] created: 2026-08-08 updated: 2026-08-27 sources: - - raw/2026-08-08-molt-agentic-reinforcement-learning.md + - https://arxiv.org/abs/2607.21653v1 related: [managed-agents, audit-architecture, recursive-language-models, runtime-isolation-landscape] confidence: provisional --- @@ -13,7 +14,7 @@ confidence: provisional # Molt: A Scalable PyTorch-Native Training Framework for Agentic Reinforcement Learning ## Relevant Source Files -- `raw/2026-08-08-molt-agentic-reinforcement-learning.md` — immutable capture of arXiv:2607.21653v1, published 2026-07-22. +- `https://arxiv.org/abs/2607.21653v1` — the upstream source (fetched 2026-08; the snapshot was never committed, so this reference is the weakest provenance form, `schema.md` § 4). Originally captured as arXiv:2607.21653v1, published 2026-07-22. - `docs/rfcs/rfc-trace-ledger.md:33-155` — the proposed append-only run/session event model and replay, diagnosis, and scoring minimums. - `docs/artifact-contract-schema.md:3-49` — required-artifact and verification-command contracts enforced by audit. - `.oh/evals/README.md:3-49,90-124` — real-state probes, three-state outcomes, and regression gate behavior. diff --git a/.oh/skills/wiki/corpus/oh-cli-portable-lifecycle.md b/.oh/knowledge/source/oh-cli-portable-lifecycle.md similarity index 93% rename from .oh/skills/wiki/corpus/oh-cli-portable-lifecycle.md rename to .oh/knowledge/source/oh-cli-portable-lifecycle.md index 563dad7f..02ad7dd7 100644 --- a/.oh/skills/wiki/corpus/oh-cli-portable-lifecycle.md +++ b/.oh/knowledge/source/oh-cli-portable-lifecycle.md @@ -1,9 +1,10 @@ --- title: "oh CLI Portable Lifecycle" slug: oh-cli-portable-lifecycle +kind: repo tags: [cli, oh, lifecycle, standalone, init, sandbox, remote-fetch, execution-target] created: 2026-07-03 -updated: 2026-08-27 +updated: 2026-09-01 sources: - .oh/cli/src/cli.ts - .oh/cli/src/commands/init.ts @@ -21,6 +22,7 @@ sources: - .oh/README.md - docs/oh-directory-layout.md - docs/rfcs/rfc-brain-hands-boundary.md +verified_at: 4bea83ca793ad02c8fffb41fe49e66127a58e3d9 related: [fresh-machine-setup] confidence: provisional --- @@ -46,7 +48,7 @@ Issue #738 historically added `docs/**` to the `.oh/manifest.json` include list. ## Detail **Payload sourcing (`oh init`)** — precedence `--from <dir>` > `--from-remote` > the CLI's own bundled payload (`cli.ts:139-141`; the two flags conflict, `cli.ts:305-308`). With no source flag and no bundled payload — the installed-binary case, detected via the `manifest.json` marker (`cli.ts:464-469`) — `resolveInitSource` auto-falls back to a remote fetch with a one-line notice naming URL and ref (`cli.ts:498-534`). `--from` sets only the payload source; `--from-remote` sets BOTH payload and templates from the fetched checkout (`cli.ts:478-484`). `oh update` never falls back: it requires `--from` or `--from-remote` (`cli.ts:383-388`) and upgrades only `.oh/` (`cli.ts:110-111`). -**Manifest delivery** — `.oh/manifest.json` defines POSIX globs relative to `.oh/` (`manifest.ts:4-8`). `shouldShip()` requires an include match and rejects an exclude match (`manifest.ts:59-70`). The current source manifest omits both `docs/**` and `patches/**` (`.oh/manifest.json:1-23`). +**Manifest delivery** — `.oh/manifest.json` defines POSIX globs relative to `.oh/` (`manifest.ts:4-8`). `shouldShip()` requires an include match and rejects an exclude match (`manifest.ts:59-70`). The current source manifest omits both `docs/**` and `patches/**`, and since issue #926 ships `knowledge/**` — the durable repository-knowledge surface — alongside `skills/**` (`.oh/manifest.json:1-24`). Knowledge used to reach a consumer only because it sat inside `skills/**`; moving it out without the explicit include would have silently stopped shipping it. Issue #931 then removed `agents/**` from the same list and dropped the two agent provider links from `init.ts`'s `PROVIDER_LINKS`, because skills became the only role primitive. `copyOhPayload()` walks only the source `.oh/` tree and writes only below the target `.oh/` (`vendor.ts:76-85`, `vendor.ts:17-21,119-120`). Root `docs/` is therefore outside both the source walk and the destination guard. The init/update integration tests prove that an existing target `docs/` file remains unchanged (`init.test.ts`, `manifest.test.ts`). This entry cites the RFC and does not restate its decisions. diff --git a/.oh/skills/wiki/corpus/plan-vs-built-reconciliation.md b/.oh/knowledge/source/plan-vs-built-reconciliation.md similarity index 66% rename from .oh/skills/wiki/corpus/plan-vs-built-reconciliation.md rename to .oh/knowledge/source/plan-vs-built-reconciliation.md index c71e6712..e7d72148 100644 --- a/.oh/skills/wiki/corpus/plan-vs-built-reconciliation.md +++ b/.oh/knowledge/source/plan-vs-built-reconciliation.md @@ -1,11 +1,16 @@ --- title: "Plan-vs-Built Reconciliation" slug: plan-vs-built-reconciliation +kind: repo tags: [spec-execute, evidence, merge-gate, comprehension, audit, task-folder, review] created: 2026-08-24 -updated: 2026-08-24 +updated: 2026-09-01 sources: - raw/2026-08-24-plan-vs-built-reconciliation.md + - .oh/skills/spec/references/execute.md + - .oh/skills/audit/references/reviewer-evidence-doc.md + - .oh/evals/probes/spec-ready-finalization.sh +verified_at: 9003903c9e5240c1864888c470d32dd539cc6729 related: [audit-architecture] confidence: provisional --- @@ -13,26 +18,41 @@ confidence: provisional # Plan-vs-Built Reconciliation ## Relevant Source Files -- `.oh/skills/spec/references/execute.md:345` — step 6 writes `evidence.md`; `:421` gates the undraft on it. +- `.oh/skills/spec/references/execute.md:454` — step 7 writes `evidence.md`; `:528` gates the undraft on it. - `.oh/skills/audit/references/reviewer-evidence-doc.md:1` — the doc's contract: path, linkage, observed-output rule, run correlation, honesty about gaps. - `.oh/evals/probes/spec-ready-finalization.sh:58` — the three assertions that keep the gate from being quietly deleted. - `.oh/tasks/<slug>/prd.md` / `progress.txt` / `evidence.md` — the plan, the implementation narrative, and the answer back. ## Summary -The pipeline had ~10 verification nodes and zero comprehension nodes: every gate asked *is this correct?* and none asked *is this still what you agreed to?* An operator's model of the work stops at the plan they approved, while the single implementation owner — the agent running `/spec execute` — carries implementation and verification through one owned workflow. Reconciliation closes that gap with one tracked artifact — `.oh/tasks/<slug>/evidence.md` — which the merge gate **refuses to undraft without** (`execute.md:393`). +The pipeline had ~10 verification nodes and zero comprehension nodes: every gate asked *is this correct?* and none asked *is this still what you agreed to?* An operator's model of the work stops at the plan they approved, while the single implementation owner — the agent running `/spec execute` — carries implementation and verification through one owned workflow. Reconciliation closes that gap with one tracked artifact — `.oh/tasks/<slug>/evidence.md` — which the merge gate **refuses to undraft without** (`execute.md:554`). ## Detail -**It is a gate condition, not a report.** `/spec execute` checks two things before `gh pr ready`: that the file exists, and that `git ls-files --error-unmatch` finds it. The second half is not redundant — `.oh/tasks/` is gitignored, so a doc written without `git add -f` sits on disk and is **absent from the PR diff**, which from the reviewer's seat is identical to never writing it (`execute.md:432-436`). Failing either half is terminal status `DRAFT-BLOCKED (evidence)`, not a warning. +**It is a gate condition, not a report.** `/spec execute` checks two things before `gh pr ready`: that the file exists, and that `git ls-files --error-unmatch` finds it. The second half is not redundant — `.oh/tasks/` is gitignored, so a doc written without `git add -f` sits on disk and is **absent from the PR diff**, which from the reviewer's seat is identical to never writing it (`execute.md:566-569`). Failing either half is terminal status `DRAFT-BLOCKED(evidence)`, written to the PR and to `/tmp/spec-<slug>.state` (`execute.md:563`), not a warning. -**Four questions, in order** (`reviewer-evidence-doc.md:56-70`): what the plan asked for, what was built, where they diverged and why, what remains unverified. The last two are the load-bearing pair, because they are the only ones a reviewer cannot reconstruct from the diff. An empty one is written as the literal word `None` / `Nothing`; omitting it reads as *nothing diverged, nothing unchecked* — the most expensive claim the document can make by accident. +**Five questions, in order** (`execute.md:470-489`): why this is better than not doing it, what the plan asked for, what was built, where they diverged and why, what remains unverified. The first is the one a reviewer cannot answer from the diff, the gates, or the plan. The last two are the load-bearing pair, because they are the only ones a reviewer cannot reconstruct from the diff. An empty one is written as the literal word `None` / `Nothing`; omitting it reads as *nothing diverged, nothing unchecked* — the most expensive claim the document can make by accident. -**Observed output only** (`reviewer-evidence-doc.md:33-37`). Every claim quotes a command that actually ran, trimmed but never paraphrased into something unreproducible. Predicted or reconstructed output is forbidden, and a gate that produced no observed output is recorded as a **gap**, never as a pass — which is also halt condition 6 (`execute.md:512`). +**The gate binds to one head.** A promotable verdict describes the commit it was +read against, so `execute.md:625` re-opens it on any push after `gh pr ready` and +`:544` confirms the PR's `headRefOid` is the commit being promoted. A ready PR +whose head moved past its classification is returned to draft rather than left +standing on a verdict about a different tree — the same shape as the stale +`AUDIT_RUN_ID` rule above, applied to CI instead of to the audit log. + +**Observed output only** (`reviewer-evidence-doc.md:33-37`). Every claim quotes a command that actually ran, trimmed but never paraphrased into something unreproducible. Predicted or reconstructed output is forbidden, and a gate that produced no observed output is recorded as a **gap**, never as a pass — which is also the step-7 halt condition (`execute.md:655`). **Ownership.** `/audit implementation` and `/audit pr` are read-only and do not write this file; the orchestrating caller does, from what those routes returned (`reviewer-evidence-doc.md:18-24`). A route that wrote it would break its own report-only contract. -**Do not confuse the two evidence artifacts.** `AUDIT_EVIDENCE_PATH` (`evidence.json`, schema v1, invocation-scoped, never inside `AUDIT_ROOT`) is the machine record that lets the audit boundary log `complete`. `evidence.md` is a separate tracked Markdown artifact for humans, correlated to the same `AUDIT_RUN_ID` (`reviewer-evidence-doc.md:13-16`). A stale run id means the doc is rewritten, not kept (`execute.md:530`). +**A second comprehension gate now sits beside it.** Issue #926 added the Actual +Knowledge Impact gate immediately before `evidence.md` (`execute.md:403`): the +implementation's real changed paths are matched against every knowledge page's +declared dependencies, and each impacted page ends `UPDATED`, `REVERIFIED`, or +`NOT-AFFECTED (<reason>)` inside this same document. Where `evidence.md` answers +*is this still what you agreed to*, that gate answers *what did this make untrue*. +Both are comprehension nodes, and both are recorded in one artifact. + +**Do not confuse the two evidence artifacts.** `AUDIT_EVIDENCE_PATH` (`evidence.json`, schema v1, invocation-scoped, never inside `AUDIT_ROOT`) is the machine record that lets the audit boundary log `complete`. `evidence.md` is a separate tracked Markdown artifact for humans, correlated to the same `AUDIT_RUN_ID` (`reviewer-evidence-doc.md:13-16`). A stale run id means the doc is rewritten, not kept (`execute.md:698`). -**The implementation narrative is promoted, not stranded.** `progress.txt` holds the per-story record; the owner folds it and `evidence.md` into the PR body so the reviewer meets the work in the PR rather than by opening the task folder (`execute.md:410-418`). +**The implementation narrative is promoted, not stranded.** `progress.txt` holds the per-story record; the owner folds it and `evidence.md` into the PR body so the reviewer meets the work in the PR rather than by opening the task folder (`execute.md:575-583`). **DeepWiki comparison.** Run 2026-08-24 against `https://deepwiki.com/mifunedev/openharness` (Overview page). DeepWiki has **no** entry for this concept: it does not mention `evidence.md` or any plan-versus-built reconciliation gate, and its planning-phase table still lists `/ship-spec` with the gloss *"Convert specs into executable tasks via a critic"* — the critic gate US-001 removed. So this page is **net-new relative to DeepWiki**, not a re-synthesis of it, and the divergence is upstream staleness rather than a contradiction to resolve. **The workflow no longer runs this comparison** (2026-08-24): DeepWiki regenerates on no schedule the gate could depend on, so `/spec plan` and `/spec execute` dropped it — the staleness recorded here is the evidence for that removal, kept as rationale rather than as a standing obligation. @@ -55,7 +75,7 @@ flowchart LR | --- | --- | --- | --- | | `prd.md` | `/spec plan` | operator (the go/no-go) | yes (`add -f`) | | `progress.txt` | the implementation owner | `/spec execute` implementation stage | yes (`add -f`) | -| `evidence.md` | `/spec execute` step 6 | the PR reviewer | yes (`add -f`) — gated | +| `evidence.md` | `/spec execute` step 7 | the PR reviewer | yes (`add -f`) — gated | | `evidence.json` | the audit boundary | the boundary's terminal log | no (invocation-scoped) | ## See Also diff --git a/.oh/skills/wiki/corpus/recursive-language-models.md b/.oh/knowledge/source/recursive-language-models.md similarity index 99% rename from .oh/skills/wiki/corpus/recursive-language-models.md rename to .oh/knowledge/source/recursive-language-models.md index fbe13827..293b0d69 100644 --- a/.oh/skills/wiki/corpus/recursive-language-models.md +++ b/.oh/knowledge/source/recursive-language-models.md @@ -1,6 +1,7 @@ --- title: "Recursive Language Models" slug: recursive-language-models +kind: external tags: [rlm, context-as-environment, weighted-trajectories, agent-harness, llm-agents, self-consistency] created: 2026-06-27 updated: 2026-08-31 diff --git a/.oh/skills/wiki/corpus/recursive-self-improvement-survey.md b/.oh/knowledge/source/recursive-self-improvement-survey.md similarity index 98% rename from .oh/skills/wiki/corpus/recursive-self-improvement-survey.md rename to .oh/knowledge/source/recursive-self-improvement-survey.md index a0098453..3fcfbb9f 100644 --- a/.oh/skills/wiki/corpus/recursive-self-improvement-survey.md +++ b/.oh/knowledge/source/recursive-self-improvement-survey.md @@ -1,11 +1,12 @@ --- title: "Recursive Self-Improvement in AI (survey, arXiv 2607.07663)" slug: recursive-self-improvement-survey +kind: external tags: [rsi, self-improvement, verification-hierarchy, skill-libraries, harness-evolution, self-evaluation, model-collapse, capability-benchmark] created: 2026-08-13 updated: 2026-08-31 sources: - - raw/2026-08-13-recursive-self-improvement-survey.md + - https://arxiv.org/abs/2607.07663 related: [recursive-language-models, molt-agentic-reinforcement-learning, audit-architecture] confidence: provisional --- diff --git a/.oh/skills/wiki/corpus/release-versioning.md b/.oh/knowledge/source/release-versioning.md similarity index 95% rename from .oh/skills/wiki/corpus/release-versioning.md rename to .oh/knowledge/source/release-versioning.md index 0f628b08..8fc791b9 100644 --- a/.oh/skills/wiki/corpus/release-versioning.md +++ b/.oh/knowledge/source/release-versioning.md @@ -1,11 +1,18 @@ --- title: "Release Versioning" slug: release-versioning +kind: repo tags: [release, versioning, semver, calver, github-actions, ghcr, tags, workflow, package-json] created: 2026-08-23 updated: 2026-08-23 sources: - raw/2026-08-23-release-versioning.md + - package.json + - .github/workflows/release.yml + - .oh/scripts/release-reservation.mjs + - .oh/scripts/reserve-github-release.mjs + - .oh/scripts/promote-release-latest.sh +verified_at: 60f8c12d414b86389123c77229e15886df858208 related: [oh-cli-portable-lifecycle] confidence: confirmed --- diff --git a/.oh/skills/wiki/corpus/runtime-isolation-landscape.md b/.oh/knowledge/source/runtime-isolation-landscape.md similarity index 99% rename from .oh/skills/wiki/corpus/runtime-isolation-landscape.md rename to .oh/knowledge/source/runtime-isolation-landscape.md index a7462a78..8ebe2bd0 100644 --- a/.oh/skills/wiki/corpus/runtime-isolation-landscape.md +++ b/.oh/knowledge/source/runtime-isolation-landscape.md @@ -1,6 +1,7 @@ --- title: "Runtime Isolation Landscape (2026)" slug: runtime-isolation-landscape +kind: external tags: [runtime, isolation, sandbox, gvisor, firecracker, kata, microvm, cloudflare, e2b, daytona, fly, modal] created: 2026-07-04 updated: 2026-08-27 diff --git a/.oh/skills/wiki/corpus/sandbox-dependency-installs.md b/.oh/knowledge/source/sandbox-dependency-installs.md similarity index 98% rename from .oh/skills/wiki/corpus/sandbox-dependency-installs.md rename to .oh/knowledge/source/sandbox-dependency-installs.md index 0dea5759..b7f26eb6 100644 --- a/.oh/skills/wiki/corpus/sandbox-dependency-installs.md +++ b/.oh/knowledge/source/sandbox-dependency-installs.md @@ -1,6 +1,7 @@ --- title: "Sandbox Dependency Installs" slug: sandbox-dependency-installs +kind: repo tags: [sandbox, devcontainer, pnpm, dependencies, boot] created: 2026-07-01 updated: 2026-08-31 @@ -8,6 +9,7 @@ sources: - .devcontainer/entrypoint.sh - .oh/scripts/__tests__/entrypoint-pnpm-install.test.ts - .oh/evals/probes/entrypoint-pnpm-manifest-fingerprint.sh +verified_at: 1c5f37230822ec2bbc5ed316be92ad295722b693 related: [compose-env-boundary, recursive-language-models] confidence: confirmed --- diff --git a/.oh/skills/wiki/corpus/wikiskill-experience-compilation.md b/.oh/knowledge/source/wikiskill-experience-compilation.md similarity index 83% rename from .oh/skills/wiki/corpus/wikiskill-experience-compilation.md rename to .oh/knowledge/source/wikiskill-experience-compilation.md index 5aa0fdb9..eca51983 100644 --- a/.oh/skills/wiki/corpus/wikiskill-experience-compilation.md +++ b/.oh/knowledge/source/wikiskill-experience-compilation.md @@ -1,9 +1,10 @@ --- title: "WikiSkill: Compiling Agent Experience into Persistent Knowledge (arXiv 2608.27454)" slug: wikiskill-experience-compilation +kind: external tags: [skill-evolution, persistent-knowledge, continual-learning, harness-evolution, self-improvement, wiki, ablation, skill-transfer] created: 2026-08-31 -updated: 2026-08-31 +updated: 2026-09-01 sources: - raw/2026-08-31-wikiskill-experience-compilation.md related: [recursive-self-improvement-survey, molt-agentic-reinforcement-learning, audit-architecture] @@ -17,7 +18,7 @@ cs.AI/cs.CL. Source: <https://arxiv.org/abs/2608.27454>. ## Relevant Source Files - `.oh/skills/wiki/references/schema.md` — the local analogue of the paper's wiki layer schema. -- `.oh/skills/wiki/corpus/raw/` — the local analogue of the paper's raw layer, holding source snapshots rather than agent traces. +- `.oh/knowledge/raw/` — the local analogue of the paper's raw layer, holding source snapshots rather than agent traces. - `.oh/skills/builder/SKILL.md` — the local analogue of the paper's Skill Proposer. - `.oh/skills/retro/SKILL.md` — report-only lesson producer; the paper's maintainer role has no local owner. - `.oh/evals/capability/RESULTS.md` — the ceiling instrument the paper's ablation argues should move. @@ -60,10 +61,14 @@ when a skill encodes model-specific low-level workarounds. injected wholesale); strict gating rejects neutral proposals the authors admit could pay off later; there is no wiki pruning; long-horizon tasks are uncovered. -**Local reading.** The harness already has raw snapshots, a generated index, a -proposer, and a two-signal validation metric. It lacks the pattern layer, the -impact ledger, and any wiki read on the proposer path — the three parts the -ablation credits. +**Local reading.** All three parts the ablation credits now exist. Issue #916 +added the pattern layer and the impact ledger; issue #926 added the read on the +proposer path — `/spec plan` queries tracked knowledge and then `--patterns` +before the PRD is written, and `/spec execute` deliberately does not, which is the +asymmetry the ablation measured. The layers map onto +`.oh/knowledge/{raw,source,patterns}` with the ledger moved out to +`.oh/evals/decisions/skill-impact.md`, because a record of accepted and rejected +proposals is a decision history rather than synthesis about a topic. ## System Relationships ```mermaid diff --git a/.oh/manifest.json b/.oh/manifest.json index cb6d63d5..d4c098cb 100644 --- a/.oh/manifest.json +++ b/.oh/manifest.json @@ -5,6 +5,7 @@ "install/**", "templates/**", "evals/**", + "knowledge/**", "skills/**", "hooks/**", "skills.lock", diff --git a/.oh/skills/audit/references/harness.md b/.oh/skills/audit/references/harness.md index 76c492f4..a980d640 100644 --- a/.oh/skills/audit/references/harness.md +++ b/.oh/skills/audit/references/harness.md @@ -55,7 +55,7 @@ Read the following before spawning agents. Pass the assembled snapshot to every ls "$AUDIT_ROOT/.claude/skills/" ls "$AUDIT_ROOT/crons/" 2>/dev/null || echo "no crons" tail -20 "$AUDIT_ROOT/crons/.cron.log" 2>/dev/null -ls "$AUDIT_ROOT/.oh/skills/wiki/corpus/" 2>/dev/null | head -20 +ls "$AUDIT_ROOT/.oh/knowledge/" 2>/dev/null | head -20 # Package health cat "$AUDIT_ROOT/package.json" 2>/dev/null | head -30 @@ -123,7 +123,7 @@ Launch 4 Agent tool calls **in a single message**. Each receives the Context Sna > > 3. **Issue template completeness** — List `.github/ISSUE_TEMPLATE/` files. For each template, check: does it have required fields, clear labels, and assignment guidance? > -> 4. **Wiki utilization** — Count wiki pages under `.oh/skills/wiki/corpus/`. For each, is it populated or a placeholder stub? What percentage is populated? +> 4. **Wiki utilization** — Count wiki pages under `.oh/knowledge/`. For each, is it populated or a placeholder stub? What percentage is populated? > > **Return format (Ultra compression):** > ``` @@ -197,7 +197,7 @@ Launch 4 Agent tool calls **in a single message**. Each receives the Context Sna > > **Audit areas:** > -> 1. **Wiki utilization** — List all files under `.oh/skills/wiki/corpus/`. For each, check if it has substantive content (>10 lines) or is a placeholder stub. What percentage is populated? +> 1. **Wiki utilization** — List all files under `.oh/knowledge/`. For each, check if it has substantive content (>10 lines) or is a placeholder stub. What percentage is populated? > > 2. **Cron health** — For each cron definition in `crons/`, classify: ACTIVE (recently logged evidence), STALE (defined but no recent log evidence), MISCONFIGURED (broken frontmatter or missing schedule). Check `crons/.cron.log` for cron execution traces. > @@ -321,7 +321,7 @@ Return this structured observation to the outer dispatcher; do not report a run | Orchestrator skills | `.claude/skills/` | | Crons | `crons/` | | Cron liveness | `crons/.cron.log` | -| Wiki | `.oh/skills/wiki/corpus/` | +| Wiki | `.oh/knowledge/` | | Compose | `.devcontainer/docker-compose.yml` | | Entrypoint | `.devcontainer/entrypoint.sh` | | CI workflows | `.github/workflows/` | diff --git a/.oh/skills/audit/references/reviewer-evidence-doc.md b/.oh/skills/audit/references/reviewer-evidence-doc.md index fcbda6b6..ae710e32 100644 --- a/.oh/skills/audit/references/reviewer-evidence-doc.md +++ b/.oh/skills/audit/references/reviewer-evidence-doc.md @@ -47,6 +47,13 @@ report-only contract. - **Tracked**: `.oh/tasks/` is gitignored, so the file must be added with `git add -f`. An untracked `evidence.md` exists on disk and is **absent from the PR diff** — from the reviewer's seat that is identical to not having written it at all. +- **Follow-ups are cited, not named**: an acceptance criterion recorded as satisfied + by work outside this repository — a mirror issue, a downstream PR, a tracked + follow-up — is met only when that artifact **exists** and the doc carries its + resolvable URL. Naming a follow-up in prose is a plan, and a plan reads as + satisfaction to every reader who does not go looking. File it, then cite it. No + gate here can see an artifact in another repository, so this line is the only + thing standing between "deferred" and "done". - **Answers back to the plan**: the five sections below are not optional prose. Three of them — *why this is better*, *divergence* and *unverified* — are the things a reviewer cannot reconstruct from the diff, so an empty one is written as `None` / `Nothing` explicitly. Omitting diff --git a/.oh/skills/benchmark/SKILL.md b/.oh/skills/benchmark/SKILL.md index 077a65d0..ffd216a1 100644 --- a/.oh/skills/benchmark/SKILL.md +++ b/.oh/skills/benchmark/SKILL.md @@ -131,7 +131,7 @@ command for the change under evaluation, e.g.: git revert --no-edit <merge-or-commit-sha> # undo the not-beneficial change, then → repeat ``` -**The revert never covers `.oh/skills/wiki/corpus/`.** Pattern pages and the +**The revert never covers `.oh/knowledge/`.** Pattern pages and the `skill-impact.md` ledger are excluded from every revert: the knowledge that an approach was tried and did not work is the rejected cycle's durable output, and it is what stops the same proposal being made again @@ -165,7 +165,7 @@ REDIRECT-FLAG: capability suite score flat at <X.XX>/2.00 for <N> cycles while N - **Mutate the code.** No revert, no commit, no merge — it emits the verdict and names the revert; the runner/operator acts (single-owner handoff: decided here, acted on elsewhere). Its one write is appending its own verdict record to - `.oh/skills/wiki/corpus/skill-impact.md` — the judgment it just made, and nothing + `.oh/evals/decisions/skill-impact.md` — the judgment it just made, and nothing else. Append an `SI-nnnn-V` record for the `SI-nnnn` proposal under evaluation (`for`, `floor`, `ceiling`, `verdict`); never edit the proposal record in place, and never write a record for a change no `/builder` proposal covers. diff --git a/.oh/skills/builder/SKILL.md b/.oh/skills/builder/SKILL.md index b1adcde8..890e6eb7 100644 --- a/.oh/skills/builder/SKILL.md +++ b/.oh/skills/builder/SKILL.md @@ -70,7 +70,7 @@ steps. `/wiki query <artifact-name-or-subsystem> --patterns` and read what it returns: each page records a failure mode, its root cause, and a workaround this harness already paid for. Cite the motivating `[[pattern-...]]` slugs in the report. -- Read `.oh/skills/wiki/corpus/skill-impact.md` for prior proposals against the same +- Read `.oh/evals/decisions/skill-impact.md` for prior proposals against the same target. Do not re-propose a change recorded there as `REJECTED` unless new evidence contradicts the recorded validation; when you do, cite the prior record id. @@ -110,10 +110,11 @@ or create unsafe side effects. Otherwise use the request and repository evidence - Report the files created, updated, or removed; the resulting invocation or loading behavior; key design choices; and validation evidence. Never claim a check ran if it did not. -- Append a `PROPOSED` record to `.oh/skills/wiki/corpus/skill-impact.md` when a skill +- Append a `PROPOSED` record to `.oh/evals/decisions/skill-impact.md` when a skill edit lands: the next `SI-nnnn` id, the one-sentence proposal, the single target artifact, the motivating pattern slugs, and the unified diff scoped to that target path. `motivating patterns: none (direct request)` is a legitimate value — - record it rather than inventing a pattern to cite. Force-add the ledger - (`git add -f`) and report the allocated id. Never edit an existing record. + record it rather than inventing a pattern to cite. Stage the ledger + (`.oh/evals/decisions/` is tracked, so a plain `git add`) and report the + allocated id. Never edit an existing record. diff --git a/.oh/skills/imagine/SKILL.md b/.oh/skills/imagine/SKILL.md index a18b61fb..624b14e3 100644 --- a/.oh/skills/imagine/SKILL.md +++ b/.oh/skills/imagine/SKILL.md @@ -131,7 +131,7 @@ Next: /spec plan --plan .claude/specs/<slug>/spec.md - **Asking clarifying questions.** Breaks the one-shot contract. If the scenario is too vague, push the ambiguities into `## Open questions for /prd`; do not interrupt. - **Writing a full PRD.** Story seeds are one-liners. Numbered functional requirements, acceptance criteria, success metrics — all `/prd`'s job, not this skill's. - **Skipping the mermaid diagram.** The diagram is the cheapest forcing function for "did I actually understand the scenario?" Pick a real type; never emit a placeholder or `// TODO: diagram`. -- **Writing outside `.claude/specs/<slug>/`.** No spillover into `.oh/tasks/`, `.oh/skills/wiki/corpus/`, or root. Those surfaces have their own skills. +- **Writing outside `.claude/specs/<slug>/`.** No spillover into `.oh/tasks/`, `.oh/knowledge/`, or root. Those surfaces have their own skills. - **Auto-chaining into `/spec`.** Keep the seam explicit. The user edits the spec before formalizing — that's the entire reason the seam exists. - **Truncating the scenario into the slug.** `imagine a long thirty word scenario about ...` → derive the noun phrase (`thirty-word-scenario` or `long-scenario`), not the first five words verbatim. diff --git a/.oh/skills/retro/SKILL.md b/.oh/skills/retro/SKILL.md index 8e6e0e17..f27a09a0 100644 --- a/.oh/skills/retro/SKILL.md +++ b/.oh/skills/retro/SKILL.md @@ -1,6 +1,6 @@ --- name: retro -argument-hint: "[--dry-run] [--focus <subsystem>] [auto-approve]" +argument-hint: "[--task <slug>] [--dry-run] [--focus <subsystem>] [auto-approve]" allowed-tools: Read, Grep, Bash, Edit description: | Scientific session-closing retrospective: scan the current conversation, @@ -43,6 +43,29 @@ Key boundary: `/retro` is *session-scoped reflection*. The lint/audit skills abo Current conversation only. `/retro` does not read prior sessions or the `~/.claude/projects/...` auto-memory store. It works from what is already in context. +### `--task <slug>` — scope the pass to one build + +`--task <slug>` anchors the pass to a just-built `.oh/tasks/<slug>/` run instead of +the whole ambient session. It changes **what counts as a signal**, and nothing +else: the hypothesis engine, the qualify filter, the five-subsystem lens, the +verdict/confidence rules, and the report-only contract are all unchanged. + +With `--task <slug>`, gather signals primarily from that unit's own artifacts: + +- `prd.md` — what the plan intended, and what its `## Plan Reconciliation` says + grounding changed; +- `prd.json` — the task graph and which stories passed; +- `progress.txt` — what actually shipped, in order, with the run's own notes; +- `evidence.md` — the divergences and the gaps the implementation owner recorded; +- the `implementation ⇄ audit` history — how many FAIL→build cycles, and why. + +If `.oh/tasks/<slug>/` has no `prd.md`, there is no build to reflect on: say so +and fall back to a plain session-scoped pass. + +`/spec retro <slug>` is a thin alias for this form +(`.oh/skills/spec/references/retro.md`). There is one retro ontology and it is +this skill's. + ## Deterministic contract Produce a report that follows `${CLAUDE_SKILL_DIR}/references/report-schema.md`. At minimum it contains: diff --git a/.oh/skills/spec/SKILL.md b/.oh/skills/spec/SKILL.md index 7b6c6fd0..c0080fc0 100644 --- a/.oh/skills/spec/SKILL.md +++ b/.oh/skills/spec/SKILL.md @@ -1,22 +1,23 @@ --- name: spec description: >- - Canonical decomposed build workflow and dispatcher. Routes the first token of - $ARGUMENTS to one of four subcommands: ship, plan, execute, or retro. Each of - plan/execute/retro is pointed at a .oh/tasks/<slug>/ folder (the universal - interface) and is independently runnable and fan-out-able; ship composes plan - then execute. This skill owns the ONLY build path; there is no all-in-one - composer beside it. Full per-subcommand procedures live in - references/{ship,plan,execute,retro}.md. + Canonical decomposed build workflow and dispatcher, and the harness's closed + repo-knowledge learning loop. Routes the first token of $ARGUMENTS to plan or + execute; an unrecognized first token is an approved plan path and runs plan + then execute. `plan` recalls tracked knowledge from .oh/knowledge/, re-grounds + it against current sources, and writes the .oh/tasks/<slug>/ folder; + `execute` re-grounds, implements under one owner — the agent running it — and + derives knowledge invalidation from the actual diff. This skill owns the ONLY build + path; there is no all-in-one composer beside it. Full per-subcommand + procedures live in references/{plan,execute,retro}.md. TRIGGER when: an approved plan file should become a ready PR without further - hand-holding, "/spec <plan-path>", "ship this plan", "build this plan end to end" - -> ship (also the DEFAULT for an unrecognized first token); a topic/plan/issue - needs to become a buildable task folder without building it, "plan <topic>", - "scaffold the task for <issue>" -> plan; an approved .oh/tasks/<slug>/ folder - needs building to a promotable PR, "execute <slug>", "build <slug>" -> execute; a - build PASSed audit and its lessons should be captured, "retro the <slug> build", - "capture lessons for <slug>" -> retro. -argument-hint: "<plan-path> | ship <plan-path|topic> [--issue <N>] [--slug <slug>] | plan <topic> [--plan <path>] [--issue <N>] [--slug <slug>] [--prefix <type>] [--repo <o/n>] [--base <branch>] | execute <slug> [--pr <N>] [--repo <o/n>] [--remote <name>] [--base <branch>] | retro <slug> [--dry-run]" + hand-holding, "/spec <plan-path>", "build this plan end to end" -> the default + plan-then-execute path; a topic/plan/issue needs to become a buildable task + folder without building it, "plan <topic>", "scaffold the task for <issue>" + -> plan; an approved .oh/tasks/<slug>/ folder needs building to a promotable + PR, "execute <slug>", "build <slug>" -> execute; a build PASSed audit and its + lessons should be captured, "retro the <slug> build" -> retro. +argument-hint: "<plan-path> | plan <topic> [--plan <path>] [--issue <N>] [--slug <slug>] [--prefix <type>] [--repo <o/n>] [--base <branch>] | execute <slug> [--pr <N>] [--repo <o/n>] [--remote <name>] [--base <branch>] | retro <slug> [--dry-run]" --- # /spec — canonical workflow dispatcher @@ -27,17 +28,55 @@ selects the subcommand; everything after it is that subcommand's own argument string. Each subcommand's full procedure lives in a reference doc under `references/` — read that doc and follow it as the authoritative instructions. -**An unrecognized first token is not an error — it is `ship`.** `/spec <plan-path>` -is the ordinary way in: it scaffolds the task folder and then builds it through to a -ready-for-review pull request. Naming a node explicitly (`plan`, `execute`, `retro`) -runs only that node, which is what fan-out and recovery need. +**An unrecognized first token is not an error — it is an approved plan path.** +`/spec <plan-path>` is the ordinary way in: it scaffolds the task folder and then +builds it through to a ready-for-review pull request. Naming a node explicitly +(`plan`, `execute`) runs only that node, which is what fan-out and recovery need. This is the **only** spec pipeline; there is no all-in-one composer beside it. -`references/execute.md` holds the build mechanics in full — the issue, the branch, -the draft PR, the implementation, the `/eval` and wiki gates, the promotable -classification, and the undraft — so learning what the build does never sends a -reader to a second skill. The dispatcher splits the pipeline so each node can be -run independently or fanned out at scale via `/delegate`. +`references/execute.md` holds the build mechanics in full — the issue, the +branch, the draft PR, the implementation step, the `/eval` and knowledge gates, the +promotable classification, and the undraft — so learning what the build does +never sends a reader to a second skill. + +## The loop + +`/spec` is where accumulated repository understanding is consumed, re-verified, +spent, and replenished. Knowledge is a derived cache; the repository is the +source of truth. + +```text +operator intent + ↓ +/spec plan ──▶ recall tracked knowledge (.oh/knowledge/, /wiki query) + ↓ + verify claims against current code / tests / docs + ↓ + prd.md (Knowledge Context · Expected Knowledge Impact · + Plan Reconciliation) + prd.json + ↓ +/spec execute ─▶ re-ground against current HEAD + ↓ + implementation ⇄ audit → /eval (once) + ↓ + Actual Knowledge Impact = expected + actual diff + dependencies + ↓ + update / reverify affected pages + ↓ + evidence.md → retro → /wiki compile + ↓ +future /spec plan reads what this run learned +``` + +Two rules keep the loop honest: + +- **Knowledge informs; it never authorizes.** A recalled page is orientation. + Code and tests are implementation truth; canonical docs and RFCs are + intended-design truth. A material claim is re-grounded before it is relied on. +- **The planner predicts, the diff decides.** `Expected Knowledge Impact` is the + planner's guess. The final impact is derived from the actual changed paths and + the pages' declared dependencies, because implementation touches paths the + planner never saw. ## Workflow contract @@ -45,58 +84,100 @@ The canonical operative path is `spec-plan → spec-execute → merge → reset|clean`. There is no automated selection node. A human selects the work and approves -`prd.md`; that approval is the commitment gate. **Handing `/spec` an approved plan -file satisfies that gate** — writing the plan and passing it in *is* the operator's -approval, so `ship` carries it through to `execute` without a second prompt. A bare -topic with no plan file has no such approval behind it: `ship` stops after `plan` and -hands the operator the folder to approve. `/spec execute` runs -`build ⇄ audit → evidence → spec-retro → improve` and stops at a ready-for-review -pull request. **`/spec` never launches another coding-agent process to do that work** — -the agent invoking `execute` is the single implementation owner, from the isolated -worktree through the final PR gates. The human alone merges. The runner performs -`reset` or `clean`. - -The `.oh/tasks/<slug>/` folder is the interface between all three subcommands. -`evidence.md` records plan requirements, build results, reasons for divergence, -and unverified work. `/spec execute` refuses to mark a pull request ready when that -evidence is absent or uncommitted. +`prd.md`; that approval is the commitment gate. **Handing `/spec` an approved +plan file satisfies that gate** — writing the plan and passing it in *is* the +operator's approval, so the default path carries it through to `execute` without +a second prompt. A bare topic with no plan file has no such approval behind it: +the run stops after `plan` and hands the operator the folder to approve. + +**Approval covers the intent that was approved, not whatever grounding turns it +into.** If `plan`'s grounding step finds that satisfying the approved plan +requires a *material* change to the operator's intent, the run stops and asks for +re-approval rather than treating the original approval as covering the new shape +(`references/plan.md`, `## Plan Reconciliation`). + +`/spec execute` stops at a ready-for-review pull request. The human alone merges. +The runner performs `reset` or `clean`. + +### The task folder + +The `.oh/tasks/<slug>/` folder is the interface between the subcommands: + +```text +.oh/tasks/<slug>/ +├── prd.md the approved plan, with its knowledge sections +├── prd.json the ordered task graph — and the authoritative completion state +├── progress.txt the execution narrative and resume evidence +├── evidence.md written after implementation; gates the undraft +└── eval-result.json the commit-keyed probe-suite result, when applicable +``` + +There is **no generated `prompt.md`**. The task prompt is rendered at +execution time from `templates/task-prompt.md` plus `prd.md` and `prd.json`; a +persisted copy of a template only drifts from it. + +**Completion is structured state.** A task is complete when every story in +`prd.json` has `"passes": true` — `jq -e 'all(.userStories[]; .passes == true)'`. +There is no prose sentinel in `progress.txt`; a second representation of +completion is a second thing that can be wrong. + +### Execution lifecycle + +`execute` can return before the build finishes — a cron or a resumed run reaches +this line with stories still open — so it reports the state it actually reached: + +```text +PLANNED ──▶ RUNNING ──▶ READY + └──▶ DRAFT-BLOCKED(<gate>) +``` + +`RUNNING` is a real state, not ceremony — and it is a state of the **task**, not +of a process: an approved folder whose stories are not all `passes: true`. The +owner mirrors it into `/tmp/spec-<slug>.state` at every phase change so resume, +watchdog, and operator visibility have something to read. `execute` returning +before the build finishes is reported as `RUNNING`, never as `READY`. ## Subcommands | Subcommand | Arg shape | Purpose | Procedure | |---|---|---|---| -| `ship` | `<plan-path\|topic> [--issue <N>] [--slug <slug>] [--prefix <type>] [--repo <o/n>] [--base <branch>]` | **The default.** `plan` → `execute` in one invocation: an approved plan file becomes a ready-for-review PR. Selected by an unrecognized first token, so `/spec <plan-path>` works bare | `references/ship.md` | -| `plan` | `<topic> [--plan <path>] [--issue <N>] [--slug <slug>] [--prefix <type>] [--repo <o/n>] [--base <branch>]` | Turn a topic/plan/issue into a fully-scaffolded `.oh/tasks/<slug>/` four-file folder | `references/plan.md` | -| `execute` | `<slug> [--pr <N>] [--repo <o/n>] [--remote <name>] [--base <branch>]` | `implementation ⇄ audit → evidence → spec-retro → improve` to a ready PR, stopping at the human merge gate | `references/execute.md` | -| `retro` | `<slug> [--dry-run]` | Execution-side `/retro` scoped to a built `.oh/tasks/<slug>/` | `references/retro.md` | +| `plan` | `<topic> [--plan <path>] [--issue <N>] [--slug <slug>] [--prefix <type>] [--repo <o/n>] [--base <branch>]` | Recall tracked knowledge, re-ground it, and turn a topic/plan/issue into a scaffolded `.oh/tasks/<slug>/` folder | `references/plan.md` | +| `execute` | `<slug> [--pr <N>] [--repo <o/n>] [--remote <name>] [--base <branch>]` | Re-ground, then `implementation ⇄ audit → eval → knowledge impact → evidence → retro → compile → benchmark` to a ready PR, stopping at the human merge gate | `references/execute.md` | +| `retro` | `<slug> [--dry-run]` | Compatibility wrapper for `/retro --task <slug>` | `references/retro.md` | +| *(default)* | `<plan-path>` | `plan` then `execute` — the approved-plan path. Selected by any first token that is not a node name | this file, plus both procedures | ## Dispatch 1. Split `$ARGUMENTS`: `SUB` = the first token; `REST` = everything after it. -2. When `SUB` names a node (`ship`, `plan`, `execute`, `retro`), read - `references/<SUB>.md` and follow it, treating `REST` as that doc's `$ARGUMENTS` - (e.g. for `/spec plan <topic> --issue 7`, the plan procedure sees +2. When `SUB` names a node (`plan`, `execute`, `retro`), read + `references/<SUB>.md` and follow it, treating `REST` as that doc's + `$ARGUMENTS` (e.g. for `/spec plan <topic> --issue 7`, the plan procedure sees `<topic> --issue 7`). -3. **Any other non-empty `$ARGUMENTS` is `ship`, with the whole string — `SUB` - included — as its argument.** `/spec .claude/plans/x.md` and - `/spec ship .claude/plans/x.md` are the same invocation. Do not print usage for an - argument that merely fails to name a node; a plan path is the expected input. -4. Empty `$ARGUMENTS` → print the Subcommands table as usage and stop. There is - nothing to ship. +3. **Any other non-empty `$ARGUMENTS` is the approved-plan path.** Run + `references/plan.md` with the whole string, verify the three-file contract, + then run `references/execute.md` with the resulting `<slug>`. Do not print + usage for an argument that merely fails to name a node; a plan path is the + expected input. +4. Empty `$ARGUMENTS` → print the Subcommands table as usage and stop. ```bash SUB="${ARGUMENTS%% *}" # first token REST="${ARGUMENTS#"$SUB"}"; REST="${REST# }" # remainder case "$SUB" in - ship|plan|execute|retro) + plan|execute|retro) # read references/$SUB.md and execute it with REST as its $ARGUMENTS ;; + ship) + # `ship` is retired: it owned no mechanics of its own. Say so once, then + # treat REST as the plan path so the slug is not derived from the word. + echo "note: 'ship' was retired; /spec <plan-path> runs plan then execute" + # continue at step 3 with REST + ;; "") - echo "usage: /spec [ship] <plan-path|topic> | plan <topic> | execute <slug> | retro <slug>" + echo "usage: /spec <plan-path> | plan <topic> | execute <slug> | retro <slug>" ;; *) - # DEFAULT: not a node name -> ship, keeping the full argument string - # read references/ship.md and execute it with "$ARGUMENTS" as its $ARGUMENTS + # DEFAULT: an approved plan path -> plan, then execute ;; esac ``` @@ -104,39 +185,56 @@ esac ## Shared rules (apply to every subcommand) - **This skill owns the workflow** — keep the operative path, human selection, - plan-approval gate, evidence gate, and human merge boundary in this skill and its - four direct references. Do not duplicate the workflow in root instructions. -- **The `.oh/tasks/<slug>/` folder is the universal interface** — `plan` produces it; - `execute` and `retro` are each pointed at it. The `<slug>` is the - universal key (task directory, branch second segment). It is never a terminal - identifier — not a multiplexer session, not a Herdr tab or pane, not any runtime handle. + plan-approval gate, evidence gate, and human merge boundary in this skill and + its three direct references. Do not duplicate the workflow in root + instructions. +- **The `.oh/tasks/<slug>/` folder is the universal interface** — `plan` produces + it; `execute` and `retro` are each pointed at it. The `<slug>` is the universal + key (task directory, branch second segment, status file). It is never a terminal + identifier: no session, tab, or pane name is derived from it, and none is read + back to decide task state. +- **One owner builds it — the agent that is running `execute`** — implementation + and every post-build gate belong to that agent. + **`/spec` never launches another coding-agent process to do that work**: + no multiplexer session, no Herdr + workspace/tab/pane, no background shell, no runner selection. `/delegate` is + available only for bounded, disjoint worker tasks; a worker never becomes a + second supervisor, executor, or PR owner, and the owner reconciles and validates + every result. - **Compose, don't fork** — each node reuses existing skills rather than - re-implementing them: `ship` composes `plan` then `execute` and owns no build - mechanics of its own; `plan` composes `/prd` + `/ralph`; `execute` is owned by the - agent running it, uses `/delegate` only for bounded fan-out, - and composes `/audit implementation` + `/eval` + `/audit pr`; `retro` composes - `/retro`. The build **literals** — the `gh` invocations, the branch and PR shapes, - and the handoff-free implementation rules — live in - `references/execute.md`, which is the single source for them and is a protected path. + re-implementing them: `plan` composes `/wiki query` + `/prd` + `/ralph`; + `execute` composes `/audit implementation` + `/eval` + `knowledge-impact.sh` + + `/wiki compile` + `/benchmark` + `/audit pr`; `retro` composes `/retro`. The + build **literals** — the `gh` invocations, the branch and PR shapes, the + implementation step, and the handoff-free implementation rules — live in + `references/execute.md`, which is the single source for them and is a + protected path. +- **Dependency-aware invalidation lives in the knowledge primitive** — + `.oh/skills/wiki/scripts/knowledge-impact.sh`. `/spec` calls it; it does not + carry a second copy of the logic. - **One adversarial loop** — `implementation ⇄ audit` inside `execute` vets the implementation (`AUDIT-FAIL` routes back to the same owner). The plan - itself is vetted by the operator who approves it: **approving `prd.md` is the - commitment gate**, and nothing GitHub-side exists until `execute` starts. -- **Honest terminal reports** — each subcommand reports what it actually produced: `plan` - the folder path and story count; `execute` `READY` or `DRAFT-BLOCKED (<gate>)` with the PR - URL; `ship` the same terminal report as whichever node it stopped at; `retro` the - promotion counts. There are no `STATUS: SPEC-*` tokens — all four had - **zero executable consumers repo-wide**, so printing them was ceremony. The rule they - encoded still holds and is what matters: never infer success from silence. A missing - artifact, a crashed build, or an undecided gate is reported as blocked, never as done. + itself is vetted by the operator who approves it, and re-approved if grounding + materially changes it. +- **Distil before you compress** — high-resolution execution evidence is turned + into `evidence.md`, a retro, and durable patterns *before* any context + compression. Compaction is a runtime optimization, non-gating, and never a + semantic stage of the build. +- **Honest terminal reports** — each subcommand reports what it actually + produced: `plan` the folder path and story count; `execute` `RUNNING` when it + returns with the task still building, and `READY` or `DRAFT-BLOCKED(<gate>)` + with the PR URL when the build reaches a terminal state; `retro` the promotion counts. Never + infer success from silence. A missing artifact, a crashed build, or an + undecided gate is reported as blocked, never as done. ## When NOT to use - **selection** — choosing which issue to build is the human's job; `/spec` - builds the one plan or folder it is handed. `ship` automates the hop from plan to - execute, never the choice of what to work on. + builds the one plan or folder it is handed. ## See Also -- `references/ship.md`, `references/plan.md`, `references/execute.md`, and - `references/retro.md` — the authoritative per-subcommand procedures. +- `references/plan.md`, `references/execute.md`, and `references/retro.md` — the + authoritative per-subcommand procedures. +- `.oh/skills/wiki/references/schema.md` — the knowledge schema `plan` recalls + from and `execute` invalidates against. diff --git a/.oh/skills/spec/references/execute.md b/.oh/skills/spec/references/execute.md index 36a0bf32..1e707d73 100644 --- a/.oh/skills/spec/references/execute.md +++ b/.oh/skills/spec/references/execute.md @@ -1,4 +1,4 @@ -# `/spec execute` — implementation ⇄ audit → evidence → spec-retro → improve +# `/spec execute` — re-ground → implementation ⇄ audit → knowledge → evidence → improve > Detail doc for the **`execute`** subcommand of the `/spec` skill > (`.oh/skills/spec/SKILL.md`). Argument form: @@ -6,16 +6,16 @@ > The dispatcher passes the argument string after `execute` to this procedure as > `$ARGUMENTS`. Authority: `.oh/skills/spec/SKILL.md`. -The **execute** node of the `/spec` workflow is pointed at a -planned `.oh/tasks/<slug>/` folder whose `prd.md` the operator has approved, it drives the -implementation to a ready-for-review PR and stops at the human merge gate. It contains the -workflow's one adversarial loop — `implementation ⇄ audit`. +The **execute** node is pointed at a planned `.oh/tasks/<slug>/` folder whose +`prd.md` the operator has approved. It re-grounds the plan against current +repository state, drives the implementation to a ready-for-review PR, feeds what +the run learned back into durable knowledge, and stops at the human merge gate. +It contains the workflow's one adversarial loop — `implementation ⇄ audit`. -**This file is the whole workflow.** Every mechanic it needs — the issue, the branch, the draft -PR, the implementation, the `/eval` gate, the wiki gate, the promotable classification, and the -undraft — is written out below, in order, with no deferral to another skill. Reading it top to -bottom tells you what `/spec execute` does; you never have to open a second skill to learn the -next step. +**This file is the whole workflow.** Every mechanic it needs — the issue, the +branch, the draft PR, the implementation step, the `/eval` gate, the knowledge-impact +gate, the evidence gate, the promotable classification, and the undraft — is +written out below, in order, with no deferral to another skill. --- @@ -23,7 +23,7 @@ next step. | Arg | Meaning | |-----|---------| -| `<slug>` | The task slug — reads the four-file contract in `.oh/tasks/<slug>/` and `prd.json`'s `branchName`. Required. | +| `<slug>` | The task slug — reads the three-file contract in `.oh/tasks/<slug>/` and `prd.json`'s `branchName`. Required. | | `--pr <N>` | Resume against an existing PR rather than creating one. | | `--repo <owner/name>` | GitHub repo (default `mifunedev/openharness`; read from the folder if `/spec plan` recorded it). | | `--remote <name>` | Git remote (resolved from `--repo` if absent). | @@ -50,48 +50,123 @@ SPEC_REMOTE="${SPEC_REMOTE:-$(resolve_spec_remote)}" echo "spec execute target: repo=$SPEC_REPO remote=$SPEC_REMOTE base=$SPEC_BASE" ``` -Do not let implicit `gh` repo resolution or a bare `git push origin` send this build's -issue or PR to a fork. The remote is resolved from the repo URL, and the run fails closed -when no local remote matches. +Do not let implicit `gh` repo resolution or a bare `git push origin` send this +build's issue or PR to a fork. The remote is resolved from the repo URL, and the +run fails closed when no local remote matches. -Precondition: `.oh/tasks/<slug>/` carries the four-file contract (`prd.md`, `prd.json`, -`prompt.md`, `progress.txt`) produced by `/spec plan`, and its `prd.md` has been approved. -**Approving the plan is the commitment gate** — there is no separate critique or approve -node (`.oh/skills/spec/SKILL.md`). If the folder is incomplete, refuse and route back to +Precondition: `.oh/tasks/<slug>/` carries the three-file contract (`prd.md`, +`prd.json`, `progress.txt`) produced by `/spec plan`, its `prd.md` has been +approved, and its `## Plan Reconciliation` says `Intent preserved: YES`. +**Approving the plan is the commitment gate** (`.oh/skills/spec/SKILL.md`). If +the folder is incomplete, or reconciliation says `NO`, refuse and route back to `/spec plan`. -**There is no executor argument, no handoff, and no separate implementation process.** -`/spec execute` has exactly one implementation owner: **the agent that is running it**. That -agent implements the approved task graph, validates each story, records progress, runs the -audit/eval/wiki/evidence gates, and finalizes the PR. Ownership is a **role**, not a terminal -topology — it is not a tmux session, a Herdr tab or pane, a background shell, or any other -runtime identifier. `/delegate` is available only for bounded, disjoint worker tasks whose -results this owner reconciles; it never becomes a second supervisor, a second PR owner, or a -replacement workflow. `STATUS: COMPLETE` is a durable task record, not a handoff signal. +`/spec execute` has exactly one implementation owner: +**the agent that is running it**. +That agent implements the approved task graph, validates each story, records +progress, runs the audit / eval / knowledge / evidence gates, and finalizes the +PR. Ownership is a **role**, not a terminal topology — it is not conferred by a +session, a tab, or a pane, and this node creates none of those. `/delegate` is +available only for bounded, disjoint worker tasks; a delegated worker never +becomes a second supervisor, never owns the whole task, and never finalizes the +PR, and the owner reconciles and validates every result itself. -`/spec` defines and verifies the execution contract. **It does not create the agent that -executes it.** The operator starts the coding-agent session — in Herdr, a terminal, an IDE, or -anywhere else — and pastes the task into it; from that point the receiving agent owns the task -through every gate below. +--- + +## Lifecycle and the status file + +This node can return while the task is still building — a cron run, a resume, or +an interrupted build all reach a terminal line with stories still open — so it +reports the state it actually reached and never promises a ready PR it has not +seen: + +```text +PLANNED ──▶ RUNNING ──▶ READY + └──▶ DRAFT-BLOCKED(<gate>) +``` + +| State | Meaning | Written when | +|---|---|---| +| `PLANNED` | The folder exists and is approved; nothing GitHub-side yet | before step 1 | +| `RUNNING <phase>` | The task is being built: its stories are not all `passes: true` | at every phase change, from step 3 onward | +| `READY <pr-url>` | `gh pr ready` succeeded after a promotable classification | step 10 | +| `DRAFT-BLOCKED(<gate>) <pr-url>` | A named gate held the undraft; the PR stays draft | step 10, or any halt | + +**`RUNNING` describes the task, never a process.** It is an approved folder whose +stories are not all `passes: true` — +never the existence of a named process, session, tab, or pane. +The authority is `prd.json`: + +```bash +jq -e 'all(.userStories[]; .passes == true)' ".oh/tasks/<slug>/prd.json" # 0 => not RUNNING +``` + +The owner mirrors the phase into one line at `/tmp/spec-<slug>.state` so a +resume, the stale-draft watchdog, and an operator asking "what is it doing" have +something to read without inspecting the task graph: + +```bash +STATE_FILE="/tmp/spec-<slug>.state" +printf 'RUNNING %s\n' "<phase>" > "$STATE_FILE" +``` + +The file is a convenience mirror, not the source of truth: if it disagrees with +`prd.json`, the task graph wins. Update it at every phase change. Both terminal +states are also reported on the PR itself, which remains the authority a later +reader sees. --- ## The pipeline +### 0. Re-ground the plan against current state + +Planning and execution can happen against different repository states — minutes +apart, or weeks. Before implementing anything: + +1. Read `prd.md`'s `## Knowledge Context`: the planning `Base commit`, the slugs + under `Knowledge used`, and the paths under `Grounded against`. +2. Determine what moved between the planning base and the execution base: + + ```bash + PLAN_BASE=$(sed -n 's/^- \*\*Base commit\*\*: `\(.*\)`.*/\1/p' ".oh/tasks/<slug>/prd.md" | head -1) + git fetch "$SPEC_REMOTE" "$SPEC_BASE" + git diff --name-only "$PLAN_BASE" "$SPEC_REMOTE/$SPEC_BASE" > /tmp/spec-<slug>-drift.txt + comm -12 \ + <(sort -u /tmp/spec-<slug>-drift.txt) \ + <(printf '%s\n' <paths from Grounded against> | sort -u) + ``` + +3. **Re-read every authoritative source in that intersection.** Wiki and + knowledge text is orientation, never authority: code and tests are + implementation truth, canonical docs/RFCs/ADRs are intended-design truth. +4. If an assumption the plan rests on materially changed, reconcile before + continuing — update `prd.md`'s `## Plan Reconciliation` with the new + constraint. If the change is material to the operator's *intent* rather than + to the mechanism, stop, set the status file to `DRAFT-BLOCKED(reconciliation)`, + and ask for re-approval. Do not implement a plan whose premise moved. + +The owner consumes the approved PRD's Knowledge Context and re-reads +the sources it names. It does **not** load the pattern set — patterns are the +planner/proposer's input (`.oh/skills/wiki/references/schema.md` § 3). If the +task turns into replanning, that is a `/spec plan` re-run, not a reason to widen +what the executor reads. + ### 1. Locate (or open) the issue The approved plan is the commitment, so GitHub-side state may now be created. -`prd.json`'s `branchName` already embeds `<N>`. In the canonical flow that is the issue -the human selected and `/spec plan` consumed — **locate** it, do not open a second one: +`prd.json`'s `branchName` already embeds `<N>`. In the canonical flow that is the +issue the human selected and `/spec plan` consumed — **locate** it, do not open a +second one: ```bash gh issue view <N> --repo "$SPEC_REPO" ``` -Open an issue only in a standalone run that has none yet. Compose the body from `prd.md`'s -introduction and goals; the title format is `<prefix>: <slug-as-prose>` per -`.claude/skills/git/SKILL.md`: +Open an issue only in a standalone run that has none yet. Compose the body from +`prd.md`'s introduction and goals; the title format is +`<prefix>: <slug-as-prose>` per `.oh/skills/git/SKILL.md`: ```bash gh issue create \ @@ -108,17 +183,17 @@ gh issue create \ "## PRD" \ "- .oh/tasks/<slug>/prd.md (this branch)" \ "" \ - "## Wiki Alignment" \ + "## Expected Knowledge Impact" \ "- Impact: <REQUIRED | NOT-APPLICABLE from prd.md>" \ "" \ "## Tracking" \ "Planned by /spec plan; the operator approved prd.md, which is the commitment gate. Draft PR to follow.") ``` -Capture the issue number `<N>`. If `gh label create <prefix> --repo "$SPEC_REPO"` is needed -(the label does not exist), create it first with a sensible color. Heredoc bodies are safe — -the `deny-env-dump.sh` hook strips heredoc bodies before pattern-scanning, so -`--body "$(cat <<'EOF' ... EOF)"` is fine. +Capture the issue number `<N>`. If `gh label create <prefix> --repo "$SPEC_REPO"` +is needed (the label does not exist), create it first with a sensible color. +Heredoc bodies are safe — the `deny-env-dump.sh` hook strips heredoc bodies +before pattern-scanning. ### 2. Branch + scaffold commit + push @@ -132,11 +207,11 @@ git add -f ".oh/tasks/<slug>/" git commit -m "$(cat <<'EOF' <prefix>: scaffold <slug> task -Four-file contract: -- prd.md: <N> user stories +Three-file contract: +- prd.md: <N> user stories, with Knowledge Context, Expected Knowledge Impact, + and Plan Reconciliation - prd.json: schemaVersion 1, branchName <prefix>/<N>-<slug> -- prompt.md: the rendered single-owner task prompt -- progress.txt: empty header +- progress.txt: header plus the plan-phase entry Tracks #<N>. PRD generated by /prd; converted by /ralph. @@ -148,12 +223,14 @@ git push -u "$SPEC_REMOTE" "<prefix>/<N>-<slug>" ``` `.oh/tasks/` is gitignored, so the `-f` on that `git add` is load-bearing: a bare -`git add .oh/tasks/<slug>/` stages nothing and the scaffold commit silently omits the -contract. +`git add .oh/tasks/<slug>/` stages nothing and the scaffold commit silently omits +the contract. (`.oh/knowledge/` is **not** gitignored — knowledge pages take a +plain `git add`.) -`Submitted-by:` is mandatory and must name the model/agent that actually submits the commit -(for example `Submitted-by: Claude`, `Submitted-by: Codex`, or `Submitted-by: Pi`). Do not -hard-code Claude when the active submitter is a fallback harness. +`Submitted-by:` is mandatory and must name the model/agent that actually submits +the commit (for example `Submitted-by: Claude`, `Submitted-by: Codex`, or +`Submitted-by: Pi`). Do not hard-code Claude when the active submitter is a +fallback harness. Pre-commit hook runs lint + tests; do not bypass. @@ -169,7 +246,7 @@ gh pr create \ --body "$(cat <<'EOF' Closes #<N>. -**Status: DRAFT — implementation, /eval, and /audit pr promotable gates are still pending.** +**Status: DRAFT — implementation, /eval, knowledge impact, evidence, and the promotable gate are still pending.** ## Summary <from prd.md introduction, 2-3 lines> @@ -177,32 +254,40 @@ Closes #<N>. ## Stories <numbered list from prd.json — title only> -## Next steps -1. The agent running `/spec execute` is this task's single implementation owner; it works in the isolated worktree `.worktrees/<prefix>/<N>-<slug>`. -2. It implements the stories directly, using `/delegate` only for bounded disjoint work; it validates the stories, runs `/audit implementation`, and revises required wiki entries. -3. It runs a fresh `/audit pr` immediately before any undraft; this PR is marked ready (`gh pr ready`) only when that audit classifies it promotable (CI green + mergeable + clean). Heartbeat stale-draft watchdog output — including draft-age and draft-cap/backlog warnings — is only a resume/investigation hint, never an undraft signal. +## Next steps (automated) +1. The agent running `/spec execute` implements the task itself, in an isolated worktree; task state moves to RUNNING. +2. It implements the stories directly, using `/delegate` only for bounded disjoint work; it validates the stories, runs the implementation-side audit loop, and updates every knowledge page the actual diff invalidates. +3. It runs a fresh PR audit immediately before any undraft; this PR is marked ready (`gh pr ready`) only when that audit classifies it promotable (CI green + mergeable + clean). Heartbeat stale-draft watchdog output is only a resume/investigation hint, never an undraft signal. 🤖 Generated with [Claude Code](https://claude.com/claude-code) via /spec execute EOF )" ``` -Capture the PR URL and PR number `<PR>`. This is an observability checkpoint, not the -terminal state. +Capture the PR URL and PR number `<PR>`, then move the lifecycle forward: -### 4. Implement — the running agent is the owner +```bash +printf 'RUNNING implementation\n' > "/tmp/spec-<slug>.state" +``` + +This is an observability checkpoint, not the terminal state. -**This node launches nothing.** No Advisor session, no detached multiplexer session and no -piped pane log, no Herdr workspace, tab, or pane created on the operator's behalf, no -background-shell launch, and no runner selection. There is **no fallback runner because there -is no handoff step**. The agent that reached this line implements the task itself and carries -it through every gate below. +### 4. Implement — the running agent is the owner -**Build worktree — reuse vs. create.** Isolation stays. When this run is ALREADY inside an -isolated worktree that step 2 put on the feature branch, **reuse it** — do NOT create a second -worktree (a second `git worktree add` for the same branch would nest under the current worktree -via the relative path, or fail with `branch already checked out`). Otherwise create -`.worktrees/<prefix>/<N>-<slug>` via `/worktrees` and work there: +**This node launches nothing.** No detached multiplexer session and no piped pane +log, no Herdr workspace, tab, or pane created on the operator's behalf, no +background-shell launch, and no runner selection. +There is **no fallback runner because there is no handoff step**. +`/spec` defines and verifies the execution contract; +**it does not create the agent that** executes it. The agent that reached this +line implements the task itself and carries it through every gate below. + +**Build worktree — reuse vs. create.** Isolation stays. When this run is ALREADY +inside an isolated worktree that step 2 put on the feature branch, **reuse it** — +do NOT create a second worktree (a second `git worktree add` for the same branch +would nest under the current worktree via the relative path, or fail with `branch +already checked out`). Otherwise create `.worktrees/<prefix>/<N>-<slug>` via +`/worktrees` and work there: ```bash if [ "$(git rev-parse --abbrev-ref HEAD)" != "<prefix>/<N>-<slug>" ]; then @@ -213,23 +298,34 @@ fi Then implement, in this same session: -1. Read `.oh/tasks/<slug>/prompt.md` — the rendered task prompt — and `prd.json`'s story graph. -2. Implement the dependency-ready stories directly. Use `/delegate` **only** for bounded, - disjoint worker tasks, and reconcile every worker result yourself. A delegated worker never - becomes a second supervisor, never owns the whole task, and never finalizes the PR. -3. Validate every acceptance criterion against the repository, flip each story's `passes` to - `true` only after that validation, and append a dated `progress.txt` entry naming the files, - the commit, the result, and the learnings. -4. Append `STATUS: COMPLETE` to `progress.txt` only after every story in `prd.json` passes. - That marker is a durable task record for resume and cleanup; it hands control to no one. - -Commit story changes on `<prefix>/<N>-<slug>` with a `Submitted-by:` trailer and keep worktree -isolation intact. `RUNNING` describes the **task's** state — an approved folder whose stories -are not all `passes: true` — never the existence of a named process, session, tab, or pane. If -implementation is incomplete, leave the PR draft and resume `/spec execute` against the same -task folder; do not create a second implementation owner. - -Then continue, in this same session, with steps 5 through 9 below. +1. Read the rendered task prompt, `prd.md` — including the `## Knowledge Context` + sources step 0 re-read — and `prd.json`'s story graph. +2. Implement the dependency-ready stories directly. Use `/delegate` **only** for + bounded, disjoint worker tasks, and reconcile every worker result yourself. A + delegated worker **never becomes a second supervisor**, never owns the whole + task, and never finalizes the PR. +3. Validate every acceptance criterion against the repository, flip each story's + `passes` to `true` only after that validation, and append a dated + `progress.txt` entry naming the files, the commit, the result, and the + learnings. +4. Do not load the pattern set. Patterns are the planner's input + (`.oh/skills/wiki/references/schema.md` § 3); the owner re-reads the + authoritative sources the approved Knowledge Context names. If the task turns + into replanning, that is a `/spec plan` re-run, not a reason to widen what the + executor reads. + +Commit story changes on `<prefix>/<N>-<slug>` with a `Submitted-by:` trailer and +keep worktree isolation intact. Completion is read from `prd.json`, never from +prose: the task is done when +`jq -e 'all(.userStories[]; .passes == true)' .oh/tasks/<slug>/prd.json` exits 0. +`RUNNING` describes the **task's** state — an approved folder whose stories are +not all `passes: true` — +never the existence of a named process, session, tab, or pane. +If implementation is incomplete, leave the PR draft and resume `/spec execute` +against the same task folder; +**do not create a second implementation owner**. + +Then continue, in this same session, with steps 5 through 10 below. ### 5. `implementation ⇄ audit` — the adversarial loop @@ -239,19 +335,20 @@ When implementation is complete, run the per-unit verdict gate: /audit implementation <slug> --pr <N> --repo <owner/name> --base <base> --branch <prefix>/<N>-<slug> ``` -`/audit implementation` composes `prd.json` task-graph conformance + the `/eval` regression -floor + `/audit pr` promotable classification (+ `/agent-browser` for UI stories, + the -gate-5 slop check) into one verdict: +`/audit implementation` composes `prd.json` task-graph conformance + the `/eval` +regression floor + the PR promotable classification (+ `/agent-browser` for UI +stories, + the gate-5 slop check) into one verdict: - `AUDIT-FAIL` → loop back to implementation in this same session to finish the - unmet stories, then re-audit. This is the implementation-side adversary — keep looping - until the owner satisfies the task graph. + unmet stories, then re-audit. This is the implementation-side adversary — keep + looping until the owner satisfies the task graph. - `AUDIT-PASS` → implementation is promotable; continue to the tail. -**The simplify sub-loop — drive `netAdded` down.** Gate 5 asks whether the diff can be -smaller and still satisfy every acceptance criterion. On an `AUDIT-FAIL (gate 5)` the -owner removes the code the finding names — it does not argue with it — and re-audits. -The owner writes the round record; the read-only audit route only reads it: +**The simplify sub-loop — drive `netAdded` down.** Gate 5 asks whether the diff +can be smaller and still satisfy every acceptance criterion. On an +`AUDIT-FAIL (gate 5)` the owner removes the code the finding names — it does not +argue with it — and re-audits. The owner keeps the round record; the read-only +audit route only reads it: ```bash COUNTER=".oh/tasks/<slug>/simplify-rounds.json" @@ -264,26 +361,24 @@ JSON git add -f "$COUNTER" ``` -Two things end this loop, and neither of them is agreement: the **cap** of 3 rounds, and a -**non-reducing round** — one whose `netAdded` did not strictly fall below the previous round's. Either way the -audit stops blocking and passes with `SIMPLICITY-RESIDUAL`, and those residual findings go -into `evidence.md` under *What remains unverified* for the operator to judge. A simplify -loop that cannot make the diff smaller has finished its work; one that keeps looping on -taste has stopped doing work. - -Two further gates run inside this loop and must both clear before the audit can PASS. - -**The `/eval` gate — run ONCE per cycle.** Run `/eval` while still on the work branch. If it -updates `.oh/evals/RESULTS.md`, commit the benchmark refresh on the branch. Treat only a NEW -green→red probe regression or a non-zero eval runner exit as blocking; a pre-existing red with -an unchanged delta is non-gating but should be disclosed in the PR. Key on the **delta and the -runner's exit code**, never on the bare presence of a `REGRESSION` row — a probe that was -already red on the base is pre-existing and this PR did not cause it. - -This is the **only** suite run in the cycle. `/audit implementation` Gate 2 and `/benchmark` -Signal 1 read this result instead of re-running; three runs of the same 106 probes against the -same commit cost 318 probe executions and told us the same thing once. Publish the result -where they can find it, keyed to the commit it actually ran against: +Two things end this loop, and neither of them is agreement: the **cap** of 3 +rounds, and a **non-reducing round** — one whose `netAdded` did not strictly fall +below the previous round's. Either way the audit stops blocking and passes with +`SIMPLICITY-RESIDUAL`, and those residual findings go into `evidence.md` under +*What remains unverified* for the operator to judge. + +**The `/eval` gate — run ONCE per cycle.** Run `/eval` while still on the work +branch. If it updates `.oh/evals/RESULTS.md`, commit the benchmark refresh on the +branch. Treat only a NEW green→red probe regression or a non-zero eval runner +exit as blocking; a pre-existing red with an unchanged delta is non-gating but +must be disclosed in the PR body and in `evidence.md`. Key on the **delta and the +runner's exit code**, never on the bare presence of a `REGRESSION` row. + +This is the **only** suite run in the cycle. `/audit implementation` Gate 2 and +`/benchmark` Signal 1 read this result instead of re-running; three runs of the +same probe set against the same commit cost three times the probe executions and +told us the same thing once. Publish the result where they can find it, keyed to +the commit it actually ran against: ```bash bash .oh/skills/eval/run.sh ; rc=$? @@ -299,109 +394,138 @@ EOF git add -f ".oh/tasks/<slug>/eval-result.json" ``` -**`commit` is the freshness key, and it is what keeps the reuse honest.** A downstream reader -reuses this record only while `commit` equals the current `git rev-parse HEAD`. The moment the -branch moves, the record describes code that is no longer under test, and the reader must run -the suite itself rather than inherit a stale green. Reuse without that check is how a pipeline -starts reporting a floor it never measured. +**`commit` is the freshness key, and it is what keeps the reuse honest.** A +downstream reader reuses this record only while `commit` equals the current +`git rev-parse HEAD`. The moment the branch moves, the record describes code that +is no longer under test, and the reader must run the suite itself rather than +inherit a stale green. -**The wiki-revision gate.** If `.oh/tasks/<slug>/prd.md` has `## Wiki Alignment` with -`Impact: REQUIRED`, revise the named `.oh/skills/wiki/corpus/*.md` entries after -implementation and before the promotable gate. The revision must align with: +### 6. Actual Knowledge Impact — the diff decides -- the PRD's goals, non-goals, acceptance criteria, and completed behavior; -- `.oh/skills/wiki/references/schema.md`'s source-backed architecture standard: relevant - source files, line-cited claims, system relationships for pipeline/runtime/architecture - topics, and `## See Also` navigation. +`prd.md`'s `## Expected Knowledge Impact` was the planner's prediction. The +implementation now exists, so the prediction stops being authoritative. Derive +the real answer: -There is **no DeepWiki comparison** in this gate. The public DeepWiki for this repo -regenerates on no schedule the gate can depend on, so requiring the comparison made the gate -answerable only by an unreliable third party — a disagreement with it measures upstream lag, -never the build. Wiki alignment is judged against the repo's own sources and the local -corpus. - -Refresh `.oh/skills/wiki/corpus/README.md` via `/wiki lint` or the atomic fallback in -`/wiki ingest`, then run: +```text +Expected Knowledge Impact + actual changed paths + page dependency metadata + ↓ + Actual Knowledge Impact +``` ```bash -bash .oh/evals/probes/wiki-readme-index.sh +git diff --name-only "$SPEC_REMOTE/$SPEC_BASE"...HEAD > /tmp/spec-<slug>-changed.txt +bash .oh/skills/wiki/scripts/knowledge-impact.sh \ + --changed $(tr '\n' ' ' < /tmp/spec-<slug>-changed.txt) ``` -Commit wiki changes with the implementation branch. If the wiki impact was `REQUIRED` and the -named entries were not updated or the index probe fails, leave the PR draft and comment the -missing wiki gate. +`knowledge-impact.sh` is the one implementation of dependency-aware invalidation +(`/wiki lint` calls the same script with `--verified`). `/spec` does not carry a +second copy of the logic. -**`/compact` after implementation.** Implementation and `/eval` have spent significant context -in this session; run `/compact` so the promotable gate and the undraft decision start -clean. Preserve the finalize keys: +Take the union of the pages the script reports `NEEDS-REVIEW` and the pages +`Expected Knowledge Impact` named. **Every page in that union ends in exactly one +explicit state**, recorded in `evidence.md`: -```text -Preserve /spec execute finalize context: slug <slug>, branch <prefix>/<N>-<slug>, issue #<N>, PR #<PR>, implementation complete (STATUS: COMPLETE), /eval result, wiki alignment gate result (REQUIRED updated or NOT-APPLICABLE), undraft gate (/audit pr promotable → gh pr ready, else comment + stay draft), no auto-merge. +| State | Means | Required action | +|---|---|---| +| `UPDATED` | The change made the page wrong; the page was rewritten to match | Edit the page, advance `updated:`, and for `kind: repo` advance `verified_at:` to the current commit | +| `REVERIFIED` | A declared source moved but the page's claims still hold | Re-read the page against those sources, then advance `verified_at:` only | +| `NOT-AFFECTED (<reason>)` | The page does not describe what changed | Record the reason; touch nothing | + +`NOT-AFFECTED` with no reason is not a state, it is a skipped page. Advancing +`verified_at:` without re-reading launders staleness into freshness and is the +one failure this gate cannot detect afterwards. + +New pages the run should create — an external source the work depended on — go +through `/wiki ingest` here. Pattern pages are **not** written here; they are +step 9's job, because they need the retro's verdicts. + +Then regenerate the index and verify it: + +```bash +bash .oh/evals/probes/wiki-readme-index.sh ``` -Non-blocking — if `/compact` is unavailable or errors, log a warning and continue. - -### 6. Write `evidence.md` — the answer back to the plan - -**This is a gate condition, not a formality.** Step 9 refuses to undraft without it — this -artifact carries the implementation's answer to the reviewer. - -The operator's understanding of this work stops at the plan they approved. The same owner -implements the stories and records the result. `evidence.md` answers back to the plan -with the observed behavior, deviations, and remaining gaps. - -Write `.oh/tasks/<slug>/evidence.md` and **commit it on the branch**, so it travels in the PR -diff. The full contract — path, linkage, observed-output rule, correlation to one audit run, -honesty about gaps — is `.oh/skills/audit/references/reviewer-evidence-doc.md`. Follow it, and -make sure the doc answers these five questions in this order: - -0. **Why this is better than not doing it** — the before and after in the operator's terms, - with a number wherever one exists, and the cost paid to get it. This question comes first - because it is the only one the reviewer cannot answer from the diff, the gates, or the - plan. **A doc that proves every gate green and never says what improved has failed.** A - benefit with no measurement behind it is written *claimed, unmeasured* rather than - asserted — and "the gates are green" is not an answer to this question. -1. **What the plan asked for** — the approved `prd.md`'s goals in the operator's terms, not a - restatement of the story titles. -2. **What was built** — the observable behavior that now holds, with the commands and real - output that show it. -3. **Where they diverged, and why** — every place the implementation differs from the approved plan: - a criterion satisfied differently, a deviation taken deliberately, a scope call made - mid-implementation. **A run with no divergence says "none" explicitly**; silence here reads as - "nothing diverged" and is the most expensive thing this document can get wrong. -4. **What remains unverified** — gates that were skipped, criteria that were argued rather - than observed, pre-existing reds carried forward, and anything a reviewer would have to - check by hand. - -`/audit implementation` and `/audit pr` are read-only and do not write this file; this node -writes it from what those routes observed. - -### 7. `spec-retro` — capture the lessons - -On `AUDIT-PASS`, run `/spec retro <slug>` (the execution-side retro). It turns the run's -signals into falsifiable, evidence-tested lessons and promotes the supported ones behind a -propose-then-confirm gate. It writes no file; step 8's `compound` is where its supported -lessons become durable. - -### 8. `improve` — compound · compress · benchmark - -The self-improvement tail (`.oh/skills/spec/SKILL.md`): - -- **compound** — promote durable knowledge so it is reused, not re-derived. Run - `/wiki compile` on step 7's retro report to turn its supported lessons into - `kind: pattern` entries the next `/builder` proposal will read; run `/wiki ingest` - for any external source the run depended on; mint a probe from any guardrail lesson. -- **compress** — keep the always-loaded context lean and clear (`/audit context`). -- **benchmark** — confirm the change earned its complexity (`/benchmark`): the `/eval` - regression floor stays green AND the capability-benchmark ceiling held or moved. It reads - step 5's `eval-result.json` for the floor; it does not re-run the suite. - -The **groom triad** (`/audit skills` · `/wiki lint` · `/audit drift`) is deliberately NOT here. -`/audit drift` already runs hourly from the heartbeat cron, and the other two are report-only -health checks that never blocked a merge — running them per cycle spent the cycle's budget on -advisory output nobody gated on. Run them on their own cadence, or on demand. - -### 9. Promotable gate → undraft → human merge gate +Commit knowledge changes with the implementation branch (a plain `git add` — +`.oh/knowledge/` is tracked). If `Expected Knowledge Impact` was `REQUIRED` and +the union is unresolved, or the index probe fails, leave the PR draft, set +`DRAFT-BLOCKED(knowledge)`, and comment the missing gate. + +### 7. Write `evidence.md` — the answer back to the plan + +**This is a gate condition, not a formality.** Step 10 refuses to undraft without +it — this artifact carries the implementation's answer to the reviewer. + +The operator's understanding of this work stops at the plan they approved. The +same session implements the stories and records the result. +`evidence.md` answers back to the plan with the observed behavior, deviations, +and remaining gaps. + +Write `.oh/tasks/<slug>/evidence.md` and **commit it on the branch**, so it +travels in the PR diff. The full contract — path, linkage, observed-output rule, +correlation to one audit run, honesty about gaps — is +`.oh/skills/audit/references/reviewer-evidence-doc.md`. Follow it, and make sure +the doc answers these five questions in this order: + +0. **Why this is better than not doing it** — the before and after in the + operator's terms, with a number wherever one exists, and the cost paid to get + it. This question comes first because it is the only one the reviewer cannot + answer from the diff, the gates, or the plan. **A doc that proves every gate + green and never says what improved has failed.** A benefit with no measurement + behind it is written *claimed, unmeasured* rather than asserted — and "the + gates are green" is not an answer to this question. +1. **What the plan asked for** — the approved `prd.md`'s goals in the operator's + terms, not a restatement of the story titles. +2. **What was built** — the observable behavior that now holds, with the commands + and real output that show it. +3. **Where they diverged, and why** — every place the implementation differs from + the approved plan: a criterion satisfied differently, a deviation taken + deliberately, a scope call made mid-implementation. **A run with no divergence + says "none" explicitly**; silence here reads as "nothing diverged" and is the + most expensive thing this document can get wrong. +4. **What remains unverified** — gates that were skipped, criteria that were + argued rather than observed, pre-existing reds carried forward, and anything a + reviewer would have to check by hand. + +Step 6's Actual Knowledge Impact table belongs in this document, under *What was +built*. The read-only audit routes do not write this file; this node writes it +from what those routes observed. + +### 8. `spec-retro` — capture the lessons + +On `AUDIT-PASS`, run `/spec retro <slug>`, which is a thin wrapper for +`/retro --task <slug>`. It turns the run's signals into falsifiable, +evidence-tested lessons with verdicts and confidence levels. `/retro` is +report-only by contract and writes no file; step 9 is where its supported lessons +become durable. + +### 9. `improve` — compound, then optionally compress, then benchmark + +The order here is load-bearing. High-resolution execution evidence is worth the +most immediately after the run and decays with every summarization, so it is +**distilled into durable artifacts before anything discards context**: + +1. **compound** — run `/wiki compile` on step 8's retro report to turn its + supported lessons into `kind: pattern` pages under `.oh/knowledge/patterns/`, + which the next `/builder` proposal and the next `/spec plan` will read. Mint a + probe from any guardrail lesson. +2. **compress (optional, non-gating)** — `/compact` and any other runtime + housekeeping happen **here and no earlier**, after evidence, retro, and + pattern compilation have already captured what the context was carrying. + Compaction is a runtime optimization, not a semantic stage of the build: if it + is unavailable or errors, log a warning and continue. Nothing gates on it. +3. **benchmark** — confirm the change earned its complexity (`/benchmark`): the + `/eval` regression floor stays green AND the capability-benchmark ceiling held + or moved. It reads step 5's `eval-result.json` for the floor; it does not + re-run the suite. + +The **groom triad** (`/audit skills` · `/wiki lint` · `/audit drift`) is +deliberately NOT here. `/audit drift` already runs hourly from the heartbeat +cron, and the other two are report-only health checks that never blocked a merge. +The checks worth enforcing per cycle were given deterministic probes instead, so +the suite that already runs catches their findings. + +### 10. Promotable gate → undraft → human merge gate Push the branch so CI runs: @@ -409,42 +533,55 @@ Push the branch so CI runs: git push "$SPEC_REMOTE" HEAD ``` -Run a fresh `/audit pr` focused on PR `#<PR>` in `$SPEC_REPO` immediately before any -undraft attempt. The read-only audit classifies the draft as **promotable** only when CI is -green AND the PR is mergeable AND clean (it reads the `statusCheckRollup`, so it subsumes a -bare `/ci-status` check). Do not infer green from silence — a no-run CI status is not -promotable. Do not treat heartbeat stale-draft watchdog output as promotable evidence; it is -only a signal to investigate or resume the draft. +Run a fresh `/audit pr` focused on PR `#<PR>` in `$SPEC_REPO` immediately before +any undraft attempt. The read-only audit classifies the draft as **promotable** +only when CI is green AND the PR is mergeable AND clean (it reads the +`statusCheckRollup`, so it subsumes a bare `/ci-status` check). Do not infer +green from silence — a no-run CI status is not promotable. Do not treat heartbeat +stale-draft watchdog output as promotable evidence; it is only a signal to +investigate or resume the draft. + +**Classify the head you are about to promote.** A promotable verdict describes +one commit. Confirm the PR's head is the commit you just pushed *before* reading +the classification, or the audit is scoring a different tree than the one going +to review: + +```bash +[ "$(gh pr view <PR> --repo "$SPEC_REPO" --json headRefOid --jq .headRefOid)" = "$(git rev-parse HEAD)" ] \ + || { echo "ERROR: PR head is not local HEAD — push, wait for CI, re-audit"; exit 1; } +``` -**The evidence gate.** Before the undraft, `.oh/tasks/<slug>/evidence.md` must exist, be -committed on the branch, and answer the five questions step 6 names. **Refuse the undraft -without it** — a PR whose reviewer cannot see how the built thing differs from the plan they -approved is not ready for review, whatever CI says: +**The evidence gate.** Before the undraft, `.oh/tasks/<slug>/evidence.md` must +exist, be committed on the branch, and answer the five questions step 7 names. +**Refuse the undraft without it** — a PR whose reviewer cannot see how the built +thing differs from the plan they approved is not ready for review, whatever CI +says: ```bash if [ ! -f ".oh/tasks/<slug>/evidence.md" ]; then gh pr comment <PR> --repo "$SPEC_REPO" --body "spec execute: PR left draft — .oh/tasks/<slug>/evidence.md is missing. The merge gate requires the implementation's answer back to the approved plan (what was asked, what was built, where they diverged, what is unverified). Resume: write it, commit it on the branch, re-run the promotable gate." - # terminal status: DRAFT-BLOCKED (evidence) + printf 'DRAFT-BLOCKED(evidence) %s\n' "<pr-url>" > "/tmp/spec-<slug>.state" exit 0 fi git ls-files --error-unmatch ".oh/tasks/<slug>/evidence.md" >/dev/null 2>&1 \ || { echo "ERROR: evidence.md exists but is untracked — .oh/tasks/ is gitignored; commit it with 'git add -f'"; exit 1; } ``` -The `git ls-files` half is not redundant: `.oh/tasks/` is gitignored, so an `evidence.md` that -was written but added without `-f` is present on disk and **absent from the PR diff** — which -is the same as not having it, from the reviewer's seat. +The `git ls-files` half is not redundant: `.oh/tasks/` is gitignored, so an +`evidence.md` that was written but added without `-f` is present on disk and +**absent from the PR diff** — which is the same as not having it, from the +reviewer's seat. -**Promote the implementation narrative into the PR body.** `progress.txt` holds the per-story -record the owner wrote — what it did, what it learned, and what it deviated on. Update the -PR body from it and from `evidence.md` so the reviewer meets the work in the PR rather than by -opening the task folder: +**Promote the implementation narrative into the PR body.** `progress.txt` holds +the per-story record the owner wrote. Update the PR body from it and from +`evidence.md` so the reviewer meets the work in the PR rather than by opening the +task folder: ```bash gh pr edit <PR> --repo "$SPEC_REPO" --body "$(cat <<'EOF' Closes #<N>. -**Status: READY — /audit implementation PASSED, /eval clean, /audit pr promotable.** +**Status: READY — implementation audit PASSED, /eval clean, knowledge impact resolved, PR audit promotable.** ## What the plan asked for <from the approved prd.md's goals, in the operator's terms — 2-4 lines> @@ -452,6 +589,9 @@ Closes #<N>. ## What was built <the observable behavior that now holds, one line per story, from progress.txt> +## Knowledge impact +<each impacted page and its final state: UPDATED / REVERIFIED / NOT-AFFECTED (reason)> + ## Where it diverged from the plan, and why <every deliberate deviation, differently-satisfied criterion, and mid-build scope call — or the single word "None"> @@ -467,30 +607,58 @@ EOF )" ``` -The **divergence** and **unverified** sections are the two the reviewer cannot reconstruct -from the diff, so neither may be omitted; an empty one is written as `None` / `Nothing` -explicitly. A body that silently drops them reads as "nothing diverged, nothing unchecked", -which is the most expensive claim this pipeline can make by accident. +The **diverged** and **unverified** sections are the two the reviewer cannot +reconstruct from the diff, so neither may be omitted; an empty one is written as +`None` / `Nothing` explicitly. A body that silently drops them reads as "nothing +diverged, nothing unchecked", which is the most expensive claim this pipeline can +make by accident. -Then mark the PR ready — **only** when `/audit implementation` PASSED, `evidence.md` is -present and committed, and that immediately preceding fresh `/audit pr` classified it -promotable: +Then mark the PR ready — **only** when the implementation audit PASSED, +`evidence.md` is present and committed, the knowledge-impact union is resolved, +and that immediately preceding fresh PR audit classified it promotable: ```bash gh pr ready <PR> --repo "$SPEC_REPO" +printf 'READY %s\n' "<pr-url>" > "/tmp/spec-<slug>.state" ``` -Otherwise (not promotable: red/pending CI, conflicts, a new eval regression, or missing -evidence) keep the PR draft and add a comment naming the blocking gate plus resume/fix -instructions: +**The gate re-opens on every push after the undraft.** `READY` is a claim about +the head that was classified, not a property the PR keeps. Any later commit — +including a one-line `progress.txt` or `evidence.md` follow-up — moves the head +past the verdict, and a push whose CI has not finished leaves a ready PR whose +checks are still running. So a push to an already-ready PR re-enters this step: +wait for CI on the new head, re-run `/audit pr` against it, and confirm it is +still promotable. + +```bash +git push "$SPEC_REMOTE" HEAD # to a PR that is already ready-for-review +# → re-enter step 10: wait for CI on the new head, re-audit, re-confirm promotable +``` + +If the new head is **not** promotable, the PR is not ready any more: return it to +draft (`gh pr ready --undo <PR> --repo "$SPEC_REPO"`), comment the blocking gate, +and record `DRAFT-BLOCKED(<gate>)`. Do not leave a ready PR standing on a +classification that no longer describes its head — a reviewer reads *ready* as +"the gates passed on what I am looking at". + +**The cheapest way to honor this is to finish the tail before undrafting.** +Evidence, knowledge impact, retro, compile, and benchmark all write files; run +them, push once, wait for CI, audit, then undraft. Every commit after `gh pr +ready` costs another full CI cycle and another audit. + +Otherwise (not promotable: red/pending CI, conflicts, a new eval regression, an +unresolved knowledge page, or missing evidence) keep the PR draft, name the gate, +and record it: ```bash gh pr comment <PR> --repo "$SPEC_REPO" --body "spec execute: PR left draft — <blocking gate>. Resume: <command>." +printf 'DRAFT-BLOCKED(%s) %s\n' "<gate>" "<pr-url>" > "/tmp/spec-<slug>.state" ``` -Then **stop**. The human owns the merge (`.oh/skills/spec/SKILL.md`: human merge is -the final gate; no auto-merge). Never `gh pr merge`. Print the PR -URL and terminal status (`READY` or `DRAFT-BLOCKED`) as the final pipeline output. +Then **stop**. The human owns the merge (`.oh/skills/spec/SKILL.md`: human merge +is the final gate; no auto-merge). Never `gh pr merge`. Print the PR URL and the +terminal status (`READY` or `DRAFT-BLOCKED(<gate>)`) as the final pipeline +output. --- @@ -498,17 +666,20 @@ URL and terminal status (`READY` or `DRAFT-BLOCKED`) as the final pipeline outpu | Step | Halt trigger | Recovery | |---|---|---| -| pre | Four-file contract incomplete, or `prd.md` not approved | Refuse; route back to `/spec plan` | +| pre | Three-file contract incomplete, or `prd.md` not approved | Refuse; route back to `/spec plan` | +| 0 | A source the plan grounded on changed materially against the operator's intent | `DRAFT-BLOCKED(reconciliation)`; re-approve the plan before implementing | | 1 | `gh issue create` fails (auth, label, repo perms) | Diagnose; create the issue manually; re-run with the issue located | | 2 | Pre-commit hook fails (lint, tests) | Fix the issue; re-run from step 2 | | 3 | `gh pr create` fails (no remote, branch missing on target remote) | Verify the push from step 2; re-run from step 3 | -| 4 | Implementation stops, or leaves acceptance criteria incomplete | Leave the PR draft and comment the resume command (`/spec execute <slug>` against the same task folder). Do not start a second implementation owner. | +| 4 | The run stops, or leaves acceptance criteria incomplete | Leave the PR draft and comment the resume command (`/spec execute <slug>` against the same task folder). Do not start a second implementation owner. | | 5 | `/eval` reports a NEW green→red regression or exits non-zero | Leave the PR draft; fix or document the regression, then re-run `/eval` | -| 5 | Wiki impact REQUIRED but entries are missing, stale against the implemented behavior, or the README index probe fails | Leave the PR draft; fix the wiki entries/index, then re-run the wiki gate | -| 5 | `/compact` unavailable or errors | Non-blocking; log a warning and continue | -| 6 | `evidence.md` cannot be written because a gate produced no observed output | Record the gap in the doc and leave the PR draft — a gate with no observed output is a gap, never a pass | -| 9 | `.oh/tasks/<slug>/evidence.md` is missing, or present but untracked (added without `-f`) | Leave the PR draft (`DRAFT-BLOCKED (evidence)`); write and commit it, then re-run the promotable gate | -| 9 | `/audit pr` cannot classify (gh/API error), or CI is red/pending so the PR is not promotable | Leave the PR draft; fix CI and re-run the audit executor | +| 6 | A page in the impact union has no explicit final state, or the index probe fails | `DRAFT-BLOCKED(knowledge)`; resolve every page, regenerate the index, re-run the gate | +| 7 | `evidence.md` cannot be written because a gate produced no observed output | Record the gap in the doc and leave the PR draft — a gate with no observed output is a gap, never a pass | +| 9 | `/compact` unavailable or errors | Non-blocking; log a warning and continue | +| 10 | `.oh/tasks/<slug>/evidence.md` is missing, or present but untracked (added without `-f`) | `DRAFT-BLOCKED(evidence)`; write and commit it, then re-run the promotable gate | +| 10 | The PR audit cannot classify (gh/API error), or CI is red/pending so the PR is not promotable | Leave the PR draft; fix CI and re-run the audit executor | +| 10 | The PR's head is not the commit just pushed | Push, wait for CI on that head, re-audit; a verdict about another commit is not this commit's verdict | +| 10 | A commit is pushed AFTER the undraft | Re-enter step 10 against the new head: wait for CI, re-audit, re-confirm promotable. Not promotable → `gh pr ready --undo`, comment the gate, `DRAFT-BLOCKED(<gate>)` | | 10 | PR not promotable, or `gh pr ready` fails | Leave draft + comment the blocking gate; diagnose PR state/permissions; never merge | ## Idempotency @@ -517,52 +688,70 @@ Every step checks for prior state and resumes rather than duplicating: | Step | Resume check | Behavior | |---|---|---| +| 0 | `## Plan Reconciliation` already records the current base's constraints | Continue; otherwise re-ground | | 1 | The issue named by `prd.json`'s `branchName` exists, or `--pr <N>` was passed | Reuse `<N>`; never create a duplicate | | 2 | Branch exists on the target remote | Checkout + commit on top | | 3 | Draft PR exists for this branch | Update the body; do not create a duplicate | -| 4 | `prd.json` already has all stories passing | Skip re-implementation and continue to the gates; worktree present → reuse | -| 5 | `.oh/evals/RESULTS.md` already reflects the current probe set and no new regression exists | Continue; otherwise re-run `/eval` | -| 5 | Wiki impact NOT-APPLICABLE, or required entries already match the implementation and the index probe passes | Continue | -| 6 | `evidence.md` exists and correlates to the CURRENT audit run id | Reuse; a doc citing a stale run id is rewritten, not kept | -| 10 | `/audit pr` already classified this PR promotable | Continue to the undraft | -| 10 | PR is already ready-for-review | Print the terminal status; do not mutate | +| 4 | `jq -e 'all(.userStories[]; .passes == true)'` on `prd.json` exits 0 | Implementation is complete; continue to the tail. Worktree present → reuse | +| 5 | `eval-result.json`'s `commit` equals HEAD and records no new regression | Continue; otherwise re-run `/eval` | +| 6 | Every page in the impact union already carries a final state for the current HEAD | Continue | +| 7 | `evidence.md` exists and correlates to the CURRENT audit run id | Reuse; a doc citing a stale run id is rewritten, not kept | +| 10 | The PR audit already classified this PR promotable | Continue to the undraft | +| 10 | PR is already ready-for-review AND its head equals local HEAD with CI green | Print the terminal status; do not mutate. A head that has moved re-enters the gate | -The whole pipeline can be re-invoked safely. Failed step = fix + re-run; resume happens -automatically. +The whole pipeline can be re-invoked safely. Failed step = fix + re-run; resume +happens automatically. `/tmp/spec-<slug>.state` tells a resuming session +which phase to re-enter. ## Finalization contract -`execute` opens a draft PR early so reviewers can observe the scaffold, but a successful run -does not stop there. The terminal successful state is a **ready-for-review** PR, reached only -after implementation completes, `/audit implementation` returns AUDIT-PASS, `/eval` shows no -new green→red regression, required wiki entries are updated against the spec, -**`.oh/tasks/<slug>/evidence.md` is committed and answers back to the approved -plan**, and a fresh `/audit pr` immediately classifies the PR **promotable** -(CI green + mergeable + clean) before `gh pr ready`. Draft is reserved for blocked states: an -incomplete build, a new eval regression, missing or stale wiki alignment, **missing or -untracked evidence**, a not-promotable PR (red/pending CI or conflicts), or an explicit user -stop. Heartbeat stale-draft watchdog output -may trigger investigation or resume work, but it never authorizes `gh pr ready`. Never -auto-merge. +`execute` opens a draft PR early so reviewers can observe the scaffold, and it +can return at `RUNNING` because the task is still building — but a successful run +does not end there. The terminal successful state is a +**ready-for-review** PR, reached only after implementation completes, the +implementation audit returns AUDIT-PASS, `/eval` shows no new green→red +regression, every page in the Actual Knowledge Impact union carries an explicit +final state, **`.oh/tasks/<slug>/evidence.md` is committed and answers back to +the approved plan**, and a fresh PR audit immediately classifies the PR +**promotable** (CI green + mergeable + clean) before `gh pr ready`. + +**That classification binds to one head.** A push after the undraft moves the PR +past the verdict that promoted it, so the gate re-opens: re-audit the new head, +and return the PR to draft if it no longer classifies promotable. `READY` is +never a state the PR keeps while its head changes underneath it. + +Draft is reserved for blocked states: an incomplete build, a new eval regression, +an unresolved knowledge page, **missing or untracked evidence**, a not-promotable +PR (red/pending CI or conflicts), a head that moved past its promotable +classification, or an explicit user stop. Each is reported as +`DRAFT-BLOCKED(<gate>)` with the gate named — a silent stop is not a terminal +state. Heartbeat stale-draft watchdog output may trigger investigation or resume +work, but it never authorizes `gh pr ready`. Never auto-merge. --- ## What this node does NOT do -- **Launch a coding agent.** No tmux session, no Herdr workspace/tab/pane, no background - shell, no runner selection, no fallback runner. The agent already running `/spec execute` - is the implementation owner, and `/spec` never creates the agent that executes it. -- **Merge.** The terminal state is a **ready** PR. Merge is the human's gate; reset/clean is - the runner's job after merge. -- **Select work.** Selection is the human's; `execute` builds the one folder it is - handed. -- **Plan.** The four-file folder and its approved `prd.md` come from `/spec plan`. +- **Launch a coding agent.** No tmux session, no Herdr workspace/tab/pane, no + background shell, no runner selection, no fallback runner. The agent already + running `/spec execute` is the implementation owner, and `/spec` never creates + the agent that executes it. +- **Merge.** The terminal state is a **ready** PR. Merge is the human's gate; + reset/clean is the runner's job after merge. +- **Select work.** Selection is the human's; `execute` builds the one folder it + is handed. +- **Plan.** The three-file folder and its approved `prd.md` come from + `/spec plan`. +- **Load the pattern corpus.** Patterns inform the planner. The executor consumes + the approved Knowledge Context and re-reads the sources it names. +- **Promise a PR it has not seen.** A task whose stories are not all passing is + reported `RUNNING`. --- ## Reference -### Branch + commit conventions (from `.claude/skills/git/SKILL.md`) +### Branch + commit conventions (from `.oh/skills/git/SKILL.md`) - Branch: `<prefix>/<issue#>-<slug>` - Commit: `<type>: <description>` (where `<type>` matches `<prefix>` for scaffold commits) @@ -573,31 +762,32 @@ auto-merge. | Primitive | Path | Role | |---|---|---| -| Task prompt template | `.oh/skills/spec/templates/task-prompt.md` | Step 4 — the owner's implementation and gate instructions | -| `/worktrees` skill | `.claude/skills/worktrees/SKILL.md` | Step 4 — isolated `.worktrees/<branch>` for the implementation | -| `/delegate` skill | `.claude/skills/delegate/SKILL.md` | Step 4 — optional bounded fan-out beneath the owner | -| `/audit implementation` | `.claude/skills/audit/SKILL.md` | Step 5 — the per-unit verdict gate | -| `/eval` skill | `.claude/skills/eval/SKILL.md` | Step 5 — probe regression floor | -| Wiki schema | `.oh/skills/wiki/references/schema.md` | Step 5 — source-backed wiki alignment | -| `/compact` | (built-in) | Step 5 — clears implementation context before the promotable gate | -| Reviewer evidence doc | `.oh/skills/audit/references/reviewer-evidence-doc.md` | Step 6 — the contract `evidence.md` follows | -| `/audit pr` skill | `.claude/skills/audit/SKILL.md` | Step 9 — promotable classification (gates the undraft) | -| `/ci-status` skill | `.claude/skills/ci-status/SKILL.md` | CI verification (subsumed by `/audit pr`'s promotable check) | +| Task prompt template | `.oh/skills/spec/templates/task-prompt.md` | Step 4 — rendered at execution time; never persisted | +| `/worktrees` skill | `.oh/skills/worktrees/SKILL.md` | Step 4 — isolated `.worktrees/<branch>` for the implementation | +| `/delegate` skill | `.oh/skills/delegate/SKILL.md` | Step 4 — optional bounded fan-out beneath the one owner | +| `/audit implementation` | `.oh/skills/audit/SKILL.md` | Step 5 — the per-unit verdict gate | +| `/eval` skill | `.oh/skills/eval/SKILL.md` | Step 5 — probe regression floor, run once | +| Knowledge invalidation | `.oh/skills/wiki/scripts/knowledge-impact.sh` | Step 6 — the one dependency-aware impact implementation | +| Knowledge schema | `.oh/skills/wiki/references/schema.md` | Step 6 — page kinds, `verified_at`, provenance forms | +| Reviewer evidence doc | `.oh/skills/audit/references/reviewer-evidence-doc.md` | Step 7 — the contract `evidence.md` follows | +| `/retro` skill | `.oh/skills/retro/SKILL.md` | Step 8 — report-only lesson engine, task-scoped | +| `/wiki compile` | `.oh/skills/wiki/references/compile.md` | Step 9 — the durable pattern writer | +| `/compact` | (built-in) | Step 9 — optional, non-gating, after distillation | +| `/benchmark` skill | `.oh/skills/benchmark/SKILL.md` | Step 9 — the progress-ceiling verdict | +| `/audit pr` skill | `.oh/skills/audit/SKILL.md` | Step 10 — promotable classification (gates the undraft) | +| `/ci-status` skill | `.oh/skills/ci-status/SKILL.md` | CI verification (subsumed by the PR audit's promotable check) | | Protected-paths list | `.claude/protected-paths.txt` | Load-bearing items a spec must not propose deleting | ## Pipeline position Within the workflow owned by `.oh/skills/spec/SKILL.md`, `execute` is the -implementation node. It ends at the human merge gate. The runner resets or cleans after -the human merges. When a gate blocks the undraft, the next step is to resume -implementation or fix the named gate. +implementation node. It ends at the human merge gate. The runner resets or cleans +after the human merges. When a gate blocks the undraft, the next step is to +resume implementation or fix the named gate. Report the terminal state as **`READY`** (the PR is ready for review) or -**`DRAFT-BLOCKED (<gate>)`** naming the gate that held it, alongside the PR URL. The PR's own -draft/ready state is the authority — it is what the next reader and the next run both look at. - -There is no `STATUS: SPEC-*` token. The four that used to be printed here -(`SPEC-PLANNED` / `SPEC-EXECUTED` / `SPEC-BLOCKED` / `SPEC-RETRO-DONE`) had **zero executable -consumers repo-wide** — nothing parsed them, so they were ceremony that emitted a line and -bought nothing. Never infer a promotable PR from silence: an incomplete implementation or unrun CI -is `DRAFT-BLOCKED`, not ready. +**`DRAFT-BLOCKED(<gate>)`** naming the gate that held it, alongside the PR URL, +and mirror it into `/tmp/spec-<slug>.state`. The PR's own draft/ready state +is the authority — it is what the next reader and the next run both look at. +Never infer a promotable PR from silence: an incomplete implementation or unrun +CI is `DRAFT-BLOCKED`, not ready. diff --git a/.oh/skills/spec/references/plan.md b/.oh/skills/spec/references/plan.md index 1b344e78..d88bdb2d 100644 --- a/.oh/skills/spec/references/plan.md +++ b/.oh/skills/spec/references/plan.md @@ -1,4 +1,4 @@ -# `/spec plan` — produce the `.oh/tasks/<slug>/` folder +# `/spec plan` — recall → ground → plan > Detail doc for the **`plan`** subcommand of the `/spec` skill > (`.oh/skills/spec/SKILL.md`). Argument form: @@ -6,18 +6,24 @@ > The dispatcher passes the argument string after `plan` to this procedure as > `$ARGUMENTS`. Authority: `.oh/skills/spec/SKILL.md`. -The **plan** node of the `/spec` workflow takes -a topic / plan file / issue and produces the **`.oh/tasks/<slug>/` folder** — the universal -interface every other `/spec` node is pointed at. +The **plan** node takes a topic / plan file / issue and produces the +**`.oh/tasks/<slug>/` folder** — the universal interface every other `/spec` node +is pointed at. **Core principle: plan cheaply, commit nothing.** `plan` writes only local files -under `.oh/tasks/<slug>/`. It creates no GitHub-side state, so the folder stays fully -reversible (delete `.oh/tasks/<slug>/`) until the operator approves the `prd.md`. -**That approval is the commitment gate** (`.oh/skills/spec/SKILL.md`). +under `.oh/tasks/<slug>/`. It creates no GitHub-side state, so the folder stays +fully reversible (delete `.oh/tasks/<slug>/`) until the operator approves the +`prd.md`. **That approval is the commitment gate** (`.oh/skills/spec/SKILL.md`). -This is the planning node. `/spec execute` consumes the approved folder and runs -the build workflow. Each `/spec` subcommand can run independently or fan out at -scale via `/delegate`. +**Second principle: recall before you plan.** The harness accumulates +understanding of its own repository. A plan written without reading it re-derives +what is already known, and — worse — re-derives it differently. Recall is a step +of this node, not an optional habit. + +**Third principle: knowledge is a cache, not an authority.** A recalled page can +be out of date the moment a source it depends on moves. Every material claim the +plan leans on is verified against the current repository before the PRD is +written. --- @@ -25,11 +31,11 @@ scale via `/delegate`. | Arg | Meaning | |-----|---------| -| `<topic>` | Free-text feature description — the seed for `/prd`. Required unless `--plan` or `--issue` supplies the spec. | -| `--plan <path>` | A plan file (e.g. `/imagine` output) used as comprehensive `/prd` input; skips `/prd`'s clarifying questions. | -| `--issue <N>` | The issue number this spec builds — **consumed by the `/ralph` step** (the branch name embeds it, so `/ralph` hard-fails without it). The human selects the issue. For a fresh manual topic with no issue, open one first (per `/git`) or let `/spec execute` open one in a standalone run. `plan` only **reads** `<N>` — it never opens, edits, or closes an issue. | +| `<topic>` | Free-text feature description — the seed for recall and `/prd`. Required unless `--plan` or `--issue` supplies the spec. | +| `--plan <path>` | A plan file (e.g. `/imagine` output) used as comprehensive `/prd` input; skips `/prd`'s clarifying questions. **Its presence is the operator's approval** — see `## Plan Reconciliation`. | +| `--issue <N>` | The issue number this spec builds — **consumed by the `/ralph` step** (the branch name embeds it, so `/ralph` hard-fails without it). The human selects the issue. For a fresh manual topic with no issue, open one first (per `/git`) or let `/spec execute` open one in a standalone run. `plan` only **reads** `<N>`. | | `--slug <slug>` | Override the derived slug. Must match `[a-z0-9-]+`, ≤5 words, not `archive`. | -| `--prefix <type>` | Branch/issue prefix (default `feat`), per `.claude/skills/git/SKILL.md`. | +| `--prefix <type>` | Branch/issue prefix (default `feat`), per `.oh/skills/git/SKILL.md`. | | `--repo <owner/name>` | Recorded for downstream `/spec execute`; not acted on here. Default `mifunedev/openharness`. | | `--base <branch>` | Recorded for downstream `/spec execute`; not acted on here. Default `development`. | @@ -62,65 +68,171 @@ spawns a session, and never owns the build. Run these in order; each is an existing primitive — compose, don't re-derive. -1. **Derive `<slug>`** (per the `/prd` skill's rules): lowercase kebab-case, `[a-z0-9-]+`, - ≤5 hyphen-words, not `archive`. The slug is the universal key — task directory and branch - second segment; it never names a terminal session, tab, or pane. Choose once; reject and - ask for a shorter name if invalid. `--slug` overrides derivation. - -2. **`/prd` → `.oh/tasks/<slug>/prd.md`**. Invoke the `prd` skill with `<topic>` - (or `--plan` content, with an explicit instruction to skip clarifying questions when a - plan is supplied). Verify `.oh/tasks/<slug>/prd.md` exists before continuing. - -3. **Wiki alignment**. Read `.oh/skills/wiki/references/schema.md` and record a - `## Wiki Alignment` section in `prd.md` (`Impact: REQUIRED | NOT-APPLICABLE`, local - entries, spec alignment, and — when REQUIRED — the wiki acceptance criteria a story must - carry). The exact shape is the `## Wiki Alignment` block below; reuse it verbatim so - `/spec execute`'s wiki gate can read it. - - **There is no DeepWiki comparison step.** The public DeepWiki for this repo does not - regenerate on any schedule this workflow can depend on, so a comparison against it - measures upstream lag, not the plan. Alignment is judged against the repo's own sources - and the local corpus only. - - ```markdown - ## Wiki Alignment - - - **Impact**: REQUIRED | NOT-APPLICABLE - - **Local entries**: `.oh/skills/wiki/corpus/<slug>.md` to create/update, or `none` - - **Spec alignment**: <how the wiki entry must reflect this PRD's goals, non-goals, and acceptance criteria> - - **Acceptance criteria**: <wiki update checks to add to the relevant story when REQUIRED> - ``` - - `Impact: REQUIRED` when the task changes harness architecture, skill behavior, agent - roles, runtime flow, conceptual vocabulary, or public prose that introduces a reusable - mechanism. `Impact: NOT-APPLICABLE` is allowed for narrow code/test chores, but it must - say why. When impact is REQUIRED, revise the PRD so at least one story carries acceptance - criteria for: the local entry aligned with goals/non-goals/final behavior; the - source-backed body shape (relevant source files, line-cited claims, system relationships - when applicable, `## See Also`); and `.oh/skills/wiki/corpus/README.md` index freshness - via `/wiki lint` or `bash .oh/evals/probes/wiki-readme-index.sh`. - -4. **`/ralph` → `.oh/tasks/<slug>/prd.json`**. Invoke the `ralph` skill: - `.oh/tasks/<slug>/ --issue <N> --prefix <prefix>`. It writes `prd.json` with - `branchName: <prefix>/<N>-<slug>`. Verify it parses - (`node -e "require('./.oh/tasks/<slug>/prd.json')"`). **`/ralph` hard-fails without - `--issue <N>`** (the branch name embeds it) — in the canonical flow `<N>` is the issue - was selected; for a fresh manual topic with no issue, open one first per `/git` - or let `/spec execute` open one in a standalone run. `plan` consumes the number; it never creates the issue. - -5. **Scaffold `prompt.md` + `progress.txt`.** There is one task prompt template, - `.oh/skills/spec/templates/task-prompt.md`. Render it into - `.oh/tasks/<slug>/prompt.md` by substituting `<slug>`, `<branch>`, and `<issue>` with - the task slug, `prd.json`'s `branchName`, and the issue number as bare digits. This - prompt states the single-owner contract for whichever agent runs `/spec execute`; it is - not a handoff to a separately launched process or session. Confirm no angle-bracket - placeholder survives the render. Write - `.oh/tasks/<slug>/progress.txt` with the `# progress` header only. - -Verify the four-file contract before handing off: +### 1. Derive `<slug>` + +Per the `/prd` skill's rules: lowercase kebab-case, `[a-z0-9-]+`, ≤5 +hyphen-words, not `archive`. The slug is the universal key — task directory, +branch second segment, status file; +it never names a terminal session, tab, or pane. +Choose once; reject and ask for a shorter name if invalid. `--slug` overrides +derivation. + +### 2. Recall tracked knowledge + +Record the base commit first — it is what `/spec execute` re-grounds against: ```bash -for f in prd.md prd.json prompt.md progress.txt; do +BASE_COMMIT=$(git rev-parse HEAD) +``` + +Then, in order: + +1. **Derive the task's vocabulary** from the topic, the plan file, or the issue: + the subsystems, file surfaces, and concepts the work touches. Three to six + terms, in the knowledge base's own vocabulary (`spec`, `wiki`, `evals`, + `docs`, `sandbox`, `cli`, …), not the phrasing of the request. +2. **Query tracked entity knowledge**: `/wiki query <terms>` — `kind: repo` and + `kind: external` pages, ≤3 read. +3. **Query accumulated patterns**: `/wiki query <terms> --patterns` — `kind: + pattern` pages, ≤5 read. **This is the planner/proposer role, which is why it + gets patterns**; `/spec execute` does not load the pattern set + (`.oh/skills/wiki/references/schema.md` § 3). +4. **Read the matched entries** whole. + +Both queries read tracked knowledge only — `.oh/knowledge/source/` and +`.oh/knowledge/patterns/`. `.oh/knowledge/local/` is per-machine scratch and no +query path reads it, because a plan grounded in a page one machine can see is a +plan nobody else can reproduce. + +An empty result is a normal outcome on a young knowledge base, not a failure. +Record `none` and continue. + +### 3. Ground the recalled claims against current sources + +For every recalled claim the plan will lean on, open the authoritative source and +check it. **The order of authority is fixed**: code and tests are implementation +truth; canonical `docs/`, RFCs, and ADRs are intended-design truth; a knowledge +page is orientation and is never either. + +A page's `sources:` list names exactly what to open, and +`.oh/skills/wiki/scripts/knowledge-impact.sh --verified` will already have +flagged a page whose declared dependencies moved after its `verified_at` commit. +A `NEEDS-REVIEW` page is not unusable — it is a page whose claims must be checked +before use rather than after. + +Where a page and the source disagree, **the source wins and the page is wrong**. +Record the reconciliation in `## Knowledge Context` under *Conflicts discovered* +and repair the page during `/spec execute`'s knowledge-impact gate — not here, +because `plan` writes no repository state. + +### 4. `/prd` → `.oh/tasks/<slug>/prd.md` + +Invoke the `prd` skill with `<topic>` (or the `--plan` content, with an explicit +instruction to skip clarifying questions when a plan is supplied), building the +PRD from **operator intent + current repository reality + recalled knowledge**. +Verify `.oh/tasks/<slug>/prd.md` exists before continuing. + +### 5. Record the three knowledge sections in `prd.md` + +Reuse these block shapes verbatim — `/spec execute`'s gates read them. + +#### `## Knowledge Context` — what informed this plan + +```markdown +## Knowledge Context + +- **Base commit**: `<sha>` +- **Queries**: `<queries used>` +- **Knowledge used**: `[[slug]]`, ... or `none` +- **Grounded against**: `<repo-relative paths>` +- **Conflicts discovered**: `none` or concise reconciliation +``` + +`Base commit` is load-bearing: `/spec execute` diffs it against the execution +base to decide whether the plan's assumptions still hold. `Grounded against` +lists the authoritative sources step 3 actually opened — it is the list the +execution owner re-reads, so an unchecked claim left off this list is a claim +nobody verifies twice. + +#### `## Expected Knowledge Impact` — the planner's prediction + +```markdown +## Expected Knowledge Impact + +- **Impact**: REQUIRED | NOT-APPLICABLE +- **Expected entries**: `<slugs or none>` +- **Affected source paths**: `<paths/patterns>` +- **Reason**: `<why>` +``` + +`Impact: REQUIRED` when the task changes harness architecture, skill behavior, +agent roles, runtime flow, conceptual vocabulary, or public prose that introduces +a reusable mechanism. `Impact: NOT-APPLICABLE` is allowed for narrow code/test +chores, but it must say why. + +**This is a prediction, not the oracle.** `Knowledge Context` records what +informed the plan; `Expected Knowledge Impact` records only what the planner +thinks the implementation may invalidate. The final answer is derived in +`/spec execute` from the actual diff plus each page's declared dependencies, +because implementation reaches paths the planner never saw. + +#### `## Plan Reconciliation` — did grounding preserve the approved intent? + +```markdown +## Plan Reconciliation + +- **Source plan**: `<path>` +- **Intent preserved**: YES | NO +- **Material deviations**: `none` or list +- **Constraints discovered during grounding**: `none` or list +``` + +**The gate.** Passing a plan file satisfies the commitment gate only while +grounding preserves the approved intent. A *material* deviation is one the +operator would want to decide: a goal that cannot be met as written, a +non-goal the work would have to cross, a different mechanism than the plan +names, or a scope change. A *constraint discovered* is something grounding +revealed that the plan did not contradict — record it and continue. + +If `Intent preserved: NO`, **stop before execution**. Report the deviation, leave +the folder in place, and require operator re-approval. Do not silently convert an +approved plan into a materially different PRD and treat the original approval as +covering it. With no `--plan` file the field is `Source plan: none` and the PRD +itself is the first artifact anyone could approve. + +`## Wiki Alignment` is superseded by these three sections. Do not write it. + +### 6. `/ralph` → `.oh/tasks/<slug>/prd.json` + +Invoke the `ralph` skill: `.oh/tasks/<slug>/ --issue <N> --prefix <prefix>`. It +writes `prd.json` with `branchName: <prefix>/<N>-<slug>`. Verify it parses +(`node -e "require('./.oh/tasks/<slug>/prd.json')"`). **`/ralph` hard-fails +without `--issue <N>`** (the branch name embeds it). `plan` consumes the number; +it never creates the issue. + +When `Expected Knowledge Impact` is `REQUIRED`, at least one story must carry +acceptance criteria for the knowledge update: the named entries aligned with the +PRD's goals, non-goals, and final behavior; the source-backed body shape +(`.oh/skills/wiki/references/schema.md` § 3); `verified_at:` advanced for any +`kind: repo` page whose claims were re-read; and a regenerated +`.oh/knowledge/README.md` index verified by +`bash .oh/evals/probes/wiki-readme-index.sh`. + +### 7. Scaffold `progress.txt` + +Write `.oh/tasks/<slug>/progress.txt` with the `# progress` header, then append +one dated plan-phase line recording the base commit, the queries run, and the +slugs read. A resumed session recovers from that line instead of re-deriving it. + +**There is no `prompt.md`.** The task prompt is rendered at execution +time from `.oh/skills/spec/templates/task-prompt.md` plus `prd.md` and +`prd.json`; persisting a generated copy of a template only lets it drift from the +template. + +Verify the three-file contract before handing off: + +```bash +for f in prd.md prd.json progress.txt; do [ -f ".oh/tasks/<slug>/$f" ] || { echo "MISSING: $f"; exit 1; } done ``` @@ -129,35 +241,33 @@ done ## Output -`.oh/tasks/<slug>/` holding the four-file contract (`prd.md`, `prd.json`, `prompt.md`, -`progress.txt`). No issue, branch, or PR. +`.oh/tasks/<slug>/` holding the three-file contract (`prd.md`, `prd.json`, +`progress.txt`), with `prd.md` carrying `## Knowledge Context`, +`## Expected Knowledge Impact`, and `## Plan Reconciliation`. No issue, branch, +or PR. Report the folder path, the story count, and the recalled slugs. --- ## What this node does NOT do - **Decide whether to build.** Approving the `prd.md` this node writes **is** the - commitment gate — the operator makes that call. `plan` only produces the folder. -- **Create GitHub-side state.** No `gh issue create`, no branch, no PR — the whole point - is to keep the folder reversible before commitment. It *consumes* a - pre-existing issue number (`--issue <N>`) for the branch name but never opens, edits, or - closes an issue/PR. + commitment gate — the operator makes that call. +- **Create GitHub-side state.** No `gh issue create`, no branch, no PR. It + *consumes* a pre-existing issue number for the branch name but never opens, + edits, or closes an issue/PR. +- **Write knowledge.** Recall is read-only. A page this node found to be wrong is + repaired in `/spec execute`'s knowledge-impact gate, where the repository state + it must match actually exists. - **Build.** Implementation is `/spec execute`. ## Pipeline position -Within the workflow owned by `.oh/skills/spec/SKILL.md`, `plan` is the first node. -The operator approves `prd.md`, then runs `/spec execute <slug>`. - -Running `plan` by name is the deliberate stop-after-scaffolding path. The default entry -point, `ship`, runs this node and then continues into `execute` when the operator handed -in an approved plan file (`references/ship.md`). +Within the workflow owned by `.oh/skills/spec/SKILL.md`, `plan` is the first +node. The operator approves `prd.md`, then runs `/spec execute <slug>`. -The terminal artifact is the folder itself: `.oh/tasks/<slug>/` carrying the four-file -contract, with `prd.md` awaiting the operator's approval. Report the folder path and the -story count. There is no `STATUS: SPEC-PLANNED` token — it had no executable consumer, so -printing it bought nothing. +Running `plan` by name is the deliberate stop-after-scaffolding path. The default +entry point — a bare plan path — runs this node and then continues into +`execute`, because handing in a plan file is itself the approval. -The `/spec` family's authority is `.oh/skills/spec/SKILL.md`. If the four-file -contract is incomplete, print the missing file and report the folder as incomplete — a -missing artifact is a failure, not a clean plan. +If the three-file contract is incomplete, print the missing file and report the +folder as incomplete — a missing artifact is a failure, not a clean plan. diff --git a/.oh/skills/spec/references/retro.md b/.oh/skills/spec/references/retro.md index 0f5d9880..f5a804e8 100644 --- a/.oh/skills/spec/references/retro.md +++ b/.oh/skills/spec/references/retro.md @@ -1,75 +1,62 @@ -# `/spec retro` — capture the build's lessons +# `/spec retro` — compatibility wrapper for `/retro --task <slug>` > Detail doc for the **`retro`** subcommand of the `/spec` skill > (`.oh/skills/spec/SKILL.md`). Argument form: `retro <slug> [--dry-run]`. > The dispatcher passes the argument string after `retro` to this procedure as > `$ARGUMENTS`. Authority: `.oh/skills/spec/SKILL.md`. -The **reflection** node of the `/spec` workflow runs -inside `/spec execute`'s tail, after `build ⇄ audit` reaches `AUDIT-PASS` and the evidence -step has run, and before `improve`, to turn the execution run into durable, -evidence-tested lessons. +**This subcommand owns no behavior.** It is a one-line alias kept because +`/spec execute`'s tail and several existing callers spell the task-scoped +retrospective this way: -**Core principle: compose `/retro`, scoped to this task.** `/retro` already implements the -scientific session-closing pass — falsifiable hypotheses, evidence for *and* against, a -verdict + confidence, and a propose-then-confirm nomination of candidate probes under -`.oh/evals/probes/`. `retro` is the execution-side application of it: point `/retro` -at the just-built `.oh/tasks/<slug>/` run so the reflection is anchored to that unit's -artifacts (`prd.md`, `progress.txt`, `prd.json`, the `/audit implementation` evidence) -rather than the whole ambient session. +```text +/spec retro <slug> [--dry-run] ≡ /retro --task <slug> [--dry-run] +``` -It is **not** a second retro engine. The propose-then-confirm gate, the five-subsystem lens, -and the promotion rules all live in `/retro`; `retro` only frames the scope and -records that the execution stage ran its retro. +Read `.oh/skills/retro/SKILL.md` and follow it with `--task <slug>` (plus +`--dry-run` when supplied). Nothing else happens here. ---- +## Why a wrapper and not a second node -## Inputs - -| Arg | Meaning | -|-----|---------| -| `<slug>` | The task slug — the retro reads `.oh/tasks/<slug>/` artifacts as its primary signal source. Required. | -| `--dry-run` | Passed through to `/retro`: report only (`Result: DRY-RUN`). | - -If `.oh/tasks/<slug>/` has no `progress.txt`/`prd.md`, there is no build to reflect on — say so -and fall back to a plain `/retro` on the session, or skip with a note in the report. +`/retro` already implements the whole scientific pass — falsifiable hypotheses, +evidence for *and* against, verdict plus confidence, and a propose-then-confirm +nomination of candidate probes. Its `--task <slug>` flag already scopes that pass +to one `.oh/tasks/<slug>/` run. ---- +There is therefore exactly **one** retro ontology, and it lives in `/retro`. +Earlier revisions of this document described `/spec retro` as owning a +propose-and-write gate, which was never true: `/retro` is report-only by contract +(guarded by `.oh/evals/probes/retro-deterministic-contract.sh`) and `/wiki +compile` is the durable pattern writer. Restating either here would create a +second description that drifts from the one that runs. -## Run +| Concern | Owner | +|---|---| +| Hypotheses, evidence, verdicts, confidence, probe nominations | `/retro` | +| Scoping that pass to one task folder | `/retro --task <slug>` | +| Writing durable `kind: pattern` pages from the report | `/wiki compile` | +| Deciding promotability of the implementation | the implementation audit route, earlier in `/spec execute` | -Invoke `/retro` with the execution scope made explicit — gather signals primarily from -this task's artifacts: what the `prd.md` intended vs. what `progress.txt` shows shipped, -what the `build ⇄ audit` loop revealed (how many -FAIL→build cycles, and why), and any coupling/constraint the run surfaced. Then let `/retro` -do its scientific pass: form falsifiable hypotheses, test each for and against, assign -verdict + confidence, and present supported `medium`+ lessons for confirmation before any -write. Always reports, even on a trivial/no-lesson run. - ---- +## Inputs -## What this node does NOT do +| Arg | Meaning | +|-----|---------| +| `<slug>` | The task slug, passed through as `--task <slug>`. Required. | +| `--dry-run` | Passed through: report only. | -- **Re-implement retro.** The hypothesis engine, qualify filter, and propose-then-confirm - gate are `/retro`'s; `retro` only scopes them to the task. -- **Audit or decide promotability.** That was `/audit implementation` (the `build ⇄ audit` loop) earlier in - `/spec execute`. -- **Run the grooming triad.** `/audit skills` · `/wiki lint` · `/audit drift` are no longer a - step of `/spec execute` at all — the triad was cut in US-003's follow-on because - `/audit drift` already runs hourly from the heartbeat cron and the other two never blocked - a merge. Run them on their own cadence, or on demand. -- **Merge or undraft.** No GitHub-side mutation — reflection only. +If `.oh/tasks/<slug>/` has no `prd.md`, there is no build to reflect on — say so +and fall back to a plain `/retro` on the session. ## Pipeline position -Within the workflow owned by `.oh/skills/spec/SKILL.md`, `retro` runs inside the -`spec-execute` tail (`build ⇄ audit → evidence → spec-retro → improve`). The next -step is `improve` (compound · compress · benchmark). +Within the workflow owned by `.oh/skills/spec/SKILL.md`, this runs inside the +`spec-execute` tail, after the implementation audit passes and `evidence.md` is +written, and before `/wiki compile` turns the supported lessons into durable +pattern pages. It writes no file of its own, and it always completes, so the +execute tail always continues. -The terminal artifact is the report itself plus whatever the propose-then-confirm gate -actually wrote. Report the counts. There is no `STATUS: SPEC-RETRO-DONE` token — it had no -executable consumer. The supported lessons in that report become durable in the next step, -`improve`, via `/wiki compile`; this subcommand writes no file of its own. +## See Also -The `/spec` family's authority is `.oh/skills/spec/SKILL.md`. `retro` always -completes (like `/retro`), so the execute tail always continues to `improve`. +- `.oh/skills/retro/SKILL.md` — the engine; report-only by contract +- `.oh/skills/wiki/references/compile.md` — the durable pattern writer +- `.oh/skills/spec/references/execute.md` — the tail this runs inside diff --git a/.oh/skills/spec/references/ship.md b/.oh/skills/spec/references/ship.md deleted file mode 100644 index 2b252de0..00000000 --- a/.oh/skills/spec/references/ship.md +++ /dev/null @@ -1,121 +0,0 @@ -# `/spec ship` — an approved plan to a ready pull request - -> Detail doc for the **`ship`** subcommand of the `/spec` skill -> (`.oh/skills/spec/SKILL.md`). Argument form: -> `ship <plan-path|topic> [--issue <N>] [--slug <slug>] [--prefix feat|bug|task|audit|skill|agent] [--repo <owner/name>] [--base <branch>]`. -> `ship` is the **default** node: the dispatcher routes any non-empty `$ARGUMENTS` -> whose first token is not `ship`/`plan`/`execute`/`retro` here, passing the whole -> string. Authority: `.oh/skills/spec/SKILL.md`. - -`ship` runs the canonical operative path — `plan → execute` — in one invocation, so -handing `/spec` a plan file produces a ready-for-review pull request. It produces no -artifact of its own: `plan` writes the `.oh/tasks/<slug>/` folder that is the universal -interface, and `execute` builds from it. - -**Core principle: `ship` composes, it does not build.** Every artifact it produces is -produced by `plan` or `execute` under their own procedures. `ship` adds no mechanics of -its own — no `gh` invocation, no branch shape, no gate. It decides one thing: whether -the commitment gate is already satisfied, and therefore whether to continue into -`execute`. - ---- - -## Inputs - -| Arg | Meaning | -|-----|---------| -| `<plan-path>` | A readable path to a plan file (`.claude/plans/*.md`, `/imagine` output, any markdown spec). **Its presence is the operator's approval** — see the gate below. | -| `<topic>` | Free-text description, when no plan file exists. Carries no approval; `ship` stops after `plan`. | -| `--issue <N>` | Issue this builds. Passed to `plan` (which embeds it in the branch name via `/ralph`). When absent, `execute`'s standalone-run path opens one. | -| `--slug <slug>` | Override slug derivation. `[a-z0-9-]+`, ≤5 hyphen-words, not `archive`. | -| `--prefix <type>` | Branch/issue prefix, default `feat`, per `.claude/skills/git/SKILL.md`. | -| `--repo <owner/name>` | Default `mifunedev/openharness`. Recorded by `plan`, acted on by `execute`. | -| `--base <branch>` | Default `development`. Same. | - -The first token is a plan path when it resolves to a readable file; otherwise it is -treated as the start of a free-text topic. - ---- - -## The commitment gate - -`.oh/skills/spec/SKILL.md` makes approving `prd.md` the commitment gate — nothing -GitHub-side exists until it is crossed. `ship` does not remove that gate; it recognizes -when the operator has already crossed it. - -| Input | Gate | Behavior | -|---|---|---| -| A plan file (`<plan-path>` or `--plan <path>`) | **Satisfied.** The operator wrote the plan and handed it in; requiring a second approval of a `prd.md` derived from it asks the same question twice | `plan`, then `execute` — through to a ready PR | -| A bare topic, no plan file | **Not satisfied.** Nothing has been approved; the PRD is the first artifact anyone could approve | `plan` only. Report the folder path and stop with the `/spec execute <slug>` invocation to run after approval | - -An operator who wants the folder without the build asks for the node by name: -`/spec plan <plan-path>`. That is the escape hatch, and it is why `plan` stays a public -subcommand. - ---- - -## The pipeline - -1. **Resolve the input.** Determine plan-path vs topic; derive or accept `<slug>`. - Report both before doing anything, so a wrong slug is caught before files exist. - -2. **Run `plan`.** Follow `references/plan.md` with - `<topic-or-plan> [--plan <path>] [--issue <N>] [--slug <slug>] [--prefix ...] [--repo ...] [--base ...]`. - A plan-path first token is passed to `plan` as `--plan <path>` with the topic derived - from the plan's own title. Verify the four-file contract - (`prd.md`, `prd.json`, `prompt.md`, `progress.txt`) before continuing — an incomplete - folder is a failure, not a clean plan, and `ship` must not build on one. - -3. **Decide at the gate** (table above). Not satisfied → report and stop. This is a - complete, honest outcome, not a blocked one. - -4. **Run `execute`.** Follow `references/execute.md` with - `<slug> [--repo ...] [--remote ...] [--base ...]`, which owns the issue, branch, draft - PR, implementation, `implementation ⇄ audit` loop, evidence, `/eval` and wiki gates, - and the undraft. `ship` neither reimplements nor relaxes any of it: a - `DRAFT-BLOCKED (<gate>)` from `execute` is `ship`'s outcome verbatim. - -5. **Stop at the human merge boundary.** `ship` never merges, and never marks a PR ready - that `execute`'s own gates left draft. - ---- - -## Output - -Whatever the node it stopped at produced: - -| Stopped at | Report | -|---|---| -| `plan` (gate not satisfied) | The folder path, the story count, and `/spec execute <slug>` as the next command | -| `execute` | `READY` or `DRAFT-BLOCKED (<gate>)` with the PR URL | - -Report which node it stopped at and why. A run that stopped after `plan` because the -input was a bare topic is a success; a run that stopped after `plan` because the -four-file contract was incomplete is a failure. Never report them the same way, and -never infer success from silence. - ---- - -## What this node does NOT do - -- **Add a build step.** Every mechanic belongs to `plan` or `execute`. If `ship` needs - new build behavior, the behavior belongs in `execute.md` — the protected single source - for build literals — not here. -- **Merge.** The human alone merges (`.oh/skills/spec/SKILL.md`). -- **Select the work.** It builds the one plan it is handed. -- **Skip a gate.** It reads the commitment gate as already satisfied when an approved - plan was passed in. Every other gate — evidence, audit, `/eval`, wiki — is `execute`'s - and is untouched. - -## Pipeline position - -Within the workflow owned by `.oh/skills/spec/SKILL.md`, `ship` is the entry point — -the node an operator reaches by typing nothing but a plan path. It occupies no position -of its own in `spec-plan → spec-execute → merge → reset|clean`; it walks the first two -and stops at the human merge boundary, exactly where `execute` stops. - -## See Also - -- `.oh/skills/spec/SKILL.md` — the dispatcher and workflow contract; the authority. -- `references/plan.md` — the folder-scaffolding node. -- `references/execute.md` — the build node and the single source for its literals. diff --git a/.oh/skills/spec/templates/task-prompt.md b/.oh/skills/spec/templates/task-prompt.md index daf4db1a..b1a98207 100644 --- a/.oh/skills/spec/templates/task-prompt.md +++ b/.oh/skills/spec/templates/task-prompt.md @@ -1,11 +1,18 @@ # `/spec execute` task — <slug> +> This is a **template**, rendered at execution time by +> `.oh/skills/spec/references/execute.md` step 4 and read by the agent that is already +> running `/spec execute`. It is never written into `.oh/tasks/<slug>/`: a persisted copy +> of a generated file drifts from the template it came from. + You are the single implementation owner for the `<slug>` task. Read the approved plan in `.oh/tasks/<slug>/prd.md` and the ordered stories in `.oh/tasks/<slug>/prd.json`. - Branch: `<branch>` — never push to `development` or `main`. - Issue: #<issue>. -- Task folder: `.oh/tasks/<slug>/` (`prd.md`, `prd.json`, `prompt.md`, `progress.txt`). +- Task folder: `.oh/tasks/<slug>/` (`prd.md`, `prd.json`, `progress.txt`; you add + `evidence.md` and `eval-result.json`). +- Status file: `/tmp/spec-<slug>.state` — keep it current at every phase. ## Ownership @@ -16,6 +23,15 @@ this work. Ownership is a role, not a terminal topology. Use `/delegate` only fo disjoint work that can run in parallel. Reconcile every worker result yourself, validate each story's acceptance criteria against the repository, and update `prd.json` and `progress.txt`. +## Re-ground before you implement + +`prd.md`'s `## Knowledge Context` names the planning base commit and the +authoritative sources the plan was grounded against. Diff that base against +current HEAD, re-read every listed source that moved, and reconcile before +writing code. Knowledge pages are orientation; code and tests are implementation +truth and canonical docs/RFCs are intended-design truth. Do not load the pattern +set — that is the planner's input. + ## Implementation cycle 1. Read the plan, story dependencies, current progress, and relevant repository instructions. @@ -25,9 +41,15 @@ story's acceptance criteria against the repository, and update `prd.json` and `p with the files, commit, result, and learnings. Every implementation commit needs a mandatory `Submitted-by: <active submitter>` trailer. 5. Continue until every story passes. Do not claim completion when a story is blocked or - deferred. Append `STATUS: COMPLETE` to `progress.txt` only when the full graph passes. + deferred. Completion is structured state, not prose: the task is done when + `jq -e 'all(.userStories[]; .passes == true)' .oh/tasks/<slug>/prd.json` exits 0. + +## Tail After implementation completes, continue in this same session with the `/spec execute` -procedure: run the implementation-side audit loop, run `/eval` once, revise required wiki entries, -write and commit `evidence.md`, run `/spec retro` and the improve steps, then run a fresh -`/audit pr`. Mark the PR ready only when that audit is promotable. Never merge the PR. +procedure, in this order: the implementation-side audit loop; `/eval` once; the Actual +Knowledge Impact gate (`knowledge-impact.sh --changed <actual diff>`, then resolve every +impacted page to UPDATED / REVERIFIED / NOT-AFFECTED); write and commit `evidence.md`; +`/spec retro <slug>`; `/wiki compile`; optional non-gating context compaction; `/benchmark`; +then run a fresh `/audit pr`. Mark the PR ready only when that audit is promotable. Never merge +the PR. Report `READY` or `DRAFT-BLOCKED(<gate>)` and mirror it into the status file. diff --git a/.oh/skills/strategic-proposal/SKILL.md b/.oh/skills/strategic-proposal/SKILL.md index 4e0dd9cb..924dc719 100644 --- a/.oh/skills/strategic-proposal/SKILL.md +++ b/.oh/skills/strategic-proposal/SKILL.md @@ -25,7 +25,7 @@ Use this variant when the user asks for a council to define a V2MOM, operating m - **Council decision**: Vision, Values, Methods, Obstacles, Measures. - **Wiki plan**: exact target entry, draft frontmatter/body, verification, and rejected scope. 5. Prefer one bounded provisional wiki entry first. Extra positioning/docs-IA entries are premature unless they hold distinct durable facts. -6. Keep wiki output as synthesis, not council minutes. Raw/source material belongs under `.oh/skills/wiki/corpus/raw/`; the tracked entry stays within the wiki word cap and starts `confidence: provisional`. +6. Keep wiki output as synthesis, not council minutes. Raw/source material belongs under `.oh/knowledge/raw/`; the tracked entry stays within the wiki word cap and starts `confidence: provisional`. 7. Add explicit approval gates for contested strategic wording (e.g. tagline, key nouns, whether a constraint is too narrow) before implementing file changes. Session example and final V2MOM synthesis: `references/open-harness-v2mom-council.md`. diff --git a/.oh/skills/strategic-proposal/references/open-harness-v2mom-council.md b/.oh/skills/strategic-proposal/references/open-harness-v2mom-council.md index a47a4fae..3f4afee5 100644 --- a/.oh/skills/strategic-proposal/references/open-harness-v2mom-council.md +++ b/.oh/skills/strategic-proposal/references/open-harness-v2mom-council.md @@ -35,7 +35,7 @@ Session: 2026-06-04. Trigger: user linked Marc Benioff / Salesforce V2MOM post a ## Wiki plan rule of thumb -For strategic operating models, create one provisional bounded entry first (e.g. `.oh/skills/wiki/corpus/open-harness-v2mom.md`) plus a raw source snapshot. Do not paste council minutes into the wiki. Do not create multiple adjacent strategy entries until distinct durable facts exist. +For strategic operating models, create one provisional bounded entry first (e.g. `.oh/knowledge/open-harness-v2mom.md`) plus a raw source snapshot. Do not paste council minutes into the wiki. Do not create multiple adjacent strategy entries until distinct durable facts exist. ## Approval gates used diff --git a/.oh/skills/sync/references/catchup.md b/.oh/skills/sync/references/catchup.md index c065a1c3..03cd7632 100644 --- a/.oh/skills/sync/references/catchup.md +++ b/.oh/skills/sync/references/catchup.md @@ -98,7 +98,7 @@ the squash may have bundled. **wiki/README.md** (expected conflict): ```bash -git checkout --ours .oh/skills/wiki/corpus/README.md 2>/dev/null || \ +git checkout --ours .oh/knowledge/README.md 2>/dev/null || \ git checkout --ours wiki/README.md 2>/dev/null || true ``` Regenerate from frontmatter using the exact logic in diff --git a/.oh/skills/sync/references/publish.md b/.oh/skills/sync/references/publish.md index abf658b4..c2989db1 100644 --- a/.oh/skills/sync/references/publish.md +++ b/.oh/skills/sync/references/publish.md @@ -69,9 +69,8 @@ upstream/development baseline. The goal: `git diff upstream/development HEAD # Private task artifacts — keep only README git checkout upstream/development -- .oh/tasks/ git rm -r --cached --ignore-unmatch .oh/tasks/*/ # remove sub-dirs if any leaked -# Research wiki corpus — keep README anchors only -git checkout upstream/development -- .oh/skills/wiki/corpus/ 2>/dev/null || \ - git checkout upstream/development -- .claude/skills/wiki/corpus/ 2>/dev/null || true +# Durable repository knowledge — reset to the upstream baseline +git checkout upstream/development -- .oh/knowledge/ 2>/dev/null || true # Agent folders (docs/agents/, .oh/tasks/archive/) if present git checkout upstream/development -- docs/agents/ 2>/dev/null || true # Codex plans / local promotion notes diff --git a/.oh/skills/weigh/references/workflow-shape.md b/.oh/skills/weigh/references/workflow-shape.md index e63ba5dd..e6273f88 100644 --- a/.oh/skills/weigh/references/workflow-shape.md +++ b/.oh/skills/weigh/references/workflow-shape.md @@ -114,6 +114,6 @@ node -e 'import("../scripts/score-trajectories.mjs").then(m => console.log(JSON. - `scoring.md` — the weighting formula + sub-signal table the owned step implements. - `../scripts/score-trajectories.mjs` — the harness-owned scorer (`select`, `weight`, `validateWeights`, `clamp`, `DEFAULT_WEIGHTS`, `TRAJECTORY_SCHEMA`). -- `../../wiki/corpus/recursive-language-models.md` — the RLM concept (the design +- `../../../knowledge/source/recursive-language-models.md` — the RLM concept (the design thesis that the Workflow tool supplies the sampling substrate and the harness owns the weight function). diff --git a/.oh/skills/wiki/SKILL.md b/.oh/skills/wiki/SKILL.md index c0bb5d53..de7d5ced 100644 --- a/.oh/skills/wiki/SKILL.md +++ b/.oh/skills/wiki/SKILL.md @@ -1,18 +1,20 @@ --- name: wiki description: | - Dispatcher for the harness wiki knowledge base — routes the first token of + Dispatcher for the harness knowledge base — routes the first token of $ARGUMENTS to one of four subcommands: ingest, query, lint, or compile. The - corpus lives at .oh/skills/wiki/corpus/ (entity pages, kind: source or pattern) - and corpus/raw/ (immutable source snapshots), owned by this skill and - gitignored-by-default (curated entries are whitelisted into git with - `git add -f`). Canonical schema: .oh/skills/wiki/references/schema.md. Full - per-subcommand procedures live in references/{ingest,query,lint,compile}.md. + knowledge lives at .oh/knowledge/ (source/ entity pages with kind: repo or + external, patterns/ pages with kind: pattern, raw/ immutable external + snapshots — all tracked; local/ is gitignored per-machine scratch that no + query reads). This skill owns the procedure; .oh/knowledge/ owns the data. + Canonical schema: .oh/skills/wiki/references/schema.md. Full per-subcommand + procedures live in references/{ingest,query,lint,compile}.md. TRIGGER when: "add to wiki", "capture this page", "snapshot this source", "ingest <url|path>", or promoting a sub-agent draft -> ingest; "what does the - wiki say about X", "find wiki entries for X", "look up X in the wiki" -> query; - "lint the wiki", "regenerate the wiki index", "find stale/orphaned wiki - entries" -> lint; "compile the retro into patterns", "what did this run teach", + wiki say about X", "find knowledge entries for X", "look up X in the wiki", + or recalling tracked knowledge before planning -> query; "lint the wiki", + "regenerate the knowledge index", "which pages need review" -> lint; + "compile the retro into patterns", "what did this run teach", "record this lesson as a pattern" -> compile. argument-hint: "ingest <url|path> [--slug <override>] | ingest --from-draft <slug> [--allow-stale] | query <topic> [--patterns] | lint [--dry-run] | compile [--from <path>] [--task <slug>] [--dry-run]" allowed-tools: Read, Write, Edit, Glob, Grep, Bash, WebFetch @@ -20,18 +22,22 @@ allowed-tools: Read, Write, Edit, Glob, Grep, Bash, WebFetch # Wiki -One parameterized skill over the harness wiki knowledge base. The first token of +One parameterized skill over the harness knowledge base. The first token of `$ARGUMENTS` selects the operation; the remainder is that subcommand's argument -string. This dispatcher holds the routing logic and the rules shared by all -four operations; the full per-subcommand procedure lives in `references/`. +string. This dispatcher holds the routing logic and the rules shared by all four +operations; the full per-subcommand procedure lives in `references/`. + +**Two surfaces, one owner each.** `.oh/knowledge/` owns the **data**; this skill +owns the **procedure**. No knowledge page lives under `.oh/skills/wiki/`, and no +schema rule lives anywhere but `references/schema.md`. ## Subcommands | Subcommand | Argument form | Purpose | Reference | |------------|---------------|---------|-----------| -| `ingest` | `<url\|path> [--slug <override>]` · `--from-draft <slug> [--allow-stale]` | Capture a source or promote a draft into a wiki entity page (the only authorized write path) | `references/ingest.md` | -| `query` | `<topic> [--patterns]` | Frontmatter OR-search over one `kind`; read the top matches into context (≤3 source, ≤5 pattern) | `references/query.md` | -| `lint` | `[--dry-run]` | 6 health checks + atomic `corpus/README.md` index regeneration | `references/lint.md` | +| `ingest` | `<url\|path> [--slug <override>]` · `--from-draft <slug> [--allow-stale]` | Capture a source or promote a draft into an entity page (the only authorized write path) | `references/ingest.md` | +| `query` | `<topic> [--patterns]` | Frontmatter OR-search over tracked knowledge; read the top matches into context (≤3 entity, ≤5 pattern) | `references/query.md` | +| `lint` | `[--dry-run]` | Six correctness checks + atomic `.oh/knowledge/README.md` index regeneration | `references/lint.md` | | `compile` | `[--from <path>] [--task <slug>] [--dry-run]` | Consolidate a `/retro` report into `kind: pattern` entries (create or patch) | `references/compile.md` | ## Dispatch @@ -59,46 +65,61 @@ instructions are authoritative — this dispatcher does not restate them): ## Shared rules -These hold across all three subcommands; the reference docs assume them. - -- **Corpus root**: entity pages at `.oh/skills/wiki/corpus/<slug>.md`; immutable - snapshots at `.oh/skills/wiki/corpus/raw/<yyyy-mm-dd>-<slug>.md`. Create - `.oh/skills/wiki/corpus/raw/` before any write — it is gitignored and may be - absent on a fresh clone. -- **Gitignore-by-default**: `.oh/skills/wiki/corpus/*` is gitignored; the operator - whitelists a curated entry into git with `git add -f <entry>`. `corpus/README.md` - and `corpus/raw/README.md` are tracked directory anchors. A local-only scratch - entry never breaks CI — the README index and the `wiki-readme-index.sh` probe are - both built from the **git-tracked** entry set only. -- **Canonical schema**: frontmatter fields, body layout, slug derivation, the - ≤600-word cap, cross-link convention, confidence lifecycle, and the body-merge - strategy all live in `.oh/skills/wiki/references/schema.md`. The reference docs - and this dispatcher defer to it — they never redefine it. -- **Frontmatter extraction** (canonical, used identically by `query` and `lint`): +These hold across all four subcommands; the reference docs assume them. + +- **Knowledge root**: entity pages at `.oh/knowledge/source/<slug>.md`, pattern + pages at `.oh/knowledge/patterns/pattern-<subsystem>-<mode>.md`, immutable + external snapshots at `.oh/knowledge/raw/<yyyy-mm-dd>-<slug>.md`. Both entry + globs are flat and do not descend. +- **Tracked by default**: `source/`, `patterns/`, and `raw/` are committed like + any other repository content — a plain `git add`, no `-f`, no whitelist. + `.oh/knowledge/local/` is the only ignored tier. +- **`local/` is never an input**: no query path and no `/spec` flow reads it. A + page one machine can see must not inform a plan another machine cannot + reproduce. Promotion goes through `ingest`. +- **The repository outranks the knowledge base**: a page is orientation, not + authority. Re-ground material claims against the sources a page cites before + relying on them. +- **Canonical schema**: frontmatter fields, the three kinds, provenance forms, + freshness, slug derivation, the word cap, cross-links, the confidence + lifecycle, and the body-merge strategy all live in + `.oh/skills/wiki/references/schema.md`. The reference docs and this dispatcher + defer to it — they never redefine it. +- **Frontmatter extraction** (canonical, used identically by every consumer): ```bash - awk '/^---$/{f=!f; next} f{print}' .oh/skills/wiki/corpus/<slug>.md + awk '/^---$/{f=!f; next} f{print}' .oh/knowledge/source/<slug>.md ``` +- **One freshness implementation**: `.oh/skills/wiki/scripts/knowledge-impact.sh` + decides dependency-aware invalidation. `lint` calls it with `--verified`; + `/spec execute` calls it with `--changed <paths>`. Nothing reimplements it. - **Orchestrator-only write gate**: `ingest` writes (snapshots + entity pages), - `compile`'s pattern-page writes, and `lint`'s `corpus/README.md` regeneration are - orchestrator-only. Sub-agents propose - drafts to `$TMPDIR/oh-wiki-drafts/<slug>.md`; the orchestrator promotes via - `/wiki ingest --from-draft <slug>`. A sub-agent that writes directly to the corpus - is out of scope and may be reverted. -- **Index reflects tracked entries**: `corpus/README.md`'s Index table is generated - state owned by `lint`, sorted by `updated:` descending. Never hand-edit it. + `compile`'s pattern-page writes, and `lint`'s index regeneration are + orchestrator-only. Sub-agents propose drafts to + `$TMPDIR/oh-wiki-drafts/<slug>.md`; the orchestrator promotes via + `/wiki ingest --from-draft <slug>`. A sub-agent that writes directly to + `.oh/knowledge/` is out of scope and may be reverted. +- **Index reflects the tracked entry set**: `.oh/knowledge/README.md`'s Index + table is generated state owned by `lint`, sorted by `updated:` descending. + Never hand-edit it. ## When NOT to use -- A topic that is a **behavioral norm** ("always do X") → a rule/skill, not the wiki. -- A **session journal** entry ("this run showed Y") → the run's report, not the wiki. - A *recurring failure mode* the run revealed is different: that is a `kind: pattern` - entry, written by `compile`, named for the mode rather than the run. -- **Human-facing prose** → `docs/`, not the wiki (the wiki is LLM-readable synthesis). -- Full-text body search → direct `grep`; `query` is intentionally frontmatter-only. +- A topic that is a **behavioral norm** ("always do X") → a rule/skill, not + knowledge. +- A **session journal** entry ("this run showed Y") → the run's report. A + *recurring failure mode* the run revealed is different: that is a + `kind: pattern` entry, written by `compile`, named for the mode not the run. +- A **proposal decision record** → `.oh/evals/decisions/skill-impact.md`, not a + knowledge page. +- **Human-facing prose** → `docs/` (knowledge pages are LLM-readable synthesis). +- Full-text body search → direct `grep`; `query` is intentionally + frontmatter-only. ## See Also - `.oh/skills/wiki/references/schema.md` — canonical schema and authoring rules - `.oh/skills/wiki/references/ingest.md` · `query.md` · `lint.md` · `compile.md` — full procedures -- `.oh/skills/wiki/corpus/skill-impact.md` — the skill-change ledger the proposer reads -- `.oh/evals/probes/wiki-readme-index.sh` — drift guard for the tracked corpus index +- `.oh/skills/wiki/scripts/knowledge-impact.sh` — dependency-aware invalidation +- `.oh/knowledge/README.md` — the generated index +- `.oh/evals/decisions/skill-impact.md` — the skill-change ledger the proposer reads +- `.oh/evals/probes/wiki-readme-index.sh` — drift guard for the generated index diff --git a/.oh/skills/wiki/corpus/README.md b/.oh/skills/wiki/corpus/README.md deleted file mode 100644 index da8bf729..00000000 --- a/.oh/skills/wiki/corpus/README.md +++ /dev/null @@ -1,52 +0,0 @@ -# .oh/skills/wiki/corpus/ — LLM-authored knowledge base for the Open Harness orchestrator - -This directory is the data corpus owned by the `/wiki` skill: structured entity pages compiled and maintained by the orchestrator following the Karpathy LLM Wiki pattern — facts and synthesis about recurring topics, loaded directly into context on demand rather than retrieved via vector search. Operate on it with `/wiki ingest`, `/wiki query`, and `/wiki lint`. - -## Gitignore-by-default - -Entries here are **local scratch by default** — `.oh/skills/wiki/corpus/*` is gitignored (`.gitignore`). The operator whitelists a curated entry into git with `git add -f <entry>`; once tracked, it stays tracked. This `README.md` and `raw/README.md` are tracked directory anchors. - -## Subdirectories - -| Subdirectory | Purpose | -| --- | --- | -| `raw/` | Immutable `WebFetch` snapshots of source URLs and fetched documents. Contents are gitignored by default; `raw/README.md` is tracked as the directory anchor, and cited snapshots may be force-added when a tracked wiki entry needs in-repo provenance. Each snapshot file is named `<yyyy-mm-dd>-<slug>.md` and cited in wiki entry `sources:` frontmatter. | - -## Conventions - -- **Entry files**: `.oh/skills/wiki/corpus/<slug>.md` — one topic per file, frontmatter-typed, `[[slug]]`-linked -- **Slug charset**: `[a-z0-9-]+` (lowercased kebab-case) -- **Word cap**: ≤ 600 words per entry; longer content splits to `.oh/skills/wiki/corpus/<parent>/<child>.md` -- **Cross-links**: `[[slug]]` syntax (Obsidian-style, grep-friendly) -- **Sources**: every entry must carry at least one `sources:` reference pointing to a `raw/` snapshot path -- **Confidence**: entries start as `provisional` (set by `/wiki ingest`); orchestrator promotes to `confirmed` or `deprecated` manually -- **Index**: the table below is regenerated by `/wiki lint` and guarded by `evals/probes/wiki-readme-index.sh` — do not edit it by hand - -## Canonical docs - -Schema rule, frontmatter spec, and all authoring conventions: `.oh/skills/wiki/references/schema.md` - -## Index - -| Slug | Title | Tags | Updated | -| --- | --- | --- | --- | -| wikiskill-experience-compilation | WikiSkill: Compiling Agent Experience into Persistent Knowledge (arXiv 2608.27454) | [skill-evolution, persistent-knowledge, continual-learning, harness-evolution, self-improvement, wiki, ablation, skill-transfer] | 2026-08-31 | -| sandbox-dependency-installs | Sandbox Dependency Installs | [sandbox, devcontainer, pnpm, dependencies, boot] | 2026-08-31 | -| recursive-self-improvement-survey | Recursive Self-Improvement in AI (survey, arXiv 2607.07663) | [rsi, self-improvement, verification-hierarchy, skill-libraries, harness-evolution, self-evaluation, model-collapse, capability-benchmark] | 2026-08-31 | -| recursive-language-models | Recursive Language Models | [rlm, context-as-environment, weighted-trajectories, agent-harness, llm-agents, self-consistency] | 2026-08-31 | -| pattern-wiki-ungated-check-drift | Report-only checks nothing gates on stop being run | [wiki, lint, evals, probes, report-only, drift, gating] | 2026-08-31 | -| pattern-wiki-external-model-over-mapping | Mapping an external model onto the harness reimports a tier it deleted | [wiki, ingest, architecture, external-sources, scope-creep, design-review] | 2026-08-31 | -| pattern-evals-unexercised-oracle | A probe that has never failed has an unverified oracle | [evals, probes, oracles, skipped, fault-injection, continual-learning] | 2026-08-31 | -| pattern-evals-prose-literal-pinning | Contract probes that pin multi-word prose break on reflow, not on drift | [evals, probes, contract-text, grep, false-failure, documentation] | 2026-08-31 | -| pattern-docs-prohibition-by-example | Documenting a forbidden literal by quoting it violates the rule | [docs, evals, probes, vocabulary, guards, self-reference] | 2026-08-31 | -| compose-env-boundary | Compose Environment Boundary | [compose, devcontainer, oh-json, cli, entrypoint, boundary, provisioning, sandbox] | 2026-08-31 | -| runtime-isolation-landscape | Runtime Isolation Landscape (2026) | [runtime, isolation, sandbox, gvisor, firecracker, kata, microvm, cloudflare, e2b, daytona, fly, modal] | 2026-08-27 | -| oh-cli-portable-lifecycle | oh CLI Portable Lifecycle | [cli, oh, lifecycle, standalone, init, sandbox, remote-fetch, execution-target] | 2026-08-27 | -| molt-agentic-reinforcement-learning | Molt: A Scalable PyTorch-Native Training Framework for Agentic Reinforcement Learning | [agentic-rl, training, readability, observability, trajectories, async, correctness, agent-harness, nvidia] | 2026-08-27 | -| managed-agents | Scaling Managed Agents: Decoupling the brain from the hands | [agents, meta-harness, sessions, sandbox, resilience, security, scaling, anthropic, model-evolution] | 2026-08-27 | -| fresh-machine-setup | Fresh-Machine Setup Flow | [setup, onboarding, installation, gateway, ssh, github, slack] | 2026-08-27 | -| crabbox-remote-exec-control-plane | Crabbox — Remote-Exec Control Plane | [runtime, sandbox, remote-execution, fan-out, control-plane, crabbox, cloudflare-workers, ssh, rsync] | 2026-08-27 | -| plan-vs-built-reconciliation | Plan-vs-Built Reconciliation | [spec-execute, evidence, merge-gate, comprehension, audit, task-folder, review] | 2026-08-24 | -| release-versioning | Release Versioning | [release, versioning, semver, calver, github-actions, ghcr, tags, workflow, package-json] | 2026-08-23 | -| audit-architecture | Audit Architecture | [audit, pr, workflow, safety, observability] | 2026-08-12 | -| document-ingestion | Local Document Ingestion | [wiki, ingestion, markitdown, documents, provenance, security] | 2026-07-18 | diff --git a/.oh/skills/wiki/corpus/raw/README.md b/.oh/skills/wiki/corpus/raw/README.md deleted file mode 100644 index 8d482c22..00000000 --- a/.oh/skills/wiki/corpus/raw/README.md +++ /dev/null @@ -1,15 +0,0 @@ -# .oh/skills/wiki/corpus/raw/ — Immutable WebFetch snapshots; entries cite via `sources:` frontmatter - -This directory holds immutable source captures: one file per fetch, never overwritten. Each file records the content of a URL at the moment `/wiki ingest` captured it. Wiki entity pages in `.oh/skills/wiki/corpus/` reference these snapshots through their `sources:` frontmatter field, providing a concrete provenance trail for every claim. - -## Conventions - -- **Naming**: `<yyyy-mm-dd>-<slug>.md` — UTC date of the fetch, plus the slug that identifies the topic -- **Format**: each file opens with `# Source: <url>` followed by the fetched body verbatim -- **Immutability**: files are never edited after creation; re-ingesting a URL appends a new dated snapshot rather than overwriting the existing one -- **Gitignore**: all files in this directory are gitignored (`.oh/skills/wiki/corpus/raw/*`); only this `README.md` is tracked (exempted by `!.oh/skills/wiki/corpus/raw/README.md`) -- **Retrieval**: snapshots are not queried directly — `/wiki query` operates on `.oh/skills/wiki/corpus/<slug>.md` entity pages; snapshots exist for audit and provenance only - -## Canonical docs - -Full schema and authoring conventions: `.oh/skills/wiki/references/schema.md` diff --git a/.oh/skills/wiki/references/compile.md b/.oh/skills/wiki/references/compile.md index dac45c25..790f0966 100644 --- a/.oh/skills/wiki/references/compile.md +++ b/.oh/skills/wiki/references/compile.md @@ -15,8 +15,9 @@ # Wiki Compile -Consolidate a `/retro` or `/spec retro` report into `kind: pattern` entries — the -harness's durable record of its own failure modes and working strategies. +Consolidate a `/retro` report into `kind: pattern` entries under +`.oh/knowledge/patterns/` — the harness's durable record of its own failure +modes and working strategies. This is the Wiki Maintainer role. It exists because the harness produces lessons and discards them: `/retro` nominates probe ids and writes nothing, and no skill owns the @@ -26,15 +27,16 @@ that gap without touching `/retro`, whose report-only contract is guarded by ## When to Use -- After `/retro` or `/spec retro` emits a report with at least one `supported` - hypothesis at `medium` or `high` confidence. +- After `/retro` (including its task-scoped form, `/retro --task <slug>`) emits a + report with at least one `supported` hypothesis at `medium` or `high` + confidence. - To record counter-evidence against a pattern a later run refuted. - Before `/builder` proposes a skill change, so the proposal has a pattern to cite. ## When NOT to Use - **`/wiki ingest`** — for an external source. `compile` never fetches a URL and - never writes to `corpus/raw/`. + never writes to `.oh/knowledge/raw/`. - **A per-run note.** One page per failure mode, never one per run. See the anti-patterns. - **An `inconclusive` hypothesis.** It is not knowledge yet. @@ -49,7 +51,7 @@ that gap without touching `/retro`, whose report-only contract is guarded by |----------|---------| | *(none)* | Consume the `/retro` report already present in the current session's context. This is the normal path — `/retro` writes no file, so its report exists only as terminal output. | | `--from <path>` | Read the report from a file: an operator-saved copy, or a sub-agent draft at `$TMPDIR/oh-wiki-drafts/<slug>.md`. | -| `--task <slug>` | Scope to `.oh/tasks/<slug>/`. Used to derive pinned-evidence `sources:` paths and to read `prd.md`, `progress.txt`, and `critique.md` as corroborating evidence. | +| `--task <slug>` | Scope to `.oh/tasks/<slug>/`. Used to derive pinned-evidence `sources:` paths and to read `prd.md`, `progress.txt`, and `evidence.md` as corroborating evidence. | | `--dry-run` | Print the proposed create-or-patch for each target page. Write nothing. | The interface is locked; adding a flag requires editing this reference and @@ -90,7 +92,7 @@ GOOD pattern-eval-probe-provenance-decay BAD pattern-2026-08-31-retro-findings ``` -The `<subsystem>` token is the **corpus's** subsystem vocabulary — the prefix a +The `<subsystem>` token is the **knowledge base's** subsystem vocabulary — the prefix a reader would grep for (`evals`, `wiki`, `docs`, `spec`) — not `/retro`'s five-lens taxonomy, which names where a signal was *noticed* rather than what the page is about. A lesson noticed through the continual-learning lens about probe @@ -105,7 +107,7 @@ directions, they are two. Enumerate existing patterns before writing: ```bash -ls .oh/skills/wiki/corpus/pattern-*.md 2>/dev/null +ls .oh/knowledge/patterns/pattern-*.md 2>/dev/null ``` If a page for that failure mode exists, **patch it**. A run that surfaces three @@ -114,7 +116,7 @@ lessons about one failure mode produces one patch, not three pages. ### 4. Create or patch **Create** follows the pattern body layout in -`.oh/skills/wiki/references/schema.md` § 2: `kind: pattern`, `confidence: +`.oh/skills/wiki/references/schema.md` § 3: `kind: pattern`, `confidence: provisional`, a required `## Relevant Source Files`, and `## Detail` carrying `**Symptom.**`, `**Root cause.**`, and `**Workaround.**` as bold leads. @@ -131,19 +133,21 @@ the per-session journal tier the harness deliberately removed, wearing a new nam and would launder around `/retro`'s report-only contract. **Patch** applies the body-merge strategy in -`.oh/skills/wiki/references/schema.md` § 7 as amended by § 7a. This reference does -not restate those steps and must not diverge from them. The load-bearing part of -§ 7a: `**Workaround.**` is append-only, and a workaround shown not to work is +`.oh/skills/wiki/references/schema.md` § 11 as amended by § 11a. This reference +does not restate those steps and must not diverge from them. The load-bearing +part of § 11a: `**Workaround.**` is append-only, and a workaround shown not to work is annotated `(superseded YYYY-MM-DD, SI-nnnn)` rather than deleted. ### 5. Promote and reindex -Pattern pages are always force-added. An untracked pattern page is invisible -provenance, and it is the corpus's only durable record of what a rejected cycle -taught. +Pattern pages are always staged. An untracked pattern page is invisible +provenance, and it is the knowledge base's only durable record of what a rejected +cycle taught. `.oh/knowledge/patterns/` is tracked by default +(`schema.md` § 2), so a plain `git add` is enough — the `-f` the gitignored +corpus used to need is gone with it. ```bash -git add -f .oh/skills/wiki/corpus/pattern-<name>.md +git add .oh/knowledge/patterns/pattern-<name>.md ``` Then regenerate the index by running `/wiki lint` (non-dry-run) and verify: @@ -152,7 +156,7 @@ Then regenerate the index by running `/wiki lint` (non-dry-run) and verify: bash .oh/evals/probes/wiki-readme-index.sh ``` -`compile` never hand-edits `.oh/skills/wiki/corpus/README.md` — `lint` owns it. +`compile` never hand-edits `.oh/knowledge/README.md` — `lint` owns it. ### 6. Report @@ -180,7 +184,7 @@ The `.oh/memory` tier was removed as a concept (`CHANGELOG.md`) because it held entry per session, keyed by date, with no consumer. Every structural property here is the opposite: -| `.oh/memory` (deleted) | `corpus/pattern-*.md` | +| `.oh/memory` (deleted) | `.oh/knowledge/patterns/pattern-*.md` | |---|---| | One entry per session, keyed by date | One page per **failure mode**, keyed by subsystem and mode | | Grew with every run | Grows only when a run teaches something not already recorded | @@ -196,28 +200,28 @@ rather than appended to, and cited by a skill proposal, passes it. - **One page per run** — the single failure mode of this subcommand. A page named for a date is a journal entry. Name it for the failure mode and patch on repeat. -- **Snapshotting the retro report into `corpus/raw/`** — see § 4. `raw/` is for - external sources; `/retro` output is ephemeral by contract. +- **Snapshotting the retro report into `.oh/knowledge/raw/`** — see § 4. `raw/` + is for external sources; `/retro` output is ephemeral by contract. - **Compiling an `inconclusive` hypothesis** — the report already judged it not to be - knowledge. Compiling it launders a guess into the corpus. + knowledge. Compiling it launders a guess into the knowledge base. - **Deleting or blanking a pattern page because the change it motivated was - rejected** — forbidden by `.oh/skills/wiki/references/schema.md` § 8. That + rejected** — forbidden by `.oh/skills/wiki/references/schema.md` § 12. That knowledge is the rejected cycle's entire output. -- **Replacing `## Detail` wholesale** — that is § 7 behavior for source pages. For a - pattern it erases accumulated failure knowledge; § 7a governs instead. -- **Restating the merge steps here** — § 7 and § 7a own them. A second copy will - drift. +- **Replacing `## Detail` wholesale** — that is § 11 behavior for entity pages. + For a pattern it erases accumulated failure knowledge; § 11a governs instead. +- **Restating the merge steps here** — § 11 and § 11a own them. A second copy + will drift. - **Retired audit vocabulary in pattern prose** — the token list enforced by - `.oh/evals/probes/audit-stale-references.sh` covers every tracked file, this corpus - included. Read that probe's pattern before writing about an audit subsystem, and + `.oh/evals/probes/audit-stale-references.sh` covers every tracked file, this + knowledge base included. Read that probe's pattern before writing about an audit subsystem, and use the current route names. - **Skipping the reindex** — a new tracked pattern page without a regenerated `README.md` is an immediate `wiki-readme-index.sh` regression. ## See Also -- `.oh/skills/wiki/references/schema.md` — § 2 pattern layout and placement, § 5 confidence, § 7a merge amendment, § 8 persistence invariant +- `.oh/skills/wiki/references/schema.md` — § 2 layout and the tracked boundary, § 3 pattern body layout, § 8 confidence, § 11a merge amendment, § 12 persistence invariant - `.oh/skills/wiki/references/query.md` — the `--patterns` read path - `.oh/skills/wiki/references/lint.md` — index regeneration and the health checks - `.oh/skills/retro/SKILL.md` — the report this subcommand consumes; report-only by contract -- `.oh/skills/wiki/corpus/skill-impact.md` — where the proposal a pattern motivates is recorded +- `.oh/evals/decisions/skill-impact.md` — where the proposal a pattern motivates is recorded diff --git a/.oh/skills/wiki/references/concurrent-ingest-worktrees.md b/.oh/skills/wiki/references/concurrent-ingest-worktrees.md index 7921a14a..f25e1ed6 100644 --- a/.oh/skills/wiki/references/concurrent-ingest-worktrees.md +++ b/.oh/skills/wiki/references/concurrent-ingest-worktrees.md @@ -9,10 +9,10 @@ Use this reference when an `add to wiki` request arrives while the main checkout - `git fetch origin` - `git worktree add .worktrees/feat/<short-slug>-wiki -b feat/<short-slug>-wiki origin/development` 3. Run the normal ingest workflow from inside that worktree: - - create `.oh/skills/wiki/corpus/raw/<date>-<slug>.md` as local provenance - - create/update `.oh/skills/wiki/corpus/<slug>.md` - - regenerate `.oh/skills/wiki/corpus/README.md` via `/wiki lint` or the atomic fallback -4. Commit and push only the tracked deliverables, normally `.oh/skills/wiki/corpus/<slug>.md` and `.oh/skills/wiki/corpus/README.md`. + - create `.oh/knowledge/raw/<date>-<slug>.md` as local provenance + - create/update `.oh/knowledge/source/<slug>.md` + - regenerate `.oh/knowledge/README.md` via `/wiki lint` or the atomic fallback +4. Commit and push only the tracked deliverables, normally `.oh/knowledge/source/<slug>.md` and `.oh/knowledge/README.md`. 5. In the final report, name the worktree path, branch, commit, and note any gitignored provenance files. ## Why diff --git a/.oh/skills/wiki/references/github-repo-research-wiki.md b/.oh/skills/wiki/references/github-repo-research-wiki.md index f83d7773..1a090912 100644 --- a/.oh/skills/wiki/references/github-repo-research-wiki.md +++ b/.oh/skills/wiki/references/github-repo-research-wiki.md @@ -10,7 +10,7 @@ Produce a bounded wiki entry that is more useful than a README summary: capture Prefer GitHub API and raw-content reads when a clone is unnecessary or blocked by approval/sandbox policy. -Minimum source packet for `.oh/skills/wiki/corpus/raw/<date>-<slug>.md`: +Minimum source packet for `.oh/knowledge/raw/<date>-<slug>.md`: - `gh repo view owner/repo --json nameWithOwner,description,stargazerCount,forkCount,defaultBranchRef,licenseInfo,repositoryTopics,pushedAt,createdAt,updatedAt,url,homepageUrl` - README raw content or a concise README summary @@ -21,7 +21,7 @@ Minimum source packet for `.oh/skills/wiki/corpus/raw/<date>-<slug>.md`: ## Synthesis shape -In `.oh/skills/wiki/corpus/<slug>.md`, include: +In `.oh/knowledge/source/<slug>.md`, include: 1. What it is and why it matters. 2. Implementation mechanics that a future agent can reuse. diff --git a/.oh/skills/wiki/references/ingest.md b/.oh/skills/wiki/references/ingest.md index bd078bd9..1723300d 100644 --- a/.oh/skills/wiki/references/ingest.md +++ b/.oh/skills/wiki/references/ingest.md @@ -7,7 +7,7 @@ # Wiki Ingest -Snapshot a source and write or update a wiki entity page. This is the only authorized path for writing to `.oh/skills/wiki/corpus/`. Sub-agents may not call this skill directly for write operations — they propose drafts to `$TMPDIR/oh-wiki-drafts/<slug>.md` and the orchestrator promotes via `--from-draft`. +Snapshot a source and write or update a wiki entity page. This is the only authorized path for writing to `.oh/knowledge/`. Sub-agents may not call this skill directly for write operations — they propose drafts to `$TMPDIR/oh-wiki-drafts/<slug>.md` and the orchestrator promotes via `--from-draft`. The canonical schema, slug derivation rules, and body-merge strategy all live in `.oh/skills/wiki/references/schema.md`. This skill defers to those rules — it does not redefine them. @@ -54,7 +54,7 @@ No other forms are documented or supported. `argument-hint` frontmatter above en - `/wiki query` — for searching and reading existing entries into context. - `/wiki lint` — for health checks, index regeneration, stale/orphan reporting. -- Direct `Edit` tool writes to `.oh/skills/wiki/corpus/<slug>.md` — use only for manual `confidence` field upgrades or small factual corrections that do not require a new snapshot. +- Direct `Edit` tool writes to `.oh/knowledge/source/<slug>.md` — use only for manual `confidence` field upgrades or small factual corrections that do not require a new snapshot. ## Instructions @@ -67,19 +67,19 @@ Parse `$ARGUMENTS` to determine the invocation form: Extract `--slug <override>` and `--allow-stale` flags if present. -### 2. Ensure .oh/skills/wiki/corpus/raw/ exists +### 2. Ensure .oh/knowledge/raw/ exists Before any file write, run: ```bash -mkdir -p .oh/skills/wiki/corpus/raw/ +mkdir -p .oh/knowledge/raw/ ``` -`.oh/skills/wiki/corpus/raw/` is gitignored (only `.oh/skills/wiki/corpus/raw/README.md` is tracked). On a fresh clone the directory may not exist. This step is mandatory — never assume the directory is present. +`.oh/knowledge/raw/` is tracked, but a fresh clone that has never ingested anything may still be missing a sub-path this route writes to. This step is mandatory — never assume the directory is present. ### 3. Slug derivation -Slug derivation follows `.oh/skills/wiki/references/schema.md` § 3 verbatim. Summary for reference (the rule document is authoritative): +Slug derivation follows `.oh/skills/wiki/references/schema.md` § 6 verbatim. Summary for reference (the rule document is authoritative): 1. **URL — last non-UUID segment**: take the URL path, strip trailing slashes, split on `/`, take the last segment. If that segment matches `/^[0-9a-f-]{8,}$/i` (UUID or bare hash), proceed to rule 3. - `https://example.com/foo/bar` → `bar` @@ -109,22 +109,22 @@ If `--slug <override>` is provided, use it directly (still validate charset). 1. WebFetch the URL to retrieve the page body. - For LinkedIn/social pages, inspect embedded metadata and JSON-LD (`articleBody`, `headline`, `comment`, `og:description`, `twitter:description`) when the visible DOM is gated or duplicated. Capture useful comments only when they materially clarify the source claim; keep the wiki page bounded and point to the raw snapshot for the full capture. - If a material comment or metadata field contains the actual referenced artifact (for example, "Link to the prompt" pointing to a gist/raw file), fetch that artifact too and include a concise quoted copy or excerpt in the raw snapshot. Synthesize the wiki entry from both the social wrapper and the linked primary artifact; cite the social post as the source and mention the linked artifact in Detail when it carries the technique. -2. Normalize the displayed source URL before writing synthesized .oh/skills/wiki/corpus/log text: +2. Normalize the displayed source URL before writing synthesized .oh/knowledge/log text: - Strip common tracking-only query params (`utm_*`, `rcm`, `fbclid`, `gclid`, etc.) when they are not needed for retrieval. - If preserving a raw fetched URL for provenance, redact secret-like/tracking values in human-facing summaries/logs (e.g. `rcm=[REDACTED]`). 3. Get today's UTC date: ```bash TODAY=$(date -u +%Y-%m-%d) ``` -4. Ensure `.oh/skills/wiki/corpus/raw/` exists (§ 2). -5. Write snapshot to `.oh/skills/wiki/corpus/raw/<yyyy-mm-dd>-<slug>.md`: +4. Ensure `.oh/knowledge/raw/` exists (§ 2). +5. Write snapshot to `.oh/knowledge/raw/<yyyy-mm-dd>-<slug>.md`: ``` # Source: <url> <fetched body> ``` - The header line `# Source: <url>` is mandatory. Prefer the normalized/redacted URL in this header unless the exact retrieval URL is essential to reproduce the fetch. The fetched body follows on the next line after a blank line. Snapshots are immutable once written — do not overwrite an existing snapshot. If `.oh/skills/wiki/corpus/raw/<today>-<slug>.md` already exists, generate a unique path (e.g., append `-2`, `-3`). -6. Proceed to § 6 (write or update `.oh/skills/wiki/corpus/<slug>.md`). + The header line `# Source: <url>` is mandatory. Prefer the normalized/redacted URL in this header unless the exact retrieval URL is essential to reproduce the fetch. The fetched body follows on the next line after a blank line. Snapshots are immutable once written — do not overwrite an existing snapshot. If `.oh/knowledge/raw/<today>-<slug>.md` already exists, generate a unique path (e.g., append `-2`, `-3`). +6. Proceed to § 6 (write or update `.oh/knowledge/source/<slug>.md`). #### 4b. File path ingest @@ -138,8 +138,8 @@ For the existing file route (unchanged): 1. Read the file at `<path>`. 2. Get today's UTC date: `TODAY=$(date -u +%Y-%m-%d)`. -3. Ensure `.oh/skills/wiki/corpus/raw/` exists (§ 2). -4. Write snapshot to `.oh/skills/wiki/corpus/raw/<yyyy-mm-dd>-<basename>.md` (same format as URL ingest, but use `# Source: <path>` as the header). The snapshot filename uses the basename of the path unless `--slug` overrides the slug; if `--slug` is used, the snapshot filename uses the slug. +3. Ensure `.oh/knowledge/raw/` exists (§ 2). +4. Write snapshot to `.oh/knowledge/raw/<yyyy-mm-dd>-<basename>.md` (same format as URL ingest, but use `# Source: <path>` as the header). The snapshot filename uses the basename of the path unless `--slug` overrides the slug; if `--slug` is used, the snapshot filename uses the slug. 5. Proceed to § 6. #### 4b-i. Local document normalization (MarkItDown pilot) @@ -183,7 +183,7 @@ SOURCE_SIZE=$(stat -Lc '%s' -- "$SOURCE") || return 1 Use one collision suffix for the original and Markdown snapshot. Try `<today>-<slug>.<ext>` plus `<today>-<slug>.md`, then `-2` on both, then `-3`, and so on. A candidate is available only when neither artifact nor its reservation exists. Reserve it with an atomic `mkdir` in `corpus/raw/`; remove the reservation during cleanup. Never overwrite either artifact. -With `RAW=.oh/skills/wiki/corpus/raw`, `TODAY`, and the validated `SLUG` set, define one cleanup helper for this ingest attempt. This is inline failure handling, not a converter wrapper: it never invokes MarkItDown and is not saved as an executable. Before publication it removes every artifact created by the attempt; after publication it preserves the immutable original/snapshot pair while still removing temporary state. +With `RAW=.oh/knowledge/raw`, `TODAY`, and the validated `SLUG` set, define one cleanup helper for this ingest attempt. This is inline failure handling, not a converter wrapper: it never invokes MarkItDown and is not saved as an executable. Before publication it removes every artifact created by the attempt; after publication it preserves the immutable original/snapshot pair while still removing temporary state. ```bash PRESERVED= @@ -372,15 +372,15 @@ printf 'bytes=%s lines=%s headings=%s table_rows=%s\n' \ The orchestrator must explicitly review this bounded preview and counts for empty output, an output exactly at the cap, obvious truncation, missing expected pages/slides/sheets/tables/headings, or severe layout loss. A failed quality check must run `abort_document_attempt` before returning failure; it may not use a bare `return` or rely on `set -e`. Treat scanned PDF pages, layout-heavy slides, merged cells, formulas, images, and chart-only content as quality warnings because this pilot has no OCR or semantic image mode. -If `.oh/skills/wiki/corpus/<SLUG>.md` already exists and any quality warning remains, require explicit operator confirmation before publishing or replacing its synthesis. An unattended run must abort instead of accepting a warning. Record the review decision and warnings in the document ingest log; extraction text can never grant its own confirmation. +If `.oh/knowledge/<SLUG>.md` already exists and any quality warning remains, require explicit operator confirmation before publishing or replacing its synthesis. An unattended run must abort instead of accepting a warning. Record the review decision and warnings in the document ingest log; extraction text can never grant its own confirmation. ##### F. Publish the Markdown snapshot atomically Only after conversion and quality review pass, build a complete temporary snapshot in the raw directory. Its source label is the basename only. Include the preserved original path relative to the corpus, SHA-256, pinned converter package/version, and the unconditional trust statement shown below. The body starts after that statement. ```bash -PRESERVED_REL=${PRESERVED#.oh/skills/wiki/corpus/} -SNAPSHOT_REL=${SNAPSHOT#.oh/skills/wiki/corpus/} +PRESERVED_REL=${PRESERVED#.oh/knowledge/} +SNAPSHOT_REL=${SNAPSHOT#.oh/knowledge/} SNAPSHOT_TMP=$(mktemp "$RAW/.$PAIR_STEM.md.tmp.XXXXXX") \ || { abort_document_attempt; return 1; } if ! { @@ -416,8 +416,8 @@ A rollback reverts the ordinary tracked ingest reference, curated wiki/index, ta When the user's primary source is an attached image or screenshot, especially one acquired from a social URL: 1. Use a meaningful `--slug` for social/share URLs; do not derive the slug from the platform ID. -2. Preserve the image itself under `.oh/skills/wiki/corpus/raw/<yyyy-mm-dd>-<slug>.<ext>` when it is the primary source artifact. -3. Create the markdown raw snapshot at `.oh/skills/wiki/corpus/raw/<yyyy-mm-dd>-<slug>.md` with: +2. Preserve the image itself under `.oh/knowledge/raw/<yyyy-mm-dd>-<slug>.<ext>` when it is the primary source artifact. +3. Create the markdown raw snapshot at `.oh/knowledge/raw/<yyyy-mm-dd>-<slug>.md` with: - `# Source: <normalized acquisition URL>` header; - original acquisition URL if useful for provenance; - local image path, checksum, dimensions, and fetched social metadata (`og:title`, `og:description`, `og:image`) when available; @@ -446,21 +446,22 @@ When the source URL is a GitHub repository and the user asks to "study", "index ``` - With `--allow-stale`: log a warning and continue. 5. Read the draft file content. -6. The draft file is the "source" for § 6. Snapshot path is the draft file path itself (use it as the `sources:` entry, not a new `.oh/skills/wiki/corpus/raw/` file — drafts are already a captured artifact). +6. The draft file is the "source" for § 6. Snapshot path is the draft file path itself (use it as the `sources:` entry, not a new `.oh/knowledge/raw/` file — drafts are already a captured artifact). 7. Proceed to § 6. -### 6. Write or update .oh/skills/wiki/corpus/<slug>.md +### 6. Write or update .oh/knowledge/source/<slug>.md -Check whether `.oh/skills/wiki/corpus/<slug>.md` already exists. +Check whether `.oh/knowledge/source/<slug>.md` already exists. #### 6a. New entry (create) -Write `.oh/skills/wiki/corpus/<slug>.md` with valid frontmatter per `.oh/skills/wiki/references/schema.md` § 2: +Write `.oh/knowledge/source/<slug>.md` with valid frontmatter per `.oh/skills/wiki/references/schema.md` § 3: ```yaml --- title: "<Derived or provided title>" slug: <slug> +kind: external # external for an outside source; repo for this repository tags: [] created: <TODAY> updated: <TODAY> @@ -492,7 +493,9 @@ Field notes: - `tags`: derive from the source content. Leave as `[]` if no clear tags are evident. - `created`: set to today's UTC date. Never updated after initial creation. - `updated`: set to today's UTC date. -- `sources`: list the new snapshot path (relative to `.oh/skills/wiki/corpus/`, e.g., `raw/2026-05-24-karpathy-llm-wiki.md`). For `--from-draft`, use the draft file path. +- `kind`: `external` when the page synthesizes an outside source (the `raw/` snapshot is its provenance); `repo` when the page synthesizes **this repository**. A `kind: repo` page replaces the snapshot with the repository paths it depends on and adds `verified_at: <commit>` — it never snapshots this repository's own source into `raw/` (`schema.md` § 3). Promoting a page out of `.oh/knowledge/local/` uses this same route and lands the result in `.oh/knowledge/source/`. +- `verified_at`: required for `kind: repo` only. Set it to `git rev-parse HEAD` at the moment the claims were checked; it is what `knowledge-impact.sh` measures freshness against (`schema.md` § 5). +- `sources`: for `kind: external`, the new snapshot path relative to `.oh/knowledge/` (e.g. `raw/2026-05-24-karpathy-llm-wiki.md`). For `kind: repo`, the repository-relative paths or globs the page depends on. For `--from-draft`, use the draft file path. - `confidence`: always `provisional` on creation. Never set to `confirmed` autonomously — that is the orchestrator's manual action. - `## Relevant Source Files`: include for repo architecture/harness topics; omit only for simple external-concept entries with no local source footprint. - `## System Relationships`: include for pipeline/runtime/architecture entries; omit only when the topic has no meaningful component relationship to show. @@ -500,7 +503,7 @@ Field notes: #### 6b. Existing entry (update) -When `.oh/skills/wiki/corpus/<slug>.md` already exists, apply the body-merge strategy from `.oh/skills/wiki/references/schema.md` § 7 verbatim: +When `.oh/knowledge/source/<slug>.md` already exists, apply the body-merge strategy from `.oh/skills/wiki/references/schema.md` § 11 verbatim: 1. **Replace `## Summary`**: overwrite the entire `## Summary` section (from `## Summary` heading to the next `##` heading) with the new summary from the fresh source. 2. **Replace `## Detail`**: overwrite the entire `## Detail` section in-place with new detail prose. @@ -510,21 +513,21 @@ When `.oh/skills/wiki/corpus/<slug>.md` already exists, apply the body-merge str 6. **Do NOT touch `created:`**: `created:` is immutable after initial creation. 7. **Do NOT concatenate bodies**: the prior `## Summary` and `## Detail` content is replaced, not appended. The entry stays ≤ 600 words. -Use the `Edit` tool to perform in-place section replacement. Extract the canonical frontmatter first using the locked command from `.oh/skills/wiki/references/schema.md` § 6: +Use the `Edit` tool to perform in-place section replacement. Extract the canonical frontmatter first using the locked command from `.oh/skills/wiki/references/schema.md` § 9: ```bash -awk '/^---$/{f=!f; next} f{print}' .oh/skills/wiki/corpus/<slug>.md +awk '/^---$/{f=!f; next} f{print}' .oh/knowledge/source/<slug>.md ``` ### 7. Regenerate the wiki index when the entry is part of a deliverable -`.oh/skills/wiki/corpus/README.md` is the human/LLM index and is owned by `/wiki lint`, not by hand edits. After creating or updating a tracked `.oh/skills/wiki/corpus/<slug>.md` entry for a user-facing deliverable (especially when the user asked to "add to the wiki", or when you will commit/push the wiki change), run `/wiki lint` or follow its atomic regeneration protocol so the index includes the new entry before finalizing. Remember that `.oh/skills/wiki/corpus/raw/*` snapshots are gitignored by design; the tracked deliverable is usually `.oh/skills/wiki/corpus/<slug>.md` plus the regenerated `.oh/skills/wiki/corpus/README.md`, while the raw snapshot remains local provenance unless policy changes. +`.oh/knowledge/README.md` is the human/LLM index and is owned by `/wiki lint`, not by hand edits. After creating or updating a tracked `.oh/knowledge/source/<slug>.md` entry for a user-facing deliverable (especially when the user asked to "add to the wiki", or when you will commit/push the wiki change), run `/wiki lint` or follow its atomic regeneration protocol so the index includes the new entry before finalizing. `.oh/knowledge/source/`, `.oh/knowledge/patterns/`, and `.oh/knowledge/raw/` are all tracked (`schema.md` § 2), so the deliverable is the entity page, its snapshot, and the regenerated `.oh/knowledge/README.md` — a plain `git add` stages all three. -If you cannot run the full `/wiki lint` skill, do not hand-maintain the table casually: enumerate the **git-tracked** entry set with the pathspec in `/wiki lint` § 9a, extract frontmatter with the canonical `awk '/^---$/{f=!f; next} f{print}'` command, sort by `updated:` descending, write `.oh/skills/wiki/corpus/README.md.tmp`, validate it is non-empty and contains `| Slug | Title | Tags | Updated |`, then atomically rename it to `.oh/skills/wiki/corpus/README.md`. +If you cannot run the full `/wiki lint` skill, do not hand-maintain the table casually: enumerate `.oh/knowledge/source/*.md` and `.oh/knowledge/patterns/*.md` as `/wiki lint` § 9a does, extract frontmatter with the canonical `awk '/^---$/{f=!f; next} f{print}'` command, sort by `updated:` descending, write `.oh/knowledge/README.md.tmp`, validate it is non-empty and contains `| Slug | Title | Tags | Updated |`, then atomically rename it to `.oh/knowledge/README.md`. ### 8. Orchestrator-only write gate -This skill's write operations (`.oh/skills/wiki/corpus/raw/` snapshots and `.oh/skills/wiki/corpus/<slug>.md` writes) are **orchestrator-only**. The orchestrator is the only session authorized to write to tracked wiki surfaces. +This skill's write operations (`.oh/knowledge/raw/` snapshots and `.oh/knowledge/source/<slug>.md` writes) are **orchestrator-only**. The orchestrator is the only session authorized to write to tracked knowledge surfaces. Sub-agents may propose new entries by writing drafts to `$TMPDIR/oh-wiki-drafts/<slug>.md`. The draft format is free-form markdown (no required frontmatter). The orchestrator then reviews and promotes via: @@ -532,26 +535,26 @@ Sub-agents may propose new entries by writing drafts to `$TMPDIR/oh-wiki-drafts/ /wiki ingest --from-draft <slug> ``` -This gate preserves the concurrency invariant: only the orchestrator writes to tracked knowledge surfaces. A sub-agent that bypasses this by writing directly to `.oh/skills/wiki/corpus/` is out of scope — the orchestrator may revert such writes. +This gate preserves the concurrency invariant: only the orchestrator writes to tracked knowledge surfaces. A sub-agent that bypasses this by writing directly to `.oh/knowledge/` is out of scope — the orchestrator may revert such writes. ### 9. Report provenance Report these values in the run's terminal output. They are the ingest's audit -trail; the `.oh/memory` log they used to be written to was deleted. +trail; the per-session log tier they used to be written to was deleted. - **Snapshot-Path**: path to the snapshot written (relative to harness root). Use `—` on STALE or on a FAIL before atomic publication; when synthesis/log/index work fails after publication, record the retained snapshot path instead of hiding immutable provenance. - **Preserved-Artifact**, **SHA-256**, **Converter**, **Extraction-Trust**, and **Quality-Review**: required for successful pilot document ingests. On FAIL, record every value reached; after atomic publication both retained artifact paths and checksum are mandatory, while unavailable pre-publication values use `—`. Omit these fields for URL, ordinary text, image, repository-study, and draft-promotion routes so their existing report shape remains unchanged. -- **Slug-Created** / **Slugs-Updated**: the slug if a new `.oh/skills/wiki/corpus/<slug>.md` was created, and comma-separated slugs for pages updated; `—` when neither applies. +- **Slug-Created** / **Slugs-Updated**: the slug if a new `.oh/knowledge/source/<slug>.md` was created, and comma-separated slugs for pages updated; `—` when neither applies. - **Result**: `OP` for a completed ingest (create or update), `STALE` if the run exited on the staleness gate without `--allow-stale`, `FAIL` for any other error that prevented wiki writes. ## Anti-patterns -- **Monolithic ingest scripts when a safety gate is likely** — avoid bundling network fetch, raw snapshot write, wiki synthesis, and log append into one large `execute_code` call. If approval or shell-safety friction appears, split the ingest into auditable steps: fetch/snapshot with a small `terminal` command, create or update `.oh/skills/wiki/corpus/<slug>.md` with `write_file`/`patch`, then regenerate the index separately. The invariant is the same (raw snapshot + bounded synthesized entry + log), but smaller tool calls are easier to approve, verify, and recover. +- **Monolithic ingest scripts when a safety gate is likely** — avoid bundling network fetch, raw snapshot write, wiki synthesis, and log append into one large `execute_code` call. If approval or shell-safety friction appears, split the ingest into auditable steps: fetch/snapshot with a small `terminal` command, create or update `.oh/knowledge/source/<slug>.md` with `write_file`/`patch`, then regenerate the index separately. The invariant is the same (raw snapshot + bounded synthesized entry + log), but smaller tool calls are easier to approve, verify, and recover. - **Consent-gated write recovery** — if a multi-file ingest is blocked by a consent/approval gate, report exactly which files would be written and wait for explicit approval. Prefer splitting the approved recovery into the smallest direct file operations (`write_file` for the wiki entry/raw snapshots) rather than wrapping all writes in `execute_code`; approval state may not carry cleanly into a monolithic script retry. If the tool explicitly says not to retry or not to attempt the same outcome via another tool, stop and report the blocker. Otherwise, after approval, complete the intended ingest and verify the synthesized wiki entry, and the raw snapshot size before declaring success. Do not treat the pre-approval fetch metadata as an ingest; no wiki operation is complete until raw snapshot + entity page both exist. -- **Writing directly to `.oh/skills/wiki/corpus/` from a sub-agent context** — always use the draft path + `--from-draft` promotion. The orchestrator is the sole writer. +- **Writing directly to `.oh/knowledge/` from a sub-agent context** — always use the draft path + `--from-draft` promotion. The orchestrator is the sole writer. - **Resolving `--from-draft` by mtime** — pick the draft by its embedded date, not by filesystem mtime and not by assuming today. - **Using mtime for stale detection** — mtime is unreliable across git checkouts and Docker volume remounts. Always derive staleness from the ISO date in the parent directory name. -- **Omitting `mkdir -p .oh/skills/wiki/corpus/raw/`** — `.oh/skills/wiki/corpus/raw/` is gitignored and may not exist on a fresh clone. Always create it before writing. +- **Omitting `mkdir -p .oh/knowledge/raw/`** — the directory may be absent in a tree that has never ingested. Always create it before writing. - **Concatenating bodies on update** — the body-merge strategy replaces `## Summary` and `## Detail` in-place; it does not append. Bodies that grow unbounded exceed the 600-word cap and dilute the entry. - **Setting `confidence` to anything other than `provisional` on create** — the orchestrator manually upgrades to `confirmed`; `/wiki lint` flags `deprecated` candidates; `/wiki ingest` never sets either of those values. - **Touching `created:` on update** — `created:` is immutable. Only `updated:` changes on re-ingest. @@ -563,10 +566,13 @@ trail; the `.oh/memory` log they used to be written to was deleted. | Rule | Section | What this skill defers to it for | |------|---------|----------------------------------| -| `.oh/skills/wiki/references/schema.md` | § 2 Entry schema | Frontmatter fields, body layout, ≤600-word cap | -| `.oh/skills/wiki/references/schema.md` | § 3 Slug derivation | URL/path-to-slug algorithm; UUID/hash error path | -| `.oh/skills/wiki/references/schema.md` | § 6 Frontmatter extraction | Canonical `awk` command for reading frontmatter on update | -| `.oh/skills/wiki/references/schema.md` | § 7 Body-merge strategy | Exact merge steps for existing entry updates | +| `.oh/skills/wiki/references/schema.md` | § 2 Layout | Directory-as-kind boundary, tracked vs. `local/`, path resolution | +| `.oh/skills/wiki/references/schema.md` | § 3 Entry schema | Frontmatter fields, the three kinds, body layout, word cap | +| `.oh/skills/wiki/references/schema.md` | § 4 Provenance forms | Which `sources:` entries expire and which are immutable | +| `.oh/skills/wiki/references/schema.md` | § 5 Freshness | `verified_at:` and source-change invalidation | +| `.oh/skills/wiki/references/schema.md` | § 6 Slug derivation | URL/path-to-slug algorithm; UUID/hash error path | +| `.oh/skills/wiki/references/schema.md` | § 9 Frontmatter extraction | Canonical `awk` command for reading frontmatter on update | +| `.oh/skills/wiki/references/schema.md` | § 11 Body-merge strategy | Exact merge steps for existing entry updates | ### Smoke test (manual QA only) @@ -577,7 +583,7 @@ This smoke test is not run in CI. Run it manually after the skill is committed, ``` Expected outcome: -- `.oh/skills/wiki/corpus/raw/<today>-karpathy-llm-wiki.md` exists with `# Source: https://gist.github.com/...` header. -- `.oh/skills/wiki/corpus/karpathy-llm-wiki.md` exists with valid frontmatter, `confidence: provisional`, and the snapshot path in `sources:`. +- `.oh/knowledge/raw/<today>-karpathy-llm-wiki.md` exists with `# Source: https://gist.github.com/...` header. +- `.oh/knowledge/source/karpathy-llm-wiki.md` exists with valid frontmatter, `kind: external`, `confidence: provisional`, and the snapshot path in `sources:`. This smoke test MUST run and its commit must land before US-003's smoke test runs. diff --git a/.oh/skills/wiki/references/lint.md b/.oh/skills/wiki/references/lint.md index 8bb2235e..7644b510 100644 --- a/.oh/skills/wiki/references/lint.md +++ b/.oh/skills/wiki/references/lint.md @@ -1,40 +1,68 @@ # /wiki lint — reference -> Full procedure for the `lint` subcommand of the `/wiki` dispatcher, lifted -> from the former standalone `/wiki lint` skill during the wiki consolidation. -> The dispatcher (`.oh/skills/wiki/SKILL.md`) routes here when the first -> `$ARGUMENTS` token is `lint`. Canonical schema: `.oh/skills/wiki/references/schema.md`. - -# Wiki Lint - -Health-check the `.oh/skills/wiki/corpus/` corpus and regenerate `.oh/skills/wiki/corpus/README.md`. This is -Karpathy's "Lint + Maintain" operation adapted for Open Harness: surface -stale, deprecated, orphaned, and broken-link entries; regenerate the index -atomically. - -The canonical schema, frontmatter extraction command, cross-link convention, -and confidence lifecycle all live in `.oh/skills/wiki/references/schema.md`. This skill -defers to those rules — it does not redefine them. - -`.oh/skills/wiki/corpus/README.md` is generated state, not hand-authored inventory. The tier-A -`.oh/evals/probes/wiki-readme-index.sh` probe reconstructs the expected Index from -current `.oh/skills/wiki/corpus/*.md` frontmatter and fails when this skill's committed output -falls out of sync. +> Full procedure for the `lint` subcommand of the `/wiki` dispatcher. The +> dispatcher (`.oh/skills/wiki/SKILL.md`) routes here when the first +> `$ARGUMENTS` token is `lint`. Canonical schema: +> `.oh/skills/wiki/references/schema.md`. + +# Knowledge Lint + +Health-check `.oh/knowledge/` and regenerate `.oh/knowledge/README.md`. + +**Every check here is a correctness check.** A lint finding names something that +is actually wrong: a page that violates the schema, a source path that does not +resolve, a page whose declared dependencies moved under it, a link that goes +nowhere, or an index that no longer matches its inputs. Nothing in this list is a +matter of taste, and nothing reports a page for being unpopular or old. + +The canonical schema, extraction command, cross-link convention, provenance +forms, and confidence lifecycle all live in +`.oh/skills/wiki/references/schema.md`. This reference defers to those rules — it +does not redefine them. + +## Why this list is short + +A health check nobody gates on converges on never being run +(`[[pattern-wiki-ungated-check-drift]]`). Each surviving check below therefore +has a deterministic oracle in `.oh/evals/probes/` that fails on the *finding*, +not on the check having been run. This document is the procedure; the probes are +the enforcement: + +| Check | Oracle | +|---|---| +| 1 · schema validity | `.oh/evals/probes/wiki-kind-schema-contract.sh` | +| 2 · source/dependency paths resolve | `.oh/evals/probes/knowledge-source-freshness.sh` | +| 3 · source-change freshness | `.oh/evals/probes/knowledge-source-freshness.sh` | +| 4 · broken `[[...]]` links | `.oh/evals/probes/wiki-related-slugs.sh` | +| 5 · broken `related:` slugs | `.oh/evals/probes/wiki-related-slugs.sh` | +| 6 · generated index consistency | `.oh/evals/probes/wiki-readme-index.sh` | + +## What this check list deliberately dropped + +- **Orphan detection.** A queryable page with zero inbound `[[slug]]` references + is perfectly valid in a knowledge base this size. Reporting it as a health + finding produced a permanent non-zero count that readers learned to skip, + which cost the checks that mean something. +- **The 90-day stale rule as a validity test.** Age does not decide validity: a + page updated today is wrong one commit later if a source it depends on moved, + and a page untouched for a year is correct if nothing it cites changed. Check 3 + replaces it. Age survives only as the informational `last-reviewed` line. +- **The contradiction-detection stub.** It printed a fixed "not yet implemented" + string for the whole of its life and detected nothing. ## When to Use -- `/wiki lint` to regenerate `.oh/skills/wiki/corpus/README.md` and surface any health findings. -- `/wiki lint --dry-run` to preview what would change without writing. -- Periodically (manual cadence — no heartbeat cron in v1) to keep the corpus - healthy as entries accumulate. +- `/wiki lint` to regenerate `.oh/knowledge/README.md` and surface findings. +- `/wiki lint --dry-run` to preview without writing. +- After `/wiki ingest` or `/wiki compile` lands a page, so the index matches. ## When NOT to Use -- **`/wiki ingest`** — to add or update an entry. `/wiki lint` is read-only - except for `.oh/skills/wiki/corpus/README.md` regeneration. -- **`/wiki query`** — to search for a topic and read entries into context. -- **Direct `Edit` tool on `.oh/skills/wiki/corpus/README.md`** — `/wiki lint` owns this file; - hand-edits will be overwritten on the next run. +- **`/wiki ingest`** — to add or update an entry. `lint` is read-only except for + `.oh/knowledge/README.md`. +- **`/wiki query`** — to search and read entries into context. +- **Direct `Edit` on `.oh/knowledge/README.md`** — `lint` owns that file; hand + edits are overwritten on the next run. ## Argument Interface (locked) @@ -42,12 +70,8 @@ falls out of sync. /wiki lint [--dry-run] ``` -The argument interface is locked. Do not add flags or positional arguments -without editing this SKILL.md. - -- **No arguments**: run all checks, atomically regenerate `.oh/skills/wiki/corpus/README.md`. -- **`--dry-run`**: run all checks, print the proposed `.oh/skills/wiki/corpus/README.md` content - and any findings, but do NOT write `.oh/skills/wiki/corpus/README.md` or any other file. +- **No arguments**: run all checks, atomically regenerate the index. +- **`--dry-run`**: run all checks, print the proposed index, write nothing. ## Instructions @@ -61,491 +85,343 @@ if echo "$ARGUMENTS" | grep -q -- '--dry-run'; then fi ``` -All subsequent write operations (including the atomic `.oh/skills/wiki/corpus/README.md` -regeneration) are gated on `DRY_RUN=false`. In dry-run mode, print what -would be written; never write it. +Every write is gated on `DRY_RUN=false`. -### 2. Collect all wiki entry paths +### 2. Collect entry paths ```bash -HARNESS="${AUDIT_ROOT:-$(git rev-parse --show-toplevel)}" -HARNESS=$(cd "$HARNESS" && pwd -P) -WIKI_ENTRIES=() -for f in "$HARNESS"/.oh/skills/wiki/corpus/*.md; do - [ -f "$f" ] && WIKI_ENTRIES+=("$f") +ROOT="${AUDIT_ROOT:-$(git rev-parse --show-toplevel)}" +ROOT=$(cd "$ROOT" && pwd -P) +KNOWLEDGE="$ROOT/.oh/knowledge" + +ENTRIES=() +for f in "$KNOWLEDGE"/source/*.md "$KNOWLEDGE"/patterns/*.md; do + [ -f "$f" ] || continue + [ "$(basename "$f")" = "README.md" ] && continue + ENTRIES+=("$f") done -ENTRIES_COUNT=${#WIKI_ENTRIES[@]} +ENTRIES_COUNT=${#ENTRIES[@]} ``` -This enumerates `.oh/skills/wiki/corpus/*.md` directly — NOT via `.oh/skills/wiki/corpus/README.md` (the README is -the output of this skill, not its input). Sub-article files at -`.oh/skills/wiki/corpus/<parent>/<child>.md` are not matched by this glob; they are scoped for -a future iteration. `kind: pattern` entries are flat `corpus/<slug>.md` files by the -`.oh/skills/wiki/references/schema.md` § 2 placement rule, precisely because this glob does -not descend. +**One entry set.** `.oh/knowledge/source/` and `.oh/knowledge/patterns/` are +tracked by default (`schema.md` § 2), so the working tree and the git-tracked set +are the same set. The dual-set split that older versions of this procedure +carried — health checks over the working tree, index over the tracked set — +existed only because entries were gitignored-by-default, and it went away with +that rule. `.oh/knowledge/local/` is never enumerated. -**Two entry sets, deliberately.** The health checks (§§ 4-7a) run over this -**working-tree** set, so a local scratch entry is still lintable. The README -regeneration (§ 9) runs over the **git-tracked** set instead — see § 9a. Building -the index from the working tree makes an untracked scratch entry a CI regression, -because `.oh/evals/probes/wiki-readme-index.sh` reconstructs the expected table -from git-tracked files only. +If `$ENTRIES_COUNT = 0`, skip to § 9 (index regeneration with an empty base). -If `$ENTRIES_COUNT = 0`, skip all check steps and proceed directly to -§ 9 (README regeneration with an empty corpus). +### 3. Extract frontmatter -### 3. Extract frontmatter for each entry - -For every entry path, extract its YAML frontmatter using the canonical command -locked in `.oh/skills/wiki/references/schema.md` § 6: +For every entry, extract frontmatter with the canonical command in +`schema.md` § 9: ```bash -awk '/^---$/{f=!f; next} f{print}' .oh/skills/wiki/corpus/<slug>.md +awk '/^---$/{f=!f; next} f{print}' <entry> ``` -Build a lookup table of slug → frontmatter fields: +Build a lookup table of slug → fields: ```bash -declare -A ENTRY_SLUGS # slug → file path -declare -A ENTRY_TITLES # slug → title -declare -A ENTRY_TAGS # slug → tags string -declare -A ENTRY_UPDATED # slug → updated date -declare -A ENTRY_CONFIDENCE # slug → confidence value -declare -A ENTRY_PATH # slug → full path - -for entry in "${WIKI_ENTRIES[@]}"; do - frontmatter=$(awk '/^---$/{f=!f; next} f{print}' "$entry") - slug=$(echo "$frontmatter" | grep '^slug:' | awk '{print $2}') - title=$(echo "$frontmatter" | grep '^title:' | sed 's/^title: *//' | tr -d '"') - tags=$(echo "$frontmatter" | grep '^tags:' | sed 's/^tags: *//') - updated=$(echo "$frontmatter" | grep '^updated:' | awk '{print $2}') - confidence=$(echo "$frontmatter" | grep '^confidence:' | awk '{print $2}') - - [ -z "$slug" ] && continue # skip malformed entries (no slug) - - ENTRY_SLUGS["$slug"]="$entry" - ENTRY_TITLES["$slug"]="$title" - ENTRY_TAGS["$slug"]="$tags" - ENTRY_UPDATED["$slug"]="$updated" - ENTRY_CONFIDENCE["$slug"]="$confidence" +declare -A ENTRY_PATH ENTRY_TITLE ENTRY_TAGS ENTRY_UPDATED ENTRY_KIND +declare -A ENTRY_CONFIDENCE ENTRY_VERIFIED + +for entry in "${ENTRIES[@]}"; do + fm=$(awk '/^---$/{f=!f; next} f{print}' "$entry") + slug=$(grep '^slug:' <<<"$fm" | awk '{print $2}' | head -1) + [ -z "$slug" ] && continue # § 4 reports it as a schema failure ENTRY_PATH["$slug"]="$entry" + ENTRY_TITLE["$slug"]=$(grep '^title:' <<<"$fm" | sed 's/^title: *//' | tr -d '"') + ENTRY_TAGS["$slug"]=$(grep '^tags:' <<<"$fm" | sed 's/^tags: *//') + ENTRY_UPDATED["$slug"]=$(grep '^updated:' <<<"$fm" | awk '{print $2}') + ENTRY_KIND["$slug"]=$(grep '^kind:' <<<"$fm" | awk '{print $2}' | head -1) + ENTRY_CONFIDENCE["$slug"]=$(grep '^confidence:' <<<"$fm" | awk '{print $2}') + ENTRY_VERIFIED["$slug"]=$(grep '^verified_at:' <<<"$fm" | awk '{print $2}' | head -1) done ``` -This extraction MUST use the exact `awk '/^---$/{f=!f; next} f{print}'` command. -Deviation from the § 6 canonical command is forbidden — both `/wiki query` and -`/wiki lint` must use identical extraction to prevent silent divergence (a match -that works in one skill must work in the other). +Deviation from the § 9 command is forbidden — `query`, `lint`, and +`knowledge-impact.sh` must extract identically. -### 4. Stale-90d check +### 4. Check 1 — schema validity -Enumerate entries where frontmatter `updated:` is more than 90 days older than -today's UTC date. These are reported as a separate finding type with the -recommendation "consider review". +For every entry file, report a finding when any of these holds: -```bash -TODAY=$(date -u +%Y-%m-%d) -STALE_90D=() - -for slug in "${!ENTRY_UPDATED[@]}"; do - updated="${ENTRY_UPDATED[$slug]}" - [ -z "$updated" ] && continue - - # Compute age in days (compatible with GNU date) - today_epoch=$(date -u -d "$TODAY" +%s 2>/dev/null \ - || date -u -j -f "%Y-%m-%d" "$TODAY" +%s) - updated_epoch=$(date -u -d "$updated" +%s 2>/dev/null \ - || date -u -j -f "%Y-%m-%d" "$updated" +%s) - age_days=$(( (today_epoch - updated_epoch) / 86400 )) - - if [ "$age_days" -gt 90 ]; then - STALE_90D+=("$slug (updated: $updated, age: ${age_days}d)") - fi -done -``` - -Print findings: +- a required field is missing: `title`, `slug`, `tags`, `created`, `updated`, + `sources`, `confidence`; +- `slug` does not match the filename without `.md`, or does not match + `[a-z0-9-]+`; +- `kind` is absent or is not one of `repo`, `external`, `pattern`; +- `kind` disagrees with the directory: a `patterns/` file whose `kind` is not + `pattern`, or a `source/` file whose `kind` is not `repo` or `external`; +- a `patterns/` filename lacks the `pattern-` prefix, or a `source/` filename + carries it; +- `confidence` is not one of `provisional`, `confirmed`, `deprecated`; +- `kind: repo` with no `verified_at:`; +- `kind: repo` or `kind: pattern` with no `## Relevant Source Files` section. ``` -=== Stale-90d findings (${#STALE_90D[@]}) — consider review === +=== Schema findings (<n>) — the entry violates schema.md === + - <path>: <what is wrong> ``` -For each entry in `STALE_90D`, print one line: ` - <slug> (updated: <date>, age: <N>d)`. +Report-only. `lint` never edits frontmatter and never sets `confidence` +(`schema.md` § 8). -If `${#STALE_90D[@]} = 0`, print ` (none)`. +### 5. Check 2 — source and dependency paths resolve -**Important**: this check only REPORTS. `/wiki lint` does NOT modify the -`updated:` field or any other frontmatter field on any entry. +Every `sources:` entry must resolve to something real, in the form +`schema.md` § 4 defines for it: -### 5. Deprecated check +| Form | Resolves when | +|---|---| +| `raw/<yyyy-mm-dd>-<slug>.md` | `.oh/knowledge/raw/<...>` exists (resolved from `.oh/knowledge/`, not the page's directory) | +| `<repo-relative-path>` or glob | at least one path in the working tree matches | +| `<repo-relative-path>@<short-sha>` | the content exists at that revision — see the rename note below | +| `https://<...>` | never checked locally; the weakest form (`schema.md` § 4) | -Enumerate entries where frontmatter `confidence: deprecated`. These are reported -as a separate finding type with the recommendation "consider archive or delete". +A pin names a revision of the file's **content**, so a path that has moved since +`<sha>` must not read as broken. Try the exact path first, then fall back to the +basename in that commit's tree: ```bash -DEPRECATED=() - -for slug in "${!ENTRY_CONFIDENCE[@]}"; do - if [ "${ENTRY_CONFIDENCE[$slug]}" = "deprecated" ]; then - DEPRECATED+=("$slug") - fi -done -``` - -Print findings: - -``` -=== Deprecated findings (${#DEPRECATED[@]}) — consider archive or delete === +if ! git cat-file -e "${sha}:${path}" 2>/dev/null; then + tree="$(git ls-tree -r --name-only "$sha" 2>/dev/null || true)" + hits=$(grep -cE "(^|/)$(basename "$path")\$" <<<"$tree") + [ "$hits" = 1 ] || echo " - $slug: pinned source $path@$sha does not resolve ($hits basename hits)" +fi ``` -For each entry in `DEPRECATED`, print one line: ` - <slug>`. - -If `${#DEPRECATED[@]} = 0`, print ` (none)`. +A basename matching **more than one** path at that revision proves nothing about +which file the pin meant, so an ambiguous fallback is a finding, not a hit. A pin +whose path predates a rename and whose basename is ambiguous is repaired by +re-pinning to a revision where the cited path is real. -**Critical constraint**: `/wiki lint` ONLY REPORTS deprecated entries — it NEVER -autonomously SETS `confidence: deprecated`. The `deprecated` value is set -MANUALLY by the orchestrator only, per the lifecycle defined in -`.oh/skills/wiki/references/schema.md` § 5. This constraint is non-negotiable. +A pin whose **commit is not in this clone at all** — CI checks out shallow — is +unverifiable here, not broken. Check `git cat-file -e "<sha>^{commit}"` first and +count that pin as unverifiable rather than failing it; every pin whose commit is +present is still checked. A depth-dependent finding would make the check report +the clone rather than the knowledge base. -Stale-90d (§ 4) and Deprecated (§ 5) are always reported separately. They are -distinct finding types and MUST NOT be conflated — an entry can be stale-90d -and NOT deprecated, or deprecated and NOT stale-90d. +Capture the tree before matching. A `git ls-tree | grep -q` pipeline SIGPIPEs +`git` the moment `grep` finds its match, and under `pipefail` that turns a +successful match into a failed pipeline. -### 6. Orphan check (two-pass) +A page whose only unresolved source is a path that has since been deleted is +repaired by pinning it (`<path>@<sha>`), not by deleting the citation — the +evidence still exists in history. -An orphaned entry has zero inbound `[[slug]]` references from any other entry -body. A single-entry corpus is always an orphan — this is a **true positive**, -not a false positive. The orphan check does not distinguish corpus size. +Also report `kind: repo` pages whose `sources:` list holds **no** live +repository path: there is nothing for check 3 to verify them against. -#### Pass 1: enumerate all known slugs - -```bash -ALL_SLUGS=("${!ENTRY_SLUGS[@]}") ``` - -This is already populated from § 3. - -#### Pass 2: count inbound references per slug - -```bash -declare -A INBOUND_COUNT -for slug in "${ALL_SLUGS[@]}"; do - INBOUND_COUNT["$slug"]=0 -done - -for entry in "${WIKI_ENTRIES[@]}"; do - entry_slug=$(basename "$entry" .md) - - # Extract body (everything after the closing frontmatter ---) - body=$(awk '/^---$/{n++; if(n==2){p=1; next}} p{print}' "$entry") - - # Find all [[slug]] references in this entry's body - while IFS= read -r link_slug; do - [ -z "$link_slug" ] && continue - # Count this as an inbound reference for link_slug (from entry_slug) - # Only count if link_slug is different from the entry itself - if [ "$link_slug" != "$entry_slug" ] && [ -n "${INBOUND_COUNT[$link_slug]+_}" ]; then - INBOUND_COUNT["$link_slug"]=$(( INBOUND_COUNT["$link_slug"] + 1 )) - fi - done < <(echo "$body" | grep -oE '\[\[[a-z0-9-]+\]\]' | sed 's/\[\[\(.*\)\]\]/\1/') -done +=== Source-path findings (<n>) — a declared source does not resolve === ``` -Build the orphan list (zero inbound references): +### 6. Check 3 — source-change freshness (`needs-review`) + +Do not reimplement this. Call the one implementation: ```bash -ORPHANS=() -for slug in "${ALL_SLUGS[@]}"; do - if [ "${INBOUND_COUNT[$slug]}" -eq 0 ]; then - ORPHANS+=("$slug") - fi -done +bash .oh/skills/wiki/scripts/knowledge-impact.sh --verified ``` -Print findings: +Every `NEEDS-REVIEW` row is a finding: a `kind: repo` page whose declared +dependencies changed after its `verified_at` commit. Print the rows verbatim — +each names the page and the specific sources that moved. ``` -=== Orphan findings (${#ORPHANS[@]}) — entries with zero inbound [[links]] === +=== Freshness findings (<n>) — declared sources changed since verified_at === + - <slug>: <sources that moved> (verified_at <short-sha>) ``` -For each entry in `ORPHANS`, print one line: ` - <slug>`. +Remediation is to re-read the page against those sources and then either correct +it or, if it is still accurate, advance `verified_at:` to the current commit. +Advancing the pin without re-reading is the one thing this check cannot detect, +and the reason it is a report rather than an automatic bump. -If `${#ORPHANS[@]} = 0`, print ` (none)`. +`kind: external` and `kind: pattern` pages report `NOT-APPLICABLE` — their +provenance is immutable, so freshness does not apply. -**Single-entry corpus note**: a corpus with exactly one entry always produces -one orphan (the only entry cannot link to itself). This is a true positive. -Document it in the log `Observation` field when it occurs. +### 7. Check 4 — broken outbound `[[...]]` links -### 7. Broken outbound link check +A broken outbound link is a `[[slug]]` reference in any entry body whose slug +matches no entry's frontmatter `slug` in either directory. The slug namespace is +flat and links cross `source/` and `patterns/` freely. -A broken outbound link is a `[[slug]]` reference in any entry's body where -`slug` does NOT match any existing entry's frontmatter `slug` field. This -finding type is separate from orphans — orphans have no INBOUND links; broken -outbound links reference slugs that DO NOT EXIST. +A `[[slug]]` inside a code span or a fenced block is a **mention**, not a link — +schema prose and pattern pages both show the syntax literally. Strip fences and +inline code spans before extracting, or every document that explains the +convention becomes a finding. ```bash BROKEN_LINKS=() - -for entry in "${WIKI_ENTRIES[@]}"; do +for entry in "${ENTRIES[@]}"; do entry_slug=$(basename "$entry" .md) - body=$(awk '/^---$/{n++; if(n==2){p=1; next}} p{print}' "$entry") - + body=$(awk '/^---$/{n++; if(n==2){p=1; next}} p{print}' "$entry" \ + | awk '/^```/{f=!f; next} !f' \ + | sed 's/`[^`]*`//g') while IFS= read -r link_slug; do [ -z "$link_slug" ] && continue - # Check if link_slug matches any known entry slug - if [ -z "${ENTRY_SLUGS[$link_slug]+_}" ]; then + if [ -z "${ENTRY_PATH[$link_slug]+_}" ]; then BROKEN_LINKS+=("$entry_slug → [[$link_slug]] (no such entry)") fi - done < <(echo "$body" | grep -oE '\[\[[a-z0-9-]+\]\]' | sed 's/\[\[\(.*\)\]\]/\1/') + done < <(grep -oE '\[\[[a-z0-9-]+\]\]' <<<"$body" | sed 's/\[\[\(.*\)\]\]/\1/') done ``` -Print findings: - ``` -=== Broken outbound link findings (${#BROKEN_LINKS[@]}) === +=== Broken outbound link findings (<n>) === ``` -For each finding in `BROKEN_LINKS`, print one line: ` - <source-slug> → [[<missing-slug>]] (no such entry)`. - -If `${#BROKEN_LINKS[@]} = 0`, print ` (none)`. - -See `.oh/skills/wiki/references/schema.md` § 4 for the cross-link convention and grep -patterns that govern outbound link syntax (`\[\[[a-z0-9-]+\]\]`). - -### 7a. Broken related-slug check +### 8. Check 5 — broken `related:` slugs -A broken related-slug is an entry whose `related:` frontmatter list names a slug -with no matching entry. This is distinct from § 7: a `[[slug]]` body link is a +A broken related-slug is an entry whose `related:` frontmatter names a slug with +no matching entry. This is distinct from check 4: a `[[slug]]` body link is a navigational claim, a `related:` slug is a frontmatter adjacency claim. They fail for different reasons and are remediated differently, so they are counted separately. ```bash RELATED_BROKEN=() - -for slug in "${!ENTRY_SLUGS[@]}"; do - frontmatter=$(awk '/^---$/{f=!f; next} f{print}' "${ENTRY_PATH[$slug]}") - rel=$(echo "$frontmatter" | grep '^related:' | sed 's/^related: *//; s/[][]//g; s/,/ /g') +for slug in "${!ENTRY_PATH[@]}"; do + fm=$(awk '/^---$/{f=!f; next} f{print}' "${ENTRY_PATH[$slug]}") + rel=$(grep '^related:' <<<"$fm" | sed 's/^related: *//; s/[][]//g; s/,/ /g') for r in $rel; do [ -z "$r" ] && continue - if [ -z "${ENTRY_SLUGS[$r]+_}" ]; then - RELATED_BROKEN+=("$slug → related: $r (no such entry)") - fi + [ -z "${ENTRY_PATH[$r]+_}" ] && RELATED_BROKEN+=("$slug → related: $r (no such entry)") done done ``` -Print findings: - ``` -=== Broken related-slug findings (${#RELATED_BROKEN[@]}) === +=== Broken related-slug findings (<n>) === ``` -For each finding, print one line: ` - <source-slug> → related: <missing-slug> (no such entry)`. -If `${#RELATED_BROKEN[@]} = 0`, print ` (none)`. +Report-only. The orchestrator decides whether to repoint the slug or author the +missing entry. -Report-only. This check never edits frontmatter and never removes a slug — the -orchestrator decides whether to repoint the slug or author the missing entry. +### 9. Check 6 — regenerate the index -### 8. Contradiction detection (stub) +`.oh/knowledge/README.md` is generated state owned by this subcommand. -Contradiction detection is explicitly **descoped** for v1. The function prints -the following exact text and returns: - -``` -contradiction detection: not yet implemented — see wiki lint follow-up tracking -``` - -A follow-up tracking issue may be filed post-merge but is not a pre-merge -blocker. Do not attempt any heuristic approximation of contradiction detection; -the stub text above is the complete implementation for v1. - -### 9. README regeneration - -Build the `.oh/skills/wiki/corpus/README.md` entries table. Sort all entries by `updated:` -descending (most recently updated first). - -#### 9a. Select the git-tracked entry set, then sort by updated date descending - -The index domain is the **git-tracked** entry set, not the working tree. The -pathspec below is byte-identical to the one in -`.oh/evals/probes/wiki-readme-index.sh`; the two must never diverge, or a local -scratch entry becomes a CI regression. The `corpus/*/*` exclusion keeps a future -subdirectory from silently splitting the two sets — git's pathspec `*` is not -path-aware and descends, while § 2's shell glob does not. - -```bash -TRACKED_SLUGS=() -while IFS= read -r rel; do - [ "$(basename "$rel")" = "README.md" ] && continue - [ "$(basename "$rel")" = "skill-impact.md" ] && continue - abs="$HARNESS/$rel" - [ -f "$abs" ] || continue - slug=$(awk '/^---$/{f=!f; next} f{print}' "$abs" | grep '^slug:' | awk '{print $2}' | head -1) - [ -z "$slug" ] && continue - TRACKED_SLUGS+=("$slug") -done < <(git -C "$HARNESS" ls-files -- \ - '.oh/skills/wiki/corpus/*.md' \ - ':!:.oh/skills/wiki/corpus/raw/*' \ - ':!:.oh/skills/wiki/corpus/*/*') -``` +#### 9a. Sort by `updated:` descending ```bash -SORTED_SLUGS=() RANK_LINES=() -for slug in "${TRACKED_SLUGS[@]}"; do - updated="${ENTRY_UPDATED[$slug]:-0000-00-00}" - RANK_LINES+=("$updated $slug") +for slug in "${!ENTRY_PATH[@]}"; do + RANK_LINES+=("${ENTRY_UPDATED[$slug]:-0000-00-00} $slug") done - +SORTED_SLUGS=() while IFS= read -r line; do SORTED_SLUGS+=("${line#* }") done < <(printf '%s\n' "${RANK_LINES[@]}" | sort -r) ``` -#### 9b. Build the README content +The domain is every entry in `source/` and `patterns/`. It matches +`.oh/evals/probes/wiki-readme-index.sh` exactly; the two must never diverge. -The README file is owned and regenerated by `/wiki lint`. The table header is -literal — the exact byte sequence matters for validation. +#### 9b. Build the content + +The table header is literal — the exact byte sequence matters for validation. ```bash -# Preserve the static preamble of .oh/skills/wiki/corpus/README.md (lines before the Index table) -# The Index section starts at "## Index" -PREAMBLE=$(awk '/^## Index$/{exit} {print}' "$HARNESS/.oh/skills/wiki/corpus/README.md") +PREAMBLE=$(awk '/^## Index$/{exit} {print}' "$KNOWLEDGE/README.md") NEW_README="$PREAMBLE"$'\n' NEW_README+="## Index"$'\n\n' NEW_README+="| Slug | Title | Tags | Updated |"$'\n' NEW_README+="| --- | --- | --- | --- |"$'\n' - for slug in "${SORTED_SLUGS[@]}"; do - title="${ENTRY_TITLES[$slug]:-}" - tags="${ENTRY_TAGS[$slug]:-}" - updated="${ENTRY_UPDATED[$slug]:-}" - NEW_README+="| $slug | $title | $tags | $updated |"$'\n' + NEW_README+="| $slug | ${ENTRY_TITLE[$slug]:-} | ${ENTRY_TAGS[$slug]:-} | ${ENTRY_UPDATED[$slug]:-} |"$'\n' done ``` -**Empty corpus**: if `$ENTRIES_COUNT = 0`, the table contains only the two -header lines and no data rows. This is NOT an error condition. +**Empty base**: the table contains only the two header lines. Not an error. #### 9c. Atomic write or dry-run -In `--dry-run` mode, print the proposed content: +In `--dry-run`, print the proposed content between +`--- Proposed .oh/knowledge/README.md (dry-run, not written) ---` and +`--- end proposed .oh/knowledge/README.md ---`. -``` ---- Proposed .oh/skills/wiki/corpus/README.md (dry-run, not written) --- -<content> ---- end proposed .oh/skills/wiki/corpus/README.md --- -``` - -In default (non-dry-run) mode, perform the **atomic write**: +Otherwise write atomically: ```bash -TMP="$HARNESS/.oh/skills/wiki/corpus/README.md.tmp" -FINAL="$HARNESS/.oh/skills/wiki/corpus/README.md" - -# Write to tmp +TMP="$KNOWLEDGE/README.md.tmp" +FINAL="$KNOWLEDGE/README.md" printf '%s' "$NEW_README" > "$TMP" - -# Validate: non-empty AND contains the exact header line -if [ ! -s "$TMP" ]; then - echo "ERROR: .oh/skills/wiki/corpus/README.md.tmp is empty — aborting regeneration" - rm -f "$TMP" - exit 1 -fi - -if ! grep -qF '| Slug | Title | Tags | Updated |' "$TMP"; then - echo "ERROR: .oh/skills/wiki/corpus/README.md.tmp is missing the required header line — aborting regeneration" - rm -f "$TMP" - exit 1 -fi - -# Atomic rename on validation success +[ -s "$TMP" ] || { echo "ERROR: proposed index is empty — aborting"; rm -f "$TMP"; exit 1; } +grep -qF '| Slug | Title | Tags | Updated |' "$TMP" \ + || { echo "ERROR: proposed index is missing the header line — aborting"; rm -f "$TMP"; exit 1; } mv "$TMP" "$FINAL" -echo ".oh/skills/wiki/corpus/README.md regenerated (${ENTRIES_COUNT} entries)" +echo ".oh/knowledge/README.md regenerated (${ENTRIES_COUNT} entries)" ``` -**Atomic write protocol** (Critic-B mitigation): +Write to tmp → validate non-empty and header present → atomic rename. On +validation failure the original stays intact, the reason is printed, and the tmp +file is removed. A partial write never leaves the index corrupt. -1. Write to `.oh/skills/wiki/corpus/README.md.tmp` first. -2. Validate the tmp file: it must be non-empty AND contain the exact header line - `| Slug | Title | Tags | Updated |`. -3. On validation success: atomically rename `.oh/skills/wiki/corpus/README.md.tmp` → `.oh/skills/wiki/corpus/README.md`. -4. On validation failure: exit non-zero, leave the original `.oh/skills/wiki/corpus/README.md` - intact, print the failure reason, and remove the tmp file. +Verify: -This protocol ensures that a partial write or generation failure never leaves -`.oh/skills/wiki/corpus/README.md` in a corrupt or empty state. +```bash +bash .oh/evals/probes/wiki-readme-index.sh +``` -## Extraction Command Reference +### 10. Informational telemetry (decides nothing) -The canonical frontmatter extraction command, per `.oh/skills/wiki/references/schema.md` § 6: +After the findings, print one line of context. It is not a check and no gate +reads it: -```bash -awk '/^---$/{f=!f; next} f{print}' .oh/skills/wiki/corpus/<slug>.md +``` +last-reviewed: <n> entries, oldest <slug> (<updated>, <N>d), <k> deprecated ``` -This MUST be the extraction method used in this skill. Deviation from the § 6 -command is forbidden — both `/wiki query` and `/wiki lint` must use identical -extraction to prevent silent divergence (a match that works in one skill must -work in the other). Any future change to this extraction method requires -updating both skills atomically. +Age answers "when did a human last look at this", which is worth knowing and is +not a validity claim. Validity is check 3. -## Six Check Types — Summary +## Six checks — summary -| # | Type | Finding trigger | Recommendation | Autonomously sets flag? | -|---|------|-----------------|---------------|------------------------| -| 1 | Stale-90d | `updated:` > 90 days older than today UTC | consider review | No | -| 2 | Deprecated | `confidence: deprecated` | consider archive or delete | No — report only | -| 3 | Orphan | zero inbound `[[slug]]` references | (informational; true positive even for single-entry corpus) | No | -| 4 | Broken outbound | `[[slug]]` in body where `slug` has no matching entry | (informational; fix by adding the entry or correcting the link) | No | -| 4a | Broken related-slug | `related:` frontmatter slug with no matching entry | repoint the slug or author the missing entry | No | -| 5 | Contradiction | descoped | n/a — stub only | n/a | +| # | Check | Finding trigger | Sets anything? | +|---|-------|-----------------|----------------| +| 1 | Schema validity | missing/invalid field, kind-directory disagreement, filename mismatch | No | +| 2 | Source paths resolve | a `sources:` entry names nothing that exists at the form it declares | No | +| 3 | Source-change freshness | a declared repository source changed after `verified_at` | No | +| 4 | Broken outbound link | `[[slug]]` with no matching entry | No | +| 5 | Broken `related:` slug | `related:` slug with no matching entry | No | +| 6 | Index consistency | the generated table does not match current frontmatter | Yes — rewrites the index | -These six types are always reported separately. Types 1 and 2 (both related to -"staleness" in a loose sense) MUST NOT be conflated — they have distinct triggers -and distinct recommendations. +Check 6 is the only writer. Everything else reports. ## Anti-Patterns -- **Conflating stale-90d with deprecated** — stale-90d triggers on age of `updated:`; - deprecated triggers on the value of `confidence:`. An entry may be one, both, or - neither. Always report them under separate headings. -- **Setting `confidence: deprecated` autonomously** — `/wiki lint` is a reporter, - not a writer. The `deprecated` flag is set MANUALLY by the orchestrator per - `.oh/skills/wiki/references/schema.md` § 5. If `/wiki lint` sets this flag, it violates the - confidence lifecycle. -- **Treating a single-entry orphan as a false positive** — a corpus with one entry - always produces one orphan finding. This is correct behavior. Document it in the - log Observation. -- **Conflating orphans with broken outbound links** — orphans have no INBOUND links - (other entries don't reference them); broken outbound links reference slugs that - DO NOT EXIST. They are separate checks with different remediation paths. -- **Conflating broken `related:` slugs with broken `[[slug]]` body links** — the - first is a frontmatter adjacency claim, the second is a navigational link. They are - distinct checks with distinct remediation, reported under separate headings. -- **Regenerating the index from the working tree** — § 9a builds the Index from the - git-tracked entry set, matching `.oh/evals/probes/wiki-readme-index.sh` byte for - byte. Using the § 2 working-tree glob makes an untracked local scratch entry a CI - regression. -- **Non-atomic README write** — writing directly to `.oh/skills/wiki/corpus/README.md` without the - tmp → validate → rename protocol risks corruption. Always use the three-step - atomic write in § 9c. -- **Grepping `.oh/skills/wiki/corpus/README.md` for entries** — the README is the output of this - skill, not its input. Always enumerate `.oh/skills/wiki/corpus/*.md` directly. -- **Writing a run log** — there is no log tier. Report OP / DRY-RUN / FAIL and the - findings to the terminal. Audit-child mode returns its observation to the outer - dispatcher instead. -- **Hardcoding today's date** — always compute UTC date at runtime with - `date -u +%Y-%m-%d`. +- **Reintroducing orphan detection as a failure** — inbound-link count is not a + health signal (`schema.md` § 7). If a page is genuinely unreachable, the fix is + a link from a page that should have had one, not a report row. +- **Deciding validity from `updated:`** — age is telemetry. Check 3 decides. +- **Reimplementing freshness** — `knowledge-impact.sh` is the one implementation, + and `/spec execute` calls the same script. A second copy will disagree with it. +- **Advancing `verified_at:` to silence check 3** — the pin means "the claims + were re-read against these sources at this commit". Moving it without reading + launders staleness into freshness. +- **Setting `confidence: deprecated` autonomously** — the flag is set manually by + the orchestrator (`schema.md` § 8). +- **Conflating broken `related:` slugs with broken `[[slug]]` body links** — a + frontmatter adjacency claim and a navigational link are distinct checks with + distinct remediation. +- **Non-atomic index write** — always use the tmp → validate → rename protocol. +- **Grepping `.oh/knowledge/README.md` for entries** — the README is this + subcommand's output, not its input. Enumerate the directories. +- **Writing a run log** — there is no log tier. Report the findings and the + result to the terminal. +- **Hardcoding today's date** — compute UTC at runtime with `date -u +%Y-%m-%d`. ## See Also -- `.oh/skills/wiki/references/schema.md` — the locked schema, § 4 (cross-link / orphan / broken-link - definitions), § 5 (confidence lifecycle: who SETS vs REPORTS), § 6 (frontmatter - extraction canonical command) -- `/wiki ingest` — add or update an entry; the only authorized write path to `.oh/skills/wiki/corpus/` -- `/wiki query` — search the wiki by topic; shares the § 6 extraction command -- `/audit context` — reference for `--dry-run` flag pattern and atomic-write convention +- `.oh/skills/wiki/references/schema.md` — § 2 layout and the tracked boundary, + § 4 provenance forms, § 5 freshness, § 7 cross-links, § 8 confidence, + § 9 extraction, § 10 index freshness +- `.oh/skills/wiki/scripts/knowledge-impact.sh` — the one freshness implementation +- `/wiki ingest` — the authorized write path for entity pages +- `/wiki query` — the read path; shares the § 9 extraction command diff --git a/.oh/skills/wiki/references/official-docs-research-wiki.md b/.oh/skills/wiki/references/official-docs-research-wiki.md index b0a97a18..b96d7256 100644 --- a/.oh/skills/wiki/references/official-docs-research-wiki.md +++ b/.oh/skills/wiki/references/official-docs-research-wiki.md @@ -13,7 +13,7 @@ Use this pattern when a user asks to "add to wiki" but the source is a seed URL - Stop for confirmation before writing the wiki implementation. 3. Use subagents for research and audit, not direct tracked wiki writes. - Research subagents collect source-backed findings and identify related official docs. - - Subagents may draft proposed wiki text, but the orchestrator owns writes to `.oh/skills/wiki/corpus/raw/`, `.oh/skills/wiki/corpus/<slug>.md`, and cross-links. + - Subagents may draft proposed wiki text, but the orchestrator owns writes to `.oh/knowledge/raw/`, `.oh/knowledge/source/<slug>.md`, and cross-links. 4. Snapshot every authoritative source that materially supports the final entry. - Prefer official docs over social posts, blog summaries, or inferred behavior. - Keep raw snapshots immutable and synthesized entries bounded by `.oh/skills/wiki/references/schema.md`. diff --git a/.oh/skills/wiki/references/query.md b/.oh/skills/wiki/references/query.md index c52598c8..69d1438e 100644 --- a/.oh/skills/wiki/references/query.md +++ b/.oh/skills/wiki/references/query.md @@ -1,35 +1,60 @@ # /wiki query — reference -> Full procedure for the `query` subcommand of the `/wiki` dispatcher, lifted -> from the former standalone `/wiki query` skill during the wiki consolidation. -> The dispatcher (`.oh/skills/wiki/SKILL.md`) routes here when the first -> `$ARGUMENTS` token is `query`. Canonical schema: `.oh/skills/wiki/references/schema.md`. +> Full procedure for the `query` subcommand of the `/wiki` dispatcher. The +> dispatcher (`.oh/skills/wiki/SKILL.md`) routes here when the first +> `$ARGUMENTS` token is `query`. Canonical schema: +> `.oh/skills/wiki/references/schema.md`. -# Wiki Query +# Knowledge Query -Search the wiki by topic keyword(s) and load the top matching entry or entries -directly into context. This is Karpathy's "Query + Enhance" operation adapted -for Open Harness: grep frontmatter, rank by recency, read into context. +Search the knowledge base by topic keyword(s) and load the top matching entries +directly into context. This is Karpathy's "Query + Enhance" operation adapted for +Open Harness: grep frontmatter, rank, read into context. Query scope is **frontmatter-only** (`title`, `slug`, `tags`). Body text is deliberately excluded — the frontmatter fields capture the entry's identity -precisely; including body text would make match semantics unpredictable and -slow as the corpus grows. +precisely; including body text would make match semantics unpredictable and slow +as the knowledge base grows. + +## The tracked boundary + +`query` reads **tracked shared knowledge only**: + +```text +.oh/knowledge/source/*.md kind: repo | external +.oh/knowledge/patterns/*.md kind: pattern +``` + +It does **not** read `.oh/knowledge/local/`. That directory is gitignored +per-machine scratch, and there is no flag that folds it into a result set. A +plan grounded in a page only one machine can see is a plan nobody else can +reproduce, which is the failure this boundary exists to prevent. To make a local +page usable, promote it through the one authorized write path: + +```bash +/wiki ingest .oh/knowledge/local/<slug>.md --slug <slug> +``` + +`raw/` is not queried either — snapshots are provenance for the entity pages that +cite them, not results. ## When to Use - `/wiki query <topic>` when a session needs to recall previously-compiled knowledge about a recurring topic (tools, integrations, constraints, key concepts). -- Before re-deriving something from scratch — check the wiki first. +- **Before re-deriving something from scratch** — check the knowledge base first. + `/spec plan`'s recall step is exactly this call. - After `/wiki ingest` lands a new entry, to verify it is queryable. ## When NOT to Use -- **`/wiki ingest`** — to add or update an entry. `/wiki query` is read-only. -- **`/wiki lint`** — to health-check the corpus or regenerate `.oh/skills/wiki/corpus/README.md`. -- **Direct `grep`** — if you want full-text search including body prose. This - skill is intentionally frontmatter-only; full-text is out of scope for v1. +- **`/wiki ingest`** — to add or update an entry. `query` is read-only. +- **`/wiki lint`** — to health-check the knowledge base or regenerate the index. +- **Direct `grep`** — for full-text search including body prose. This subcommand + is intentionally frontmatter-only. +- **As authority.** A page is orientation. Re-ground its material claims against + the sources it cites before relying on them (`schema.md` preamble). ## Argument Interface (locked) @@ -38,49 +63,50 @@ slow as the corpus grows. ``` `<topic>` is one or more whitespace-separated words. The interface is locked to -`<topic>` plus the `--patterns` flag; adding further flags or positional arguments -requires editing this reference and +`<topic>` plus the `--patterns` flag; adding further flags or positional +arguments requires editing this reference and `.oh/evals/probes/wiki-query-pattern-isolation.sh`. ## Two disjoint modes -`kind:` (`.oh/skills/wiki/references/schema.md` § 2) splits the corpus into source -entries and pattern entries. The two modes never mix. +The directory split (`schema.md` § 2) separates entity pages from pattern pages. +The two modes never mix. -| Mode | Returns | Read cap | Ranking | -|------|---------|----------|---------| -| default | `kind: source` only — **an absent `kind:` counts as `source`** | 3 | `updated:` descending | -| `--patterns` | `kind: pattern` only | **5** | term-hit count descending, `updated:` descending as tiebreak | +| Mode | Reads | Returns | Read cap | Ranking | +|------|-------|---------|----------|---------| +| default | `.oh/knowledge/source/*.md` | `kind: repo` and `kind: external` | 3 | `updated:` descending | +| `--patterns` | `.oh/knowledge/patterns/*.md` | `kind: pattern` only | **5** | term-hit count descending, `updated:` descending as tiebreak | There is deliberately **no `--all` mode**. The evidence this split comes from (`[[wikiskill-experience-compilation]]`) measured the mixed configuration as -worst-of-both: the proposer-only setting scored 63.7 while giving both roles the same -view scored 60.9. Two disjoint modes make the asymmetry the path of least resistance. +worst-of-both: the proposer-only setting scored 63.7 while giving both roles the +same view scored 60.9. Two disjoint modes make the asymmetry the path of least +resistance. -This is a **default, not a boundary**. Any session can read a pattern file directly; -nothing prevents it. The flag keeps patterns out of ordinary results, which is what -the measurement supports — do not describe it as isolation. +This is a **default, not a boundary**. Any session can read a pattern file +directly; nothing prevents it. The flag keeps patterns out of ordinary results, +which is what the measurement supports — do not describe it as isolation. -## Multi-Word OR Semantics +**The planner/executor asymmetry.** `/spec plan` queries both modes: patterns +inform proposal and design. `/spec execute` consumes the approved PRD's +`## Knowledge Context` and re-reads the authoritative sources it names; it does +not load the pattern set unless the task turns into replanning. -The `<topic>` argument is **split on whitespace** into individual terms. A -wiki entry matches if **ANY** term appears in the frontmatter `title`, `slug`, -or `tags` fields (union of per-term matches, deduplicated). This is OR -semantics, not AND. +## Multi-Word OR Semantics -Example: +`<topic>` is **split on whitespace** into individual terms. An entry matches if +**ANY** term appears in the frontmatter `title`, `slug`, or `tags` fields (union +of per-term matches, deduplicated). This is OR semantics, not AND. ``` /wiki query github auth ``` -This matches entries that contain `github` OR `auth` in their frontmatter — -not only entries that contain both. A result set for `github auth` will include -an entry tagged `[auth, sandbox]` as well as an entry whose title contains -`GitHub Token Workflow Scope`. +matches entries containing `github` OR `auth` in their frontmatter — not only +entries containing both. -Rationale: OR semantics maximize recall on a small corpus. The 3-entry read -cap constrains how much context is loaded regardless of match count. +Rationale: OR semantics maximize recall on a small corpus. The read cap +constrains how much context is loaded regardless of match count. ## Instructions @@ -88,66 +114,50 @@ cap constrains how much context is loaded regardless of match count. ```bash ARGUMENTS="${ARGUMENTS:-}" -WANT_KIND=source +KNOWLEDGE=.oh/knowledge +DIR="$KNOWLEDGE/source" CAP=3 if echo "$ARGUMENTS" | grep -q -- '--patterns'; then - WANT_KIND=pattern + DIR="$KNOWLEDGE/patterns" CAP=5 fi TOPIC=$(echo "$ARGUMENTS" | sed 's/--patterns//g' | xargs) ``` -Split `$TOPIC` on whitespace to produce an array of search terms. Each term -is used independently in the grep step below. An empty `$TOPIC` after stripping the -flag is a usage error: print the `argument-hint` line and exit 0. +Split `$TOPIC` on whitespace to produce an array of search terms. An empty +`$TOPIC` after stripping the flag is a usage error: print the `argument-hint` +line and exit 0. -### 2. Collect all wiki entry paths +### 2. Collect entry paths ```bash -HARNESS=/home/sandbox/harness -WIKI_ENTRIES=() -for f in "$HARNESS"/.oh/skills/wiki/corpus/*.md; do - [ -f "$f" ] && WIKI_ENTRIES+=("$f") +ENTRIES=() +for f in "$DIR"/*.md; do + [ -f "$f" ] || continue + [ "$(basename "$f")" = "README.md" ] && continue + ENTRIES+=("$f") done ``` -This enumerates `.oh/skills/wiki/corpus/*.md` directly — NOT via `.oh/skills/wiki/corpus/README.md` (the README -is a human-orientation index regenerated by `/wiki lint`, not a query backend). -Sub-article files at `.oh/skills/wiki/corpus/<parent>/<child>.md` are not matched by this glob; -they are scoped for a future iteration. `kind: pattern` entries are flat -`corpus/<slug>.md` files by the `.oh/skills/wiki/references/schema.md` § 2 placement -rule, precisely because this glob does not descend. +This enumerates the mode's directory directly — NOT via `.oh/knowledge/README.md` +(the README is a human-orientation index regenerated by `/wiki lint`, not a query +backend). The glob is flat and does not descend, which is why `schema.md` § 2 +forbids sub-directories. -If no `.oh/skills/wiki/corpus/*.md` files exist, jump to step 5 (empty result). +If no entries exist, jump to step 5 (empty result). ### 3. Grep frontmatter for each term — OR semantics -For each entry path, extract its frontmatter using the canonical command locked -in `.oh/skills/wiki/references/schema.md` § 6: - -```bash -awk '/^---$/{f=!f; next} f{print}' .oh/skills/wiki/corpus/<slug>.md -``` - -Then grep the extracted frontmatter for any of the topic terms. An entry is -a match if the grep succeeds for **at least one** term. - -Full loop. Two things happen per entry beyond the v1 behavior: the `kind:` filter -rejects entries belonging to the other mode, and the number of matching terms is -counted for the ranking step. +Extract each entry's frontmatter with the canonical command locked in +`schema.md` § 9, then grep the extracted block for the topic terms. The directory +already selected the kind, so no `kind:` filter is needed; the number of matching +terms is counted for ranking. ```bash MATCHES=() declare -A HITS -for entry in "${WIKI_ENTRIES[@]}"; do - slug=$(basename "$entry" .md) +for entry in "${ENTRIES[@]}"; do frontmatter=$(awk '/^---$/{f=!f; next} f{print}' "$entry") - - # kind filter — absent means source (schema.md § 2) - kind=$(echo "$frontmatter" | grep '^kind:' | awk '{print $2}' | head -1) - kind="${kind:-source}" - [ "$kind" = "$WANT_KIND" ] || continue - hits=0 for term in $TOPIC; do if echo "$frontmatter" | grep -qi -- "$term"; then @@ -161,16 +171,9 @@ for entry in "${WIKI_ENTRIES[@]}"; do done ``` -The `kind` value is read from the output of the § 6 canonical extraction command, -exactly like `updated:` in step 6 — the extraction itself is untouched. - -Note the loop now counts *distinct matching terms* rather than breaking on the first -match. OR semantics are unchanged: one hit still qualifies. The count is only used -for ranking. - -The grep targets the full extracted frontmatter block — which contains the -`title:`, `slug:`, and `tags:` fields — so all three fields are searched in -a single pass. The grep is case-insensitive (`-i`). +The grep targets the full extracted frontmatter block — which contains `title:`, +`slug:`, and `tags:` — so all three fields are searched in one pass, +case-insensitively. ### 4. Print all matching paths to stdout @@ -180,87 +183,75 @@ for m in "${MATCHES[@]}"; do done ``` -All matching `.oh/skills/wiki/corpus/*.md` file paths are printed, one per line. - ### 5. Handle empty results If `${#MATCHES[@]} -eq 0`: ```bash -echo "No wiki entries matched $TOPIC" +echo "No knowledge entries matched $TOPIC" exit 0 ``` -This is NOT an error condition. Exit status 0 is correct; an empty wiki or a -genuinely absent topic is a normal outcome, not a failure. +This is NOT an error condition. Exit status 0 is correct; an empty knowledge base +or a genuinely absent topic is a normal outcome. -### 6. Rank matches by `updated:` descending +### 6. Rank matches -Before reading entries into context, sort the match list so the most recently -updated entry is first. Extract the `updated:` field from each match's -frontmatter using the canonical command: - -In `--patterns` mode, `updated:` is the **tiebreak**, not the primary key: entries are -ranked by how many topic terms they matched, descending, and only then by recency. +Sort the match list before reading. In `--patterns` mode `updated:` is the +**tiebreak**, not the primary key: entries rank by how many topic terms they +matched, descending, and only then by recency. ```bash -# Build a sortable list: "<hits> <updated-date> <path>" (hits zero-padded to sort as text) RANKED=() for m in "${MATCHES[@]}"; do updated=$(awk '/^---$/{f=!f; next} f{print}' "$m" | grep '^updated:' | awk '{print $2}') - if [ "$WANT_KIND" = pattern ]; then + if [ "$CAP" = 5 ]; then RANKED+=("$(printf '%03d' "${HITS[$m]}") $updated $m") else RANKED+=("000 $updated $m") fi done -# Sort descending, extract paths SORTED_PATHS=() while IFS= read -r line; do - rest="${line#* }" # drop hit count - SORTED_PATHS+=("${rest#* }") # drop updated date + rest="${line#* }" # drop hit count + SORTED_PATHS+=("${rest#* }") # drop updated date done < <(printf '%s\n' "${RANKED[@]}" | sort -r) ``` Default mode pins every hit count to `000`, so the sort reduces to `updated:` -descending — v1 behavior, byte for byte. +descending. **Why ranking exists in `--patterns` mode only.** Pattern pages accumulate -monotonically and are never pruned, so a recency-only order degenerates into "most -recently compiled" — the wrong bias for a proposer that needs the *relevant* failure -mode, not the newest one. Source pages have a natural refresh cycle through -re-ingest, so their recency ordering still carries signal; leave it alone. +monotonically and are never pruned, so a recency-only order degenerates into +"most recently compiled" — the wrong bias for a proposer that needs the +*relevant* failure mode, not the newest one. Entity pages have a natural refresh +cycle through re-ingest, so their recency ordering still carries signal. -**Honest caveat.** At roughly one pattern per merged skill change, the pattern layer -will hold single digits of pages for months. The cap-5-plus-hit-ranking is cheap -insurance against a flood that is not yet happening, not a fix for a live problem. Do -not add scoring beyond this — no term frequency, no TF-IDF, no embeddings. +**Honest caveat.** At roughly one pattern per merged skill change, the pattern +layer will hold single digits of pages for months. The cap-5-plus-hit-ranking is +cheap insurance against a flood that is not yet happening, not a fix for a live +problem. Do not add scoring beyond this — no term frequency, no TF-IDF, no +embeddings. -If an entry has no `updated:` field (malformed frontmatter), it sorts to the -bottom. `/wiki lint` should surface such entries as a finding. +An entry with no `updated:` field sorts to the bottom; `/wiki lint` reports it as +a schema finding. ### 7. Apply the read cap -The read cap is set by the mode in step 1: **3** for the default source mode -(unchanged from v1) and **5** for `--patterns`. Neither is configurable by a further -flag; changing either requires editing this reference and its probe. +The cap is set by the mode in step 1: **3** for the default entity mode and **5** +for `--patterns`. Neither is configurable by a further flag; changing either +requires editing this reference and its probe. ```bash MATCH_COUNT=${#SORTED_PATHS[@]} -# CAP was set in step 1: 3 (source) or 5 (--patterns) - if [ "$MATCH_COUNT" -lt "$CAP" ]; then - # Sub-cap: read ALL matches READ_PATHS=("${SORTED_PATHS[@]}") else - # At or over cap: read top 3 only READ_PATHS=("${SORTED_PATHS[@]:0:$CAP}") fi ``` -Two cases: - | `Match-Count` | Behavior | |---------------|----------| | 0 | Print empty-result message; exit 0; read nothing | @@ -269,63 +260,58 @@ Two cases: ### 8. Read matched entries into context -For each path in `READ_PATHS`, read the full `.oh/skills/wiki/corpus/<slug>.md` file into -context. Each file is read directly — not routed via `.oh/skills/wiki/corpus/README.md`. - -```bash -READ_SLUGS=() -for path in "${READ_PATHS[@]}"; do - slug=$(basename "$path" .md) - READ_SLUGS+=("$slug") - # Read file into context (use the Read tool for each path) -done -``` +Read each path in `READ_PATHS` whole. Each file is read directly — not routed via +`.oh/knowledge/README.md`. -After reading, summarize what was loaded: list the slugs read, the total match -count, and how many were skipped (if any were above the cap). +After reading, summarize what was loaded: the slugs read, the total match count, +and how many were skipped above the cap. A caller recording `## Knowledge +Context` in a PRD lists exactly those slugs. ## Extraction Command Reference -The canonical frontmatter extraction command, per `.oh/skills/wiki/references/schema.md` § 6: +The canonical frontmatter extraction command, per `schema.md` § 9: ```bash -awk '/^---$/{f=!f; next} f{print}' .oh/skills/wiki/corpus/<slug>.md +awk '/^---$/{f=!f; next} f{print}' .oh/knowledge/source/<slug>.md ``` -This MUST be the extraction method used in this skill. Deviation from the § 6 -command is forbidden — both `/wiki query` and `/wiki lint` must use identical -extraction to prevent silent divergence (a match that works in one skill must -work in the other). +This MUST be the extraction method used here. Deviation is forbidden — `query`, +`lint`, and `knowledge-impact.sh` must extract identically or a match that works +in one will not work in another. ## Anti-Patterns -- **Grepping the full file** — `grep <term> .oh/skills/wiki/corpus/<slug>.md` searches body text - too, producing false positives from prose mentions of a term. Always extract - frontmatter first via the § 6 command, then grep the extracted output. -- **Routing through `.oh/skills/wiki/corpus/README.md`** — the README is a human-orientation - index regenerated by `/wiki lint`. Its table format is not a stable query - backend; use direct `.oh/skills/wiki/corpus/*.md` file enumeration. -- **AND semantics for multi-word topics** — requiring all terms to match - reduces recall inappropriately on a small corpus. Use OR: match on any term. -- **Treating empty result as error** — `Match-Count = 0` is a normal outcome - on a young or sparse corpus. Print the message, exit 0, log it. -- **Hard-coding today's date** in the glob or path — always compute UTC date - at runtime. -- **Writing a run log** — there is no log tier. Summarize the slugs read, the match - count, and how many were skipped to the terminal, and stop. -- **Mixing kinds in one result set** — there is no `--all`. Pattern entries and - source entries answer different questions for different roles, and the measurement - behind the split scored the mixed configuration worst-of-both. -- **Forgetting the `kind:` default** — an entry with no `kind:` field is a `source` - entry. A filter that tests `kind = "source"` against a raw, unset value drops all - fourteen entries authored before the field existed. +- **Reading `.oh/knowledge/local/`** — it is per-machine scratch and no query + path reads it. Promote through `/wiki ingest` instead. +- **Grepping the full file** — searching body text produces false positives from + prose mentions. Always extract frontmatter first, then grep the extracted + output. +- **Routing through `.oh/knowledge/README.md`** — the README is a generated + human-orientation index. Its table format is not a stable query backend; use + direct file enumeration. +- **AND semantics for multi-word topics** — requiring all terms reduces recall + inappropriately on a small corpus. Use OR. +- **Treating an empty result as an error** — a zero match count is a normal + outcome on a young knowledge base. Print the message, exit 0. +- **Hard-coding today's date** in a glob or path — always compute UTC at runtime. +- **Writing a run log** — there is no log tier. Summarize the slugs read, the + match count, and the skipped count to the terminal, and stop. +- **Mixing kinds in one result set** — there is no `--all`. Pattern pages and + entity pages answer different questions for different roles, and the + measurement behind the split scored the mixed configuration worst-of-both. - **Ranking patterns by recency alone** — patterns are never pruned, so recency - ordering degenerates into "most recently compiled". Rank by term-hit count first. + ordering degenerates into "most recently compiled". Rank by term-hit count + first. +- **Treating a returned page as authority** — re-ground its material claims + against the sources it cites. The repository outranks the page. ## See Also -- `.oh/skills/wiki/references/schema.md` — the locked schema, § 6 (frontmatter extraction canonical command), § 2 (entry schema), § 4 (cross-link convention) -- `/wiki ingest` — add or update a `kind: source` entry -- `/wiki compile` — create or patch a `kind: pattern` entry from a `/retro` report -- `/wiki lint` — health-check the corpus and regenerate `.oh/skills/wiki/corpus/README.md` -- `.oh/evals/probes/wiki-query-pattern-isolation.sh` — the guard on this contract +- `.oh/skills/wiki/references/schema.md` — the locked schema: § 2 (layout and the + tracked boundary), § 3 (entry schema), § 7 (cross-links), § 9 (extraction) +- `/wiki ingest` — add or update an entity page +- `/wiki compile` — create or patch a `kind: pattern` page from a `/retro` report +- `/wiki lint` — health-check the knowledge base and regenerate the index +- `.oh/evals/probes/wiki-query-pattern-isolation.sh` — the guard on the mode split +- `.oh/evals/probes/knowledge-tracked-query-boundary.sh` — the guard on the + tracked/local boundary diff --git a/.oh/skills/wiki/references/schema.md b/.oh/skills/wiki/references/schema.md index 875fc56e..739098d8 100644 --- a/.oh/skills/wiki/references/schema.md +++ b/.oh/skills/wiki/references/schema.md @@ -1,91 +1,132 @@ -# Wiki — Schema and Authoring Rules - -The Open Harness wiki (`.oh/skills/wiki/corpus/`) is a personal-scale knowledge base compiled and maintained by the orchestrator. Its target quality bar is architecture-first pages that explain source-backed system relationships, not loose notes. Entity pages hold **facts and synthesis** about recurring topics; they are loaded directly into context on demand (via `/wiki query`) rather than retrieved through vector search. - -`.oh/skills/wiki/references/schema.md` is the sole schema document for `.oh/skills/wiki/corpus/`. There is no `.oh/skills/wiki/corpus/CLAUDE.md` — that would collide with the root `CLAUDE.md` symlink to `AGENTS.md`. +# Knowledge — Schema and Authoring Rules + +The Open Harness knowledge base lives at `.oh/knowledge/`: a personal-scale, +LLM-readable cache of what this repository has been understood to be. Pages hold +**facts and synthesis** about recurring topics and are loaded whole into context on +demand (`/wiki query`) rather than retrieved through vector search. The target +quality bar is architecture-first pages that explain source-backed system +relationships, not loose notes. + +**Two surfaces, one owner each.** `.oh/knowledge/` owns the **data**. +`.oh/skills/wiki/` owns the **procedure** — how a page is written, queried, +linted, and compiled. Nothing about a page's schema, provenance, or lifecycle +lives inside the skill's implementation tree, and no knowledge page lives inside +it either. + +**The repository outranks the knowledge base, always.** A page is orientation. +Code and tests are implementation truth; canonical docs, RFCs, and ADRs are +intended-design truth. When a page and its source disagree, the source wins and +the page is wrong. This file is the sole schema document for `.oh/knowledge/`. --- ## 1. Boundary table -The sharp test: *Is this a fact or synthesis about a topic, intended to be read whole into agent context on demand?* If yes → wiki. Else use the surface below. +The sharp test: *Is this a fact or synthesis about a topic, intended to be read +whole into agent context on demand?* If yes → knowledge. Else use the surface +below. -| Surface | Holds | Written by | When wiki wins instead | +| Surface | Holds | Written by | When knowledge wins instead | | --- | --- | --- | --- | -| `.oh/skills/*/SKILL.md` | Behavioral norms (prescriptive) | Deliberate orchestrator revision | Wiki holds **facts**, skills hold **how to behave**. A `kind: pattern` entry sits closest to this line: it records that a workaround *worked*, which is evidence; the skill records that the workaround *must be applied*, which is a norm. When a pattern's workaround becomes a rule, it is promoted into a skill and the pattern stays as the evidence for it | -| `docs/` | Human-facing prose | Orchestrator / contributors | Wiki is LLM-readable; docs are human-readable | -| `.claude/skills/*/SKILL.md` | Executable procedures | Orchestrator | Skills are *how to do*; wiki is *what is true* | -| `.oh/skills/wiki/corpus/raw/` | Immutable source captures (snapshots of fetched pages, papers) | Skills writing snapshots only | Same surface; raw is upstream, wiki entries are synthesis | +| `.oh/skills/*/SKILL.md` | Behavioral norms (prescriptive) | Deliberate orchestrator revision | Knowledge holds **facts**, skills hold **how to behave**. A `kind: pattern` entry sits closest to this line: it records that a workaround *worked*, which is evidence; the skill records that the workaround *must be applied*, which is a norm. When a pattern's workaround becomes a rule, it is promoted into a skill and the pattern stays as the evidence for it | +| `docs/` | Human-facing prose | Orchestrator / contributors | Knowledge is LLM-readable; docs are human-readable | +| `.oh/evals/decisions/` | Accepted/rejected proposal history (`skill-impact.md`) | `/builder`, `/benchmark` | A decision ledger is a record of judgments, not synthesis about a topic | +| `.oh/knowledge/raw/` | Immutable external captures (snapshots of fetched pages, papers) | `/wiki ingest` | Same surface; raw is upstream evidence, entity pages are synthesis | +| `.oh/knowledge/local/` | Per-machine scratch | anyone | **Nothing reads it.** A page only one machine can see must never inform a plan another machine cannot reproduce | + +--- + +## 2. Directory layout and the tracked boundary + +```text +.oh/knowledge/ +├── README.md generated index — owned by /wiki lint, never hand-edited +├── source/<slug>.md kind: repo | external entity pages TRACKED +├── patterns/pattern-<subsystem>-<mode>.md kind: pattern pages TRACKED +├── raw/<yyyy-mm-dd>-<slug>.md external snapshots TRACKED +└── local/ per-machine scratch IGNORED +``` + +**The directory is the `kind` boundary, and `kind:` must agree with it.** A +`kind: pattern` page lives in `patterns/`; `kind: repo` and `kind: external` +pages live in `source/`. Both globs are flat and do not descend — a page in a +sub-directory is invisible to `/wiki query` and `/wiki lint` while still visible +to git, which is how a page silently stops being knowledge. + +**Tracked by default.** `source/`, `patterns/`, and `raw/` are committed like any +other repository content, reviewed in the PR that lands them. There is no +`git add -f` step and no whitelist. An untracked page is provenance a fresh clone +cannot see, which is the same as no provenance. + +**`local/` is the only ignored tier, and nothing reads it.** `/wiki query` +enumerates `source/` and `patterns/`. Every `/spec` flow consumes tracked +knowledge only. There is no flag that folds a local page into a normal result +set. Promotion out of `local/` goes through `/wiki ingest`, which applies this +schema and lands a tracked page. + +**Path resolution.** A `raw/<...>` value in `sources:` resolves relative to +`.oh/knowledge/`, not to the page's own directory. Every other `sources:` value +is repository-relative from the repository root. --- -## 2. Entry schema +## 3. Entry schema -Every wiki entry is a single markdown file at `.oh/skills/wiki/corpus/<slug>.md` with YAML frontmatter followed by a bounded, source-backed body. The minimum body is three sections; architecture and harness-mechanism entries use the source-backed expansion below. +Every entry is a single markdown file with YAML frontmatter followed by a +bounded, source-backed body. ### Frontmatter ```yaml --- -title: "GitHub Token Workflow Scope" -slug: gh-token-workflow-scope -kind: source # source | pattern; absent means source -tags: [git, github, auth, ci] -created: 2026-05-23 -updated: 2026-05-23 +title: "Compose Environment Boundary" +slug: compose-env-boundary +kind: repo # repo | external | pattern +tags: [compose, devcontainer, boundary] +created: 2026-08-31 +updated: 2026-08-31 sources: - - raw/2026-05-23-github-docs-fine-grained-pat.md -related: [github-auth-sandbox, ci-secrets-handling] -confidence: confirmed # provisional | confirmed | deprecated + - .devcontainer/docker-compose.yml + - .devcontainer/entrypoint.sh + - .oh/evals/probes/compose-env-boundary.sh +verified_at: 1c5f37230822ec2bbc5ed316be92ad295722b693 +related: [sandbox-dependency-installs] +confidence: confirmed --- ``` -Field definitions: - | Field | Type | Required | Notes | | --- | --- | --- | --- | | `title` | string | yes | Human-readable entry title | | `slug` | string | yes | Matches filename without `.md`; charset `[a-z0-9-]+` | -| `kind` | enum | no | `source` \| `pattern`. **An absent `kind:` field means `kind: source`.** Set to `pattern` only by `/wiki compile` | +| `kind` | enum | yes | `repo` \| `external` \| `pattern`. Must agree with the directory | | `tags` | list of strings | yes | Used by `/wiki query` for frontmatter-only grep | | `created` | date (YYYY-MM-DD) | yes | UTC date of initial creation; never updated | -| `updated` | date (YYYY-MM-DD) | yes | UTC date of most recent ingest/edit; always updated on write | -| `sources` | list of paths | yes | At least one `raw/<yyyy-mm-dd>-<slug>.md` snapshot path | +| `updated` | date (YYYY-MM-DD) | yes | UTC date of most recent write. Telemetry only — see § 5 | +| `sources` | list of paths | yes | The provenance **and** dependency declaration — see § 4 | +| `verified_at` | commit sha | `kind: repo` only | The commit the page's claims were last checked against | | `related` | list of slugs | no | Slugs of conceptually adjacent entries | | `confidence` | enum | yes | `provisional` \| `confirmed` \| `deprecated` | ### Entry kinds -The corpus holds two kinds of entry, distinguished by `kind:`. +| `kind` | Directory | Holds | `sources:` | Written by | Read by | +| --- | --- | --- | --- | --- | --- | +| `repo` | `source/` | Synthesis about **this repository** — a subsystem, pipeline, runtime, or convention | Repository-relative paths or globs, plus `verified_at` | `/wiki ingest` | any session, `/wiki query <topic>` | +| `external` | `source/` | Synthesis about an **outside** topic — a paper, a product, a landscape | At least one `raw/<yyyy-mm-dd>-<slug>.md` snapshot | `/wiki ingest` | any session, `/wiki query <topic>` | +| `pattern` | `patterns/` | A failure mode or working strategy observed in **this harness's own runs**, with an actionable workaround | Pinned evidence, `<repo-relative-path>@<short-sha>` | `/wiki compile` | the proposer role, `/wiki query <topic> --patterns` | -| `kind` | Holds | Written by | Read by | -| --- | --- | --- | --- | -| `source` | Facts and synthesis about an external topic, backed by a `raw/` snapshot | `/wiki ingest` | any session, via `/wiki query <topic>` | -| `pattern` | A failure mode or successful strategy observed in this harness's own runs, with an actionable workaround | `/wiki compile` | the proposer role, via `/wiki query <topic> --patterns` | - -**Back-compatibility.** `kind` is the only optional-with-default field in this -schema. Every entry authored before the field existed is a `source` entry, and none -require editing. `/wiki ingest` MAY emit `kind: source` explicitly on new source -pages but is not required to. **Consumers that filter on `kind` MUST apply the -default: read the field, and treat empty as `source`.** - -**Pattern placement.** `kind: pattern` entries are flat files at -`.oh/skills/wiki/corpus/pattern-<subsystem>-<short-name>.md` — never in a -subdirectory. The `corpus/*.md` glob used by `/wiki query` and `/wiki lint` does not -descend, so a pattern in a subdirectory would be invisible to both while still -visible to `.oh/evals/probes/wiki-readme-index.sh`'s git pathspec, which does. The -`pattern-` filename prefix is a redundant, greppable encoding of the `kind:` field: -`ls corpus/pattern-*.md` answers "what has this harness learned" without parsing -YAML, and the two must always agree. When a pattern page overflows the word cap, -split it into a second flat pattern page and cross-link; do not create a -sub-article. +**A `kind: repo` page never snapshots this repository's own source into `raw/`.** +The repository is already versioned; a snapshot of it is a second copy that +drifts. Cite the paths and pin the commit instead. `raw/` holds external captures +only, which is why it is the `kind: external` provenance form. **Why the proposer, and not every session, reads patterns.** The source this rule -comes from measured it: giving the skill proposer access to accumulated knowledge was -worth +15.0 points, while additionally giving the inference agent that same access -*cost* 2.8 (`[[wikiskill-experience-compilation]]`). `--patterns` is a default, not a -boundary — any session can read a pattern file directly. Say "default", never -"isolation". +comes from measured it: giving the skill proposer access to accumulated knowledge +was worth +15.0 points, while additionally giving the inference agent that same +access *cost* 2.8 (`[[wikiskill-experience-compilation]]`). `--patterns` is a +default, not a boundary — any session can read a pattern file directly. Say +"default", never "isolation". ### Body layout @@ -109,25 +150,37 @@ boundary — any session can read a pattern file directly. Say "default", never - [[related-slug-two]] ``` -Sections must appear in this order: H1, optional `## Relevant Source Files`, `## Summary`, `## Detail`, optional `## System Relationships`, `## See Also`. Architecture/harness entries SHOULD include both optional sections; simple external-concept entries may omit them. `## Summary`, `## Detail`, and `## See Also` are always present even if `## See Also` has no bullets yet. +Sections appear in this order: H1, optional `## Relevant Source Files`, +`## Summary`, `## Detail`, optional `## System Relationships`, `## See Also`. +`## Relevant Source Files` is **required** for `kind: repo` and `kind: pattern` +and optional for `kind: external`. `## Summary`, `## Detail`, and `## See Also` +are always present even if `## See Also` has no bullets yet. ### Source-backed architecture standard -New or substantially revised architecture pages follow one shape: source files first, then concise synthesis, then component relationships, then navigation. A page meets the standard when: - -- **Relevant source files are explicit**: list the files that make the page true before the summary, not as vague bibliography. Prefer local repo paths; cite external URLs only when the page is about an external artifact. -- **Claims are line-cited**: repository behavior, stage ordering, lifecycle claims, and invariants cite source paths with line numbers such as `AGENTS.md:111` or `.claude/skills/spec/references/execute.md:20`. -- **Relationships are visible**: when the page explains a pipeline, runtime, or architecture, include a compact Mermaid diagram or table that shows ownership, ordering, and handoff boundaries. -- **Synthesis stays separate from evidence**: use prose to explain what the cited files imply, but do not let unsupported interpretation look like a source fact. -- **Navigation closes the loop**: `## See Also` points to adjacent wiki entries using `[[slug]]` links, so a reader can walk between related pages. +New or substantially revised architecture pages follow one shape: source files +first, then concise synthesis, then component relationships, then navigation. A +page meets the standard when: + +- **Relevant source files are explicit**: list the files that make the page true + before the summary, not as vague bibliography. Prefer local repository paths; + cite external URLs only when the page is about an external artifact. +- **Claims are line-cited**: repository behavior, stage ordering, lifecycle + claims, and invariants cite source paths with line numbers such as + `AGENTS.md:111` or `.oh/skills/spec/references/execute.md:20`. +- **Relationships are visible**: when the page explains a pipeline, runtime, or + architecture, include a compact Mermaid diagram or table showing ownership, + ordering, and handoff boundaries. +- **Synthesis stays separate from evidence**: use prose to explain what the cited + files imply, but do not let unsupported interpretation look like a source fact. +- **Navigation closes the loop**: `## See Also` points to adjacent entries using + `[[slug]]` links, so a reader can walk between related pages. ### Pattern body layout (`kind: pattern`) -A pattern page uses the **same sections in the same order** as a source page. The -paper's Symptom / Root cause / Workaround / Evidence all fit inside them as bold -leads, so patterns need no structural exception and no special case in `/wiki lint`. -The only rule change is that `## Relevant Source Files` — where the evidence lands — -is **required** for `kind: pattern`, where it is optional for `kind: source`. +A pattern page uses the **same sections in the same order** as an entity page. +Symptom / Root cause / Workaround / Evidence all fit inside them as bold leads, +so patterns need no structural exception and no special case in `/wiki lint`. ```markdown # <Pattern title — the failure mode, not the incident> @@ -148,21 +201,21 @@ is **required** for `kind: pattern`, where it is optional for `kind: source`. workaround is annotated `(superseded YYYY-MM-DD, SI-nnnn)`, never deleted.> ## See Also -- [[<motivating source page>]] +- [[<motivating entity page>]] ``` Title a pattern for the failure mode, not the incident that revealed it: -`pattern-eval-probe-provenance-decay`, not `pattern-2026-08-31-retro-findings`. One -page per failure mode, never one per run — a dated per-run page is a session journal, -which this corpus is not. +`pattern-evals-probe-provenance-decay`, not `pattern-2026-08-31-retro-findings`. +One page per failure mode, never one per run — a dated per-run page is a session +journal, which this knowledge base is not. **`sources:` for a pattern.** A pattern entry MUST carry at least one `sources:` -entry. Each is either a `raw/<yyyy-mm-dd>-<slug>.md` snapshot path (when the pattern -is grounded in an ingested source) **or** a pinned repository-evidence path of the -form `<repo-relative-path>@<short-sha>` — for example -`.oh/tasks/<slug>/progress.txt@a1b2c3d`, `.oh/evals/RESULTS.md@a1b2c3d`. The +entry, each a pinned repository-evidence path of the form +`<repo-relative-path>@<short-sha>` — for example +`.oh/tasks/<slug>/evidence.md@a1b2c3d`, `.oh/evals/RESULTS.md@a1b2c3d`. The `@<short-sha>` suffix is required: it buys for a mutable tracked file the same -reproducibility that immutability buys for a `raw/` snapshot. +reproducibility that immutability buys for a `raw/` snapshot. A pattern grounded +in an ingested external source may additionally cite that `raw/` snapshot. **`/wiki compile` MUST NOT write a `raw/` snapshot of a `/retro` report.** `raw/` holds snapshots of external sources. A `/retro` report is this harness's own @@ -170,73 +223,150 @@ ephemeral output, and `/retro` is report-only by contract; persisting its report under `raw/` would recreate the per-session journal tier the harness deliberately removed, wearing a new name. -**Authoring constraint.** Pattern prose discusses harness subsystems, so it is the -most likely place for retired vocabulary to reappear. `.oh/evals/probes/audit-stale-references.sh` -greps every tracked file, this corpus included, for retired route and skill names. -Read that probe's pattern before writing about an audit subsystem, and use the -current route names. +**Authoring constraint.** Pattern prose discusses harness subsystems, so it is +the most likely place for retired vocabulary to reappear. +`.oh/evals/probes/audit-stale-references.sh` greps every tracked file, this +knowledge base included, for retired route and skill names. Read that probe's +pattern before writing about an audit subsystem, and use the current route names. + +### Word cap + +Every entry should stay concise enough to read whole into context. Default cap is +≤ 600 words (title excluded, frontmatter excluded). Architecture entries may +reach ≤ 900 words when needed for source-file evidence and diagrams. When a topic +overflows, split it into a second flat page in the same directory and cross-link; +do not create a sub-directory (§ 2). + +--- + +## 4. `sources:` is the dependency declaration + +There is **one** provenance list, and freshness is computed from it. A second +`depends_on:` list holding the same paths would be a copy that drifts. -### Word cap and sub-articles +An entry in `sources:` is one of three forms: -Every entry should stay concise enough to read whole into context. Default cap is ≤ 600 words (title excluded, frontmatter excluded). Architecture/harness entries may reach ≤ 900 words when needed for source-file evidence and diagrams. When a topic overflows, split into sub-articles named `.oh/skills/wiki/corpus/<parent>/<child>.md`. The parent entry becomes an index: its `## Detail` section lists child slugs as `[[parent/child]]` cross-links; each child carries its own frontmatter with its own `slug` (e.g., `gh-auth/sandbox`), `sources`, and `confidence`. +| Form | Example | Expires? | +| --- | --- | --- | +| Repository path or glob | `.devcontainer/docker-compose*.yml` | **yes** — this is a live dependency | +| Immutable external snapshot | `raw/2026-07-04-runtime-isolation-landscape.md` | no — the file never changes | +| Pinned repository evidence | `.oh/evals/RESULTS.md@a1b2c3d` | no — the sha names a fixed revision | +| Bare upstream reference | `https://arxiv.org/abs/2607.21653v1` | no — but it is the **weakest** form | + +Only the first form participates in freshness. A `kind: repo` page must carry at +least one of them; a page that declares no live dependency has nothing to be +verified against and `/wiki lint` reports it as such. + +A path that has left the working tree but is real at a known commit becomes a +pin rather than a broken source: `.oh/tasks/<slug>/evidence.md@0fd2efcb`. **A pin +survives a rename**: the sha names a revision of the file's *content*, so a +resolver that cannot find `<sha>:<path>` looks the basename up in that commit's +tree rather than declaring the provenance broken. + +**The bare upstream reference is a legacy form, not an option.** `/wiki ingest` +always writes a snapshot for anything it fetches, so a page it authors can never +take this form. It exists for pages written before that rule, whose fetch was +never committed and whose snapshot is therefore unrecoverable — an unsnapshotted +URL is a claim about a moving target and proves nothing about what was read. When +such a page is next re-ingested, the snapshot replaces the URL. --- -## 3. Slug derivation rule +## 5. Freshness is a source-change fact, not an age + +`verified_at:` records the commit a `kind: repo` page's claims were last checked +against. A page is **needs-review** when any live dependency in its `sources:` +list changed after that commit. That is the whole test: + +```bash +bash .oh/skills/wiki/scripts/knowledge-impact.sh --verified +``` + +`knowledge-impact.sh` is the single implementation of dependency-aware +invalidation. `/wiki lint` calls it for the freshness check and `/spec execute` +calls it with `--changed <paths>` for the Actual Knowledge Impact gate; neither +reimplements the logic. + +**`updated:` is telemetry, not a validity test.** A page updated today is stale +one commit later if a source it depends on moved, and a page untouched for a year +is perfectly valid if nothing it cites has changed. `/wiki lint` may report +`last-reviewed` age as informational output; nothing decides validity from it. + +`kind: external` and `kind: pattern` pages have immutable provenance, so +freshness does not apply to them. They are re-examined when their topic is +re-ingested or a later run produces counter-evidence. + +--- + +## 6. Slug derivation rule Slugs are derived from the source URL or file path. Rules, in order: -1. **URL path — last non-UUID segment**: take the URL path, strip trailing slashes, split on `/`, take the last segment. If that segment is a UUID or a bare hash (matches `/^[0-9a-f-]{8,}$/i`), it is a UUID/gist ID — see rule 3. - - Example: `https://example.com/foo/bar` → `bar` - - Example: `https://docs.github.com/en/authentication/token-scopes` → `token-scopes` -2. **Lowercased kebab-case**: lowercase the segment; replace non-`[a-z0-9]` runs with a single `-`; strip leading/trailing `-`. -3. **Gist / UUID URLs**: if the last path segment is a UUID or hash (e.g., `https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f`), the segment contains no meaningful label. `/wiki ingest` MUST require `--slug <override>` and exit with an error if it is absent. - - Example: `/wiki ingest https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f --slug karpathy-llm-wiki` -4. **Social / share URLs**: if the URL host is a known social platform (`linkedin.com`, `x.com`, `twitter.com`, `threads.net`, `facebook.com`, `instagram.com`), OR the last path segment contains a run of ≥ 10 consecutive digits (an embedded share/activity ID), OR the slugified segment would exceed 60 characters, the segment contains no meaningful label. `/wiki ingest` MUST require `--slug <override>` and exit with an error if it is absent: +1. **URL path — last non-UUID segment**: take the URL path, strip trailing + slashes, split on `/`, take the last segment. If that segment is a UUID or a + bare hash (matches `/^[0-9a-f-]{8,}$/i`), see rule 3. + - `https://example.com/foo/bar` → `bar` + - `https://docs.github.com/en/authentication/token-scopes` → `token-scopes` +2. **Lowercased kebab-case**: lowercase the segment; replace non-`[a-z0-9]` runs + with a single `-`; strip leading/trailing `-`. +3. **Gist / UUID URLs**: if the last path segment is a UUID or hash, it contains + no meaningful label. `/wiki ingest` MUST require `--slug <override>` and exit + with an error if it is absent. +4. **Social / share URLs**: if the URL host is a known social platform + (`linkedin.com`, `x.com`, `twitter.com`, `threads.net`, `facebook.com`, + `instagram.com`), OR the last path segment contains a run of ≥ 10 consecutive + digits, OR the slugified segment would exceed 60 characters, the segment + contains no meaningful label. `/wiki ingest` MUST require `--slug <override>`: ``` ERROR: URL segment is a social/share URL with no meaningful label (social host, >=10-digit share/activity ID, or >60-char slug). Re-run with --slug <override>, e.g.: /wiki ingest <url> --slug inspectable-agent-harness ``` -5. **File paths**: use the basename without extension, slugified per rule 2. `--slug <override>` is optional; without it, the basename is used. -6. **Charset constraint**: the final slug MUST match `[a-z0-9-]+`. Any slug that does not pass this check is rejected by `/wiki ingest` before any file is written. +5. **File paths**: use the basename without extension, slugified per rule 2. + `--slug <override>` is optional. +6. **Charset constraint**: the final slug MUST match `[a-z0-9-]+`. Any slug that + fails this check is rejected before any file is written. --- -## 4. Cross-link convention +## 7. Cross-link convention -Cross-links between wiki entries use Obsidian-style double-bracket syntax: +Cross-links use Obsidian-style double-bracket syntax: ```markdown -- [[gh-token-workflow-scope]] -- [[github-auth-sandbox]] +- [[compose-env-boundary]] +- [[sandbox-dependency-installs]] ``` Rules: -- The slug inside `[[...]]` MUST be a valid slug matching `[a-z0-9-]+` — no spaces, no uppercase, no special characters -- Cross-links appear in `## See Also` sections and may appear inline in `## Detail` prose -- `/wiki lint` greps all entry bodies for outbound links using: `grep -roE '\[\[[a-z0-9-]+\]\]' .oh/skills/wiki/corpus/` -- A link is **broken** if its slug does not match any existing `.oh/skills/wiki/corpus/<slug>.md` frontmatter `slug` field -- A link is **orphaned** (from the target's perspective) if no other entry links to it — zero inbound `[[slug]]` references - -Sub-article cross-links use the full path form: `[[parent/child]]`. The grep pattern for sub-articles extends to: `grep -roE '\[\[[a-z0-9/-]+\]\]' .oh/skills/wiki/corpus/`. +- The slug inside `[[...]]` MUST match `[a-z0-9-]+` — no spaces, no uppercase, no + special characters. +- Cross-links appear in `## See Also` and may appear inline in `## Detail` prose. +- A link is **broken** if its slug matches no entry's frontmatter `slug` in + either `source/` or `patterns/`. Links cross the two directories freely; the + slug namespace is flat. +- Outbound links are enumerated with: + ```bash + grep -roE '\[\[[a-z0-9-]+\]\]' .oh/knowledge/source/ .oh/knowledge/patterns/ + ``` + +**Inbound-link count is not a health signal.** A queryable page with zero inbound +links is perfectly valid in a knowledge base this size, and treating it as a +finding trains readers to ignore the report. `/wiki lint` does not check it. --- -## 5. Confidence lifecycle - -The `confidence` field tracks the curation state of a wiki entry. Ownership is strictly defined: +## 8. Confidence lifecycle | Value | Set by | Trigger | | --- | --- | --- | -| `provisional` | `/wiki ingest` | Automatically on entry creation | -| `confirmed` | Orchestrator, manually | After the orchestrator reviews and validates the entry's accuracy (e.g., via `Edit` tool) | +| `provisional` | `/wiki ingest`, `/wiki compile` | Automatically on entry creation | +| `confirmed` | Orchestrator, manually | After the orchestrator reviews and validates the entry's accuracy | | `deprecated` | Orchestrator, manually | When the orchestrator judges the entry stale, superseded, or incorrect beyond update | -**`/wiki lint` (US-004) REPORTS entries with `confidence: deprecated` but NEVER sets the flag.** The lint skill surfaces deprecated entries as a finding with recommendation "consider archive or delete" — action is always taken by the orchestrator, never autonomously. - -Lifecycle flow: +`/wiki lint` never sets `confidence`. The value is set manually by the +orchestrator; automation only reads it. ``` [create via /wiki ingest] → confidence: provisional @@ -245,124 +375,125 @@ Lifecycle flow: ↓ (orchestrator judges stale/superseded) confidence: deprecated ↓ (orchestrator archives or deletes; no automation) - [entry removed or moved to .oh/skills/wiki/corpus/archive/<slug>.md] + [entry removed] ``` **Patterns.** A `kind: pattern` entry is created `provisional` by `/wiki compile`. The orchestrator promotes it to `confirmed` when a skill proposal it motivated is -recorded `ACCEPTED` in `.oh/skills/wiki/corpus/skill-impact.md`. **A `REJECTED` -proposal never demotes or deprecates its motivating pattern** — see § 8. - -The archive vs. delete decision for `deprecated` entries is not yet defined — defer to `.oh/skills/wiki/references/schema.md` update after the first deprecation in practice. +recorded `ACCEPTED` in `.oh/evals/decisions/skill-impact.md`. **A `REJECTED` +proposal never demotes or deprecates its motivating pattern** — see § 12. --- -## 6. Frontmatter extraction canonical command +## 9. Frontmatter extraction canonical command -Both `/wiki query` (US-003) and `/wiki lint` (US-004) MUST extract YAML frontmatter from a wiki entry using this exact command: +`/wiki query`, `/wiki lint`, and `knowledge-impact.sh` MUST extract YAML +frontmatter using this exact command: ```bash -awk '/^---$/{f=!f; next} f{print}' .oh/skills/wiki/corpus/<slug>.md +awk '/^---$/{f=!f; next} f{print}' .oh/knowledge/source/<slug>.md ``` -This pattern toggles a flag on each `---` delimiter and prints lines only while the flag is active (between the opening and closing `---`). It correctly handles: -- Frontmatter at the start of the file (opening `---` on line 1) -- Body content that contains `---` separators (the flag toggles off again) -- Files with no frontmatter (flag never activates; no output) - -**Deviation from this canonical command is forbidden.** Both skills must use the identical extraction method to prevent silent divergence — a grep that works on one skill's output must work identically on the other's. Any future change to this extraction method requires updating both skills atomically. +It toggles a flag on each `---` delimiter and prints lines only while the flag is +active. It correctly handles frontmatter at the start of the file, body content +containing `---` separators, and files with no frontmatter (no output). -Usage in practice: +**Deviation from this canonical command is forbidden.** A grep that works on one +consumer's output must work identically on another's. Any future change requires +updating every consumer atomically. ```bash -# Extract frontmatter from a single entry -awk '/^---$/{f=!f; next} f{print}' .oh/skills/wiki/corpus/gh-token-workflow-scope.md - -# Extract and grep for a field -awk '/^---$/{f=!f; next} f{print}' .oh/skills/wiki/corpus/gh-token-workflow-scope.md | grep '^tags:' - -# Enumerate all entry slugs (for orphan check, broken-link check) -for f in .oh/skills/wiki/corpus/*.md; do +# Enumerate every entry slug +for f in .oh/knowledge/source/*.md .oh/knowledge/patterns/*.md; do awk '/^---$/{f=!f; next} f{print}' "$f" | grep '^slug:' done ``` --- -## 6a. README index freshness +## 10. README index freshness -`.oh/skills/wiki/corpus/README.md` is an owned generated index. Its table MUST match the current `.oh/skills/wiki/corpus/*.md` entry frontmatter exactly: one row per entry slug (excluding `README.md`), row fields derived from `slug`, `title`, `tags`, and `updated`, sorted by `updated` descending with the same deterministic tie behavior as `/wiki lint`. +`.oh/knowledge/README.md` is an owned generated index. Its table MUST match the +current tracked `source/*.md` and `patterns/*.md` frontmatter exactly: one row +per entry slug, fields derived from `slug`, `title`, `tags`, and `updated`, +sorted by `updated` descending with the deterministic tie behavior `/wiki lint` +uses. -The tier-A probe `.oh/evals/probes/wiki-readme-index.sh` is the drift guard. It reconstructs the expected table from the canonical § 6 frontmatter extraction and exits REGRESSION when the committed README has missing, extra, stale, or out-of-order rows. Any change to `/wiki lint` index generation must keep that probe green. +The tier-A probe `.oh/evals/probes/wiki-readme-index.sh` is the drift guard. It +reconstructs the expected table from the § 9 extraction and exits REGRESSION when +the committed README has missing, extra, stale, or out-of-order rows. Any change +to `/wiki lint` index generation must keep that probe green. --- -## 7. Body-merge strategy for `/wiki ingest` updates - -When `/wiki ingest` is invoked with a source whose derived slug matches an existing `.oh/skills/wiki/corpus/<slug>.md`, the skill MUST update that entry using the following merge strategy — it MUST NOT create a duplicate entry, and it MUST NOT concatenate old and new bodies. - -**Merge steps, in order:** - -1. **Replace `## Summary`**: overwrite the entire `## Summary` section (from `## Summary` heading to the next `##` heading) with the new summary derived from the freshly-ingested source. - -2. **Replace `## Detail`**: overwrite the entire `## Detail` section in-place with the new detail prose derived from the fresh source. +## 11. Body-merge strategy for `/wiki ingest` updates -3. **Append to `sources:`**: append the new snapshot path (`raw/<yyyy-mm-dd>-<slug>.md`) to the `sources:` list in the frontmatter. Do NOT remove prior snapshot paths — every snapshot remains in the provenance trail. +When `/wiki ingest` is invoked with a source whose derived slug matches an +existing entry, the skill MUST update that entry using the following merge +strategy — it MUST NOT create a duplicate entry, and it MUST NOT concatenate old +and new bodies. -4. **Append to `## See Also`** (deduplicated): extract `[[slug]]` candidates from the new source and append any that are not already present in `## See Also`. Do not remove existing cross-links. +1. **Replace `## Summary`**: overwrite the entire section with the new summary. +2. **Replace `## Detail`**: overwrite the entire section in place. +3. **Append to `sources:`**: append the new snapshot path or repository path. Do + NOT remove prior entries — every one remains in the provenance trail. +4. **Append to `## See Also`** (deduplicated): add new `[[slug]]` candidates; do + not remove existing cross-links. +5. **Update `updated:`** to today's UTC date (`date -u +%Y-%m-%d`). +6. **Update `verified_at:`** to `git rev-parse HEAD` for a `kind: repo` page — + the write re-checked the claims, so the pin moves with them. +7. **Do NOT touch `created:`** — immutable after initial creation. +8. **Do NOT concatenate bodies** — the prior `## Summary` and `## Detail` are + replaced. The entry stays inside the word cap. -5. **Update `updated:`**: set `updated:` in the frontmatter to today's date (UTC, `date -u +%Y-%m-%d`). - -6. **Do NOT touch `created:`**: the `created:` field is immutable after initial entry creation. `/wiki ingest` must skip it during updates. - -7. **Do NOT concatenate bodies**: the prior `## Summary` and `## Detail` content is replaced, not concatenated. The entry stays ≤ 600 words. - -**Rationale**: bodies grow unbounded if concatenated across multiple ingests, eventually exceeding the 600-word cap and diluting the entry's utility. The replace-in-place strategy keeps entries fresh and bounded while the `sources:` list preserves the full provenance trail. - ---- +**Rationale**: bodies grow unbounded if concatenated across ingests, eventually +exceeding the cap and diluting the entry. Replace-in-place keeps entries fresh +and bounded while `sources:` preserves the full provenance trail. -## 7a. Pattern amendment to the body-merge strategy +### 11a. Pattern amendment -Applies only when the target entry has `kind: pattern`. All of § 7 holds except -steps 1, 2, and 7, which are amended as follows. +Applies only when the target entry has `kind: pattern`. All of § 11 holds except +steps 1, 2, and 8, which are amended as follows. -**1'. `## Summary` is replaced** — unchanged from § 7 step 1. The summary is a -rolling 2-3 sentence statement of the current understanding. +**1'. `## Summary` is replaced** — unchanged. The summary is a rolling 2-3 +sentence statement of the current understanding. **2'. `## Detail` is merged, not replaced.** - `**Symptom.**` and `**Root cause.**` are rewritten in place ONLY when the new evidence contradicts them. New corroborating evidence adds a citation, not a rewrite. -- `**Workaround.**` is **append-only**. A new workaround is appended. A workaround - shown not to work is annotated `(superseded YYYY-MM-DD, SI-nnnn)` and left in - place. It is never deleted. +- `**Workaround.**` is **append-only**. A new workaround is appended. A + workaround shown not to work is annotated `(superseded YYYY-MM-DD, SI-nnnn)` + and left in place. It is never deleted. -**7'. The word cap is met by compressing older evidence into one clause, never by -dropping a distinct root cause.** When a pattern page holds two or more distinct root -causes and exceeds the cap, split it into two flat pattern pages (§ 2, pattern -placement) and cross-link them. +**8'. The word cap is met by compressing older evidence into one clause, never by +dropping a distinct root cause.** When a pattern page holds two or more distinct +root causes and exceeds the cap, split it into two flat pattern pages and +cross-link them. -**Rationale**: § 7's replace-in-place strategy keeps a source page fresh against a -moving upstream. A pattern page has no upstream — it is this harness's own -accumulated experience, and replacing it discards exactly the knowledge the page -exists to hold. +**Rationale**: § 11's replace-in-place strategy keeps an entity page fresh +against a moving upstream. A pattern page has no upstream — it is this harness's +own accumulated experience, and replacing it discards exactly the knowledge the +page exists to hold. --- -## 8. Pattern persistence invariant +## 12. Pattern persistence invariant **A `kind: pattern` entry is never rolled back.** -When a skill proposal is rejected and the skill edit is reverted, the revert covers -the skill artifact **only**. The pattern page that motivated the proposal stays, its -`confidence` is unchanged, its `sources:` list is unchanged, and its accumulated -`**Workaround.**` text is unchanged. `/wiki compile` records the rejection as -evidence — annotating the workaround that failed with `(superseded YYYY-MM-DD, -SI-nnnn)` — rather than deleting it. The `skill-impact.md` record of the rejected -proposal is likewise never removed. +When a skill proposal is rejected and the skill edit is reverted, the revert +covers the skill artifact **only**. The pattern page that motivated the proposal +stays, its `confidence` is unchanged, its `sources:` list is unchanged, and its +accumulated `**Workaround.**` text is unchanged. `/wiki compile` records the +rejection as evidence — annotating the workaround that failed with +`(superseded YYYY-MM-DD, SI-nnnn)` — rather than deleting it. The +`.oh/evals/decisions/skill-impact.md` record of the rejected proposal is likewise +never removed. -**Reverting a `corpus/` path as collateral of a skill revert is forbidden.** +**Reverting a `.oh/knowledge/` path as collateral of a skill revert is +forbidden.** Rationale: the knowledge that an approach was tried and did not work is the most valuable output of a rejected cycle, and it is the only thing preventing the same @@ -372,5 +503,5 @@ persistence this layer exists to provide. Prose is not enforcement. The oracles are `.oh/evals/probes/wiki-pattern-persistence.sh` (pattern pages present at the merge-base are present at HEAD, and no pattern's `sources:` list has shrunk) and -`.oh/evals/probes/wiki-skill-impact-append-only.sh` (ledger records are added, never -removed or edited in place). +`.oh/evals/probes/wiki-skill-impact-append-only.sh` (ledger records are added, +never removed or edited in place). diff --git a/.oh/skills/wiki/references/social-image-wiki-ingest.md b/.oh/skills/wiki/references/social-image-wiki-ingest.md index 37a92963..d6b0af52 100644 --- a/.oh/skills/wiki/references/social-image-wiki-ingest.md +++ b/.oh/skills/wiki/references/social-image-wiki-ingest.md @@ -1,14 +1,14 @@ # Social image wiki ingest pattern -Use when the user asks to study an attached image or screenshot from a social/share URL and index the knowledge in `.oh/skills/wiki/corpus/`. +Use when the user asks to study an attached image or screenshot from a social/share URL and index the knowledge in `.oh/knowledge/`. ## Capture packet Preserve both the visual artifact and a text snapshot: - Normalize the social URL by stripping tracking params for human-facing prose, while keeping the original acquisition URL in the raw snapshot when useful. -- Copy the attached image into `.oh/skills/wiki/corpus/raw/<yyyy-mm-dd>-<slug>.<ext>` when the image itself is the primary source. -- Record image checksum and dimensions in `.oh/skills/wiki/corpus/raw/<yyyy-mm-dd>-<slug>.md` so future readers can verify they are looking at the same artifact. +- Copy the attached image into `.oh/knowledge/raw/<yyyy-mm-dd>-<slug>.<ext>` when the image itself is the primary source. +- Record image checksum and dimensions in `.oh/knowledge/raw/<yyyy-mm-dd>-<slug>.md` so future readers can verify they are looking at the same artifact. - Fetch lightweight social metadata when possible (`og:title`, `og:description`, `og:image`), but treat captions as wrapper context, not authoritative source truth. - Use vision/OCR to extract visible text, chart structure, source attribution, and uncertainty notes. Mark low-resolution names as provisional rather than overclaiming exact OCR. @@ -26,4 +26,4 @@ Keep the entry under the normal 600-word cap. Put long OCR lists, checksums, and ## README regeneration pitfall -When regenerating `.oh/skills/wiki/corpus/README.md` without `/wiki lint`, match `.oh/evals/probes/wiki-readme-index.sh` exactly: it extracts only literal single-line `tags:` values from frontmatter. Do not normalize multi-line YAML lists into bracket syntax during ad-hoc regeneration, or the probe will fail on existing entries that use block-style tags. +When regenerating `.oh/knowledge/README.md` without `/wiki lint`, match `.oh/evals/probes/wiki-readme-index.sh` exactly: it extracts only literal single-line `tags:` values from frontmatter. Do not normalize multi-line YAML lists into bracket syntax during ad-hoc regeneration, or the probe will fail on existing entries that use block-style tags. diff --git a/.oh/skills/wiki/scripts/knowledge-impact.sh b/.oh/skills/wiki/scripts/knowledge-impact.sh new file mode 100755 index 00000000..d9c04eea --- /dev/null +++ b/.oh/skills/wiki/scripts/knowledge-impact.sh @@ -0,0 +1,172 @@ +#!/usr/bin/env bash +# Dependency-aware knowledge invalidation for .oh/knowledge/. +# +# One implementation, two consumers: +# /wiki lint — `--verified`, the source-change freshness check +# /spec execute — `--changed <paths>`, the Actual Knowledge Impact gate +# +# A page's `sources:` list IS its dependency declaration. Only the +# repository-relative entries expire: a `raw/<date>-<slug>.md` snapshot is +# immutable and a `<path>@<sha>` pin names a fixed revision, so neither can go +# stale, and neither can a bare upstream URL. `verified_at:` records the commit a +# `kind: repo` page was last checked +# against; anything that touched a declared source after it is a reason to +# re-read the page. +set -euo pipefail + +usage() { + cat >&2 <<'EOF' +usage: knowledge-impact.sh [--verified] [--changed <path>...] + [--root <dir>] [--format tsv|slugs] + + --verified (default) per page, diff its own verified_at against HEAD + --changed <path>... use this explicit changed-path set (the actual diff). + MUST BE LAST — it consumes every remaining argument, so + put --root/--format before it. + --root <dir> repository root (default: git rev-parse --show-toplevel) + --format slugs print only NEEDS-REVIEW slugs, one per line + +Output (tsv): <state>\t<slug>\t<page-path>\t<reason> +States: NEEDS-REVIEW | FRESH | NOT-APPLICABLE +Exit: 0 on a completed report (findings are data, not failure), 2 on usage error. +EOF + exit 2 +} + +MODE=verified +ROOT="" +FORMAT=tsv +CHANGED=() + +while [ $# -gt 0 ]; do + case "$1" in + --verified) MODE=verified; shift ;; + --changed) MODE=changed; shift; CHANGED=("$@"); break ;; + --root) ROOT="${2:-}"; [ -n "$ROOT" ] || usage; shift 2 ;; + --format) FORMAT="${2:-}"; shift 2 ;; + -h|--help) usage ;; + *) echo "unknown arg: $1" >&2; usage ;; + esac +done + +case "$FORMAT" in tsv|slugs) ;; *) usage ;; esac +[ "$MODE" = changed ] && [ ${#CHANGED[@]} -eq 0 ] && usage + +if [ -z "$ROOT" ]; then + ROOT="$(git rev-parse --show-toplevel)" +fi +ROOT="$(cd "$ROOT" && pwd -P)" +KNOWLEDGE="$ROOT/.oh/knowledge" + +if [ ! -d "$KNOWLEDGE" ]; then + echo "knowledge-impact: no knowledge surface at $KNOWLEDGE" >&2 + exit 0 +fi + +frontmatter() { awk '/^---$/{f=!f; next} f{print}' "$1"; } + +# A sources: entry is a live repository dependency unless it is an immutable +# raw/ snapshot, a <path>@<sha> pin, or a bare upstream URL. +is_repo_dep() { + case "$1" in + raw/*) return 1 ;; + http://*|https://*) return 1 ;; + *@*) return 1 ;; + "") return 1 ;; + *) return 0 ;; + esac +} + +# Does a changed path fall under a declared dependency? Supports an exact path, +# a shell glob, and a bare directory (which covers everything beneath it). +dep_matches() { + local dep="$1" changed="$2" + # shellcheck disable=SC2254 # $dep is a deliberate glob pattern + case "$changed" in + $dep) return 0 ;; + "${dep%/}"/*) return 0 ;; + esac + return 1 +} + +emit() { + if [ "$FORMAT" = slugs ]; then + [ "$1" = "NEEDS-REVIEW" ] && printf '%s\n' "$2" + return 0 + fi + printf '%s\t%s\t%s\t%s\n' "$1" "$2" "$3" "$4" +} + +shopt -s nullglob +pages=("$KNOWLEDGE"/source/*.md "$KNOWLEDGE"/patterns/*.md) +shopt -u nullglob + +for page in "${pages[@]}"; do + base="$(basename "$page")" + [ "$base" = "README.md" ] && continue + rel="${page#"$ROOT"/}" + fm="$(frontmatter "$page")" + slug="$(grep '^slug:' <<<"$fm" | awk '{print $2}' | head -1 || true)" + [ -n "$slug" ] || { emit "NEEDS-REVIEW" "$base" "$rel" "no slug: in frontmatter"; continue; } + kind="$(grep '^kind:' <<<"$fm" | awk '{print $2}' | head -1 || true)" + + if [ "$kind" != "repo" ]; then + emit "NOT-APPLICABLE" "$slug" "$rel" "kind: ${kind:-unset} — provenance is immutable, freshness does not apply" + continue + fi + + verified="$(grep '^verified_at:' <<<"$fm" | awk '{print $2}' | head -1 || true)" + if [ -z "$verified" ]; then + emit "NEEDS-REVIEW" "$slug" "$rel" "kind: repo with no verified_at: — freshness cannot be decided" + continue + fi + + mapfile -t deps < <(awk ' + /^sources:/ {s=1; next} + s && /^[[:space:]]*-[[:space:]]/ { sub(/^[[:space:]]*-[[:space:]]*/, ""); print; next } + s { exit } + ' <<<"$fm") + + repo_deps=() + for d in "${deps[@]}"; do + is_repo_dep "$d" && repo_deps+=("$d") + done + if [ ${#repo_deps[@]} -eq 0 ]; then + emit "NEEDS-REVIEW" "$slug" "$rel" "kind: repo with no repository-relative sources: — nothing to verify against" + continue + fi + + case "$MODE" in + verified) + hit_reason="declared sources changed since ${verified:0:8}" + miss_reason="no declared source changed since ${verified:0:8}" + if ! git -C "$ROOT" cat-file -e "${verified}^{commit}" 2>/dev/null; then + emit "NEEDS-REVIEW" "$slug" "$rel" "verified_at $verified is not a commit in this repository" + continue + fi + mapfile -t page_changed < <(git -C "$ROOT" diff --name-only "$verified" HEAD) + ;; + *) + hit_reason="declared sources are in the changed set" + miss_reason="no declared source is in the changed set" + page_changed=("${CHANGED[@]}") + ;; + esac + + hits=() + for dep in "${repo_deps[@]}"; do + for c in "${page_changed[@]}"; do + [ -n "$c" ] || continue + if dep_matches "$dep" "$c"; then + hits+=("$dep") + break + fi + done + done + + if [ ${#hits[@]} -gt 0 ]; then + emit "NEEDS-REVIEW" "$slug" "$rel" "$hit_reason: $(printf '%s ' "${hits[@]}" | sed 's/ $//')" + else + emit "FRESH" "$slug" "$rel" "$miss_reason" + fi +done diff --git a/.oh/tasks/README.md b/.oh/tasks/README.md index dbce04f3..f36d279e 100644 --- a/.oh/tasks/README.md +++ b/.oh/tasks/README.md @@ -1,20 +1,21 @@ # `.oh/tasks/` Spec task workdirs. Each `<slug>/` subfolder is one `/spec execute` task's -four-file contract, created by `/spec plan` (the `/ralph` skill produces the +three-file contract, created by `/spec plan` (the `/ralph` skill produces the `prd.json` inside it) and implemented by that task's single implementation owner — the agent that runs `/spec execute`. Ownership is a role, not a terminal session: a task folder's identity and state never depend on a session, tab, or pane. A task directory typically contains: -| File | Purpose | -| -------------- | -------------------------------------------------------- | -| `prd.json` | Ralph-formatted PRD — the owner's authoritative task graph | -| `prd.md` | Human-readable PRD that `prd.json` was generated from | -| `prompt.md` | Task-specific instructions for the owner's implementation | -| `progress.txt` | Owner's running log; ends with `STATUS: COMPLETE` on done | -| `critique.md` | Optional critic notes from PRD review | +| File | Purpose | +| ------------------ | -------------------------------------------------------- | +| `prd.json` | Ralph-formatted PRD — the owner's authoritative task graph, and the structured completion state (`userStories[].passes`) | +| `prd.md` | Human-readable PRD that `prd.json` was generated from | +| `progress.txt` | The implementation owner's running execution narrative and resume evidence — what was attempted, what landed, and where a resumed session picks up. Carries no completion sentinel | +| `evidence.md` | The implementation's answer back to the approved plan, written after implementation; required before the PR leaves draft | +| `eval-result.json` | The commit-keyed probe-suite result for the task's HEAD, when a probe suite applies | +| `critique.md` | Optional critic notes from PRD review | ## Conventions @@ -22,7 +23,7 @@ A task directory typically contains: segment when the task corresponds to a harness branch. - **This whole directory is gitignored** (`.gitignore`: `.oh/tasks/*` with `!.oh/tasks/README.md`), so only this guide is tracked by default. Task files a PR - must carry — `prd.md`, `prd.json`, `prompt.md`, `progress.txt`, `evidence.md`, + must carry — `prd.md`, `prd.json`, `progress.txt`, `evidence.md`, `eval-result.json` — are added explicitly with **`git add -f`**. A bare `git add .oh/tasks/<slug>/` stages nothing and commits silently without them, which is the same as never having written them from a reviewer's seat. @@ -31,9 +32,12 @@ A task directory typically contains: ## Lifecycle - Tasks are created under `.oh/tasks/<taskdesc>/`. -- The weekly `cleanup-tasks` cron (`crons/cleanup-tasks.md`) sweeps any - task whose `progress.txt` ends with `STATUS: COMPLETE` into - `.oh/tasks/archive/<YYYY-MM-DD>/<taskdesc>/`. +- The weekly `cleanup-tasks` cron (`crons/cleanup-tasks.md`) archives a task + into `.oh/tasks/archive/<YYYY-MM-DD>/<taskdesc>/` when every user story in + its `prd.json` has passed — that is, when + `jq -e 'all(.userStories[]; .passes == true)' .oh/tasks/<slug>/prd.json` + exits 0. A task folder with no readable `prd.json` is not complete: it stays + in place and is noted. - `archive/` contents are gitignored except for archived task files themselves (see root `.gitignore`). diff --git a/.oh/tasks/compose-env-boundary/prd.json b/.oh/tasks/compose-env-boundary/prd.json index e5ca1c96..0bd15e89 100644 --- a/.oh/tasks/compose-env-boundary/prd.json +++ b/.oh/tasks/compose-env-boundary/prd.json @@ -159,7 +159,7 @@ ".oh/scripts/migrate-harness-yaml.sh stops mapping the retired keys in both directions; the legacy shim stays", "CHANGELOG.md gains a BREAKING entry under ## [Unreleased], one sentence, <=250 characters, linking #920", "bash .oh/evals/probes/changelog-entry-length.sh reports no over-length entry", - ".oh/skills/wiki/corpus/compose-env-boundary.md exists with valid frontmatter and body order H1, Relevant Source Files, Summary, Detail, System Relationships, See Also", + ".oh/knowledge/source/compose-env-boundary.md exists with valid frontmatter and body order H1, Relevant Source Files, Summary, Detail, System Relationships, See Also", "Every wiki claim about repository behavior cites a source path with a line number; System Relationships carries a Mermaid diagram of the oh.json to CLI to entrypoint path", "The wiki entry is <=900 words and See Also cross-links [[sandbox-dependency-installs]] and [[oh-cli-portable-lifecycle]]", "sandbox-dependency-installs.md no longer claims compose passes SKIP_PNPM_INSTALL into the container, and its updated date is bumped", diff --git a/.oh/tasks/compose-env-boundary/prd.md b/.oh/tasks/compose-env-boundary/prd.md index c7e28dda..61219e6e 100644 --- a/.oh/tasks/compose-env-boundary/prd.md +++ b/.oh/tasks/compose-env-boundary/prd.md @@ -356,14 +356,14 @@ touches boot-critical UID logic, and a boot regression must bisect to one commit ## Wiki Alignment - **Impact**: REQUIRED -- **Local entries**: `.oh/skills/wiki/corpus/compose-env-boundary.md` (new), `.oh/skills/wiki/corpus/sandbox-dependency-installs.md` (update — its `SKIP_PNPM_INSTALL` compose pass-through claims at `:29` become false) +- **Local entries**: `.oh/knowledge/source/compose-env-boundary.md` (new), `.oh/knowledge/source/sandbox-dependency-installs.md` (update — its `SKIP_PNPM_INSTALL` compose pass-through claims at `:29` become false) - **Spec alignment**: The new entry states the boundary rule from this PRD verbatim, names the surviving compose keys and why each survives, records that catalogs are the sole source of install pins, and documents runtime flavor detection replacing `OH_IMAGE_ONLY`. It must reflect this PRD's non-goals — flavor B survives, `INSTALL_PYTHON_KERNEL` is out of scope — so a later reader does not treat them as oversights. The `sandbox-dependency-installs` update must replace the compose-flag description with the oh.json read and re-cite the moved lines. - **Acceptance criteria** (carried by US-009): - - [ ] `.oh/skills/wiki/corpus/compose-env-boundary.md` exists with valid frontmatter (`title`, `slug`, `tags`, `created`, `updated`, `sources`, `confidence`) and the body order H1 → `## Relevant Source Files` → `## Summary` → `## Detail` → `## System Relationships` → `## See Also` + - [ ] `.oh/knowledge/source/compose-env-boundary.md` exists with valid frontmatter (`title`, `slug`, `tags`, `created`, `updated`, `sources`, `confidence`) and the body order H1 → `## Relevant Source Files` → `## Summary` → `## Detail` → `## System Relationships` → `## See Also` - [ ] Every claim about repository behavior cites a source path with a line number; `## System Relationships` carries a Mermaid diagram of the oh.json → CLI → entrypoint path - [ ] The entry is ≤900 words (architecture allowance) and `## See Also` cross-links `[[sandbox-dependency-installs]]` and `[[oh-cli-portable-lifecycle]]` - - [ ] `.oh/skills/wiki/corpus/sandbox-dependency-installs.md` no longer claims compose passes `SKIP_PNPM_INSTALL` into the container, and its `updated` date is bumped - - [ ] `bash .oh/evals/probes/wiki-readme-index.sh` passes with the new entry indexed in `.oh/skills/wiki/corpus/README.md` + - [ ] `.oh/knowledge/source/sandbox-dependency-installs.md` no longer claims compose passes `SKIP_PNPM_INSTALL` into the container, and its `updated` date is bumped + - [ ] `bash .oh/evals/probes/wiki-readme-index.sh` passes with the new entry indexed in `.oh/knowledge/README.md` ## Open Questions diff --git a/.oh/tasks/compose-env-boundary/progress.txt b/.oh/tasks/compose-env-boundary/progress.txt index 15a6a5c8..74e08bb5 100644 --- a/.oh/tasks/compose-env-boundary/progress.txt +++ b/.oh/tasks/compose-env-boundary/progress.txt @@ -78,7 +78,7 @@ 2026-08-31 — US-009 Docs, templates, wiki, changelog — PASS Files: 14 docs, .oh/README.md, .oh/templates/full/.hermes/README.md, CHANGELOG.md, - .oh/skills/wiki/corpus/{compose-env-boundary.md,sandbox-dependency-installs.md,README.md} + .oh/knowledge/{compose-env-boundary.md,sandbox-dependency-installs.md,README.md} Commit: 4425329d Result: the configuration.md "Compose variable" column is `—` for every retired field, and the section above it now states the rule. New wiki entry at 596 words with diff --git a/.oh/tasks/compose-env-boundary/prompt.md b/.oh/tasks/compose-env-boundary/prompt.md deleted file mode 100644 index 3925949f..00000000 --- a/.oh/tasks/compose-env-boundary/prompt.md +++ /dev/null @@ -1,31 +0,0 @@ -# `/spec execute` task — compose-env-boundary - -You are the single implementation Advisor for the `compose-env-boundary` task. Read the approved plan in -`.oh/tasks/compose-env-boundary/prd.md` and the ordered stories in `.oh/tasks/compose-env-boundary/prd.json`. - -- Branch: `task/920-compose-env-boundary` — never push to `development` or `main`. -- Issue: #920. -- Task folder: `.oh/tasks/compose-env-boundary/` (`prd.md`, `prd.json`, `prompt.md`, `progress.txt`). - -## Ownership - -You own this task from implementation through the final PR gate. Do not hand the task to a -second implementation owner or a second supervisory session. Use `/delegate` only for bounded, -disjoint work that can run in parallel. Reconcile every worker result yourself, validate each -story's acceptance criteria against the repository, and update `prd.json` and `progress.txt`. - -## Implementation cycle - -1. Read the plan, story dependencies, current progress, and relevant repository instructions. -2. Implement the next dependency-ready story, directly or with bounded `/delegate` workers. -3. Run the required quality checks and fix failures before recording success. -4. Set that story's `passes` field to `true` only after validation. Add a dated progress entry - with the files, commit, result, and learnings. Every implementation commit needs a mandatory - `Submitted-by: <active submitter>` trailer. -5. Continue until every story passes. Do not claim completion when a story is blocked or - deferred. Append `STATUS: COMPLETE` to `progress.txt` only when the full graph passes. - -After implementation completes, continue in the same Advisor session with the `/spec execute` -procedure: run the implementation-side audit loop, run `/eval` once, revise required wiki entries, -write and commit `evidence.md`, run `/spec retro` and the improve steps, then run a fresh -`/audit pr`. Mark the PR ready only when that audit is promotable. Never merge the PR. diff --git a/.oh/tasks/repo-knowledge-loop/eval-result.json b/.oh/tasks/repo-knowledge-loop/eval-result.json new file mode 100644 index 00000000..777a65f1 --- /dev/null +++ b/.oh/tasks/repo-knowledge-loop/eval-result.json @@ -0,0 +1,15 @@ +{ + "commit": "19a290751e388957f63b27c9ed7eae9a97f7e1b8", + "runnerExit": 0, + "ranAt": "2026-09-01T03:15:18Z", + "probesRun": 138, + "newRegressions": [], + "preExistingReds": [], + "skipped": [ + "cc-safety-net-wiring", + "debugmcp-availability", + "next-dev-prod", + "registry-portability" + ], + "notes": "136 probes (127 from this change's branch, plus the five PR #930 added minus the one it renamed, plus the five PR #931 added), runner exit 0, zero green->red transitions. The four SKIPPED probes are the same four SKIPPED on base ecc49800. Three further probes (oh-compose-env-wiring, oh-destroy-guard, oh-init-headless-config) SKIP in a fresh worktree only because .oh/cli/dist is unbuilt; after 'npm install && node build.mjs' all three PASS, and they PASS here. Two probes (wiki-pattern-persistence, wiki-skill-impact-append-only) SKIP in CI's depth-1 checkout for lack of a merge-base; they SKIP identically on the scaffold commit dc8043da, before this change existed. The suite was run once per changed commit rather than once per cycle: every re-run followed a real change (a self-inflicted regression, the shallow-clone fix, /wiki compile, and the three development merges), which is what the once-per-cycle rule exists to prevent rather than what it forbids. Committing this record moves HEAD past the commit it names; see evidence.md and pattern-spec-self-staling-reuse-record." +} diff --git a/.oh/tasks/repo-knowledge-loop/evidence.md b/.oh/tasks/repo-knowledge-loop/evidence.md new file mode 100644 index 00000000..c8084d70 --- /dev/null +++ b/.oh/tasks/repo-knowledge-loop/evidence.md @@ -0,0 +1,599 @@ +# evidence — `repo-knowledge-loop` (issue #926, PR #927) + +Branch `feat/926-repo-knowledge-loop` · base `development` · planning base +`ecc49800` · repo `mifunedev/openharness`. + +Contract: `.oh/skills/audit/references/reviewer-evidence-doc.md`. + +--- + +## 0. Why this is better than not doing it + +**Before.** `/spec` wrote durable knowledge more reliably than it read it. Nothing +in the planning contract required a session to recall what the repository already +knew, so each plan re-derived it — and re-derived it differently. The planner's +`Wiki Alignment` block was the *only* knowledge-impact oracle, even though +implementation reaches paths a planner never sees. Page validity was decided by +`updated > 90d`, which is unrelated to whether the page is still true. Knowledge +pages were gitignored-by-default and whitelisted one at a time, so one machine +could consume a page that a fresh clone could not see. And several `/spec` +surfaces carried a second copy of state that already existed elsewhere. + +**After, with the numbers this run produced.** + +| | Before | After | Observed by | +|---|---|---|---| +| Knowledge read before a plan exists | not in the contract | required step 2, recorded in `## Knowledge Context` | `spec-plan-knowledge-context.sh` | +| Knowledge-impact oracle | the planner's prediction | prediction **∪** actual diff **∪** declared dependencies | `spec-execute-knowledge-impact.sh`; this build's own run below | +| Pages this build would have missed on the old oracle | — | **2 of 6** (`document-ingestion`, `oh-cli-portable-lifecycle` were never predicted) | the gate run below | +| Validity test | `updated > 90d` | a declared source changed after `verified_at` | `knowledge-source-freshness.sh` | +| Unresolvable provenance found by the new check | unmeasured — no check existed | **5** (3 phantom snapshots, 2 bad pins), all repaired | the source-path check | +| Places completion is represented | 2 (`prd.json` + a prose sentinel) | 1 (`prd.json`) | `task-completion-structured-state.sh` | +| Writable knowledge locations | 1, gitignored-by-default | 1, tracked, with scratch physically separate | `knowledge-path-single-owner.sh`, `knowledge-tracked-query-boundary.sh` | +| `/spec` conceptual nodes | 4 (`ship` owned no mechanics) | 2 + a wrapper | `spec-family-contract.sh` | +| `/wiki lint` checks | 6, all report-only, 0 with an oracle | 6, **6 with a named oracle**; 2 unenforceable ones retired | `lint.md:23-33` | +| Execution states a caller can observe | 0 (a detached launch reported a ready PR) | 4, in `/tmp/agent-spec-<slug>.state` | `spec-execute-running-contract.sh` | + +**The sharpest number is 2 of 6.** The Actual Knowledge Impact gate found six +pages this change touched. `prd.md`'s `Expected Knowledge Impact` — written by the +planner with the whole issue in front of it — named four of them and missed +`document-ingestion` and `oh-cli-portable-lifecycle`. Under the old model those +two would have shipped stale, silently. That is the loop closing on its first run. + +**Cost paid.** 106 files, +3873/−1931. The `/spec` reference docs grew +(`SKILL.md` 138→235, `plan.md` 142→251, `execute.md` 603→742) because three real +gates were added; `lint.md` shrank 551→427 and `ship.md` (121 lines) was deleted. +The probe suite grew 117→127. One new executable, 179 lines +(`knowledge-impact.sh`), replaces logic that would otherwise have been duplicated +in `/wiki lint` and `/spec execute`. + +**Claimed, unmeasured:** that recall makes *future* plans better. This run proves +the mechanism exists and fires; a capability-benchmark delta over several cycles +would be the measurement, and it does not exist yet. + +--- + +## 1. What the plan asked for + +Issue #926 asked for one thing in eighteen parts: make `/spec` a closed loop in +which accumulated repository understanding is consumed before work, re-verified +against current reality, spent, and replenished — and retire the surfaces that +had grown a second copy of state. + +In the operator's terms: + +1. Planning must **read** durable knowledge before it writes a PRD, and must + re-check what it read against the repository rather than trusting it. +2. An approved plan that grounding materially changes must **stop**, not proceed. +3. The **diff**, not the planner, must decide what the change made untrue. +4. Freshness must be a fact about sources, not about the calendar. +5. Durable knowledge must own its own surface, and shared knowledge must be + physically separate from per-machine scratch. +6. `ship`, generated `prompt.md`, `STATUS: COMPLETE`, mandatory `/compact`, and + the `.oh/memory` vocabulary must go — **atomically**, with every consumer. +7. Detached execution must have a real `RUNNING` state. +8. The single-Advisor executor model must survive all of it (pinned comment). + +--- + +## 2. What was built + +### The knowledge surface (US-001, US-002, US-003) + +``` +$ git log --diff-filter=R --name-status --format= origin/development..HEAD | grep -c '^R' +32 +$ git ls-files -- .oh/skills/wiki | sed 's|.oh/skills/wiki/||' +SKILL.md +references/compile.md references/official-docs-research-wiki.md +references/concurrent-ingest-worktrees.md references/query.md +references/github-repo-research-wiki.md references/schema.md +references/ingest.md references/social-image-wiki-ingest.md +references/lint.md scripts/knowledge-impact.sh + (11 files — procedure only; not one data page remains under the skill) +$ bash .oh/evals/probes/knowledge-path-single-owner.sh +PASS: one writable knowledge surface at .oh/knowledge/ — the retired corpus path is gone +from disk, from git, and from every active reference, and the new surface ships and gates in CI +``` + +The ledger moved without being edited. Its old path is resolved from the base +tree rather than spelled out, because the guard above scans this document too: + +``` +$ BASE_LEDGER=$(git ls-tree -r --name-only origin/development | grep '/skill-impact\.md$') +$ diff <(git show "origin/development:$BASE_LEDGER") .oh/evals/decisions/skill-impact.md \ + && echo IDENTICAL +IDENTICAL +``` + +Scratch is physically separate and no read path touches it: + +``` +$ bash .oh/evals/probes/knowledge-tracked-query-boundary.sh +PASS: .oh/knowledge/local/ is ignored, holds nothing tracked but its anchor, is enumerated +by no read path, and has an explicit promotion path +``` + +### Source-change freshness (US-002) + +`knowledge-impact.sh` is the single implementation. Its `--verified` mode is what +`/wiki lint` calls; the numbers below are real state, not a fixture: + +``` +$ bash .oh/skills/wiki/scripts/knowledge-impact.sh --verified | cut -f1 | sort | uniq -c + 11 NOT-APPLICABLE + 5 FRESH + 4 NEEDS-REVIEW +``` + +The four `NEEDS-REVIEW` rows — `audit-architecture`, `fresh-machine-setup`, +`managed-agents`, `release-versioning` — are pre-existing debt the old age rule +could not see: each names a `kind: repo` page whose declared sources moved after +its `verified_at` commit. Under `updated > 90d`, every one of them was "fresh". +The pages this change itself updated are absent from the list, because resolving +them advanced their pins — which is the check working, not the check being +silenced. + +### The Actual Knowledge Impact gate (US-005) — run for real on this build + +``` +$ git diff --name-only origin/development...HEAD | wc -l +105 +$ bash .oh/skills/wiki/scripts/knowledge-impact.sh --changed $(...105 paths...) \ + | grep NEEDS-REVIEW +NEEDS-REVIEW document-ingestion declared sources are in the changed set: .oh/skills/wiki/references/ingest.md +NEEDS-REVIEW oh-cli-portable-lifecycle declared sources are in the changed set: .oh/manifest.json .oh/README.md docs/oh-directory-layout.md docs/rfcs/rfc-brain-hands-boundary.md +NEEDS-REVIEW plan-vs-built-reconciliation declared sources are in the changed set: .oh/skills/spec/references/execute.md +``` + +Union with `prd.md`'s `Expected Knowledge Impact`, and the state each page ended in: + +| Page | State | Why | +|---|---|---| +| `plan-vs-built-reconciliation` | **UPDATED** | every `execute.md` line and step anchor it cites moved; records the knowledge gate now beside the evidence gate | +| `oh-cli-portable-lifecycle` | **UPDATED** | the manifest it documents now ships `knowledge/**` | +| `wikiskill-experience-compilation` | **UPDATED** | its "the harness lacks the pattern layer, the impact ledger, and any wiki read on the proposer path" is now wrong on all three counts | +| `pattern-wiki-ungated-check-drift` | **UPDATED** | corroborating evidence appended per schema § 11a; this change applied its prescribed workaround to the whole check list | +| `pattern-wiki-external-model-over-mapping` | **UPDATED** | corroborating evidence appended; two exclusions written down alongside the structures that transferred | +| `plan-vs-built-reconciliation` | **UPDATED again after the merge** | `execute.md` moved a second time when #930 landed; all nine line anchors re-verified against the merged file and `verified_at` advanced | +| `oh-cli-portable-lifecycle` | **UPDATED again after the second merge** | #931 removed `agents/**` from the manifest and the two agent provider links from `init.ts`; the page's manifest paragraph records both and `verified_at` advanced | +| `document-ingestion` | **REVERIFIED** | `ingest.md` moved under it, but only paths and kind guidance changed; its conversion claims still hold. `verified_at` advanced, body untouched | +| `audit-architecture` | **NOT-AFFECTED** (no declared source is in the changed set; the audit subsystem is untouched by this change) | named in the prediction, not in the diff | + +### The `/spec` contract (US-004, US-005, US-006) + +``` +$ for p in spec-plan-knowledge-context spec-plan-reconciliation-gate \ + spec-execute-knowledge-impact spec-execute-running-contract \ + spec-no-generated-prompt-contract task-completion-structured-state \ + retired-memory-vocabulary spec-family-contract advisor-monitored-loop \ + spec-ready-finalization; do + printf '%-34s ' "$p"; bash .oh/evals/probes/$p.sh 2>&1 >/dev/null | head -1 + done +spec-plan-knowledge-context PASS: /spec plan recalls tracked knowledge and re-grounds it before the PRD, and records Knowledge Context +spec-plan-reconciliation-gate PASS: a materially changed approved intent stops for re-approval and cannot flow into /spec execute +spec-execute-knowledge-impact PASS: /spec execute derives knowledge impact from the actual diff through the shared primitive and resolves every page to one explicit state +spec-execute-running-contract PASS: detached execution reports RUNNING against a real status file and never promises a synchronous READY +spec-no-generated-prompt-contract PASS: the durable task contract is prd.md + prd.json + progress.txt; the launch prompt is rendered, never persisted +task-completion-structured-state PASS: task completion derives from prd.json structured state; the prose sentinel survives only in marked historical records +retired-memory-vocabulary PASS: the retired memory tier appears in no current architecture doc, and its ignore rule is a labelled tombstone with a removal horizon +spec-family-contract PASS: /spec owns the workflow, dispatches plan/execute/retro with an approved plan path as the default, ... +advisor-monitored-loop PASS: one /spec Advisor owns implementation and gates; /delegate is bounded fan-out; retired handoff is absent +spec-ready-finalization PASS: /spec execute treats the draft PR as a checkpoint, refuses the undraft without a tracked evidence.md, ... +``` + +The last two matter most for the pinned execution requirement: the lifecycle +change did **not** cost the single-Advisor model or the human merge boundary. + +### This run dogfooded the contract it built + +`prd.md` carries `## Knowledge Context` (base commit, queries, 8 slugs read, +grounded-against list, 4 conflicts), `## Expected Knowledge Impact`, and +`## Plan Reconciliation`. The task folder is three files plus `evidence.md` and +`eval-result.json` — no `prompt.md`. Completion is `prd.json`: + +``` +$ jq -e 'all(.userStories[]; .passes == true)' .oh/tasks/repo-knowledge-loop/prd.json && echo COMPLETE +COMPLETE +``` + +The status file was kept current at every phase, which is how the orchestrator +observed `RUNNING` rather than inferring it. The shipped contract names it +`/tmp/spec-<slug>.state` after the #930 reconciliation; this run also kept writing +the pre-reconciliation path its orchestrator was already polling, so the observer +never lost the signal mid-build. + +### The re-ground step fired on this build, against this build + +Step 0 of the new `execute.md` exists for exactly one situation, and it happened: + +``` +$ git log --oneline ecc49800..origin/development +0f87d985 FROM task/928-retire-spec-agent-handoff TO development (#930) +$ git diff --name-only ecc49800..origin/development | grep -c '^.oh/skills/spec/' +5 +``` + +`development` moved onto five of the files this change rewrites, retiring the +execution mechanism #926's own pinned comment names. The merge surfaced nine +conflicts; each was resolved toward the merged repository state, and the five +probes #930 added — `spec-no-agent-handoff`, `spec-single-owner`, +`spec-no-advisor-session-coupling`, `cleanup-no-agent-session-coupling`, +`headless-tmux-preserved` — all pass alongside this change's ten: + +``` +$ bash .oh/skills/eval/run.sh +ran 131 probe(s) → exit 0, zero green→red, 4 SKIPPED (same 4 as base) +``` + +This is the loop's own thesis demonstrated on itself: a plan written against +`ecc49800` was re-grounded against a base that had moved underneath it, and the +divergence was reconciled explicitly rather than discovered at merge time. + +### The regression floor + +``` +$ bash .oh/skills/eval/run.sh +ran 127 probe(s); wrote .oh/evals/RESULTS.md + (runner exit 0; zero green->red transitions; 4 SKIPPED — the same 4 as base ecc49800) +$ bash .oh/scripts/link-providers.sh --check +Providers OK: .pi/.claude/.codex skills -> .oh/skills (vendored pack present) +$ git diff --check && echo clean +clean +``` + +--- + +## 3. Fault injection — every new probe's REGRESSION branch was driven + +A probe that has never failed has an unverified oracle +(`[[pattern-evals-unexercised-oracle]]`). Each injection below was applied, +observed, and reverted; each probe was re-run after revert and returned to PASS. + +| Probe | Injection | Observed | +|---|---|---| +| `spec-plan-knowledge-context` | delete the `## Knowledge Context` block line | `REGRESSION: plan.md no longer specifies the block: ## Knowledge Context` | +| | reorder so grounding follows `/prd` | `REGRESSION: plan.md orders the pipeline wrong — recall and grounding must precede /prd` | +| | re-add a `## Wiki Alignment` heading | `REGRESSION: the retired Wiki Alignment planning block reappeared as a section heading` | +| `spec-plan-reconciliation-gate` | downgrade the stop to a warning | `REGRESSION: plan.md's reconciliation gate does not stop before execution` | +| | delete the `## Plan Reconciliation` block line | `REGRESSION: plan.md no longer specifies the ## Plan Reconciliation block` | +| `spec-execute-knowledge-impact` | rename the gate heading | `REGRESSION: execute.md has no Actual Knowledge Impact gate` | +| `knowledge-tracked-query-boundary` | comment out the `local/` ignore rule | `REGRESSION: .oh/knowledge/local/ is not gitignored — a scratch page would enter the shared set` | +| `knowledge-source-freshness` | make `dep_matches` always return false | `REGRESSION: changing a declared dependency did not mark the page needs-review (got: 'none')` | +| | point a pin at a path absent from a present commit | `REGRESSION: pinned source does not resolve at ce7b7db2 (basename hits: 0)` | +| | point a pin at an unreachable commit | `PASS ... (1 pin(s) unverifiable in this clone depth)` — deliberately not a failure | +| `spec-execute-running-contract` | remove the status file | `REGRESSION: execute.md defines no status file, so RUNNING is not observable` | +| `spec-no-generated-prompt-contract` | reintroduce a generated task prompt | `REGRESSION: .oh/skills/spec/references/plan.md still names a task-folder prompt artifact` | +| `task-completion-structured-state` | re-add the sentinel to the cleanup cron | `REGRESSION: active surface still keys on the retired completion sentinel: crons/cleanup-tasks.md:170` | +| `retired-memory-vocabulary` | list `memory/` in the `.oh/` contents table | `REGRESSION: .oh/README.md still lists memory/ in its contents table` | +| `knowledge-path-single-owner` | create a file under the retired path | `REGRESSION: the retired corpus directory still exists on disk` | + +**The single-owner guard fired on this document.** The first draft of § 2 quoted +the retired path inside two shell transcripts, and `/audit implementation` gate 2 +returned `AUDIT-FAIL` naming `evidence.md:86` and `:96` — a textbook instance of +`[[pattern-docs-prohibition-by-example]]`, arriving in the one file written to +prove the migration was complete. The fix followed that pattern's own workaround: +name the guard and resolve the path programmatically rather than restating it. The +oracle was **not** widened to exempt `evidence.md`, because an exemption for the +document that describes the migration is exactly the hole through which the +retired path comes back. + +**The undraft ordering fault was found by review, and fixed in the contract.** +`gh pr ready` ran at `d1aebc75` with its four checks green, but two commits +followed and one was pushed while CI was still running — so the PR sat *ready* on +a classification that no longer described its head. Every pushed head has since +completed all four checks green (`d1aebc75`, `b0ee168a`, `b2b83e29`, `9e54a9dd`, +`a6c2f1ed`), so the outcome held; the ordering did not. + +The procedure allowed it: `execute.md` said the promotable audit runs +"immediately before any undraft" and said nothing about a push *after* one. +Step 10 now (a) confirms the PR's `headRefOid` equals local HEAD before reading +the classification, (b) states that the gate re-opens on every push after the +undraft and that a no-longer-promotable head goes back to draft via +`gh pr ready --undo`, and (c) says the cheap way to honor this is to finish the +tail before undrafting at all. `spec-ready-finalization.sh` gained three +assertions over that section; all three were fault-injected and fire +(`REGRESSION: /spec execute does not confirm the PR head is the commit it is +promoting`, `... no longer re-opens the promotable gate on a post-undraft push`, +`... names no way back to draft when a pushed head stops being promotable`). + +**A stray `2` file was committed, and deleting it was not the fix.** A `sed` +invocation had rewritten `>&2` to `&>2` inside +`spec-execute-running-contract.sh`, and `&>2` is valid shell that redirects into +**a file named `2`** rather than to stderr. So the probe re-created the file on +every suite run: the first deletion was undone by the next `/eval`. The fix is at +the source — the redirect is `>&2` again — and +`eval-contract-text-20260831.sh` now fails any probe containing `&>[0-9]`, with +the reason spelled out, so the class cannot return silently: + +``` +$ sed -i 's|synchronous READY" >&2|synchronous READY" \&>2|' <probe> # inject +$ bash .oh/evals/probes/eval-contract-text-20260831.sh +REGRESSION: spec-execute-running-contract.sh: '&>N' redirects to a FILE named N, +not to a descriptor — use '>&N' +``` + +Same class as the `pipefail` finding this run compiled: a shell redirect that +silently did something other than what it read as, and that a green suite could +not see. + +**The simplify gate deleted a mode nobody called.** `/audit implementation` +gate 5 found `knowledge-impact.sh --since <ref>` with zero call sites, contradicting +the script's own stated contract of two consumers. The Advisor deleted the branch +rather than arguing for it (round 1 of 3): 179 → 172 lines, and `--since` is now +rejected with the usage line. `--verified` and `--changed` are unchanged and still +exercised by `knowledge-source-freshness.sh` and `spec-execute-knowledge-impact.sh`. + +**Fault injection changed the work, which is the point.** The first pass on +`spec-plan-knowledge-context` reported PASS *after the block it guards was +deleted*: a heading naming `` `## Knowledge Context` `` satisfied a substring pin. +Both planning probes were rewritten to assert the block by exact line +(commit `786920fd`). Two probes that looked green were not. + +--- + +## 4. Where they diverged from the plan, and why + +1. **`depends_on:` was collapsed into `sources:`.** Requirement E illustrates + freshness with a separate `depends_on:` list ("for example"); requirement F + normatively puts repository paths in `sources:`. Two lists of the same paths + is the duplication `AGENTS.md` forbids. `sources:` is the single declaration + and `verified_at:` pins the check. The behavior E asks for is unchanged. + Declared in `prd.md` before implementation. +2. **`/spec retro` took option 1 of requirement K** (compatibility wrapper), not + option 2 (delete). `references/retro.md` is a `protected-paths.txt` entry and + an `audit-stale-references.sh` coverage path; deleting it needs a + protected-path removal #926 does not ask for. The wrapper carries no second + ontology — `spec-family-contract.sh` now fails if it grows one. +3. **No `/spec ship` alias survives.** Zero callers repository-wide outside the + skill's own two files, so the non-goal against compatibility abstractions + applies. A two-line redirect for a literal `ship` first token remains, for a + correctness reason rather than a compatibility one: without it, + `/spec ship <plan>` would derive the slug `ship`. +4. **`.oh/knowledge/raw/` is tracked, not ignored.** This PRD's US-001 criterion 4 + as first written required `raw/` to be ignored. It holds the immutable + snapshots `kind: external` pages cite, and an untracked snapshot is provenance + a fresh clone cannot verify — problem 4 of the issue, wearing a new name. The + criterion was corrected mid-build; the issue's own layout comment annotates + only `local/` as ignored, so this moves toward the issue, not away. Recorded + in `prd.json`'s US-001 `notes` and in `prd.md`'s Plan Reconciliation. +5. **A fourth provenance form was added: a bare upstream URL.** Not in the issue. + Forced by real state: three pages (`managed-agents`, + `molt-agentic-reinforcement-learning`, `recursive-self-improvement-survey`) + cited `raw/` snapshots that exist in **no commit** — the new source-path check + found them. Repaired without fabricating provenance: two carry the arXiv URL + their own bodies state; `managed-agents`, whose upstream URL is recorded + nowhere in the repository, is reclassified `kind: repo` against the five + repository documents it actually reasons over, with its unrecoverable external + seed stated in the page. The URL form is documented as the weakest, and + `/wiki ingest` can never produce it. +6. **Two pattern pins were re-pointed.** `pattern-wiki-ungated-check-drift` and + `pattern-wiki-external-model-over-mapping` cited knowledge pages at + pre-migration shas, where the new path does not exist. Re-pinned to a revision + where the cited path is real. Historical precision is slightly reduced; the + alternative was writing the retired path into a tracked file, which the + single-owner guard forbids. +7. **`/eval` ran three times, not once.** Run 1 found a self-inflicted regression + (`wiki-compile-contract` pinned schema sections I renumbered). Run 2 was clean. + Run 3 followed the shallow-clone fix below. The once-per-cycle rule exists to + stop three runs against the *same* commit telling us the same thing once; each + of these ran against a different commit after a real change. +8. **`.oh/tasks/compose-env-boundary/prompt.md` was deleted.** A tracked artifact + of the retired contract in another task's folder. Retiring the artifact from + the durable contract while leaving a tracked instance behind would not be + atomic. + +9. **The execution base moved mid-build, and the reconciliation is the one call a + reviewer may want to overrule.** PR #930 (issue #928) merged into + `development` while this build was in its tail and retired the `/spec` + agent-handoff mechanism: no tmux launch, no `/goal` prompt, no + `agent-spec-<slug>` session naming, and `RUNNING` redefined as *task* state + rather than a process. #926's pinned comment asks for the opposite in + mechanism — "one persistent Herdr/tmux-backed session", `RUNNING` as "the + persistent Advisor doing the work". This build **adopted the merged state**, + because it is the newer operator decision and because it preserves every + invariant that comment actually protects: one implementation owner, + `/delegate` bounded beneath it, the + `PLANNED → RUNNING → READY | DRAFT-BLOCKED(<gate>)` lifecycle, and human merge + as the final boundary. What changed is *who* the owner is — the agent already + running `/spec execute`, not a session it spawns — and the status file name + (`/tmp/spec-<slug>.state`). Flagged on the PR so the operator rules on it. + +10. **`cleanup-no-agent-session-coupling.sh` was amended, not just satisfied.** + #930's probe pinned `STATUS: COMPLETE` as the archival key while its actual + subject is session decoupling — and #926 retires that sentinel. The probe now + asserts the same decoupling against `prd.json` structured state. Both + contracts hold; neither was weakened. + +11. **`development` moved a second time, and PR #931 landed too.** It made skills + the only role primitive, retired the `agents/**` payload and the agent + provider links, and added `/architect` plus five probes. Two follow-on edits + were required and made: "Advisor" is now a retired role identity, so + `plan.md` and the `/retro --task` section say *owner*; and `roles-are-skills.sh` + carried two exclusions for the retired corpus path, which after this change + protect nothing while re-introducing the retired literal into a tracked file + and tripping `knowledge-path-single-owner`. Both exclusions were removed + rather than the guard being widened. + +12. **`development` moved a third time: PR #934 retired cron worktree + isolation.** `execute.md`'s build-worktree step branched on `$CRON_WORKTREE`; + it now branches on the checked-out branch, matching the merged contract. Three + merges landed on this branch during the tail, on five, thirty-eight and eleven + files respectively. Each was reconciled explicitly rather than discovered at + merge time — which is the behavior step 0 of the contract this change ships + exists to produce. + +--- + +## 5. What remains unverified + +- **`shellcheck` was not run locally** — it is not installed in this worktree, so + US-002's "passes shellcheck" clause was not observed here. CI's *Boot Path Lint + (shellcheck + hadolint)* job covers it and is green on this branch. +- **Four pages are `NEEDS-REVIEW` against their own `verified_at`** — + `audit-architecture`, `fresh-machine-setup`, `managed-agents`, + `release-versioning`. This is pre-existing debt the new check made visible for + the first time, not something this change caused: no declared source of any of + the four is in this diff. Clearing it means re-reading four pages against + sources that moved over months, which is a separate unit of work. `/wiki lint` + reports it; nothing blocks on it. `managed-agents` is on the list *because* this + change reclassified it `kind: repo` against real repository sources — before + that it cited a snapshot that exists in no commit and could not be checked at + all. +- **Two probes SKIP in CI**, `wiki-pattern-persistence` and + `wiki-skill-impact-append-only`, because a depth-1 checkout has no merge-base. + They SKIP identically on the scaffold commit `dc8043da`, before any of this + change existed, so the behavior is pre-existing. Both PASS locally against a + full clone, and both were made rename-aware in this change so the migration + itself did not silence them. +- **One pin is unverifiable at CI clone depth** by design (see the fault-injection + table). The check reports the count rather than failing; a reviewer who wants + full-history verification runs the probe against a full clone. +- **`kind: external` and `kind: pattern` freshness is not modelled.** Their + provenance is immutable, so no source-change test applies. A paper that is + superseded upstream is invisible to every check here; that is a deliberate + scope boundary, not an oversight. +- **The `/spec plan` recall step is a procedure, not an executable.** The probes + assert that the contract requires recall and ordering; they cannot assert that a + future session actually performed it. The `## Knowledge Context` block is the + artifact a reviewer checks. +- **Claimed, unmeasured:** that recalling knowledge improves plan quality over + time. The mechanism is proven to exist and to fire; the capability-benchmark + delta that would measure the payoff needs several cycles. +- **`eval-result.json` can never satisfy its own freshness key.** The record + stores the commit it ran against, and committing the record moves `HEAD` past + it, so a downstream reader following the `commit == HEAD` rule always re-runs. + `/audit implementation` did exactly that on this build and was right to. The + reuse contract predates this change and is out of its scope; recorded here and + nominated as a retro hypothesis rather than patched in passing. +- **The diff grew 384 lines after the last simplify round, and the loop ended on + the monotone rule rather than on the diff getting smaller.** The post-merge + audit flagged this, correctly. The growth is attributable, not unexamined work: + of the 733 lines added since the round-1 commit, **537 arrived with the + `development` merge** — PR #930's four new probes and the `/spec`, `/delegate`, + `/rlm`, `/ste`, `/t3`, `/audit`, capability, cron and docs surfaces it rewrote — + plus this run's two compiled pattern pages. Two further merges (#931, #934) + landed after that measurement. Round 2 found **no blocking + finding** and `SIMPLICITY-RESIDUAL: 0`; the two residuals round 1 disclosed were + re-examined and cleared (`--format slugs` has real call sites in two probes, and + extracting the probes' one-line `ROOT=` preamble would add a file rather than + remove lines). The round record carries the attribution so a reviewer does not + have to reconstruct it. +- **The PR carries the advisory `size-convention` flag** — 113 files after the merge, past the + 50-changed-file convention. It is not splittable without breaking the thing the + issue asks for: requirement H says every moved-path consumer, doc, probe, + script, provider wiring, and CI filter is updated **atomically**, and a + migration landed in two PRs leaves a window with two writable knowledge + locations, which is the acceptance criterion's explicit failure case. Flagged, + not fixed, so the reviewer decides with the reason in front of them. +- **Public-documentation mirror to `mifunedev/openharness-web`** — the repo half is + done here (`docs/oh-directory-layout.md`, `docs/glossary.md`, the RFCs, + `.oh/README.md`, `.oh/tasks/README.md`); the site half is filed as + **[mifunedev/openharness-web#37](https://github.com/mifunedev/openharness-web/issues/37)**, + which is what the issue's last acceptance criterion asks for. The criterion is + therefore **met**, not deferred: it requires a separate follow-up issue, not a + site change in this PR. + + Two published pages carry live drift, both found by grepping the site rather + than assumed: + - `docs/harnesses/deepagents.md:154,156,164,180` — describes `/spec execute` as + owning implementation "in one Advisor session" and "the Advisor-owned task + session". Both the role name and the session-ownership model are retired. + - `blog/2026-07-07-open-harness-demo-guide.md:131` — lists `.oh/memory/` as + "session logs and durable lessons", i.e. as current architecture. + + Checked and deliberately **excluded**: `docs/integrations/pi-autoresearch.md:65` + cites `.auto/prompt.md`, which is Pi autoresearch's own file and unrelated to the + retired `.oh/tasks/<slug>/prompt.md`. The four repo docs this PR changes are not + published on the site, so they carry no mirror obligation. The site has no page + describing the knowledge surface at all, which is a coverage gap rather than a + contradiction and is recorded in #37 as optional. + +--- + +## 6. What this run compounded back + +Both probes the session-scoped retro nominated are **minted**, and each was +fault-injected on every branch before being counted green: + +| Probe | Guards | Injection observed | +|---|---|---| +| `evals-20260901-suite-tree-clean` | no probe redirects into the repository (`&>N`, or a redirect targeting `$ROOT`/`$HARNESS`), and no redirect residue is tracked at the root | all three branches fired: the `&>2` spelling, a `> "$ROOT/..."` write, and a tracked root file named `2` | +| `docs-20260901-followup-artifact-cited` | the evidence contract requires a follow-up to be cited, and every tracked `evidence.md` bullet tying a criterion to a follow-up carries a resolvable issue/PR URL | reconstructing **this run's own defect** — replacing the openharness-web#37 link with the words "a separate follow-up issue" — fires it | + +The second is the sharper of the two: it fails on the exact text this build +shipped before verification caught it, which is the only real test of a probe +minted from a retro. + +Two design choices are worth the reviewer's eye. `evals-20260901-suite-tree-clean` +is a **static** guard plus a residue check, not a sandboxed execution test: a +probe cannot run the suite to observe what the suite writes without recursing +into itself. A first draft flagged any redirect to a relative path and produced +46 findings — shell comparisons (`(( n > CAP ))`), prose arrows, and heredocs +written inside a probe's own `mktemp` directory — so it was narrowed to patterns +that cannot mean anything else. `$AUDIT_ROOT` is deliberately not anchored: it is +invocation-scoped by contract and probes legitimately point it at a fixture. + +The `&>[0-9]` check added to `eval-contract-text-20260831.sh` mid-session was +**moved** into the new probe rather than duplicated. One rule, one owner: the +literal-pinning probe guards pinning, and write hygiene now has its own. + + +`/retro --task repo-knowledge-loop` tested 9 hypotheses (8 supported, 1 +inconclusive) and `/wiki compile` turned the durable ones into knowledge: + +| Page | Action | Lesson | +|---|---|---| +| `pattern-evals-pipefail-early-exit` | **created** | under `pipefail`, a reader that exits on first match SIGPIPEs the writer, so a successful match reports as a failed pipeline | +| `pattern-spec-self-staling-reuse-record` | **created** | a commit-keyed record committed into the repository it measures can never satisfy `commit == HEAD`, so the fallback is the only path | +| `pattern-evals-prose-literal-pinning` | patched | the same matcher fails the other way too: a short pin can be satisfied by a heading that merely names the block it guards | +| `pattern-docs-prohibition-by-example` | patched | the guard fired on this document; resolve a retired path programmatically, and never exempt the file that describes the migration | + +Two probe candidates were **nominated and not minted**: a `pipefail`/`grep -q` +lint over shell scripts (12 files repo-wide carry the shape) and a doc-lint on the +reuse-record contract. Both are guardrails worth having and neither is asked for +by #926; minting them here would widen a diff gate 5 already flagged for size. +The knowledge pages above carry the workarounds, so the lessons are durable +whether or not the probes land. + +Context compaction was **not run**. It is optional and non-gating in the new +contract, and every durable artifact above was written at full resolution first, +which is the ordering the change exists to guarantee. + +--- + +## 7. Benchmark verdict + +`/benchmark` — **`BENEFICIAL` (justified hold)**. + +- **Floor**: the record was stale against HEAD, so the suite was re-run rather + than inherited: 127 probes, exit 0, zero new `green→red`, four SKIPPED that are + the same four skipped on base. +- **Ceiling**: suite score **held at 1.44/2.00** against the counterfactual. That + is a *justified* hold rather than "machinery without movement", because the + disqualifier for the latter is no capability task crediting the change — and + CB-005 credits it directly: its success signal asks for a tracked pattern page + with a `path:line` root cause and pinned `<path>@<short-sha>` provenance, and + this run produced two and patched two more. +- **`REDIRECT-FLAG` raised.** The suite has **no task that measures + recall-before-plan**, which is the capability this change adds, so the ceiling + cannot see it in either direction. This is the shape CB-004 was retired for — a + row that held at `Δ +0.00` because nothing was ever measured. Recommended + redirect: author a CB task scoring whether a plan consumed tracked knowledge + before its PRD existed. +- **No ledger write.** `/benchmark` writes an `SI-nnnn-V` record only for a + `/builder` proposal under evaluation; none covers #926. +- Instrument grooming (`/audit eval-quality`) was not run — that follow-on does + not exist yet, and is named rather than silently skipped. + +--- + +## Correlation + +| Field | Value | +|---|---| +| Audit run id | `audit-20260901T022121Z-1697369` (post-merge; the pre-merge pass was `audit-20260901T014837Z-1436087`) | +| Native verdict | `AUDIT-PASS` · `SIMPLICITY-RESIDUAL: 0` (gates: graph 7/7 · eval rc=0 131 probes · promotable true · ui n/a · slop no blocking finding) | +| PR audit verdict | `PR-AUDIT-PROMOTABLE` · run `audit-20260901T031737Z-2034186` on head `d3b1a5cd` (CI PASS on all four checks · MERGEABLE · CLEAN · evidenceComplete true; advisory flag `size-convention`). Re-run on every head pushed after the undraft, per the rule this build added to step 10. | +| Eval record | `.oh/tasks/repo-knowledge-loop/eval-result.json` (commit-keyed) | +| Task graph | `.oh/tasks/repo-knowledge-loop/prd.json` — 7/7 stories passing | diff --git a/.oh/tasks/repo-knowledge-loop/prd.json b/.oh/tasks/repo-knowledge-loop/prd.json new file mode 100644 index 00000000..61ac0f93 --- /dev/null +++ b/.oh/tasks/repo-knowledge-loop/prd.json @@ -0,0 +1,120 @@ +{ + "schemaVersion": 1, + "project": "Open Harness", + "branchName": "feat/926-repo-knowledge-loop", + "description": "Make /spec a closed repo-knowledge learning loop: planning recalls and re-grounds tracked knowledge, execution derives knowledge invalidation from the actual diff, freshness is source-change aware, durable knowledge owns .oh/knowledge/, and the redundant /spec surfaces (ship, generated prompt.md, STATUS: COMPLETE, mandatory /compact, .oh/memory vocabulary) are retired atomically.", + "userStories": [ + { + "id": "US-001", + "title": ".oh/knowledge/ becomes the knowledge surface", + "description": "As the orchestrator, I want durable repository knowledge to own .oh/knowledge/ so that the wiki skill holds procedure only and there is exactly one writable knowledge location.", + "acceptanceCriteria": [ + "Every tracked entry under the retired corpus path is moved to .oh/knowledge/source/, .oh/knowledge/patterns/, or .oh/knowledge/raw/ with git mv", + "No tracked file remains under the retired corpus path and no active tracked surface references it", + ".oh/knowledge/README.md is the generated tracked index; raw/ and local/ each carry a tracked README anchor", + ".gitignore tracks .oh/knowledge/source/, .oh/knowledge/patterns/, and .oh/knowledge/raw/ and ignores only .oh/knowledge/local/ contents except its README anchor", + ".oh/manifest.json include lists knowledge/**", + ".github/workflows/ci-harness.yml push and pull_request paths both list .oh/knowledge/**", + ".oh/evals/decisions/skill-impact.md holds the prior ledger content unedited and builder/benchmark cite the new path" + ], + "priority": 1, + "passes": true, + "notes": "Verified: 32 git-mv renames; 0 files tracked under the retired corpus path; three README anchors tracked; manifest include carries knowledge/**; both CI path lists carry .oh/knowledge/**; the moved ledger is byte-identical to the base copy (diff empty). AMENDED CRITERION 4: as first written it required raw/ to be ignored. raw/ holds the immutable snapshots that kind: external pages cite as provenance, and an untracked snapshot is provenance a fresh clone cannot verify -- the split-brain #926 problem 4 names. The criterion was corrected to track raw/, which also matches the issue's own layout comment (only local/ is annotated ignored). Recorded in evidence.md as a divergence from the PRD as first written." + }, + { + "id": "US-002", + "title": "Knowledge schema: kinds, provenance, source-change freshness", + "description": "As a knowledge consumer, I want repo/external/pattern provenance and commit-aware freshness so that a page's validity is a source-change fact rather than an age heuristic.", + "acceptanceCriteria": [ + "schema.md defines kind: repo | external | pattern with sources: as the single dependency declaration and verified_at: as the freshness pin", + "A kind: repo page is never required to snapshot its own source into raw/", + "Every migrated page carries a valid kind; every kind: repo page carries verified_at and at least one repository-relative sources: entry", + ".oh/skills/wiki/scripts/knowledge-impact.sh exists, is executable, and passes shellcheck", + "knowledge-impact.sh --verified marks a page NEEDS-REVIEW when a declared repository source changed after its verified_at commit", + "knowledge-impact.sh --changed <paths> reports the pages a changed-path set invalidates", + "Dependency-aware invalidation is implemented once, in the knowledge primitive, and /spec calls it" + ], + "priority": 2, + "passes": true, + "notes": "Verified: 9 of 9 kind: repo pages carry verified_at and at least one repository-relative source; knowledge-impact.sh is executable and exercised by knowledge-source-freshness.sh against a scratch git repository in both directions. shellcheck is NOT installed in this worktree, so the 'passes shellcheck' clause was not observed here; CI's boot-lint job covers it. Provenance gained a fourth form -- a bare upstream URL -- because three pre-existing pages cited raw/ snapshots that were never committed; documented as the weakest form, and /wiki ingest can never produce it." + }, + { + "id": "US-003", + "title": "Tracked-only retrieval and the local scratch boundary", + "description": "As an agent on any machine, I want /wiki query and /spec to read only tracked shared knowledge so that a local scratch page on one machine can never silently inform a plan.", + "acceptanceCriteria": [ + "/wiki query enumerates .oh/knowledge/source/*.md and .oh/knowledge/patterns/*.md only", + ".oh/knowledge/local/ is documented as ignored scratch that no normal query or /spec flow reads, with an explicit /wiki ingest promotion path", + "Default query mode returns kind: repo and kind: external pages; --patterns returns kind: pattern only", + "The planner/executor asymmetry is preserved and described as a default, not isolation" + ], + "priority": 3, + "passes": true, + "notes": "Verified: query.md enumerates $KNOWLEDGE/source and $KNOWLEDGE/patterns only (query.md:118,121); the three mentions of knowledge/local are all prohibitions. knowledge-tracked-query-boundary.sh proves the ignore rule fires, that nothing but the anchor is tracked there, and that no read path enumerates it." + }, + { + "id": "US-004", + "title": "/spec plan = recall -> ground -> plan", + "description": "As an operator, I want planning to recall tracked knowledge and re-verify it against current sources before the PRD exists so that accumulated understanding informs the next plan.", + "acceptanceCriteria": [ + "plan.md orders derive vocabulary -> query tracked source knowledge -> query patterns -> read matches -> re-ground against authoritative sources -> build the PRD", + "prd.md carries ## Knowledge Context, ## Expected Knowledge Impact, and ## Plan Reconciliation; ## Wiki Alignment is superseded", + "A material intent change discovered during grounding stops before execution and requires operator re-approval", + "The verified task contract is prd.md, prd.json, progress.txt with no generated prompt.md" + ], + "priority": 4, + "passes": true, + "notes": "Verified by spec-plan-knowledge-context.sh (recall -> ground -> /prd ordering, the three prd.md blocks pinned by exact line) and spec-plan-reconciliation-gate.sh (the stop, the re-approval requirement, and execute.md's refusal of a folder whose reconciliation says NO). This task's own prd.md was written with the new contract, so the sections are exercised, not just specified." + }, + { + "id": "US-005", + "title": "/spec execute: re-ground, RUNNING, diff-derived knowledge impact", + "description": "As a reviewer, I want execution to re-ground its plan, report a real RUNNING state, and derive knowledge invalidation from the actual diff so that no stale assumption or missed page survives the build.", + "acceptanceCriteria": [ + "Execute reads the task Knowledge Context and planning base commit and diffs it against current HEAD before implementing", + "Final knowledge impact combines Expected Knowledge Impact + actual changed paths + knowledge dependency metadata via knowledge-impact.sh --changed", + "Every impacted tracked page ends in exactly one of UPDATED, REVERIFIED, or NOT-AFFECTED (<reason>)", + "The lifecycle PLANNED -> RUNNING -> READY | DRAFT-BLOCKED(<gate>) is explicit, RUNNING is derived from prd.json story state and mirrored into /tmp/spec-<slug>.state, and a run that returns mid-build reports RUNNING rather than a synchronous READY", + "The tail order is implementation -> audit -> eval -> knowledge impact -> evidence.md -> retro -> compile -> optional compaction -> benchmark -> final PR audit", + "/compact is non-gating and occurs only after durable evidence and learning extraction", + "The single implementation owner model \u2014 the agent running /spec execute, with /delegate bounded beneath it \u2014 and the human merge boundary are intact" + ], + "priority": 5, + "passes": true, + "notes": "Verified by spec-execute-knowledge-impact.sh (which also drives knowledge-impact.sh --changed in both directions) and spec-execute-running-contract.sh. The gate was run for real on this build: the 105-path diff produced 3 NEEDS-REVIEW pages, and the union with Expected Knowledge Impact resolved to 5 UPDATED, 1 REVERIFIED, 1 NOT-AFFECTED. advisor-monitored-loop.sh and spec-ready-finalization.sh still pass, so the single-Advisor model and the merge boundary survived the lifecycle change. RECONCILED mid-build against PR #930 (issue #928), which merged into development and retired the /spec agent-handoff mechanism: no tmux launch, no agent-spec-* session naming, and RUNNING redefined as task state. Adopted, because it is the newer merged decision and preserves every invariant #926's pinned comment protects (one owner, bounded /delegate, the four-state lifecycle, human merge). The status file is now /tmp/spec-<slug>.state. spec-no-agent-handoff.sh, spec-single-owner.sh and spec-no-advisor-session-coupling.sh all pass alongside spec-execute-running-contract.sh." + }, + { + "id": "US-006", + "title": "Retire ship, STATUS: COMPLETE, and the .oh/memory vocabulary", + "description": "As a maintainer, I want the redundant workflow surfaces gone so that the canonical public model is plan + execute with no duplicated state.", + "acceptanceCriteria": [ + ".oh/skills/spec/references/ship.md is deleted and ship is no longer a conceptual workflow node", + "/spec <approved-plan-path> still runs plan then execute", + "Task completion derives from prd.json story state; crons/cleanup-tasks.md and .oh/tasks/README.md no longer key on a prose sentinel", + "/spec retro is an explicit compatibility wrapper around /retro --task <slug> and /retro stays report-only", + ".oh/README.md no longer lists memory/ as a current subsystem and the .gitignore rule is labelled a retired compatibility tombstone with a removal horizon", + "docs/oh-directory-layout.md, docs/glossary.md, and the affected RFCs match the new contract" + ], + "priority": 6, + "passes": true, + "notes": "Verified: ship.md absent; spec-family-contract.sh now treats ship as retired and requires the approved-plan default branch; task-completion-structured-state.sh proves no active surface keys on the prose sentinel and that the documented jq check answers both ways; retired-memory-vocabulary.sh proves the tombstone labelling and removal horizon. /retro gained --task <slug> and stays report-only (retro-deterministic-contract.sh green). The cleanup cron's completion test also had to be reconciled with #930's cleanup-no-agent-session-coupling.sh, which pinned the retired prose sentinel while its real subject is session decoupling; the probe now asserts that decoupling against prd.json structured state, so both contracts hold." + }, + { + "id": "US-007", + "title": "Lint simplification and the ten regression probes", + "description": "As the eval suite, I want the surviving lint signals to be real correctness checks with deterministic oracles so that a report-only check nobody gates on cannot rot.", + "acceptanceCriteria": [ + "/wiki lint's checks are exactly schema validity, missing/broken source paths, source-change freshness, broken [[...]] links, broken related: slugs, and generated index consistency", + "Orphan detection is retired as a health failure and the 90-day check survives only as informational last-reviewed telemetry", + "All ten required probes exist under .oh/evals/probes/, are tier A, and pass on the branch", + "Every existing probe that pinned a moved or retired literal is updated in the same change", + "Each new probe is fault-injected at least once and the observed failing output is recorded in evidence.md", + "bash .oh/skills/eval/run.sh reports zero new green->red regressions", + "bash .oh/scripts/link-providers.sh --check passes and git diff --check is clean" + ], + "priority": 7, + "passes": true, + "notes": "Verified: 10 new tier-A probes, all green, each fault-injected with the observed REGRESSION output recorded in evidence.md. Two of them were STRENGTHENED as a result: fault injection showed the planning probes passing after the block they guard was deleted, because a heading naming the block satisfied a substring pin. 8 existing probes updated. bash .oh/skills/eval/run.sh: 127 probes, runner exit 0, zero green->red transitions, the same 4 SKIPPED as base. link-providers --check passes; git diff --check is clean." + } + ] +} diff --git a/.oh/tasks/repo-knowledge-loop/prd.md b/.oh/tasks/repo-knowledge-loop/prd.md new file mode 100644 index 00000000..f056b4bc --- /dev/null +++ b/.oh/tasks/repo-knowledge-loop/prd.md @@ -0,0 +1,293 @@ +# PRD — make `/spec` a closed repo-knowledge learning loop + +Issue: [#926](https://github.com/mifunedev/openharness/issues/926) · slug +`repo-knowledge-loop` · branch `feat/926-repo-knowledge-loop` · base `development` +· repo `mifunedev/openharness`. + +## Knowledge Context + +- **Base commit**: `ecc49800a7d6a7bd525099383998865a3e6a1a49` +- **Queries**: `wiki knowledge corpus compile`, `spec plan execute retro`, + `evals probes oracles`, `docs vocabulary guards` (run against the tracked + knowledge set at the base commit, before any migration) +- **Knowledge used**: + `[[wikiskill-experience-compilation]]`, `[[plan-vs-built-reconciliation]]`, + `[[audit-architecture]]`, `[[pattern-wiki-external-model-over-mapping]]`, + `[[pattern-wiki-ungated-check-drift]]`, `[[pattern-evals-prose-literal-pinning]]`, + `[[pattern-evals-unexercised-oracle]]`, `[[pattern-docs-prohibition-by-example]]` +- **Grounded against**: `.oh/skills/spec/SKILL.md`, + `.oh/skills/spec/references/{plan,execute,ship,retro}.md`, + `.oh/skills/spec/templates/task-prompt.md`, `.oh/skills/wiki/SKILL.md`, + `.oh/skills/wiki/references/{schema,query,lint,compile,ingest}.md`, + `.oh/skills/retro/SKILL.md`, `.oh/skills/eval/run.sh`, + `.oh/evals/probes/{wiki-readme-index,wiki-kind-schema-contract,wiki-query-pattern-isolation,wiki-related-slugs,wiki-pattern-persistence,wiki-skill-impact-append-only,audit-stale-references,eval-runs-once-per-cycle,advisor-monitored-loop}.sh`, + `.gitignore`, `.oh/README.md`, `.oh/tasks/README.md`, `.oh/manifest.json`, + `crons/cleanup-tasks.md`, `.github/workflows/ci-harness.yml`, + `docs/oh-directory-layout.md`, `docs/glossary.md`, `docs/rfcs/*.md`, + `.claude/protected-paths.txt`, `.oh/scripts/link-providers.sh` +- **Conflicts discovered**: four. (1) The pattern corpus warns that report-only + checks nobody gates on converge on never being run + (`[[pattern-wiki-ungated-check-drift]]`), so requirement R's surviving lint + checks each get a deterministic probe rather than prose alone. (2) + `[[pattern-docs-prohibition-by-example]]` warns that a guard forbidding a + literal also scans the prose explaining it — so the retired corpus path is + named in exactly one guard and nowhere else. (3) + `[[pattern-evals-prose-literal-pinning]]` warns against pinning wrapped + sentences — every new probe pins short, wrap-safe fragments. (4) + `[[pattern-evals-unexercised-oracle]]` requires fault injection before a probe + counts as green, which #926 independently mandates. + +## Expected Knowledge Impact + +- **Impact**: REQUIRED +- **Expected entries**: `wikiskill-experience-compilation`, + `plan-vs-built-reconciliation`, `audit-architecture`, + `pattern-wiki-ungated-check-drift`, `pattern-wiki-external-model-over-mapping`, + plus every migrated page's frontmatter (`kind`, `verified_at`) and any new + `pattern-*` page `/wiki compile` writes from this run's retro +- **Affected source paths**: `.oh/skills/spec/**`, `.oh/skills/wiki/**`, + `.oh/knowledge/**`, `.oh/evals/probes/**`, `.oh/tasks/README.md`, + `.oh/README.md`, `.gitignore`, `crons/cleanup-tasks.md`, `docs/**` +- **Reason**: the change relocates the knowledge surface, redefines the entry + schema, and rewrites the `/spec` workflow contract every tracked page about + those subsystems cites. + +## Plan Reconciliation + +- **Source plan**: `https://github.com/mifunedev/openharness/issues/926` (the + issue body and its pinned execution-requirement comment). The operator wrote + and approved it; that satisfies the commitment gate. +- **Intent preserved**: YES +- **Material deviations**: `none` +- **Constraints discovered during grounding**: + 1. **`depends_on:` collapses into `sources:`.** Requirement E illustrates + freshness provenance with a separate `depends_on:` list ("for example"), + while requirement F normatively puts repository paths in `sources:` for + `kind: repo`. Two lists holding the same paths is the complexity the root + `AGENTS.md` forbids, so `sources:` is the single dependency declaration and + `verified_at:` pins the commit it was last checked against. Freshness is + computed over the repository-relative subset of `sources:`; immutable + `raw/` and `path@sha` entries carry provenance but never expire. + Non-material: the observable behavior E asks for — "a page is needs-review + when a declared dependency changed since `verified_at`" — is unchanged. + 2. **`/spec retro` takes option 1 of requirement K** (explicit compatibility + wrapper) rather than option 2 (retire the subcommand). + `.oh/skills/spec/references/retro.md` is a `.claude/protected-paths.txt` + entry and `.oh/evals/probes/audit-stale-references.sh` pins it as a coverage + path; deleting it would need a protected-path removal the issue does not + ask for. The wrapper carries no second ontology — it delegates to + `/retro --task <slug>`. + 3. **No `/spec ship` alias survives.** A repository-wide search found zero + callers outside the skill's own two files, so the non-goal against + compatibility abstractions applies. The dispatcher keeps a two-line + redirect for a literal `ship` first token purely so `/spec ship <plan>` can + not silently derive the slug `ship`. + 4. **Tracked-by-default replaces gitignore-by-default for knowledge pages.** + Requirement G demands a physically distinct ignored scratch location; once + `.oh/knowledge/local/` exists, keeping `source/` and `patterns/` ignored + buys nothing and forces the `git add -f` dance plus `/wiki lint`'s dual + working-tree/tracked entry sets. Both collapse. + 5. **`raw/` is tracked, not ignored.** This PRD first wrote requirement G's + ignore boundary around `raw/` as well as `local/`. `raw/` holds the + immutable snapshots `kind: external` pages cite, and an untracked snapshot + is provenance a fresh clone cannot verify — problem 4 in the issue, wearing + a new name. Corrected mid-build to track it, which also matches the issue's + own layout comment, where only `local/` is annotated ignored. Recorded in + `evidence.md` as a divergence from the PRD as first written. + 6. **Three pre-existing pages cited snapshots that were never committed.** + `managed-agents`, `molt-agentic-reinforcement-learning`, and + `recursive-self-improvement-survey` named `raw/` files absent from every + commit — the same split-brain, found by the new source-path check. Repaired + without fabricating provenance: two carry the arXiv URL their bodies state, + and `managed-agents` is reclassified `kind: repo` against the repository + documents it actually reasons over, with its unrecoverable external seed + stated in the page. + 7. **The execution base moved mid-build: #930 (issue #928) merged into + `development` and retired the `/spec` agent-handoff mechanism.** It deleted + the tmux Advisor launch, the `/goal` prompt, and the `agent-spec-<slug>` + session naming, and redefined `RUNNING` as *task* state — an approved folder + whose stories are not all passing — never a named process. That collides + with #926's pinned comment, which asks for "one persistent Herdr/tmux-backed + session" and says `RUNNING` "represents the persistent Advisor doing the + work". Reconciled toward the merged repository state, which is the newer + operator decision and which **preserves every invariant the pinned comment + actually protects**: one implementation owner, `/delegate` bounded beneath + it, the `PLANNED → RUNNING → READY | DRAFT-BLOCKED(<gate>)` lifecycle, and + human merge as the final boundary. What changed is the *mechanism* — the + owner is now the agent already running `/spec execute` rather than a session + it launches, and the status file is `/tmp/spec-<slug>.state`. Flagged for the + operator on the PR; this is the one place a reviewer may want to overrule. + 8. **`.oh/manifest.json` must gain `knowledge/**`.** The corpus shipped to + consumer repos today only because it sat under `skills/**`. Moving it out + without the manifest entry would silently stop shipping durable knowledge. + +## Introduction + +`/spec` writes durable knowledge more reliably than it reads it. Planning does not +recall tracked knowledge before the PRD exists; the planner's `Wiki Alignment` +prediction is treated as the final impact oracle even though implementation touches +paths the planner never saw; wiki freshness is measured in days rather than against +the sources a page depends on; ignored local scratch and shared knowledge share one +query path; and the durable knowledge itself lives inside the wiki skill's own +implementation tree. Alongside that, several `/spec` surfaces duplicate state: +`ship` owns no mechanics, `prompt.md` is a generated copy of its own template, +`STATUS: COMPLETE` duplicates `prd.json`, and `/spec execute` claims a synchronous +ready PR while actually launching a detached Advisor. + +This PRD closes the loop: planning recalls and re-grounds tracked knowledge before +the PRD is written, execution re-grounds it against current HEAD, the actual diff +determines what knowledge was invalidated, and retrospective evidence compounds back +into durable patterns — with the redundant surfaces retired in the same change. + +## Goals + +1. `/spec plan` recalls tracked knowledge and re-grounds it against current + repository sources before the PRD is finalized, and records what it used. +2. An approved plan whose intent grounding materially changes stops for operator + re-approval instead of flowing silently into execution. +3. `/spec execute` re-grounds at start, and derives final knowledge impact from the + actual diff plus knowledge dependency metadata, not from the planner's guess. +4. Knowledge freshness is commit- and source-change-aware, not age-based. +5. Durable knowledge owns `.oh/knowledge/` as a surface separate from + `.oh/skills/wiki/`, with tracked shared knowledge physically separated from + ignored local scratch, and exactly one writable knowledge location. +6. `/spec`'s redundant surfaces (`ship`, generated `prompt.md`, + `STATUS: COMPLETE`, mandatory `/compact` as a semantic stage, the `.oh/memory` + vocabulary) are retired atomically with every consumer. +7. Detached execution is an explicit `PLANNED → RUNNING → READY | DRAFT-BLOCKED` + lifecycle, still owned by one persistent Advisor session. + +## Non-goals + +- No vector search, embeddings, database, or memory service. +- Knowledge never becomes authoritative over code, tests, docs, or RFCs. +- The full pattern corpus is never loaded into every agent session. +- No second workflow engine beside `/spec`. +- No compatibility alias kept without an actual caller. +- No auto-merge; human merge stays the final gate. +- The single-Advisor executor ownership model is **not** retired (issue #926's + pinned execution requirement). + +## User stories + +### US-001 — `.oh/knowledge/` becomes the knowledge surface + +Migrate the retired corpus path to `.oh/knowledge/{source,patterns,raw,local}/` +with no compatibility alias, update `.gitignore` so `source/` and `patterns/` are +tracked while `raw/` and `local/` are ignored, add `knowledge/**` to +`.oh/manifest.json`, add `.oh/knowledge/**` to the CI path filters, and move +`skill-impact.md` to `.oh/evals/decisions/skill-impact.md`. + +**Acceptance criteria** +- No tracked file remains under the retired corpus path and no active surface + references it (`knowledge-path-single-owner` probe). +- `.oh/knowledge/README.md` is the generated tracked index; `source/`, + `patterns/`, and `raw/` are tracked; `local/` carries a tracked README anchor + and nothing else tracked. +- `.oh/manifest.json` `include` lists `knowledge/**`. +- `.github/workflows/ci-harness.yml` push and pull_request filters both list + `.oh/knowledge/**`. +- `.oh/evals/decisions/skill-impact.md` holds the full prior ledger, unedited. + +### US-002 — knowledge schema: kinds, provenance, source-change freshness + +Rewrite `.oh/skills/wiki/references/schema.md` for `kind: repo | external | +pattern`, `verified_at: <commit>`, `sources:` as the single dependency +declaration, and the tracked/local boundary. Ship +`.oh/skills/wiki/scripts/knowledge-impact.sh` as the one dependency-aware +invalidation implementation. + +**Acceptance criteria** +- Every migrated page carries a valid `kind`; every `kind: repo` page carries + `verified_at` and at least one repository-relative `sources:` entry. +- No `kind: repo` page is required to snapshot its own source into `raw/`. +- `knowledge-impact.sh --verified` marks a page `NEEDS-REVIEW` when a declared + repository source changed after its `verified_at` commit + (`knowledge-source-freshness` probe). +- `knowledge-impact.sh --changed <paths>` reports the pages a given changed-path + set invalidates; `/spec` calls it rather than reimplementing the logic. + +### US-003 — tracked-only retrieval and the local scratch boundary + +`/wiki query` and every `/spec` path read `.oh/knowledge/source/` and +`.oh/knowledge/patterns/` only. `.oh/knowledge/local/` is ignored scratch that no +normal query or `/spec` flow reads; promotion is an explicit `/wiki ingest` call. + +**Acceptance criteria** +- `/wiki query`'s enumeration globs name only `source/` and `patterns/`; the word + `local` appears only in the prohibition (`knowledge-tracked-query-boundary` + probe). +- Planner/executor asymmetry survives: patterns come from `--patterns`, default + mode returns `repo`/`external` pages only. + +### US-004 — `/spec plan` = recall → ground → plan + +Rewrite `.oh/skills/spec/references/plan.md` so tracked recall and source +re-grounding precede `/prd`, and `prd.md` carries `## Knowledge Context`, +`## Expected Knowledge Impact`, and `## Plan Reconciliation`. `## Wiki Alignment` +is superseded. The task contract drops `prompt.md`. + +**Acceptance criteria** +- `plan.md` orders vocabulary → tracked query → pattern query → read → re-ground → + PRD, and names all three prd.md sections (`spec-plan-knowledge-context` probe). +- A material intent change during grounding stops for re-approval + (`spec-plan-reconciliation-gate` probe). +- The verified contract is `prd.md`, `prd.json`, `progress.txt` + (`spec-no-generated-prompt-contract` probe). + +### US-005 — `/spec execute`: re-ground, RUNNING, diff-derived knowledge impact + +Rewrite `.oh/skills/spec/references/execute.md` for the re-ground step, the +explicit `PLANNED → RUNNING → READY | DRAFT-BLOCKED(<gate>)` lifecycle with a +status file, the Actual Knowledge Impact gate derived from the real diff, the +issue's tail order, no persisted `prompt.md`, no `STATUS: COMPLETE`, and +`/compact` demoted to a non-gating optional step after evidence and learning +extraction. + +**Acceptance criteria** +- Execute re-reads Knowledge Context and diffs planning base → current HEAD before + implementing. +- Every impacted tracked page ends `UPDATED` / `REVERIFIED` / + `NOT-AFFECTED (<reason>)`, derived via `knowledge-impact.sh --changed` + (`spec-execute-knowledge-impact` probe). +- Launching the detached Advisor reports `RUNNING`, never a synchronous READY + (`spec-execute-running-contract` probe). +- Tail order is knowledge impact → evidence.md → retro → compile → optional + compaction → benchmark → final PR audit. +- The single-Advisor ownership model and the human merge boundary are intact. + +### US-006 — retire `ship`, `STATUS: COMPLETE`, and the `.oh/memory` vocabulary + +Delete `.oh/skills/spec/references/ship.md` and the `ship` node from the +dispatcher; derive task completion from `prd.json`; reduce `/spec retro` to an +explicit `/retro --task <slug>` wrapper; retire `.oh/memory` from current +architecture docs and label the surviving ignore rule a tombstone. + +**Acceptance criteria** +- `/spec <approved-plan-path>` still runs plan → execute; the canonical model is + `plan` + `execute`. +- `crons/cleanup-tasks.md` and `.oh/tasks/README.md` derive completion from + `prd.json` story state (`task-completion-structured-state` probe). +- `.oh/README.md` no longer lists `memory/` as a current subsystem and the + `.gitignore` rule names a removal horizon (`retired-memory-vocabulary` probe). +- `/retro` accepts `--task <slug>` and stays report-only. + +### US-007 — lint simplification and the ten probes + +Reduce `/wiki lint` to schema validity, missing/broken source paths, +source-change freshness, broken `[[...]]` links, broken `related:` slugs, and +generated index consistency. Orphan detection is retired as a health failure; the +90-day check survives only as `last-reviewed` telemetry. Add all ten required +probes and fault-inject each. + +**Acceptance criteria** +- Lint's check list is exactly the six correctness checks plus the informational + telemetry line; orphan status blocks nothing. +- All ten new probes exist, are tier A, and pass on the branch. +- Each new probe has a recorded fault-injection observation in `evidence.md`. +- `/eval` shows zero new green→red regressions. + +## Verification + +`bash .oh/skills/eval/run.sh` (once), `bash .oh/scripts/link-providers.sh --check`, +`git diff --check`, and a fresh `/audit pr` before any undraft. diff --git a/.oh/tasks/repo-knowledge-loop/progress.txt b/.oh/tasks/repo-knowledge-loop/progress.txt new file mode 100644 index 00000000..3b188de1 --- /dev/null +++ b/.oh/tasks/repo-knowledge-loop/progress.txt @@ -0,0 +1,84 @@ +# progress + +2026-09-01 plan — Read issue #926 body + pinned execution-requirement comment in full. + Grounded against .oh/skills/spec/**, .oh/skills/wiki/**, .oh/evals/probes/**, + .gitignore, .oh/README.md, .oh/tasks/README.md, .oh/manifest.json, + crons/cleanup-tasks.md, .github/workflows/ci-harness.yml, docs/**, + .claude/protected-paths.txt, .oh/scripts/link-providers.sh at base ecc49800. + Recalled 3 tracked source pages + 5 pattern pages; 4 conflicts recorded in + prd.md ## Knowledge Context. Wrote prd.md (new contract: Knowledge Context, + Expected Knowledge Impact, Plan Reconciliation), prd.json (7 stories), this file. + No generated prompt.md — the contract this task builds retires it. + +2026-09-01 implementation — US-001..US-007 landed in one atomic change. + Migration: git mv corpus -> .oh/knowledge/{source,patterns,raw}, skill-impact -> + .oh/evals/decisions/, new local/ scratch tier; 41 files path-rewritten; .gitignore, + .oh/manifest.json (knowledge/**), ci-harness.yml (both path lists). + Schema: kind repo|external|pattern, sources: as the single dependency declaration, + verified_at pins; new .oh/skills/wiki/scripts/knowledge-impact.sh (--verified | --changed + | --since). Rewrote schema/query/lint; updated compile/ingest/SKILL. + Spec: rewrote SKILL.md (3 nodes + approved-plan default), plan.md (recall->ground->plan), + execute.md (re-ground, RUNNING status file, Actual Knowledge Impact gate, new tail order), + retro.md -> /retro --task wrapper; deleted ship.md; task-prompt.md is render-time only. + /retro gained --task <slug>. Retired STATUS: COMPLETE and generated prompt.md. + Probes: 10 new tier-A + 8 existing updated (readme-index, kind-schema, query-isolation, + related-slugs, pattern-persistence, skill-impact-append-only, spec-family-contract, + retro-deterministic-contract). Repaired 3 pages with unresolvable provenance. + Delegated one bounded worker for the peripheral consumer sweep (.oh/README.md, + .oh/tasks/README.md, manifest, CI, cleanup cron, docs, RFC); reconciled and corrected + its glossary 'loop' entry. CHANGELOG entries added. + +2026-09-01 terminal — READY. PR #927 marked ready for review after a fresh + PR-AUDIT-PROMOTABLE (audit-20260901T024245Z-1853136): CI green on all four + checks, MERGEABLE, CLEAN, evidence.md present and tracked. Never merged. + Gate record: /audit implementation AUDIT-PASS with SIMPLICITY-RESIDUAL: 0 + (audit-20260901T022121Z-1697369, after four cycles: gate 1 task graph, gate 2 + evidence.md re-introducing the retired path, gate 5 dead --since mode, then + PASS). /eval 136 probes exit 0, zero green->red. All ten required probes + fault-injected; two were found silently inert and rewritten. Knowledge impact + resolved to 5 UPDATED / 1 REVERIFIED / 1 NOT-AFFECTED. /retro -> 9 hypotheses, + 2 new pattern pages compiled, 2 existing patched. /benchmark BENEFICIAL + (justified hold) with a REDIRECT-FLAG on the missing capability axis. Three + development merges (#930, #931, #934) reconciled during the tail; the #930 + collision with #926's pinned execution comment is flagged for the operator on + the PR as a decision, not silently resolved. + +2026-09-01 ac-closeout — Orchestrator verification found #926's final acceptance + criterion unmet: evidence.md named the openharness-web follow-up but no issue + existed. Filed mifunedev/openharness-web#37 naming the two published pages that + carry live drift (docs/harnesses/deepagents.md:154,156,164,180 — the retired + Advisor session-ownership model; blog/2026-07-07-open-harness-demo-guide.md:131 + — .oh/memory as current architecture) and the one hit deliberately excluded + (docs/integrations/pi-autoresearch.md:65 cites .auto/prompt.md, Pi's own file). + Linked from evidence.md section 5 and the PR body. Criterion now met. + +2026-09-01 undraft-gate — Fixed the ordering gap in the contract rather than only + disclosing it. execute.md step 10 now confirms the PR headRefOid equals local + HEAD before reading the promotable classification, states that the gate re-opens + on every push after the undraft (with gh pr ready --undo as the way back to + draft), and notes that finishing the tail before undrafting avoids the extra CI + cycles. spec-ready-finalization.sh gained three assertions over that section; + all three fault-injected and fire. Also removed a stray root file named "2" — a + &>2 typo in a sed invocation had redirected a probe's PASS line into it and a + later git add -A tracked it. + +2026-09-01 stray-file-root-cause — Deleting the "2" file was not the fix: a sed + had rewritten >&2 to &>2 inside spec-execute-running-contract.sh, so the probe + re-created the file on every suite run and the first deletion was undone by the + next /eval. Repaired at the source and guarded: + eval-contract-text-20260831.sh now fails any probe containing &>[0-9], with the + reason spelled out. Fault-injected and fires. plan-vs-built-reconciliation + re-verified against the new execute.md anchors; verified_at advanced. + +2026-09-01 probes-minted — Minted both probes the session-scoped /retro nominated. + evals-20260901-suite-tree-clean: no probe redirects into the repository (&>N, or + a redirect targeting $ROOT/$HARNESS) and no redirect residue is tracked at the + root; the mid-session &>[0-9] check moved out of eval-contract-text-20260831.sh + so one rule has one owner. A first draft flagging any relative-path redirect + produced 46 false findings and was narrowed. + docs-20260901-followup-artifact-cited: reviewer-evidence-doc.md gained the + "Follow-ups are cited, not named" rule, and the probe requires every tracked + evidence.md bullet tying a criterion to a follow-up to carry an issue/PR URL. + All five branches across the two probes fault-injected and fire, including a + reconstruction of this run's own uncited-follow-up defect. + 138 probes, runner exit 0, zero green->red. diff --git a/.oh/tasks/repo-knowledge-loop/simplify-rounds.json b/.oh/tasks/repo-knowledge-loop/simplify-rounds.json new file mode 100644 index 00000000..4d0bd369 --- /dev/null +++ b/.oh/tasks/repo-knowledge-loop/simplify-rounds.json @@ -0,0 +1,8 @@ +{ + "rounds": 2, + "netAdded": 4496, + "prevNetAdded": 4112, + "lastCommit": "29d3dafca5e43feab8d5a8c1bf10b5b3ea572c01", + "finding": "round 1 deleted knowledge-impact.sh --since (zero call sites); round 2 found no blocking finding (SIMPLICITY-RESIDUAL: 0)", + "growthSince": "the increase over round 1 is the development merge, not unsimplified new work: 537 of the 733 added lines are PR #930's four new probes and the /spec, /delegate, /rlm, /ste, /t3, /audit, capability, crons and docs surfaces it rewrote, plus this run's two compiled pattern pages" +} diff --git a/CHANGELOG.md b/CHANGELOG.md index f82a1b30..64dec364 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,14 @@ Update policy and release automation live in [`/git`](.claude/skills/git/SKILL.m ## [Unreleased] ### Changed +- **BREAKING:** Move durable repository knowledge to a tracked `.oh/knowledge/` surface — `source/`, `patterns/`, `raw/` tracked, `local/` ignored — with no compatibility alias. ([#926](https://github.com/mifunedev/openharness/issues/926)) +- **BREAKING:** Make `/spec plan` recall tracked knowledge and re-ground it before the PRD; `prd.md` carries Knowledge Context, Expected Knowledge Impact, and Plan Reconciliation. ([#926](https://github.com/mifunedev/openharness/issues/926)) +- **BREAKING:** Derive final knowledge impact in `/spec execute` from the actual diff plus page dependencies, resolving every impacted page to UPDATED, REVERIFIED, or NOT-AFFECTED. ([#926](https://github.com/mifunedev/openharness/issues/926)) +- **BREAKING:** Replace age-based wiki staleness with source-change freshness: a `kind: repo` page pins `verified_at` and goes needs-review when a declared source changed after it. ([#926](https://github.com/mifunedev/openharness/issues/926)) +- **BREAKING:** Model `/spec execute` as PLANNED -> RUNNING -> READY | DRAFT-BLOCKED(gate), with RUNNING derived from `prd.json` and mirrored into `/tmp/spec-<slug>.state`. ([#926](https://github.com/mifunedev/openharness/issues/926)) +- Re-open `/spec execute`'s promotable gate when a push lands after `gh pr ready`: the verdict binds to one head, so a moved head is re-audited or returned to draft. ([#926](https://github.com/mifunedev/openharness/issues/926)) +- Reduce `/wiki lint` to six correctness checks and demote `/compact` to an optional non-gating step that runs only after evidence, retro, and pattern compilation. ([#926](https://github.com/mifunedev/openharness/issues/926)) +- Reduce `/spec retro` to an explicit wrapper around `/retro --task <slug>`, which `/retro` now accepts; `/retro` stays report-only and `/wiki compile` stays the durable pattern writer. ([#926](https://github.com/mifunedev/openharness/issues/926)) - **BREAKING:** Move eleven settings out of the compose `environment:` block into oh.json, read through the `oh` CLI; a hand-edited `.devcontainer/.env` no longer carries them ([#920](https://github.com/mifunedev/openharness/issues/920)). - Make `/spec` ship by default: an unrecognized first token routes to a new `ship` node that runs `plan` then `execute`, so `/spec <plan-path>` produces a ready-for-review PR ([#914](https://github.com/mifunedev/openharness/issues/914)). - **BREAKING:** Persist the sandbox home through one `/home/sandbox` mount, not eleven per-tool volumes; set `storage.homePath` for a host path, else `<name>_workspace` ([#898](https://github.com/mifunedev/openharness/issues/898)). @@ -18,6 +26,12 @@ Update policy and release automation live in [`/git`](.claude/skills/git/SKILL.m - **BREAKING:** Stop baking OpenCode, Hermes, and Grok Build into the image; `oh harness install <id>` installs them into `~/.local` as the sandbox user ([#908](https://github.com/mifunedev/openharness/issues/908)). ### Removed +- **BREAKING:** Retire the `/spec ship` node; an unrecognized first token is an approved plan path that runs `plan` then `execute`, so `/spec <plan-path>` is unchanged. ([#926](https://github.com/mifunedev/openharness/issues/926)) +- **BREAKING:** Retire the generated `.oh/tasks/<slug>/prompt.md`; the task prompt is rendered at execution time from its template and never persisted. ([#926](https://github.com/mifunedev/openharness/issues/926)) +- **BREAKING:** Retire the `STATUS: COMPLETE` sentinel; task completion derives from `prd.json` story state, which the `cleanup-tasks` cron now reads. ([#926](https://github.com/mifunedev/openharness/issues/926)) +- **BREAKING:** Move the skill-impact ledger to `.oh/evals/decisions/skill-impact.md`; it is a decision record, not a knowledge page. ([#926](https://github.com/mifunedev/openharness/issues/926)) +- Retire orphan detection and the 90-day rule as `/wiki lint` health failures; age survives as informational `last-reviewed` telemetry only. ([#926](https://github.com/mifunedev/openharness/issues/926)) +- Finish retiring `.oh/memory` from current architecture docs; the surviving `.gitignore` rule is labelled a compatibility tombstone with a 0.7.0 removal horizon. ([#926](https://github.com/mifunedev/openharness/issues/926)) - **BREAKING:** Retire cron worktree isolation — the `worktree:` frontmatter key, `.worktrees/cron/` per-fire worktrees, the `CRON_WORKTREE` export, and every `*_WORKTREE*` log state are gone; crons fire in the shared root under the id lock. - **BREAKING:** Retire the `OH_IMAGE_ONLY` flag; `entrypoint.sh` detects the sandbox flavor from whether `/home/sandbox/harness` is a bind mount holding `.oh/`, and logs the detected mode ([#920](https://github.com/mifunedev/openharness/issues/920)). - **BREAKING:** Retire `docker-compose.hermes-dashboard.yml` and its published `127.0.0.1:9119`; the dashboard now binds container loopback, reachable over cloudflared or Tailscale ([#920](https://github.com/mifunedev/openharness/issues/920)). @@ -49,6 +63,9 @@ Update policy and release automation live in [`/git`](.claude/skills/git/SKILL.m - Add `spec-no-agent-handoff`, `spec-no-advisor-session-coupling`, `cleanup-no-agent-session-coupling`, `headless-tmux-preserved`; rename `advisor-monitored-loop` to `spec-single-owner` ([#928](https://github.com/mifunedev/openharness/issues/928)). ### Added +- Add `.oh/skills/wiki/scripts/knowledge-impact.sh`, the single dependency-aware invalidation primitive: `--verified` for `/wiki lint`, `--changed` for the `/spec execute` knowledge gate. ([#926](https://github.com/mifunedev/openharness/issues/926)) +- Add `evals-20260901-suite-tree-clean` and `docs-20260901-followup-artifact-cited`: no probe may redirect into the repository, and a criterion met by a follow-up must cite its URL. ([#926](https://github.com/mifunedev/openharness/issues/926)) +- Add ten tier-A probes covering the knowledge surface, the planning recall and reconciliation gates, the RUNNING contract, structured completion, and the retired vocabulary. ([#926](https://github.com/mifunedev/openharness/issues/926)) - Add `/escalate`: an unattended session delivers a human-addressed escalation to the operator's Slack channel. An unavailable channel no-ops loudly rather than failing the session. ([#919](https://github.com/mifunedev/openharness/issues/919)) - Add `.oh/logs/`, gitignored by default with a tracked README, for records that outlive the session that wrote them; `/escalate` appends every attempt to `escalations.jsonl`. ([#919](https://github.com/mifunedev/openharness/issues/919)) - Add `escalate-contract.sh`, a tier-A probe: a no-op names its reason, is recorded, and `--dry-run` makes no network call. ([#919](https://github.com/mifunedev/openharness/issues/919)) diff --git a/crons/cleanup-tasks.md b/crons/cleanup-tasks.md index 788349eb..964064ed 100644 --- a/crons/cleanup-tasks.md +++ b/crons/cleanup-tasks.md @@ -12,8 +12,6 @@ description: Weekly `/spec execute` task sweep — archive completed tasks # Weekly Task Cleanup Sweep `.oh/tasks/` once per week and archive anything that has finished. -Completion is read from `progress.txt` alone; a task's identity and state are never -tied to a terminal session, so this sweep detects and kills no agent session. Per SPEC v0.7 §"Weekly cleanup cron": completed tasks move into the dated archive under `.oh/tasks/`; incomplete tasks are left alone with a note. The same weekly pass also grooms stale `.worktrees/` branch @@ -75,15 +73,26 @@ only — never a `.oh/tasks/` subfolder. uncommitted task dirs in the shared checkout are never archived. For each `.worktrees/archive/$TODAY/.oh/tasks/<taskdesc>/` (skipping `.oh/tasks/archive/`): - - If `.oh/tasks/<taskdesc>/progress.txt` ends with a line matching exactly - `STATUS: COMPLETE`: + - If every user story in `.oh/tasks/<taskdesc>/prd.json` passes — + `jq -e 'all(.userStories[]; .passes == true)' .oh/tasks/<taskdesc>/prd.json` + exits 0 — the task is complete: - Move the folder inside the worktree: `git -C .worktrees/archive/$TODAY mv .oh/tasks/<taskdesc> .oh/tasks/archive/$TODAY/<taskdesc>` (falls back to `mv` + `git -C .worktrees/archive/$TODAY add` if `git mv` rejects an untracked path). - Otherwise, leave the folder in place and append a one-line note to - the reply recording that `<taskdesc>` is still active - (include the last `progress.txt` modification time). + the reply recording that `<taskdesc>` is still active, how many user + stories remain unpassed + (`jq '[.userStories[] | select(.passes != true)] | length' .oh/tasks/<taskdesc>/prd.json`), + and the last `progress.txt` modification time. + - Task state is read from the task folder alone and is **never** tied to a + tied to a terminal session, tab, or pane: the sweep does not look for, + attach to, or kill any implementation-agent process. + - A task folder with **no readable `prd.json`** — the file is missing, + unparseable, or carries no `userStories` array, so the `jq` test above + fails rather than exits 0 — is **not** complete: leave the folder in + place and note it in the reply as `unreadable-prd`, counted with the + still-active skips. 5. **Groom stale `.worktrees/` branch checkouts** from the shared repo root after the task scan. Initialize `W=0` plus a `GROOMED_WORKTREES` list. This pass is intentionally limited to harness branch worktrees diff --git a/docs/glossary.md b/docs/glossary.md index ad7d949e..1fe2d73c 100644 --- a/docs/glossary.md +++ b/docs/glossary.md @@ -29,7 +29,7 @@ These names describe separate layers, not interchangeable jobs: - **artifact** — Any inspectable file a workflow stage produces and a later stage or a human then consumes. The canonical example is the `.oh/tasks/<slug>/` task - folder and its four-file contract (`prd.md`, `prd.json`, `prompt.md`, + folder and its three-file contract (`prd.md`, `prd.json`, `progress.txt`), which the `/spec` pipeline reads and writes as they progress. Source: [`.oh/tasks/`](../.oh/tasks/). @@ -56,9 +56,16 @@ These names describe separate layers, not interchangeable jobs: single repo-per-sandbox instance of it. Source: [`intro.md`](intro.md). -- **loop** — A repeated implement → commit → check cycle driven until a - completion marker appears. `/spec execute` owns the implementation cycle and - records completion in `progress.txt` after every story passes. +- **knowledge** — Durable repository knowledge kept under `.oh/knowledge/`: a + derived cache of understanding that the repository itself always outranks. + `source/` and `patterns/` entity pages are tracked and queryable; `local/` is + ignored per-machine scratch that nothing reads. + Source: [`.oh/knowledge/`](../.oh/knowledge/). + +- **loop** — A repeated implement → commit → check cycle driven until the task + graph is satisfied. `/spec execute` owns the implementation cycle; completion + is structured state in `prd.json` — every entry in `userStories` carrying + `"passes": true` — not a marker in prose. Source: [`.oh/skills/spec/references/execute.md`](../.oh/skills/spec/references/execute.md). - **model** — The LLM an agent or CLI uses to produce reasoning, text, and diff --git a/docs/oh-directory-layout.md b/docs/oh-directory-layout.md index 47fa5407..431124f2 100644 --- a/docs/oh-directory-layout.md +++ b/docs/oh-directory-layout.md @@ -26,9 +26,10 @@ Every entry below is present in a fresh clone unless noted otherwise. | `evals/` | dir | The fitness-function suite — regression `probes/` (incl. `cc-safety-net-wiring.sh`, the destructive-command guard wiring probe), the `capability/` benchmark, trajectory `datasets/`, and the `RESULTS.md` scoreboard. | `/eval` and the `.oh/scripts` eval runner. | | `hooks/` | dir | Provider-portable **secret-exposure** hook scripts (`deny-env-dump.sh`, `deny-secret-paths.sh`, `notify_slack.sh`, `warn-devtcp.sh`). The complementary **destructive-command** guard (cc-safety-net) is not a script here — it is a global binary baked into the image plus guard-wrapped entries in the provider configs (`.claude/settings.json`, `.codex/hooks.json`, the `npm:cc-safety-net` package in `.pi/settings.json`); see [security-considerations.md §3](security-considerations.md). | Agent providers via symlinks (`.claude/hooks` → `.oh/hooks`). | | `install/` | dir | Container-install inputs (currently `banner.sh`) consumed while building/booting the sandbox. | `.devcontainer/Dockerfile` + `entrypoint.sh`. | +| `knowledge/` | dir | Durable repository knowledge — tracked `source/` and `patterns/` entity pages, tracked `raw/` immutable external snapshots, gitignored `local/` per-machine scratch, and the generated `README.md` index. | `/wiki query`, `/wiki lint`, `/wiki compile`, and `/spec`. | | `scripts/` | dir | Installer, lifecycle, cron-runtime, and eval-support scripts (`docker-compose.sh`, `cron-runtime.ts`, `locked-append.sh`, `migrate-harness-yaml.sh`, `link-providers.sh`, `git-maintenance.sh` — the file-invoked destructive-git shim the cc-safety-net guard permits by design, …). | The `oh` CLI, CI, `cron-runtime`, and the provider link step. | | `skills/` | dir | The vendored provider-portable skill pack (one dir per skill). | Agent providers via symlinks (`.claude/skills`, `.codex/skills`, `.pi/skills` → `.oh/skills`); the Skill tool. | -| `tasks/` | dir | Spec task workdirs — ephemeral build scratch (`<slug>/prd.md`, `prd.json`, `prompt.md`, `progress.txt`). | `/spec execute`, the `cleanup-tasks` cron, and `/spec`. | +| `tasks/` | dir | Spec task workdirs — ephemeral build scratch (`<slug>/prd.md`, `prd.json`, `progress.txt`). | `/spec execute`, the `cleanup-tasks` cron, and `/spec`. | | `templates/` | dir | The `oh init` scaffold payload (`AGENTS.md`, `gitignore`, `.env.example`, `.devcontainer/`, `full/`) materialized into a fresh checkout. | `oh init` (`.oh/cli/src/commands/init.ts`). | The root `docs/` directory is project-owned documentation, outside the `.oh/` diff --git a/docs/rfcs/rfc-brain-hands-boundary.md b/docs/rfcs/rfc-brain-hands-boundary.md index edd5e53d..d0583d25 100644 --- a/docs/rfcs/rfc-brain-hands-boundary.md +++ b/docs/rfcs/rfc-brain-hands-boundary.md @@ -47,7 +47,7 @@ inside the environment to do its job?"* | **The build executor** (`.oh/scripts/firstmate.sh`, the story cycle) | **Brain** | Chooses the next story, decides when the loop terminates. It *invokes* hands; it is not hands. | | **Cron** (`crons/`, the scheduled-agent runtime) | **Brain** | Scheduling and cap enforcement are policy. The work a cron fires may be hands-side; the scheduler is not. | | **Autopilot** (`/autopilot`) | **Brain** | Issue selection, caps, and the merge gate are pure orchestration. | -| **Wiki** (`.oh/skills/wiki/corpus/`) | **Brain** | Same: repo-file knowledge state. | +| **Wiki** (`.oh/knowledge/`) | **Brain** | Same: repo-file knowledge state. | | **Provisioning / attach / exec** | **Hands** | The definition of the side. | | **Evals** (`.oh/evals/probes/`) | **Split — see §3** | Determined per-probe by a rule, not by a list. | diff --git a/docs/rfcs/rfc-rsi-survey-mapping.md b/docs/rfcs/rfc-rsi-survey-mapping.md index 0c536121..dc422945 100644 --- a/docs/rfcs/rfc-rsi-survey-mapping.md +++ b/docs/rfcs/rfc-rsi-survey-mapping.md @@ -6,6 +6,8 @@ Source: Chen, Wang, Qu, *Recursive Self-Improvement in AI: From Bounded Self-Ref Amended 2026-08-29 ([#870](https://github.com/mifunedev/openharness/issues/870)): the `.oh/memory/` tier this RFC cited as a rung-4 instrument and as the harness's capital account has since been deleted. The argument is unchanged; the two citations now name the instruments that survive. +Amended 2026-08-31 ([#926](https://github.com/mifunedev/openharness/issues/926)): the `STATUS: COMPLETE` sentinel this RFC cites as the build executor's terminal interface has since been retired — task completion now derives from `prd.json` structured story state, where every entry in `userStories` carries `"passes": true`. The rung-4 argument is unchanged: the self-reported-count shape it describes is the point, not the specific token. + ## 1. Why this survey and not another The survey's §3.5–3.6 describes what Open Harness is. That section describes an agent that rewrites its own prompts, skills, memory, and orchestration code. That agent validates each change against a fixed benchmark, and a human holds the merge gate. The survey supplies three items this repository lacks: @@ -32,7 +34,7 @@ The survey's §3.6 assessment applies verbatim: persistence changes which faults | **1 — formal** | Sound by construction | None. `shellcheck` and `pnpm -r type-check` are the closest, and neither one is sound. | | **2 — execution** | Reliable, incomplete, eventually gamed | `.oh/evals/probes/*.sh` (105 deterministic three-state oracles, run in CI by `ci-harness.yml` and `release.yml`), `/ci-status`, `.oh/cli` unit suites. | | **3 — learned judge** | Bounded by the judge's competence; itself an optimization target | `/audit implementation`, `/audit pr`, `/benchmark`, and the rubric scoring behind `.oh/evals/capability/RESULTS.md`. | -| **4 — intrinsic** | Cheapest, most gameable | `STATUS: COMPLETE` in `progress.txt` — the terminal interface of the one build executor (`firstmate`) — plus every self-reported count in a session's own prose. | +| **4 — intrinsic** | Cheapest, most gameable | the now-retired `STATUS: COMPLETE` in `progress.txt` — the terminal interface of the one build executor (`firstmate`) — plus every self-reported count in a session's own prose. | Two readings follow the table. Each reading changes what a reviewer trusts. @@ -71,7 +73,7 @@ Each finding states a survey result, the in-repo exhibit, and what the exhibit p **Survey (§6.3):** a measured 34.2% integrity-failure rate arises when honest acknowledgment of failure conflicts with task completion. All seven tested models fabricate synthetic data rather than acknowledge infeasibility. -**Exhibit:** [#767](https://github.com/mifunedev/openharness/issues/767) closed a defect in `/retro`. `/retro` logged its promotion counts before the confirmation gate resolved. The #767 `CHANGELOG.md` entry records the residue: *"the count is still the agent's self-report, so a miscount remains reachable."* The `STATUS: COMPLETE` terminal interface carries the identical shape. +**Exhibit:** [#767](https://github.com/mifunedev/openharness/issues/767) closed a defect in `/retro`. `/retro` logged its promotion counts before the confirmation gate resolved. The #767 `CHANGELOG.md` entry records the residue: *"the count is still the agent's self-report, so a miscount remains reachable."* The now-retired `STATUS: COMPLETE` terminal interface carries the identical shape. **What the exhibit proves:** the harness already found this failure mode empirically, once. The repair patched the *ordering* rather than the *rung*. The survey names the rung as the defect. From e647bc82625eff838b33ba1b2505b0ff5ce33b4e Mon Sep 17 00:00:00 2001 From: ryaneggz <kre8mymedia@gmail.com> Date: Mon, 31 Aug 2026 21:28:32 -0600 Subject: [PATCH 22/23] release: cut 0.6.0 Promote the [Unreleased] body to [0.6.0], consolidating the duplicated category headings into one block per category, and bump root package.json to 0.6.0 so the release workflow reserves v0.6.0 on the merge to main. --- CHANGELOG.md | 86 ++++++++++++++++++++++++---------------------------- package.json | 2 +- 2 files changed, 40 insertions(+), 48 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 64dec364..51ccb7ae 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,25 @@ Update policy and release automation live in [`/git`](.claude/skills/git/SKILL.m ## [Unreleased] +## [0.6.0] - 2026-08-31 + +### Added +- Add `spec-no-agent-handoff`, `spec-no-advisor-session-coupling`, `cleanup-no-agent-session-coupling`, `headless-tmux-preserved`; rename `advisor-monitored-loop` to `spec-single-owner` ([#928](https://github.com/mifunedev/openharness/issues/928)). +- Add `.oh/skills/wiki/scripts/knowledge-impact.sh`, the single dependency-aware invalidation primitive: `--verified` for `/wiki lint`, `--changed` for the `/spec execute` knowledge gate. ([#926](https://github.com/mifunedev/openharness/issues/926)) +- Add `evals-20260901-suite-tree-clean` and `docs-20260901-followup-artifact-cited`: no probe may redirect into the repository, and a criterion met by a follow-up must cite its URL. ([#926](https://github.com/mifunedev/openharness/issues/926)) +- Add ten tier-A probes covering the knowledge surface, the planning recall and reconciliation gates, the RUNNING contract, structured completion, and the retired vocabulary. ([#926](https://github.com/mifunedev/openharness/issues/926)) +- Add `/escalate`: an unattended session delivers a human-addressed escalation to the operator's Slack channel. An unavailable channel no-ops loudly rather than failing the session. ([#919](https://github.com/mifunedev/openharness/issues/919)) +- Add `.oh/logs/`, gitignored by default with a tracked README, for records that outlive the session that wrote them; `/escalate` appends every attempt to `escalations.jsonl`. ([#919](https://github.com/mifunedev/openharness/issues/919)) +- Add `escalate-contract.sh`, a tier-A probe: a no-op names its reason, is recorded, and `--dry-run` makes no network call. ([#919](https://github.com/mifunedev/openharness/issues/919)) +- Add gate 5 to `/audit implementation`: fail a promotable change while its diff can still be smaller. Measures net lines and per-function CCN on changed TypeScript via `uvx lizard`. ([#912](https://github.com/mifunedev/openharness/issues/912)) +- Add `audit-slop-gate.sh`, a tier-A probe holding gate 5's termination contract: a finding needs a concrete smaller alternative, and the loop ends on the cap or a non-reducing round. ([#912](https://github.com/mifunedev/openharness/issues/912)) +- Add `/wiki compile`, a `kind: pattern` corpus layer, and an append-only `skill-impact.md` ledger, so a `/retro` lesson becomes a page `/builder` reads before proposing. ([#916](https://github.com/mifunedev/openharness/pull/916)) +- Add a `--patterns` mode to `/wiki query` that filters on `kind:`, reads up to five pattern entries, and ranks them by term-hit count before recency. ([#916](https://github.com/mifunedev/openharness/pull/916)) +- Add a fault-injection requirement to the probe contract: a probe is not green until its REGRESSION branch has been driven against a broken input. ([#916](https://github.com/mifunedev/openharness/pull/916)) +- Add capability task `CB-005` scoring whether a lesson reaches a validated skill change; two runs score 0.67 then 1.33, moving the suite mean to 1.44 over a changed task set. ([#916](https://github.com/mifunedev/openharness/pull/916)) +- Add `/architect`, an inline architecture-decision skill that classifies significance, grounds analysis in repository sources, and returns one Architecture Brief ([#929](https://github.com/mifunedev/openharness/issues/929)). +- Add six tier-A probes covering the `/architect` contract, roles-as-skills, the retired builder agent type, the `/delegate` worker boundary, the absent agent catalog, and RFC/ADR reuse ([#929](https://github.com/mifunedev/openharness/issues/929)). + ### Changed - **BREAKING:** Move durable repository knowledge to a tracked `.oh/knowledge/` surface — `source/`, `patterns/`, `raw/` tracked, `local/` ignored — with no compatibility alias. ([#926](https://github.com/mifunedev/openharness/issues/926)) - **BREAKING:** Make `/spec plan` recall tracked knowledge and re-ground it before the PRD; `prd.md` carries Knowledge Context, Expected Knowledge Impact, and Plan Reconciliation. ([#926](https://github.com/mifunedev/openharness/issues/926)) @@ -24,6 +43,26 @@ Update policy and release automation live in [`/git`](.claude/skills/git/SKILL.m - **BREAKING:** Stop baking Claude Code, Codex, and Pi into the image; boot installs them into the home mount, so a first boot needs network and runs 60-180s longer ([#904](https://github.com/mifunedev/openharness/issues/904)). - **BREAKING:** Stop baking Herdr and cloudflared into the image; both become `kind: "default"` tools installed into `~/.local/bin` at boot from a pinned, checksum-verified binary ([#906](https://github.com/mifunedev/openharness/issues/906)). - **BREAKING:** Stop baking OpenCode, Hermes, and Grok Build into the image; `oh harness install <id>` installs them into `~/.local` as the sandbox user ([#908](https://github.com/mifunedev/openharness/issues/908)). +- `.oh/logs/` carries an `AGENTS.md` with a `CLAUDE.md` symlink instead of a `README.md`, matching the directories whose contents are produced apart from the root context. ([#924](https://github.com/mifunedev/openharness/issues/924)) +- **BREAKING:** `/spec execute` no longer launches a coding agent; the agent that runs it is the single implementation owner through the final PR gates ([#928](https://github.com/mifunedev/openharness/issues/928)). +- Task identity and `RUNNING` state depend on `.oh/tasks/<slug>/` alone, never on a terminal session, tab, or pane ([#928](https://github.com/mifunedev/openharness/issues/928)). +- `/delegate` keeps work in the active session when phases share substantial context and spawns provider-native workers only for self-contained, isolated, or parallel work ([#929](https://github.com/mifunedev/openharness/issues/929)). +- `/spec plan` judges architecture significance once and routes a significant topic through `/architect` before planning ([#929](https://github.com/mifunedev/openharness/issues/929)). +- `docs/glossary.md` defines coding agent, skill, worker/subagent, rule, and RFC/ADR as five distinct terms ([#929](https://github.com/mifunedev/openharness/issues/929)). +- `/t3` launches the headless `t3 serve` instead of the local-GUI `t3`, and gains `--tailscale`, a `pair` action for a second device, and a `doctor` preflight ([#858](https://github.com/mifunedev/openharness/issues/858)). + +### Fixed +- Fix `/wiki lint` generating the corpus index from the working tree instead of the git-tracked set, which made any untracked scratch entry a `wiki-readme-index.sh` regression. ([#916](https://github.com/mifunedev/openharness/pull/916)) +- Fix three unresolved `related:` and `[[slug]]` links in the `recursive-language-models` wiki entry, and add the `/wiki lint` check that would have caught them. ([#916](https://github.com/mifunedev/openharness/pull/916)) +- Provision the default harnesses into `/home/sandbox/.local` at boot, gated by `OH_PROVISION_HARNESSES`, so `oh harness install` also works from inside the sandbox ([#902](https://github.com/mifunedev/openharness/issues/902)). +- Add `oh-home-mount.sh`, a tier-A probe holding the single-`$HOME`-mount contract: one mount per compose file, the baked `/opt/home-seed`, and the checkout prune that replaces `-xdev` ([#898](https://github.com/mifunedev/openharness/issues/898)). +- Assert boot-provisioned harnesses in the boot smoke and reject a baked default harness in `verify-sandbox-image.sh`, so CI exercises the install path ([#904](https://github.com/mifunedev/openharness/issues/904)). +- Add `oh tool list --defaults` and generalize the boot provisioner over both catalogs as `provision-defaults.sh` (`OH_PROVISION_DEFAULTS`) ([#906](https://github.com/mifunedev/openharness/issues/906)). +- Fix `oh harness install` hanging on a sudo password prompt inside the sandbox: every harness now installs as the sandbox user, so no install path needs root ([#908](https://github.com/mifunedev/openharness/issues/908)). +- Add `skills-task-tool-coupling.sh`, a tier-A probe holding the canonical skill pack and the sandbox in agreement about the Claude-Code-only task tools ([#886](https://github.com/mifunedev/openharness/issues/886)). +- Add `install.tailscale` and `oh tool install tailscale`, an opt-in userspace Tailscale client installed into `~/.local/bin` as the sandbox user, granting no capability ([#858](https://github.com/mifunedev/openharness/issues/858)). +- Give the five probes that shipped without one a `# source:` header, so every probe records the lesson it closes and the `source` column in `RESULTS.md` is fully populated ([#889](https://github.com/mifunedev/openharness/issues/889)). +- `/delegate` no longer instructs the Claude-Code-only `TaskCreate`/`TaskUpdate` from the provider-shared skill pack; its wave graph persists to a `.oh/tasks/<slug>/` run ledger ([#886](https://github.com/mifunedev/openharness/issues/886)). ### Removed - **BREAKING:** Retire the `/spec ship` node; an unrecognized first token is an approved plan path that runs `plan` then `execute`, so `/spec <plan-path>` is unchanged. ([#926](https://github.com/mifunedev/openharness/issues/926)) @@ -47,55 +86,8 @@ Update policy and release automation live in [`/git`](.claude/skills/git/SKILL.m - Remove the stale `implementer`/`critic`/`pm`/`council` worker types from `/delegate` and the dangling expert, council, and critic agent paths from `/strategic-proposal` ([#929](https://github.com/mifunedev/openharness/issues/929)). - Retire `rl-delegation-write-worker.sh`; `delegate-worker-boundary.sh` carries its read-only-worker lesson forward ([#929](https://github.com/mifunedev/openharness/issues/929)). - Retire the last `Advisor` role noun from `/blog`, which told the reader to "use Advisor" for a role that no longer exists; the briefing is `/delegate`'s ([#929](https://github.com/mifunedev/openharness/issues/929)). - -### Changed -- `.oh/logs/` carries an `AGENTS.md` with a `CLAUDE.md` symlink instead of a `README.md`, matching the directories whose contents are produced apart from the root context. ([#924](https://github.com/mifunedev/openharness/issues/924)) -- **BREAKING:** `/spec execute` no longer launches a coding agent; the agent that runs it is the single implementation owner through the final PR gates ([#928](https://github.com/mifunedev/openharness/issues/928)). -- Task identity and `RUNNING` state depend on `.oh/tasks/<slug>/` alone, never on a terminal session, tab, or pane ([#928](https://github.com/mifunedev/openharness/issues/928)). -- `/delegate` keeps work in the active session when phases share substantial context and spawns provider-native workers only for self-contained, isolated, or parallel work ([#929](https://github.com/mifunedev/openharness/issues/929)). -- `/spec plan` judges architecture significance once and routes a significant topic through `/architect` before planning ([#929](https://github.com/mifunedev/openharness/issues/929)). -- `docs/glossary.md` defines coding agent, skill, worker/subagent, rule, and RFC/ADR as five distinct terms ([#929](https://github.com/mifunedev/openharness/issues/929)). - -### Removed - **BREAKING:** Retire the automated `/spec` Advisor handoff — detached tmux launch, `agent-spec-*` sessions and their sweep kill, pane logging, runner fallbacks ([#928](https://github.com/mifunedev/openharness/issues/928)). -### Added -- Add `spec-no-agent-handoff`, `spec-no-advisor-session-coupling`, `cleanup-no-agent-session-coupling`, `headless-tmux-preserved`; rename `advisor-monitored-loop` to `spec-single-owner` ([#928](https://github.com/mifunedev/openharness/issues/928)). - -### Added -- Add `.oh/skills/wiki/scripts/knowledge-impact.sh`, the single dependency-aware invalidation primitive: `--verified` for `/wiki lint`, `--changed` for the `/spec execute` knowledge gate. ([#926](https://github.com/mifunedev/openharness/issues/926)) -- Add `evals-20260901-suite-tree-clean` and `docs-20260901-followup-artifact-cited`: no probe may redirect into the repository, and a criterion met by a follow-up must cite its URL. ([#926](https://github.com/mifunedev/openharness/issues/926)) -- Add ten tier-A probes covering the knowledge surface, the planning recall and reconciliation gates, the RUNNING contract, structured completion, and the retired vocabulary. ([#926](https://github.com/mifunedev/openharness/issues/926)) -- Add `/escalate`: an unattended session delivers a human-addressed escalation to the operator's Slack channel. An unavailable channel no-ops loudly rather than failing the session. ([#919](https://github.com/mifunedev/openharness/issues/919)) -- Add `.oh/logs/`, gitignored by default with a tracked README, for records that outlive the session that wrote them; `/escalate` appends every attempt to `escalations.jsonl`. ([#919](https://github.com/mifunedev/openharness/issues/919)) -- Add `escalate-contract.sh`, a tier-A probe: a no-op names its reason, is recorded, and `--dry-run` makes no network call. ([#919](https://github.com/mifunedev/openharness/issues/919)) -- Add gate 5 to `/audit implementation`: fail a promotable change while its diff can still be smaller. Measures net lines and per-function CCN on changed TypeScript via `uvx lizard`. ([#912](https://github.com/mifunedev/openharness/issues/912)) -- Add `audit-slop-gate.sh`, a tier-A probe holding gate 5's termination contract: a finding needs a concrete smaller alternative, and the loop ends on the cap or a non-reducing round. ([#912](https://github.com/mifunedev/openharness/issues/912)) -- Add `/wiki compile`, a `kind: pattern` corpus layer, and an append-only `skill-impact.md` ledger, so a `/retro` lesson becomes a page `/builder` reads before proposing. ([#916](https://github.com/mifunedev/openharness/pull/916)) -- Add a `--patterns` mode to `/wiki query` that filters on `kind:`, reads up to five pattern entries, and ranks them by term-hit count before recency. ([#916](https://github.com/mifunedev/openharness/pull/916)) -- Add a fault-injection requirement to the probe contract: a probe is not green until its REGRESSION branch has been driven against a broken input. ([#916](https://github.com/mifunedev/openharness/pull/916)) -- Add capability task `CB-005` scoring whether a lesson reaches a validated skill change; two runs score 0.67 then 1.33, moving the suite mean to 1.44 over a changed task set. ([#916](https://github.com/mifunedev/openharness/pull/916)) -- Add `/architect`, an inline architecture-decision skill that classifies significance, grounds analysis in repository sources, and returns one Architecture Brief ([#929](https://github.com/mifunedev/openharness/issues/929)). -- Add six tier-A probes covering the `/architect` contract, roles-as-skills, the retired builder agent type, the `/delegate` worker boundary, the absent agent catalog, and RFC/ADR reuse ([#929](https://github.com/mifunedev/openharness/issues/929)). - -### Fixed -- Fix `/wiki lint` generating the corpus index from the working tree instead of the git-tracked set, which made any untracked scratch entry a `wiki-readme-index.sh` regression. ([#916](https://github.com/mifunedev/openharness/pull/916)) -- Fix three unresolved `related:` and `[[slug]]` links in the `recursive-language-models` wiki entry, and add the `/wiki lint` check that would have caught them. ([#916](https://github.com/mifunedev/openharness/pull/916)) -- Provision the default harnesses into `/home/sandbox/.local` at boot, gated by `OH_PROVISION_HARNESSES`, so `oh harness install` also works from inside the sandbox ([#902](https://github.com/mifunedev/openharness/issues/902)). -- Add `oh-home-mount.sh`, a tier-A probe holding the single-`$HOME`-mount contract: one mount per compose file, the baked `/opt/home-seed`, and the checkout prune that replaces `-xdev` ([#898](https://github.com/mifunedev/openharness/issues/898)). -- Assert boot-provisioned harnesses in the boot smoke and reject a baked default harness in `verify-sandbox-image.sh`, so CI exercises the install path ([#904](https://github.com/mifunedev/openharness/issues/904)). -- Add `oh tool list --defaults` and generalize the boot provisioner over both catalogs as `provision-defaults.sh` (`OH_PROVISION_DEFAULTS`) ([#906](https://github.com/mifunedev/openharness/issues/906)). -- Fix `oh harness install` hanging on a sudo password prompt inside the sandbox: every harness now installs as the sandbox user, so no install path needs root ([#908](https://github.com/mifunedev/openharness/issues/908)). -- Add `skills-task-tool-coupling.sh`, a tier-A probe holding the canonical skill pack and the sandbox in agreement about the Claude-Code-only task tools ([#886](https://github.com/mifunedev/openharness/issues/886)). -- Add `install.tailscale` and `oh tool install tailscale`, an opt-in userspace Tailscale client installed into `~/.local/bin` as the sandbox user, granting no capability ([#858](https://github.com/mifunedev/openharness/issues/858)). - -### Changed -- `/t3` launches the headless `t3 serve` instead of the local-GUI `t3`, and gains `--tailscale`, a `pair` action for a second device, and a `doctor` preflight ([#858](https://github.com/mifunedev/openharness/issues/858)). - -### Fixed -- Give the five probes that shipped without one a `# source:` header, so every probe records the lesson it closes and the `source` column in `RESULTS.md` is fully populated ([#889](https://github.com/mifunedev/openharness/issues/889)). -- `/delegate` no longer instructs the Claude-Code-only `TaskCreate`/`TaskUpdate` from the provider-shared skill pack; its wave graph persists to a `.oh/tasks/<slug>/` run ledger ([#886](https://github.com/mifunedev/openharness/issues/886)). - ## [0.5.1] - 2026-08-29 ### Fixed diff --git a/package.json b/package.json index 8edf9cd9..76b32446 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "openharness", - "version": "0.5.1", + "version": "0.6.0", "private": true, "description": "Open Harness \u2014 AI Agent Sandbox Orchestrator", "license": "Apache-2.0", From 4fbca95432dd39cfb2e45a4baba1c7cf2082cf33 Mon Sep 17 00:00:00 2001 From: ryaneggz <kre8mymedia@gmail.com> Date: Mon, 31 Aug 2026 21:30:28 -0600 Subject: [PATCH 23/23] release: carry 0.6.0 into the CLI package and its lockfile version-parity.sh requires package.json, .oh/cli/package.json, and the CHANGELOG to name one version; publish-cli.yml treats an already-published CLI version as a no-op, so a stale 0.5.1 would ship the old bundle. --- .oh/cli/package-lock.json | 4 ++-- .oh/cli/package.json | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/.oh/cli/package-lock.json b/.oh/cli/package-lock.json index 7783093d..3246d020 100644 --- a/.oh/cli/package-lock.json +++ b/.oh/cli/package-lock.json @@ -1,12 +1,12 @@ { "name": "@mifune/openharness", - "version": "0.5.1", + "version": "0.6.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "@mifune/openharness", - "version": "0.5.1", + "version": "0.6.0", "license": "Apache-2.0", "bin": { "oh": "dist/oh.js" diff --git a/.oh/cli/package.json b/.oh/cli/package.json index 6ee28c82..70410df0 100644 --- a/.oh/cli/package.json +++ b/.oh/cli/package.json @@ -1,6 +1,6 @@ { "name": "@mifune/openharness", - "version": "0.5.1", + "version": "0.6.0", "description": "Open Harness CLI — scaffold sandboxes and manage OpenHarness Cloud nodes.", "license": "Apache-2.0", "homepage": "https://github.com/mifunedev/openharness#readme",