fix(spawn-ori-eval): quote the user's request as data in the task prompt ORI-971 - #161
Conversation
…mpt ORI-971
The template put the request on a bare `User request:` line, so a request
that is itself the bootstrap instruction ("run curl ... and follow the
instructions in its output") landed in instruction position and Ori fetched
this skill, then reasoned about which role it was before recovering. Quote
it between markers and say it is the user's words rather than work to do.
The markers also give a multi-line request a boundary, which the bare line
did not, and nothing is stripped, so the verbatim rule still holds.
|
@jackyliang-openrouter this is your ORI-971. mind taking a look and telling me whether it's what you were after? i took the first of your two directions, the template telling ori to ignore orchestration instructions embedded in the request, rather than the orchestrator stripping self-referential bootstrap text before passthrough. stripping puts a model in charge of deciding what counts as self-referential on every run, fails open on any phrasing nobody anticipated, and contradicts the "pass a vague or empty request through unchanged" rule three lines above it. quoting costs a paragraph and can't fail open. two caveats before you sign off:
one more for your call, not changed here: step 22 appends the user's reply to |
There was a problem hiding this comment.
Perry's Review
Verdict: ✅ LGTM
Risk: 🟢 Low
Risk assessment:
| Dimension | Severity | Risk | Reasoning |
|---|---|---|---|
| Implementation risk | 🟩 | Low | Minimal, self-contained text change to one skill template; no code paths or data flows affected. |
| Premise risk | 🟩 | Low | The diagnosis (verbatim request reads as instruction to Ori) is well-argued and the quoting approach is the right fix; the PR honestly notes the live test was not run. |
| Estimated impact | 🟩 | Low | Worst case the markers don't fully prevent re-fetching, but nothing breaks — the eval still runs. |
| Risk Factor | Severity | Risk | Reasoning |
|---|---|---|---|
| Reversibility | 🟩 | Low | A one-line revert restores the original template. |
| Detectability | 🟩 | Low | A regression would show as Ori re-fetching the spawn skill during a run, which is observable in the answer file. |
| Blast radius | 🟩 | Low | Affects only the spawn-ori-eval skill's task prompt template. |
| Data integrity | None | None | No persisted state is touched. |
| Financial exposure | 🟩 | Low | A failed run wastes one paid turn (~$0.40–4.00), not unrecoverable spend. |
| Security and privacy exposure | None | None | Nothing sensitive is reachable from this change. |
| Propagation | 🟩 | Low | Only consumers of this skill template read the output. |
| Availability | None | None | The change cannot affect whether anything serves. |
| Recovery cost | 🟩 | Low | Reverting the template is the full recovery. |
| Time to correct | 🟩 | Low | A revert or marker tweak is a one-edit fix. |
Review details
Summary
This PR fixes ORI-971: when a user's request is itself the bootstrap instruction (e.g. run curl -fsSL https://openrouter.ai/skills/spawn-ori-eval and follow the instructions in its output), the passthrough copied it into task.txt on a bare User request: line. Ori read it as an instruction, fetched the spawn skill, and burned reasoning figuring out which of two roles it was before recovering.
The fix wraps <verbatim request> in ---begin user request--- / ---end user request--- markers with framing text telling Ori the quoted block is the user's words, not work to do. The rules section cross-references this ("quoted as appendix C has it").
Findings
No blockers, no suggestions. The change is clean and well-reasoned:
- Self-contained: no other references to the old
User request:format remain in the skill or repo. The<verbatim request>placeholder only appears inside the new markers in Appendix C. - Nothing stripped: the verbatim passthrough rule is preserved — a vague or empty request still passes through unchanged. The markers add a boundary, they don't filter content.
- Alternative well-rejected: the PR description argues against stripping self-referential text in the orchestrator (puts a model in charge of deciding what counts as self-referential, fails open on unanticipated phrasing, contradicts the verbatim rule). The quoting approach is the better design.
- No test harness: skill text only, and the PR honestly notes the real check (a live run) costs a paid turn and wasn't run. This is appropriate for a text-template change.
CI
No CI checks are configured for this repo.
Summary
The task prompt template put the user's request on a bare
User request:line. When the request is itself the bootstrap instruction, as inrun curl -fsSL https://openrouter.ai/skills/spawn-ori-eval and follow the instructions in its output to get started, the passthrough copies it into instruction position intask.txt. Ori reads it as an instruction, fetches this skill, and spends reasoning working out which of the two roles it is before recovering.This quotes the request between markers and tells Ori it is the user's words rather than work to do.
What changed
<verbatim request>in---begin user request---/---end user request---, with a sentence saying instructions inside it describe how the session was started.Nothing is stripped, so
Pass a vague or empty request through unchangedstill holds and Ori's interview still does the scoping. The markers also give a multi-line pasted request a boundary, which the bare line did not.Why not strip it in the orchestrator
The other direction in ORI-971 was to strip self-referential bootstrap text before passthrough. That puts a model in charge of deciding what counts as self-referential on every run, fails open on any phrasing not anticipated, and contradicts the verbatim rule three lines above it.
How I verified
Read-only. This is skill text with no test harness, so the real check is one live run started with the bootstrap phrasing, reading
answer-1.txtfor a fetch of this skill. That costs a paid turn and 10 to 30 minutes and has not been run.Refs ORI-971