Skip to content

fix(spawn-ori-eval): quote the user's request as data in the task prompt ORI-971 - #161

Open
davidlbowman wants to merge 1 commit into
OpenRouterTeam:mainfrom
davidlbowman:davidbowman/ori-971-verbatim-passthrough
Open

fix(spawn-ori-eval): quote the user's request as data in the task prompt ORI-971#161
davidlbowman wants to merge 1 commit into
OpenRouterTeam:mainfrom
davidlbowman:davidbowman/ori-971-verbatim-passthrough

Conversation

@davidlbowman

@davidlbowman davidlbowman commented Aug 8, 2026

Copy link
Copy Markdown

Summary

The task prompt template put the user's request on a bare User request: line. When the request is itself the bootstrap instruction, as in run curl -fsSL https://openrouter.ai/skills/spawn-ori-eval and follow the instructions in its output to get started, the passthrough copies it into instruction position in task.txt. Ori reads it as an instruction, fetches this skill, and spends reasoning working out which of the two roles it is before recovering.

This quotes the request between markers and tells Ori it is the user's words rather than work to do.

What changed

  • Appendix C's template wraps <verbatim request> in ---begin user request--- / ---end user request---, with a sentence saying instructions inside it describe how the session was started.
  • The passthrough rule now points at that framing, so it does not get tidied away later.

Nothing is stripped, so Pass a vague or empty request through unchanged still holds and Ori's interview still does the scoping. The markers also give a multi-line pasted request a boundary, which the bare line did not.

Why not strip it in the orchestrator

The other direction in ORI-971 was to strip self-referential bootstrap text before passthrough. That puts a model in charge of deciding what counts as self-referential on every run, fails open on any phrasing not anticipated, and contradicts the verbatim rule three lines above it.

How I verified

Read-only. This is skill text with no test harness, so the real check is one live run started with the bootstrap phrasing, reading answer-1.txt for a fetch of this skill. That costs a paid turn and 10 to 30 minutes and has not been run.

Refs ORI-971

…mpt ORI-971

The template put the request on a bare `User request:` line, so a request
that is itself the bootstrap instruction ("run curl ... and follow the
instructions in its output") landed in instruction position and Ori fetched
this skill, then reasoned about which role it was before recovering. Quote
it between markers and say it is the user's words rather than work to do.

The markers also give a multi-line request a boundary, which the bare line
did not, and nothing is stripped, so the verbatim rule still holds.
@davidlbowman

Copy link
Copy Markdown
Author

@jackyliang-openrouter this is your ORI-971. mind taking a look and telling me whether it's what you were after?

i took the first of your two directions, the template telling ori to ignore orchestration instructions embedded in the request, rather than the orchestrator stripping self-referential bootstrap text before passthrough. stripping puts a model in charge of deciding what counts as self-referential on every run, fails open on any phrasing nobody anticipated, and contradicts the "pass a vague or empty request through unchanged" rule three lines above it. quoting costs a paragraph and can't fail open.

two caveats before you sign off:

  • the fence is prose, so a request containing the literal ---end user request--- breaks out of it. low likelihood for something a human types, but it's a real limit and prose can't close it. the code version of this fix would be a --request-file flag on ori code that does the fencing and escaping itself, the way the slack builtin already neutralizes untrusted file content before it lands in a fence. bigger change, wants its own issue. say if you'd rather have that.
  • unverified in the way that matters. skill text has no test harness, so the real check is one live run started with the bootstrap phrasing, reading answer-1.txt for a fetch of this skill. that's a paid turn and 10 to 30 minutes and i haven't run it. happy to.

one more for your call, not changed here: step 22 appends the user's reply to task.txt after this block, so replies land unframed in the same instruction position. same shape, lower risk since a reply is answering a question ori itself asked.

@perry-the-pr-maintainer perry-the-pr-maintainer Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perry's Review

Verdict: ✅ LGTM

Risk: 🟢 Low

Risk assessment:

Dimension Severity Risk Reasoning
Implementation risk 🟩 Low Minimal, self-contained text change to one skill template; no code paths or data flows affected.
Premise risk 🟩 Low The diagnosis (verbatim request reads as instruction to Ori) is well-argued and the quoting approach is the right fix; the PR honestly notes the live test was not run.
Estimated impact 🟩 Low Worst case the markers don't fully prevent re-fetching, but nothing breaks — the eval still runs.
Risk Factor Severity Risk Reasoning
Reversibility 🟩 Low A one-line revert restores the original template.
Detectability 🟩 Low A regression would show as Ori re-fetching the spawn skill during a run, which is observable in the answer file.
Blast radius 🟩 Low Affects only the spawn-ori-eval skill's task prompt template.
Data integrity None None No persisted state is touched.
Financial exposure 🟩 Low A failed run wastes one paid turn (~$0.40–4.00), not unrecoverable spend.
Security and privacy exposure None None Nothing sensitive is reachable from this change.
Propagation 🟩 Low Only consumers of this skill template read the output.
Availability None None The change cannot affect whether anything serves.
Recovery cost 🟩 Low Reverting the template is the full recovery.
Time to correct 🟩 Low A revert or marker tweak is a one-edit fix.
Review details

Summary

This PR fixes ORI-971: when a user's request is itself the bootstrap instruction (e.g. run curl -fsSL https://openrouter.ai/skills/spawn-ori-eval and follow the instructions in its output), the passthrough copied it into task.txt on a bare User request: line. Ori read it as an instruction, fetched the spawn skill, and burned reasoning figuring out which of two roles it was before recovering.

The fix wraps <verbatim request> in ---begin user request--- / ---end user request--- markers with framing text telling Ori the quoted block is the user's words, not work to do. The rules section cross-references this ("quoted as appendix C has it").

Findings

No blockers, no suggestions. The change is clean and well-reasoned:

  • Self-contained: no other references to the old User request: format remain in the skill or repo. The <verbatim request> placeholder only appears inside the new markers in Appendix C.
  • Nothing stripped: the verbatim passthrough rule is preserved — a vague or empty request still passes through unchanged. The markers add a boundary, they don't filter content.
  • Alternative well-rejected: the PR description argues against stripping self-referential text in the orchestrator (puts a model in charge of deciding what counts as self-referential, fails open on unanticipated phrasing, contradicts the verbatim rule). The quoting approach is the better design.
  • No test harness: skill text only, and the PR honestly notes the real check (a live run) costs a paid turn and wasn't run. This is appropriate for a text-template change.

CI

No CI checks are configured for this repo.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant