Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,7 @@ One gateway process serves both modes simultaneously, chosen per request by URL
- **A request with no tools never reaches the advisor.** It has no next action to steer — it is a harness one-shot (summarise a fetched page, title a chat), not an agent loop. Measured before the gate: 6 of 7 consults in one session came from 2-message toolless calls, 50s of Opus-tier advisor time spent where no advice could be acted on. Gated in both halves, since either alone leaks: `interceptConsultAdvisor` returns before injecting the tool or the executor prompt, and the gateway only wires `onAdvisorToolUse` when the tool was actually offered. Guarded by `test/translate.sh` (G5) and `test/dispatch.sh` (C1-SIDECALL).
- The advisor continuation is exempt from the 120s stream-idle watchdog (`continuationActive`) and streams like the first call. Both are needed: Corti has gone silent for well over 120s *mid-generation* after emitting a token, so streaming alone does not keep the watchdog fed. Pings still go out during the continuation (the advisor phase is over, so they no longer displace the "Advising" indicator). Its ceiling is an absolute deadline computed from what is left of `NONSTREAM_TIMEOUT_MS` after the advisor phase — a fresh full budget would outlive the client's own deadline, so the graceful failure note would never land.
- Advisor sessions: children are marked via a `-noadvisor-` token placed *before* the mode marker (matched with `includes("-noadvisor-")`, not `endsWith`); they skip the 120s stream-idle watchdog and the advisor intercept (recursion guard). Debug logs are per-session (`x-claude-code-session-id`); advisor children log into the parent's file via `x-corti-advisor-for` through `ANTHROPIC_CUSTOM_HEADERS`.
- **A non-multimodal model must never receive an image block (A1, third form).** `corti-s1` (opus) is blind: a `Read` on a `.png` returns a base64 image block that Corti rejects with `400 "…is not a multimodal model"`, killing the session. `interceptImages` describes each image via a sighted side model (`corti-s1-mini-instant`, the instant sonnet tier — multimodal and non-reasoning, so no thinking cap and low latency; `CORTI_VISION_MODEL` overrides) and replaces the block with text, so the blind primary never sees it. It is **capability-driven** — the gateway reads `capabilities.image_input` from the catalog, so a future multimodal `corti-s1` passes images through untouched with no code change (the whole point of using capabilities, not a hardcoded blind list). The sidecall asks for a JSON object (`response_format: { type: "json_object" }`, which Corti honors) with typed fields — a verbatim `text_content` transcription, a structural `description`, a `palette` of approximate hex bound to what carries it, and an `uncertainties` list — so the lossiness is a machine-readable signal, not a hunch; `runDescribeImage` parses and labels each field, degrading to raw text if the model ignored the schema. **Every field is a property of the image, never of the request**: the cache key is the image hash, so a request-dependent field is computed from the first turn's question and then replayed, stale, on every later turn. An `answer` field was tried and removed for exactly that reason — it also drove the vision model to answer as the calling agent ("I cannot save files to your directory") and to narrate the image as a spec sheet. The prompt lives in `lib/vision-describe-prompt.txt` (fingerprinted via `lib/*.txt`), carrying per-type checklists; `image_type` is deliberately not requested, since it came back as UI mockup/document/screenshot/slide for the same image at temperature 0.1 and nothing routes on it. Descriptions are cached per image by a hash of the image data alone — never scoped to a `tool_use_id`, since a re-Read of a pasted image carries a fresh id and would re-describe identical bytes into a different palette the model then trusts — and must be byte-identical across turns — the same A1 prefix-cache failure as WebSearch if they drift. The describe sidecall is context-aware: `interceptImages` extracts the user's question (or the model's stated intent, if there's no user text) and passes it so the vision model knows which parts of the image matter — it steers what the description covers, never what it answers. `describeContext` strips `<system-reminder>` spans first: the harness puts a CLAUDE.md replay in the same user message as the image, and a measured 3656-char reminder ahead of a 117-char question meant the vision model was told the reminder was the request. Decoding is pinned (`temperature: 0.1`, `top_p: 0.9`) and the image part is sent before the text part; unpinned, the same prompt swung a background read between `#111111` and `#141413`, making any prompt change unmeasurable. The cache key is the image hash only, so the description is computed once on the turn the image first appears with its question and reused after. The describe sidecall re-enters the gateway's own openai endpoint with `skipImages: true`, the recursion guard checked before any capability lookup so a misconfigured catalog (the vision model itself blind) can't loop; it carries no tools, so the advisor gate (G5) skips it too. A describe failure degrades to a placeholder, never a crash. Describing is lossy — it matches the manual delegation the user used to repeat by hand, but pixel-precise UI work should still route the turn at a multimodal model. The gateway threads `describeImage` only for image-bearing requests (lazy capability fetch on the first one), so non-image requests pay nothing. Guarded by `test/translate.sh` (image: cases).
- Three context readouts legitimately disagree: `/context` shows the harness's own estimate of the raw Anthropic body; the statusline shows the model's real usage from the *last successful* turn; the proxy's `count_tokens` is a local char/4 estimate. Divergence alone is not a bug.
- `test/models.sh` covers `lib/models.mjs` tier/caps logic against captured fixtures; it should stay green.

Expand Down
Loading