configured anthropic-messages: native cached dialect for hosted endpoints (mu-iah94) - #580
Merged
Merged
Conversation
…ints (mu-iah94) Wire-verified during the mu-nd9p2 opus arm: a [[providers.endpoints]] anthropic-messages entry pointed at api.anthropic.com sent 62 requests with ZERO cache_control blocks — 286k input tokens at full rate, cache_creation/cache_read 0 throughout, ~10x the cached price on repeat prefixes. Cause: the Configured factory arm built every anthropic-messages endpoint as OllamaProvider, whose wrapper sets supports_prompt_caching=false (correct for ollama — its anthropic compatibility endpoint accepts no cache_control field) and degrades thinking to the on/off OllamaSwitch (no effort levels). The factory now picks the dialect: hosted/metered endpoints get the native AnthropicProvider (with_api_base + the session cache_ttl per mu-f1a0, cache markers on, adaptive thinking with effort levels); local ollama-compat endpoints keep the existing path. Selection: new optional [[providers.endpoints]] field `prompt_caching` decides explicitly; omitted, an endpoint with a resolved API key is assumed hosted (auth-less = local server, matching the api_key_env docs). The flag rides ProviderSelector::Configured as an additive optional field (serde default; absent on old selectors). Labels shift for hosted endpoints: provider_label was "ollama" (actively misleading in errors from an api.anthropic.com lane), now "anthropic". Event-path labels still use the configured name (mu-v8ye resolves that in session.rs, unaffected). Deploy note: the config parser rejects unknown fields, so a config that sets `prompt_caching` needs a post-this-commit binary; configs that omit it parse everywhere. Test pins the four dialect cases (keyed/auth-less x explicit override).
Contributor
Author
|
Board outcome (consensus panel, live-seat adjudication per cc-nesting policy): both live seats APPROVE, stable — glm-5.2 rounds 2–4, kimi-k3 rounds 3–4. The formal ESCALATE is the standing seat mechanics: claude seats structurally skipped under CLAUDECODE nesting; gpt-5.5 codex seat unparsed/timeout (the broken-pipe infra issue tracked on mu-provider-drift-2026q3-y43la). Artifacts: Pre-PR checks green (fmt/clippy/workspace tests/verify-claims). One earlier clippy round caught a duplicated |
sahuagin
approved these changes
Sep 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Wire-verified during the mu-nd9p2 opus arm: a
[[providers.endpoints]]anthropic-messages entry pointed at api.anthropic.com sent 62 requests with zerocache_controlblocks — 286k input tokens at full rate (~10x the cached price on repeat prefixes). Cause: the Configured factory arm built every anthropic-messages endpoint asOllamaProvider, whose wrapper setssupports_prompt_caching=false(right for ollama — its anthropic-compat endpoint accepts no cache_control) and degrades thinking to on/off.Fix: the factory picks the dialect. Hosted endpoints get the native
AnthropicProvider(with_api_base+ the sessioncache_ttlper mu-f1a0 → cache markers on, adaptive thinking with effort levels); local ollama-compat endpoints keep the existing path. Selection: new optional endpoint fieldprompt_cachingdecides explicitly; omitted, a keyed endpoint is assumed hosted (auth-less = local, matching theapi_key_envdocs). The flag ridesProviderSelector::Configuredas an additive optional serde field.Side effect: hosted-endpoint errors now label "anthropic" instead of the actively-misleading "ollama". Deploy note: the config parser rejects unknown fields — a config setting
prompt_cachingneeds a post-this-commit binary; configs omitting it parse everywhere.Test pins all four dialect cases (keyed/auth-less × explicit override). Bead: mu-iah94.