Skip to content

Price 1-hour-TTL cache writes at 2x input (fixes #162) - #163

Open
MildlyMeticulous wants to merge 1 commit into
phuryn:mainfrom
MildlyMeticulous:fix/cache-1h-ttl-pricing
Open

Price 1-hour-TTL cache writes at 2x input (fixes #162)#163
MildlyMeticulous wants to merge 1 commit into
phuryn:mainfrom
MildlyMeticulous:fix/cache-1h-ttl-pricing

Conversation

@MildlyMeticulous

Copy link
Copy Markdown

Fixes #162.

What was wrong

Anthropic bills cache writes at 1.25x base input for the 5-minute TTL and 2x for the 1-hour TTL. scanner.py read only the aggregate cache_creation_input_tokens, and every cost path applied cache_write — the 5-minute rate — to the whole bucket. 1-hour writes were therefore billed 37.5% under their real rate.

Claude Code writes most of its cache with a 1-hour TTL, so this isn't a rounding error. The API returns the split alongside the aggregate:

"usage": {
  "cache_creation_input_tokens": 3904,
  "cache_creation": {
    "ephemeral_5m_input_tokens": 0,
    "ephemeral_1h_input_tokens": 3904
  }
}

Measured effect

Same 425-file corpus, python cli.py scan && python cli.py stats, before and after:

Est. total cost
before $3,379.26
after $3,702.44
difference +$323.18 (9.6%)

(The issue quotes a larger percentage against raw cache-write cost across all records; this is the smaller, more relevant figure measured through the tool's own deduplicated pipeline.)

Approach

The goal was to change pricing without disturbing token accounting or the two duplicated pricing tables.

  • cache_creation_tokens still holds the aggregate. Only the 1-hour portion is new, in a cache_creation_1h_tokens column on turns. Every existing token total and display is untouched.
  • The 5-minute portion is derived as the remainder. Rows recorded before this change have 1h = 0 and therefore bill exactly as they did — the migration needs no backfill and no rescan.
  • The 1-hour rate is derived as 2 x input rather than adding a cache_write_1h entry to both PRICING tables. Those tables are duplicated between cli.py and dashboard.py (Pricing table duplicated across cli.py and dashboard.py — make Python the single source of truth #82), so deriving avoids adding a third place to keep in sync.
  • calc_cost / calcCost take an optional trailing argument. Omitting it bills everything at the 5-minute rate, so any caller not updated here behaves as before.
  • Existing databases migrate in place through the repo's own _ensure_column helper.

Edge cases handled

  • Older logs with no cache_creation object keep the whole bucket on the 5-minute rate.
  • A breakdown claiming more 1-hour tokens than the aggregate is clamped to the aggregate, so a malformed record can't inflate cost. I saw 22 such records in ~41k; this also covers the case where 5m + 1h doesn't reconcile.
  • Negative values are clamped to zero.
  • Both parse paths are covered — scanner.py has a second, more-indented assistant-record path for sidechain/subagent files that would otherwise have silently kept the old behaviour.

Tests

153 passed (147 existing, unchanged, plus 6 new):

  • 1-hour writes bill at 2x input
  • 5-minute-only writes are unchanged at 1.25x
  • omitting the new argument is backwards compatible
  • mixed TTLs split correctly
  • a 1-hour portion exceeding the total is clamped
  • a negative 1-hour portion is ignored

Note on the tables

cache_write remains the 5-minute rate in both tables and is untouched, so no existing entry changes meaning.

Anthropic bills cache writes at 1.25x base input for the 5-minute TTL and
2x for the 1-hour TTL. The scanner read only the aggregate
`cache_creation_input_tokens` and every cost path applied `cache_write`
(the 5-minute rate) to the whole bucket, so 1-hour writes were billed
37.5% under their real rate.

Claude Code writes most of its cache with a 1-hour TTL, so this is not a
rounding error. On a 425-file corpus, `cli.py stats` moves from
$3379.26 to $3702.44 - a $323.18 (9.6%) increase in reported cost.

The API returns the split alongside the aggregate:

    "usage": {
      "cache_creation_input_tokens": 3904,
      "cache_creation": {
        "ephemeral_5m_input_tokens": 0,
        "ephemeral_1h_input_tokens": 3904
      }
    }

Changes:
- scanner.py records the 1-hour portion in a new `cache_creation_1h_tokens`
  column on `turns`, in both the main and sidechain parse paths. Existing
  databases migrate in place via the existing `_ensure_column` helper.
- `cache_creation_tokens` still holds the aggregate, so token totals and
  every existing display are unchanged. The 5-minute portion is derived as
  the remainder, which keeps rows recorded before this change billing
  exactly as they did.
- calc_cost / calcCost take an optional trailing 1-hour argument and derive
  the 1-hour rate as 2x input rather than adding a rate to each of the two
  duplicated pricing tables. Omitting the argument preserves the old
  behaviour.
- Logs with no `cache_creation` object (older sessions) keep the whole
  bucket on the 5-minute rate, and a breakdown claiming more 1-hour tokens
  than the aggregate is clamped so it cannot inflate cost.

Adds six regression tests covering the 2x rate, the unchanged 5-minute
path, backwards compatibility when the argument is omitted, mixed TTLs,
and both malformed-breakdown clamps.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cache-creation cost under-reported ~49%: 1-hour TTL cache writes billed at the 5-minute rate

1 participant