Skip to content

fix: account for all observed requests in an ACP prompt - #486

Open
jhgaylor wants to merge 2 commits into
agentclientprotocol:mainfrom
jhgaylor:fix/prompt-usage-accounting
Open

fix: account for all observed requests in an ACP prompt#486
jhgaylor wants to merge 2 commits into
agentclientprotocol:mainfrom
jhgaylor:fix/prompt-usage-accounting

Conversation

@jhgaylor

@jhgaylor jhgaylor commented Sep 7, 2026

Copy link
Copy Markdown

Problem

PromptResponse.usage and _meta.quota.token_count report tokenUsage.last, the final model request of the turn. A tool-using prompt makes several requests; every request before the last one is dropped from the report.

Change

  • Add PromptTokenUsage: captures a thread-total baseline when the prompt starts, then accumulates total - previous deltas from each thread/tokenUsage/updated notification for the active turn.
  • Baseline source: latest thread total seen on the transport (covers load/resume/fork snapshots that arrive before the ACP handler exists), falling back to the in-memory session total. New sessions start at zero.
  • PromptResponse.usage, _meta.quota.token_count, and _meta.quota.model_usage now carry the prompt total instead of the last request.
  • New _meta.usageAccounting (version, source, scope, completeness) documents the contract.
  • Plan-to-implementation continuation is one prompt: both turns accumulate into the same total.
  • session/update.usage_update (context occupancy) is unchanged.

Edge cases

Input Result
Duplicate / replayed snapshot delta 0, no double count
Rate-limit-only notification repeating history delta 0
Delayed notification from a previous turn ignored (turn id mismatch)
Resume with no snapshot before first observation first snapshot sets baseline, not charged; completeness: partial
Negative delta, non-integer, total != input + cached + output, reasoning > output (Codex counter replacement or synthetic context-full total) delta dropped, known usage kept, partial
Cancelled turn or typed failure observed usage retained, partial
No notifications during prompt usage: null, partial
Native child threads, background title generation out of scope, not counted

Behavior change for existing consumers

_meta.quota.token_count and model_usage[].token_count change meaning from last-request to prompt-total. Reports from earlier adapter versions cannot be reconstructed.

Validation

  • npm run typecheck, npm run build
  • vitest run --no-file-parallelism --retry=0: 563 passed, 26 skipped
  • New prompt-usage-accounting.test.ts covers: multiple requests, duplicate snapshots, previous-turn replay, restored history baseline, missing baseline, missing usage, cancellation, typed failure, counter replacement, synthetic context-full total
  • plan-review-events.test.ts covers plan + implementation accumulation
  • Live probe, Codex 0.153.3, two model requests per prompt across three prompts including process restart/resume:
Prompt Prompt total Final request only
1 28,828 14,452
2 29,127 14,602
3 29,425 14,750

Follow-up commit

5845c95 removes child subagent thread entries from the transport-level usage map when the root session closes. The listener records every thread id, and closeSession only deleted the root id.

Signed-off-by: Jake Gaylor <jhgaylor@gmail.com>
The transport listener records thread/tokenUsage/updated totals for every
thread, including native subagent children. closeSession only removed the
root entry, so child entries stayed for the process lifetime.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014uE8inPFvA8x4jA1LQRGmA
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant