Skip to content

feat(schema): track cache creation tokens - #1158

Open
March-77 wants to merge 1 commit into
cloudwego:mainfrom
March-77:agent/track-cache-creation-tokens
Open

feat(schema): track cache creation tokens#1158
March-77 wants to merge 1 commit into
cloudwego:mainfrom
March-77:agent/track-cache-creation-tokens

Conversation

@March-77

Copy link
Copy Markdown

What type of PR is this?

feat

Check the PR title.

  • This PR title matches the format: <type>(optional scope): <description>.
  • The description is user-oriented and clear.
  • A separate user-documentation PR is not required; the additive Go API field is documented inline.

(Optional) Translate the PR title into Chinese.

feat(schema): 追踪提示词缓存写入 token

(Optional) More detailed description for this PR(en: English/zh: Chinese).

en:

Providers can report and bill prompt-cache writes separately from cache reads, but Eino previously exposed only CachedTokens. This made cache-creation volume unavailable to provider integrations and downstream cost accounting.

This PR:

  • adds provider-neutral CacheCreationTokens fields to schema.PromptTokenDetails and the model callback DTO;
  • preserves cache-creation counts through both message concatenation paths using the existing cumulative-usage maximum semantics;
  • adds JSON round-trip and legacy-payload compatibility coverage;
  • verifies cache-read and cache-write maxima independently in standard and agentic message tests.

The change is additive. Existing payloads that omit cache_creation_tokens continue to decode with the Go zero value.

Validation:

  • go test ./schema ./components/model
  • go vet ./...
  • golangci-lint run --new-from-rev=origin/main --timeout 5m (0 issues)
  • git diff --check

go test ./... was also attempted on Windows. The changed packages pass, while unrelated existing filesystem/path and CRLF-sensitive tests fail under Windows. A WSL rerun could not start because the local WSL instance timed out before go version.

zh(optional):

新增独立的缓存写入 token 统计,并在普通及 Agentic 流式消息合并中保留该累计值,便于准确计算提示词缓存写入成本。

(Optional) Which issue(s) this PR fixes:

Fixes #1150

(optional) The PR that updates user documentation:

N/A

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

feat(schema): track LLM cache-creation (write) tokens in PromptTokenDetails

1 participant