Skip to content

fix(provider): count Anthropic cache_creation tokens in input usage - #9881

Open
xiaoyuyu6420 wants to merge 1 commit into
AstrBotDevs:masterfrom
xiaoyuyu6420:fix/anthropic-cache-creation-usage
Open

fix(provider): count Anthropic cache_creation tokens in input usage#9881
xiaoyuyu6420 wants to merge 1 commit into
AstrBotDevs:masterfrom
xiaoyuyu6420:fix/anthropic-cache-creation-usage

Conversation

@xiaoyuyu6420

@xiaoyuyu6420 xiaoyuyu6420 commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Problem

Anthropic's Usage.input_tokens only counts tokens after the last cache breakpoint — it excludes both cache reads and cache writes. ProviderAnthropic._extract_usage maps input_tokens into TokenUsage.input_other and cache_read_input_tokens into input_cached, which is correct for reads — but cache_creation_input_tokens (tokens written into a freshly created cache entry) are dropped entirely.

Per Anthropic's official docs (linked in the code comment):

total_input_tokens = cache_read_input_tokens + cache_creation_input_tokens + input_tokens

So on any request that creates a new cache entry (e.g. first turn after a cache ephemeral_1h breakpoint on a long system prompt), usage.input and usage.total undercount the real input — which feeds context-occupancy stats (current_context_tokens in the tool-loop runner).

The OpenAI provider handles this category correctly: input_other = prompt_tokens - cached, so input_other includes cache-write tokens. Anthropic was the only provider dropping them.

Changes

  • astrbot/core/provider/sources/anthropic_source.py: fold cache_creation_input_tokens into input_other in both _extract_usage (non-streaming / message_start) and _update_usage (streaming message_delta), mirroring OpenAI's accounting where input_other holds all non-cache-read input including cache writes.
  • tests/test_anthropic_source.py (new): 5 unit tests covering extract/update with and without cache fields, and omitted-field preservation in streaming deltas.

Verification

$ uv run pytest tests/test_anthropic_source.py tests/test_anthropic_kimi_code_provider.py tests/unit/test_provider_stats.py -q
38 passed

Summary by Sourcery

Ensure Anthropic cache writes are included in input token accounting so context and usage statistics remain accurate.

Bug Fixes:

  • Correct Anthropic input and total token usage by including cache-creation tokens in non-cached input accounting.

Tests:

  • Add unit coverage for Anthropic usage extraction and streaming updates with cache fields and omitted usage fields.

Anthropic's input_tokens only covers tokens after the last cache breakpoint;
cache_creation_input_tokens (tokens written into a new cache entry) were
dropped entirely from TokenUsage, so usage.input/total undercounted requests
that create a fresh cache entry. Fold cache_creation_input_tokens into
input_other in both _extract_usage and the streaming _update_usage, matching
the OpenAI provider's accounting where input_other holds all non-cache-read
input (including cache writes).

Official formula: total_input = cache_read + cache_creation + input_tokens.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've reviewed your changes and they look great!

Sourcery assessment

Approved.


Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant