Pull requests / #2290

#2290 Anthropic API: cache_creation_input_tokens, the prompt tokens read now and kept for the next request

open · @xrip · 0 Kommentare · Auf GitHub

Server & APIModels & quantsWindows

Beschreibung

Fixes #2289.

## The change

`/v1/messages` usage now reports the tokens a request reads as a cache write, since they stay in the session (and the conversation cache) for the next request:

| field | before | after |
|---|---|---|
| `cache_read_input_tokens` | the reused prefix | the same |
| `cache_creation_input_tokens` | absent | the tokens read now, but the last |
| `input_tokens` | the tokens read now | 1, the last prompt token |

`input_tokens` stays at least 1: Claude Code takes 0 as "not given" and keeps `message_start`'s whole prompt beside the cache counts, which counts the prompt twice (#1013).

The three fields still add up to the prompt, so a client's context fill is unchanged. Only `anthropic_events`' `message_delta` changes; the OpenAI and Responses paths are untouched.

## Tests

`python -m pytest serve/test_server.py serve/test_request_accounting.py serve/test_parallel.py`: 345 passed (Windows, Python 3.12). The usage assertions now check the three-way split, and `input_tokens >= 1`.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Mehr auf der Site

Links zu Install, Modellen, Releases.