Pull requests / #2290
#2290 Anthropic API: cache_creation_input_tokens, the prompt tokens read now and kept for the next request
open · @xrip · 0 comentarios · En GitHub
Server & APIModels & quantsWindows
Descripción
Fixes #2289. ## The change `/v1/messages` usage now reports the tokens a request reads as a cache write, since they stay in the session (and the conversation cache) for the next request: | field | before | after | |---|---|---| | `cache_read_input_tokens` | the reused prefix | the same | | `cache_creation_input_tokens` | absent | the tokens read now, but the last | | `input_tokens` | the tokens read now | 1, the last prompt token | `input_tokens` stays at least 1: Claude Code takes 0 as "not given" and keeps `message_start`'s whole prompt beside the cache counts, which counts the prompt twice (#1013). The three fields still add up to the prompt, so a client's context fill is unchanged. Only `anthropic_events`' `message_delta` changes; the OpenAI and Responses paths are untouched. ## Tests `python -m pytest serve/test_server.py serve/test_request_accounting.py serve/test_parallel.py`: 345 passed (Windows, Python 3.12). The usage assertions now check the three-way split, and `input_tokens >= 1`. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
En el sitio
Enlaces a install, modelos, releases.