Pull requests / #1675
#1675 serve: say llama.cpp's phrase in the context-overflow 400 (#1615)
open · @nekomario28 · 0 Kommentare · Auf GitHub
Server & APIModels & quantsDocumentation
Beschreibung
Fixes #1615. Agent harnesses detect a context overflow by matching the error text against the phrases servers already use (llama.cpp's `exceeds the available context size`, OpenAI's `exceeds the context window`, `context_length_exceeded`, ...). Strata's two refusal messages matched none, so a prompt bigger than the context failed the turn instead of running the client's compact-and-retry. ## Change Both `ValueError`s keep their wording and guidance and now also carry llama.cpp's phrase: `the request exceeds the available context size`. Docs and tests match the kept prefix (`exceeds the context`), so nothing else had to move. Not done (the issue marks it optional): llama.cpp's `"type": "exceed_context_size_error"` + `n_prompt_tokens`/`n_ctx` fields — the emit sites all share the plain `invalid_request_error` shape and the text alone is enough for the reporter's matcher; happy to add the typed error if you'd like it. ## Test plan - Extended `test_explicit_budget_over_the_context_is_rejected` (OpenAI + Anthropic 400s) and the `room < 1` branch test to assert the phrase. - `serve.test_server.MaxTokens` + `VisionTempFiles`: 13 tests pass.
Mehr auf der Site
Links zu Install, Modellen, Releases.