#4314 [Engine] reject a boolean or a fractional token limit with 400 (Fix #4037)
closed · @XiaoBinGan · 0 comments · View on GitHub
Server & APIModels & quantsSecurity
Description
## Problem and result
`POST /v1/chat/completions` with `"max_tokens": true` answers one token, `2.5` answers two, and
`false` answers the whole context, because both routes convert the field with `int(...)`:
`int(True)` is 1, `int(2.5)` is 2, and `False` falls through the `or` chain to 0, which
`Service.prepare` reads as the remaining context. `/v1/messages` does the same for `max_tokens`. The
request therefore runs with a generation budget nobody asked for, and nothing says so.
`/v1/responses` already refuses the same values with `"max_output_tokens must be a positive whole
number"`, so the three routes disagreed.
After this change both routes answer HTTP 400 and name the field, and `/v1/responses` behaves as it
did. Whole numbers, the documented `0` / `-1` sentinels, an absent field, a whole-valued float
(`8.0`) and a numeric string (`"192"`, which some clients send) keep working.
## Related discussion
#4037 — the report, with the lines (`serve/server.py:7148` and `:7396` at the
commit it was filed against).
## Approach
One helper next to `check_request_sampling`, which the same two handlers already call for the same
kind of 400:
```python
def check_token_limit(req, *names) -> int:
for name in names:
value = req.get(name)
if value is None:
continue
if isinstance(value, bool):
raise ValueError(f"{name}: a whole number of tokens (true and false are not)")
...
if value == 0:
continue # 0 is "the rest of the context": the next field, or no cap, decides
return value
return 0
```
The two call sites become `check_token_limit(req, "max_completion_tokens", "max_tokens")` and
`check_token_limit(req, "max_tokens")`. The first field naming a non-zero cap still wins, as the `or`
chain it replaces did, so nothing changes for a request that sends both. A `ValueError` is already the
established 400 on these paths (`stop_strings`, `check_request_sampling`, a bad `tool_choice`), so no
error plumbing was touched. `serve/responses.py` is not changed: it already validates and a
`ResponsesError` carries its own field name.
## Validation
**Passed:** `python -m unittest serve.test_request_hardening -v` (Python 3.11.16, from a source checkout):
**11 tests, OK**, including the new `test_a_boolean_or_fractional_token_limit_is_a_400`. It posts
`true`, `2.5` and `false` as `max_tokens` on `/v1/chat/completions` and `/v1/messages`, and as
`max_completion_tokens` on chat, and asserts **400** with the field named; then posts `8`, `8.0`,
`"8"`, `0`, `-1` and `{max_completion_tokens: 8, max_tokens: 4}` and asserts **200**.
**Passed:** no regression in the neighbours that share these handlers:
`serve.test_responses` + `serve.test_security` **69 tests, OK (1 skipped)**;
`serve.test_request_accounting` + `serve.test_effort_paths` + `serve.test_vision_token_limit`
**30 tests, OK**.
**Failed:** none.
**Not run:** the same requests against a **live engine**: every check above uses the suite's `MockEngine`,
which is what the existing hardening tests use. The code path is the request parser, before any engine
call, so a real model would only repeat the same 400; I did not run it against a model and I am not
claiming I did.
**Not run:** `python -m pytest tests/terminal` and the `apps/host` UI tests: no file in those trees is
touched.
## Limits
- A whole-valued float (`8.0`) is accepted, because `int(8.0)` changes nothing; a fractional one is
refused. That is the line the issue drew ("reject boolean and fractional token limits").
- A numeric string is still read, so `"2.5"` is a 400 (it was a `ValueError` from `int()` before, so
the answer is the same) but the message now names the field.
- Negative values other than `-1` are not refused here. The issue asked to preserve the documented
sentinels, not to add range checks, and `Service.prepare` already treats `<= 0` as "the rest".
Related on this site
Quick links to related pages.