Pull requests / #635
#635 serve: a malformed "tools" value is a 400 naming the field, not a closed connection (#592)
closed · @Avicennasis · 0 comentarios · En GitHub
Descripción
Fixes #592. One fail-closed change on the request seam, with tests against the mock engine (no model, no GPU).
## `serve/frontend.py`: a malformed `"tools"` is a 400 naming the field, where it closed the connection
`openai_to_messages` (`serve/frontend.py:185`) handed every entry of a request's `tools` on unchanged unless it was a `{"type": "function"}` object, and `anthropic_to_messages` (`:235`) did `t["name"]` on each; the next consumer, `server.py:2422`'s `own = {t.get("name") for t in tools or []}`, assumed an object. So `"tools": "auto"`, `["get_weather"]`, or `[{"type": "function", "function": "get_weather"}]` raised `AttributeError` / `TypeError` / `KeyError` in the request thread and the connection closed with no reply - the `502` behind a proxy in #592. Reproduced here with the mock engine on `main`: 6 of the 8 shapes I sent (the issue's three and a string `function`, over both endpoints) came back `RemoteDisconnected`, with `AttributeError: 'str' object has no attribute 'get'` x3, `TypeError: string indices must be integers` x2 and `KeyError: 'name'` x1 on the server's console. The issue's third shape, `[{"name": "get_weather"}]`, already answered 200 on both APIs: a bare object with a name is the unwrapped form, and it stays so.
Both converters now go through `_tool_list`, next to the #460 `_object_list` it builds on. A value that is not a list of objects (a JSON-encoded string is decoded first, as `messages` is), an entry whose `"function"` is not an object or that has no `"name"` string, or a `"parameters"` / `"input_schema"` that is not an object is a `ValueError`, which the dispatcher already turns into the 400 the other field refusals use:
```
$ curl -s -w ' [%{http_code}]' -X POST http://127.0.0.1:$PORT/v1/chat/completions -H "Content-Type: application/json" \
-d '{"messages":[{"role":"user","content":"hi"}],"tools":"auto","max_tokens":8}'
{"error": {"type": "invalid_request_error", "message": "tools must be a list of objects (a string was sent that is not JSON)"}} [400]
```
That is the mock engine on a free port; the issue's `[{"type":"function","function":"get_weather"}]` answers `tools[0]: "function" must be an object` [400], a function object without a name `tools[0] has no "name" string` [400], and `[{"name":"f","parameters":[]}]` `tools[0] (f): "parameters" must be an object` [400]. No field, `null` and `[]` are still no tools. `server.py:2422` is unchanged: the list it gets is now always objects with a name, so the consumer needs no second guard. `/v1/messages/count_tokens` reads the same request and refuses the same way.
Not overlapping with #510: that one is the history side, `json.loads` on a `tool_calls` entry's `arguments` at `frontend.py:181`; this is the request's `tools` list, four lines down, and the hunks do not touch.
Tests: `serve/test_server.py` (`ClientShapes`, `python -m unittest serve.test_server`): 13 malformed shapes over `/v1/chat/completions` and `/v1/messages`, plus `count_tokens`, are a 400 whose message names `tools`; a well-formed tool on each API still renders into the prompt (the name and a schema property are read back from the mock engine's prompt); the bare form answers 200 on both; the empty forms are `None`. On `main` with the same test file: 9 errors (the connection closed with no reply, as in the issue) and 5 failures (shapes such as `{"name": 5}`, `{"name": null}` or `"parameters": []` that were accepted with 200 and rendered as they came).
## What I ran, and where
Linux x86_64 (Ubuntu 24.04, AMD Ryzen 7 5800X - no AVX-512, no NVIDIA GPU), Python 3.12.3. Nothing under `src/` is touched, so nothing was built.
- `python -m unittest serve.test_server`: 120 tests OK in 55 s (118 as before plus these 2; the 2 run in 0.6 s).
- `python -m unittest serve.test_server.ClientShapes` against `main`'s `serve/frontend.py` with the same test file: 9 errors, 5 failures, as above.
- `git diff --check` clean.
Branch is on `99f3dbd` (0.1.38); happy to rebase if it has moved by the time you look.
En el sitio
Enlaces a install, modelos, releases.