#6285 [Engine] serve: a tool call that does not match the declared schema reaches the client unchanged and is dropped

open · @erycsonero · 1 commentaires · Sur GitHub

Setup & installServer & APIAMD / HIPNVIDIA / CUDAModels & quantsSecurityLinux

Description

### Area

`[Engine]` — the server/API (`terminal/serve/frontend.py`)

### Desktop app, Terminal or source?

Source (`setup.sh`, git `baf16363b` + local patches). The probes below were re-run against `main` on 19/11.

### What happened

**Problem:** a tool call whose arguments do not match the tool's declared schema is passed through unchanged, so a
client that validates the arguments against the schema drops the call — one lost turn each. Four malformed shapes do
this; a fifth (the parameterless call) is already correct and is here as a control.

| shape written by the model | `main` today | what the schema declares |
|---|---|---|
| `<parameter=command>ls -la` — *only when the tool declares **one required** parameter, `commands`, an array of strings, and the written name is that parameter minus the trailing `s`* | `{"command":"ls -la"}` | `{"commands":["ls -la"]}` |
| `<parameter=arguments>{"commands": ["ls"]}` | `{"arguments":{"commands":["ls"]}}` | `{"commands":["ls"]}` |
| `<parameter=commands>[{"cmd":"ls","is_background":false}]` | `{"commands":[{"cmd":…}]}` | `{"commands":["ls"]}` |
| `<parameter=queries>[{"query":"x","path":"y"}]` on an array-of-strings tool | `{"queries":[{"query","path"}]}` | `{"queries":["x"]}` |
| `<parameter=…>` absent entirely (control: correct today) | `{}` ✅ | `{}` |

Row 1 is **not** a general rule that a parameter loses its plural `s`. It applies only to those three conditions at once;
a tool with two required parameters, or whose required parameter is not an array of strings, is left alone.

Measured over two runs of a long agentic task (an OpenLDAP delivery, ~300 tool calls each): **7 refusals in 270
requests** and **9 in 330** — each refusal one lost turn, counted from the server's diagnostic line (`is missing the
required …`) and cross-checked against the client's own error text. The borrowing class is the dominant one: in an
earlier 457-request run the same diagnostic logged **28** refusals of it (20 × `run_commands` missing `commands`, 8 ×
`read_files` missing `files`), i.e. ~6 % of that run's requests.

### Smallest steps or command that reproduce it

The parser only — no engine, no model, a second to run:

```python
from terminal.serve.frontend import OutputParser

RUN = {"name": "run_commands",
       "parameters": {"type": "object", "properties": {"commands": {"type": "array", "items": {"type": "string"}}},
                      "required": ["commands"]}}
text = '<tool_call><function=run_commands><parameter=command>ls -la</parameter></function></tool_call>'

p = OutputParser(thinking=False, tools=[RUN], stream_tools=True)
events = p.feed(text) + p.finish("stop")
print("".join(e.text for e in events if e.kind == "tool_args"))   # {"command":"ls -la"} -> dropped by the client
```

Expected: `{"commands":["ls -la"]}` — the only parameter the schema declares, as an array of strings.
Actual: `{"command":"ls -la"}`, i.e. a key the schema does not declare and none of the required ones.

### Reproduction status

Reproduced locally.

### Strata version

0.2.14.1 (`baf16363b`) on the local tree; the probe was re-run against `main` on 19/11 (`terminal/serve/frontend.py`).

### Graphics card, memory and system

RTX 5060 Ti 16 GB · 64 GB RAM · Ubuntu (Linux). Not relevant to the parser paths above.

### Model, size and settings

Qwen3.8-Flash-Next GSQ-RCO IQ3_XXS · engine context **131 072** (`--max-context`) · vision on (engine side).
The client (Cline) was configured with a **150 000**-token window on purpose: slightly larger than the engine's,
so it compacts later — as close as possible to the real 131 072 limit — instead of compacting early and
wasting context. That is a client setting; the compactions in those runs landed at **94–104k**. Neither number
affects the parser paths above.

### Logs

Sanitized diagnostic lines from one of the runs (no private content, no paths):

```text
[strata] tool call suspicious: run_commands is missing the required commands - this client refuses the whole call
[strata] tool call suspicious: run_commands.command is not declared in the schema (str, 12)
[strata] tool call suspicious: search_codebase.queries holds 1 dict element(s) where the schema says string
         (1: query,case_sensitive,context_length,path)
```

That diagnostic is local instrumentation, not on `main`; the counts above come from it and from the client's own error
text.

### Scope I would propose

- **In scope**: the borrowed dialect, the array of strings written as objects (and the same with a filter beside the
  text, e.g. `{query, path}`), and — if wanted — the live rewrite that applies those rules *while the call streams*, since
  a client validates the streamed arguments, not only the final object.
- **Out of scope, and why**: the envelope unwrap (that is **#5805**, and the envelope-content repair only exists *inside*
  it), the refused-name filter (**#6120**), anything about `<invoke name=…>` (**#5969** already handles it in code), and
  the parameterless case (already `{}`).
- **Deliberately left alone**: inventing a missing value, or executing a payload that names a *different* tool. Two shapes
  stay as they are — a call wrapped in another tool's envelope whose payload does not fit, and a call the model never
  closed (#211 behaviour). Both cost one turn and the model retries by itself.

### The safety rule I would ask a reviewer to hold me to

The rewrite only looks at a parameter **declared** as `array` of `string`, and it collapses an element in exactly two
cases:

- it is an object whose **only** string is non-empty and sits under one of `cmd`, `command`, `command_text`, `cmd_line`,
  `shell`, `text`, and every other value in that object is a boolean or a number —
  `{"cmd": "ls", "is_background": false}` → `"ls"`;
- it is an object with exactly one non-empty string under one of `query`, `q`, `pattern`, `regex`, `term`, `search`; the
  rest of that object is dropped, filters included — `{"query": "x", "path": "y"}` → `"x"`.

Anything else — a second string under a name that is not in those lists, two of those names in the same element, a list
or an object beside the text in the first case, an element that is not an object — leaves the **whole value** exactly as
it was. There is no partial rewrite and no invented string, and that is why most of the tests are negative ones.

### Tests

Eight `unittest` files (85 tests) that reproduce the shapes and pin the negative cases; they run against **the parser
only — no engine, no model**. I can send them as a patch or a gzipped attachment.

### The question

Are the dialect and the array-of-objects rules welcome as a PR, and should the live rewrite come with them or wait? If
**#5805** lands the envelope first and its author wants it, I would adapt *these two rules* to that code rather than ship
a second mechanism for the same job. Either way the tests are yours to use or discard.

Sur le site

Liens install, modèles, releases.