Issues / #2451
#2451 ### More data for #1431: wrong parameter syntax inside an otherwise valid call (Coder IQ1_M on an Arc Pro B60)
open · @mbaaku · 0 comments · View on GitHub
Setup & installServer & APINVIDIA / CUDAModels & quantsLinux
Description
**Hardware note:** this is an **Intel Arc Pro B60** (`bmg-g21`, SYCL backend, built with `AOT=bmg-g21`). The startup banner says the SYCL backend was tested on an Arc Pro B70 and an Arc A750, so the B60 counts as untested. I have no B70 or CUDA card to check whether the rate is backend-specific, so treat that as an open question.
**Setup**
- Strata v0.1.44.1 (`296b59d`), SYCL backend, 1× Intel Arc Pro B60 (24 GB), Debian Linux 13 , Docker 29.8.1 (`ONEAPI_DEVICE_SELECTOR=level_zero:0`, `STRATA_EXPERT_SPLIT=1`)
- Model: Qwen3.8-Flash-Next **Coder IQ1_M** (`--kv int8`, `--max-context 131072`, 1 slot)
- Run config: `"tool_call_recovery": true`, `"fit_max_tokens": true`; reasoning effort left at the model default
- Client: Continue CLI (`cn`) through LiteLLM, OpenAI chat completions, streaming, **21 tools** (built-in `Bash`, `Read`, `List`, … plus MCP tools)
- Raw outputs captured with `STRATA_DEBUG=1`
<details>
<summary>Full configuration (compose environment, .env, setup command, engine arguments, run config)</summary>
`compose.yaml`, service `strata` (image built from the repo's `Dockerfile.b60`), environment:
~~~yaml
environment:
STRATA_DEBUG: "1"
AOT: bmg-g21
REPO: ${STRATA_ROOT}/Strata
BUILD_DIR: ${STRATA_ROOT}/Strata/build-sycl-aot
JOBS: "28"
STRATA_EXPERT_SPLIT: "1"
ONEAPI_DEVICE_SELECTOR: level_zero:0
SYCL_CACHE_PERSISTENT: "0"
ZES_ENABLE_SYSMAN: "1"
STRATA_SYCL_ROOT: ${STRATA_ROOT}
STRATA_SYCL_IMAGE: strata-sycl-b60
STRATA_VERIFY_DEVICE_PLAN: "1"
STRATA_VERIFY_NO_HOST: "1"
STRATA_MIRROR_MIB: "36864"
~~~
`.env` next to `compose.yaml` (`STRATA_ROOT` and `STRATA_API_KEY` omitted; the commented lines are the other quants used on the same setup):
~~~bash
#STRATA_MODEL=IQ3_S
#STRATA_MODEL=UD-IQ4_XS
STRATA_MODEL=IQ1_M
#STRATA_FAMILY=qwen
#STRATA_FAMILY=unsloth
STRATA_FAMILY=coder
STRATA_CONTEXT=131072
~~~
Container command (after an incremental SYCL build when the checked-out commit changed):
~~~bash
exec ./setup.sh \
--backend sycl \
--family ${STRATA_FAMILY} \
--model ${STRATA_MODEL} \
--gguf-dir "${STRATA_ROOT}/models/${STRATA_MODEL}" \
--models-dir "${STRATA_ROOT}/models" \
--data-dir "${STRATA_ROOT}/data" \
--context ${STRATA_CONTEXT} \
--kv int8 \
--vision no \
--host 0.0.0.0 \
--port 8080 \
--api-key "${STRATA_API_KEY}" \
--yes
~~~
Engine arguments from `strata-coder-iq1_m.json` (paths omitted):
~~~text
--pack <path> --native <path> --ple-gguf <path> --expert-profile <path> --expert-cache auto --spec 4
--spec-min-p 0.5 --mtp <path> --max-context 131072 --kv int8 --kv-resident 32768 --stream-experts
--vram-reserve-mib 2048 --prefill 4096
~~~
Other run-config keys: `"backend": "sycl"`, `"tool_call_recovery": true`, `"reasoning_close_retry": true`, `"fit_max_tokens": true`.
</details>
**What happens**
The model opens the call correctly (`<tool_call>` + `<function=NAME>`), but writes the parameter tag in a different syntax. `parse_tool_call` only reads `<parameter=NAME>`, so the call reaches the client with **empty arguments** (`{}`), even though the full value is in the output. `tool_call_recovery` does not cover these forms.
It starts after roughly 15–25k prompt tokens and comes in streaks (2–3 broken calls in a row). It hits the `Bash` tool far more often than MCP tools with longer argument lists. Sampling made no difference: greedy and temperature 0.7 / top_p 0.8 / top_k 20 gave the same rate. No VRAM warnings in the log; `reasoning_close_retry` never fired.
Measured:
- 2026-10-13 (v0.1.43.1): 17 of 61 `Bash` calls broken in one 64-turn session
- 2026-10-14 (v0.1.44.1): 35 of 45 `Bash` calls broken in the first 30 min after a restart
**Tool schemas**
- `Bash` (Continue built-in): required string `command`, optional `timeout`
- `retry_code` (MCP): required strings `attempt_id` and `feedback`
**Raw outputs (exact, from the server log)**
1. Name as an XML-style attribute, quoted (most common on 14.10):
~~~text
<tool_call>
<function=Bash>
<parameter name="command">grep -rn "correction_fit\|correction_generate\|job_type\|correction" lotto_graph/services/lstm/correction/runner.py lotto_graph/api/routes/jobs.py 2>/dev/null | grep -v __pycache__ | sed -n '1,40p'
</parameter>
</function>
</tool_call><|im_end|>
~~~
2. Name as an attribute, unquoted:
~~~text
<tool_call>
<function=Bash>
<parameter name=command>
ls lotto_graph/worker.py 2>/dev/null; grep -rln "def.*job\|job_type\|correction" lotto_graph/worker.py 2>/dev/null
</parameter>
</function>
</tool_call><|im_end|>
~~~
3. All arguments as JSON in a nameless `<parameter>` (feedback shortened here):
~~~text
<tool_call>
<function=retry_code>
<parameter>
{"attempt_id": "bf3b0e1cb16b", "feedback": "Checked against the real repository file: `get_model` DOES exist on CorrectionRepository, so leave that call as it is. …"}
</parameter>
</function>
</tool_call>
~~~
4. Not recoverable, no value at all (observed bodies of `<function=Bash>`):
~~~text
<parameter name> command </parameter>
<parameter name> Bash </parameter>
<parameter> </parameter>
(empty body)
<|end|>
~~~
5. Also seen, rarer:
- `<arg_value>{"command": "grep -rn …"}` inside `<function=Bash>`
- `<function=Bash>` with `<parameter=dirpath>` (a parameter of the `List` tool)
- `<parameter=previous_thinking>`, `<parameter=parameter>`
Breakdown: on 13.10, 6 of the 17 broken calls carried the full value in a wrong syntax (forms 1–3) and 11 had none (form 4). On 14.10, 19 of 35 were forms 1–2, 3 were the `<arg_value>` JSON form and 13 had no value.
**Local workaround for forms 1–3 (patch below)**
A small patch on v0.1.44.1 (`serve/frontend.py`), active only with `"tool_call_recovery": true` and declared tools:
- inside `<function=…>…</function>` it rewrites `<parameter name="X">`, `<parameter name='X'>` and `<parameter name=X>` to `<parameter=X>`;
- it expands `<parameter>{json}</parameter>` into one `<parameter=key>` per key (non-string values as JSON);
- a new `ParamForms` class does this before `OutputParser` sees the text, holding a tag split across stream deltas until it is complete;
- text outside a call (reasoning, answer) is untouched.
It adds `serve/test_param_forms.py`: 9 tests built from the samples above, streamed in 1/3/7-character pieces and whole, inside and after the thinking, plus checks that the template form is unchanged, that the patch is off without `tool_call_recovery`, and that text outside a call is untouched. The existing parser tests (`test_tool_name_space`, `test_parser_track`, `test_reasoning_rescue`, `test_reasoning_tools`, `test_responses`) give the same results with and without it.
It does nothing for form 4, where the value is never generated; that probably needs the grammar-constrained emission proposed earlier in this issue. Happy to open a PR if this approach fits.
<details>
<summary>Patch against v0.1.44.1 (<code>git apply</code>)</summary>
~~~diff
diff --git a/serve/frontend.py b/serve/frontend.py
index c36a44d..909f364 100644
--- a/serve/frontend.py
+++ b/serve/frontend.py
@@ -870,6 +870,100 @@ def parse_tool_call(body: str, schema: dict | None = None) -> ToolCall:
RCALL_MAX = 32768 # the most of a `<tool_call>` inside the reasoning that is held while waiting for its end (#804)
+class ParamForms:
+ """LOCAL PATCH (not upstream; see NOTATKI-ZMIANY.md next to the Strata checkout, 2026-10-14): rewrite parameter
+ tags the model writes in forms next to the template's into `<parameter=NAME>`, before OutputParser sees the
+ text. Inside a `<function=...>` ... `</function>` only, and only with "tool_call_recovery" on:
+
+ <parameter name="NAME"> <parameter name='NAME'> <parameter name=NAME> -> <parameter=NAME>
+ <parameter>{"a": 1, "b": "x"}</parameter> -> one <parameter=K> per key
+
+ Seen live (Qwen3.8-Flash-Next Coder IQ1_M, a coding agent with 21 tools): these calls carried the whole command
+ but came out with empty arguments, because parse_tool_call only reads `<parameter=NAME>`. A tag split across
+ deltas is held back until it is complete, like the parser's own tags."""
+
+ NAMED = re.compile(r"""<parameter\s+name\s*=\s*(?:"([^"<>]*)"|'([^'<>]*)'|([^\s"'<>]+))\s*>""")
+ BARE = "<parameter>"
+ BARE_MAX = 65536 # the most of a nameless parameter held while waiting for its </parameter>
+
+ def __init__(self):
+ self.buf = ""
+ self.inside = False # between a <function=...> and its </function>
+ self.rewritten = 0
+
+ def push(self, delta: str) -> str:
+ self.buf += delta
+ return self._drain(final=False)
+
+ def flush(self) -> str:
+ return self._drain(final=True)
+
+ @staticmethod
+ def _value(v) -> str:
+ return v if isinstance(v, str) else json.dumps(v, ensure_ascii=False)
+
+ def _drain(self, final: bool) -> str:
+ out = []
+ while self.buf:
+ if not self.inside:
+ i = self.buf.find(FUNC_START)
+ if i < 0:
+ keep = 0
+ if not final: # the end may be the start of a `<function=`
+ keep = next((n for n in range(min(len(FUNC_START) - 1, len(self.buf)), 0, -1)
+ if FUNC_START.startswith(self.buf[-n:])), 0)
+ out.append(self.buf[:len(self.buf) - keep])
+ self.buf = self.buf[len(self.buf) - keep:]
+ break
+ out.append(self.buf[:i + len(FUNC_START)])
+ self.buf = self.buf[i + len(FUNC_START):]
+ self.inside = True
+ continue
+ i = self.buf.find("<")
+ if i < 0:
+ out.append(self.buf)
+ self.buf = ""
+ break
+ out.append(self.buf[:i])
+ self.buf = self.buf[i:]
+ if self.buf.startswith(FUNC_END):
+ out.append(FUNC_END)
+ self.buf = self.buf[len(FUNC_END):]
+ self.inside = False
+ continue
+ m = self.NAMED.match(self.buf)
+ if m:
+ out.append(PARAM_START + next(g for g in m.groups() if g is not None).strip() + ">")
+ self.buf = self.buf[m.end():]
+ self.rewritten += 1
+ continue
+ if self.buf.startswith(self.BARE):
+ close = self.buf.find(PARAM_END)
+ if close < 0 and not final and len(self.buf) < self.BARE_MAX:
+ break # wait for the whole nameless parameter
+ obj = None
+ if close >= 0:
+ try:
+ obj = json.loads(self.buf[len(self.BARE):close])
+ except ValueError:
+ obj = None
+ if isinstance(obj, dict) and obj:
+ out.append("".join(f"<parameter={k}>\n{self._value(v)}\n</parameter>\n" for k, v in obj.items()))
+ self.buf = self.buf[close + len(PARAM_END):].lstrip("\n")
+ self.rewritten += 1
+ continue
+ out.append(self.BARE)
+ self.buf = self.buf[len(self.BARE):]
+ continue
+ if not final and ">" not in self.buf and (
+ any(t.startswith(self.buf) for t in (FUNC_END, self.BARE, "<parameter "))
+ or self.buf.startswith("<parameter ")):
+ break # an unfinished tag that may still become one of the above
+ out.append("<")
+ self.buf = self.buf[1:]
+ return "".join(out)
+
+
class OutputParser:
"""Incremental parser of the model's text. Feed deltas; get events. A tag split across deltas is held back
until it is complete, so clients never see `<tool_` or `</thi`."""
@@ -898,6 +992,7 @@ class OutputParser:
self.recover = recover
Related on this site
Quick links to related pages.