#6218 [Engine] No character-before-the-tag rule settles </think>: four shapes, three tickets, two opposite fixes

closed · @florian862-art · 3 comments · View on GitHub

Server & APIMulti-GPUNVIDIA / CUDAModels & quantsLinux

Description

<!-- Titel: [Engine] No character-before-the-tag rule settles </think>: four shapes, three tickets, two opposite fixes -->

### Area

[Engine]  (speed, kernels, models, the server and API)

### Desktop app, Terminal or source?

Docker

### What happened

`_quoted_think_end()` in `serve/frontend.py` decides whether a `</think>` ends the thinking or is
the model quoting its own tag, by looking at the character **before** the tag:

```python
before = self.buf[i - 1] if i else self.line[-1:]
if before not in ('"', "'", "`"):
    return False                                  # gate: no quote right before -> real end
after = self.buf[i + len(THINK_END):]
if not after:
    return None
return after[0] not in "\r\n"                     # witness: a real end is followed by a line break
```

I am not reporting a fifth variant of that decision. I am reporting that **the character before the
tag does not carry the information the gate is asked to read**, and that this is now measurable:
three tickets ask this one function for four things, and the two candidate fixes on the table each
satisfy exactly one of them and break the other.

The four shapes, all of them real, all of them about the *same* function:

| shape | where it comes from | what it needs |
|---|---|---|
| `it returns "</think>\n\n2", no call` | #1526, closed, pinned in `serve/test_reasoning_tools.py` | **quoted** |
| `plan</think>answer` | #4077, open | **real end** |
| `The user said "only."</think>2` | #5605, open | **real end** |
| ``the marker `\n</think>\n\n` appears once`` | this report | **quoted** |

Measured on `main` (`c77572806`), each shape fed whole and in 7- and 1-character deltas:

| rule in the gate | #1526 | #4077 | #5605 | ours | |
|---|---|---|---|---|---|
| `main` as it stands | quoted ✓ | real ✓ | quoted ✗ | real ✗ | 6/12 |
| #5605's suggested rule (quote must *open* a quotation) | quoted ✓ | real ✓ | real ✓ | real ✗ | 9/12 |
| look 4 characters back for the quote | quoted ✓ | real ✓ | quoted ✗ | quoted ✓ | 9/12 |

`main` gets 6 of 12, each proposed fix 9 of 12 — and the two fixes are **opposite**: #5605 tightens
the gate, ours widens it. #5605's rule makes our case strictly worse (verified, see the repro), and
our widening makes #5605's case strictly worse. Neither is mergeable as a fix for both.

Why the character before the tag cannot settle it: it is the *last character of the reasoning*, and
the reasoning is free text. A model that ends a sentence on a closing quote and a model that names
its own control token produce the same character there. The template's actual contract is entirely
on the other side of the tag — `chat_template.jinja` renders
`reasoning_content + '</think>' + '\n\n' + content`, so a real end is always followed by a line
break — and `after` already implements that. The `before` gate is a heuristic bolted onto a rule
that does not need it, and every ticket in this family is that bolt failing in a different direction.

I am not proposing a rule, because I do not have one that reaches 12/12. What I can add below is
end-to-end evidence from a live model, which the three other tickets do not have.

### Smallest steps or command that reproduce it

Parser level — no model, no GPU, no server. Save as `vier_formen.py` in the repo root and run it
against three variants of `serve/frontend.py` (unpatched, #5605's suggestion, the 4-character
lookback):

```python
import sys; sys.path.insert(0, ".")
from serve.frontend import OutputParser

T = "</think>"
CASES = [
    ("#1526 quoted tag stays reasoning",
     'it returns "' + T + '\\n\\n2", no call\n' + T + "\n\nanswer",
     "answer", 'it returns "' + T + '\\n\\n2", no call\n'),
    ("#4077 bare tag ends the thinking",
     "plan" + T + "answer", "answer", "plan"),
    ("#5605 sentence closes on a quote",
     'The user said "only."' + T + "2", "2", 'The user said "only."'),
    ("OURS  escaped mention",
     "the marker `\\n" + T + "\\n\\n` appears once\n" + T + "\n\nanswer",
     "answer", "the marker `\\n" + T + "\\n\\n` appears once\n"),
]

def split(text, width):
    p = OutputParser(thinking=True)
    ev = []
    for i in range(0, len(text), width):
        ev += p.feed(text[i:i + width])
    ev += p.finish()
    j = lambda k: "".join(e.text for e in ev if e.kind == k)
    return j("content"), j("reasoning")

for name, text, want_c, want_r in CASES:
    marks = []
    for w in (100_000, 7, 1):
        c, r = split(text, w)
        marks.append("ok" if (c, r) == (want_c, want_r) else "FEHLER")
    print(f"{name:36} {'  '.join(marks)}")
```

On `main` (`c77572806`):

```text
#1526 quoted tag stays reasoning     ok  ok  ok
#4077 bare tag ends the thinking     ok  ok  ok
#5605 sentence closes on a quote     FEHLER  FEHLER  FEHLER
OURS  escaped mention                FEHLER  FEHLER  FEHLER
```

Our case on its own, to see what the client gets:

```python
import sys; sys.path.insert(0, ".")
from serve.frontend import OutputParser

text = "the marker `\\n" + "</think>" + "\\n\\n` appears once\n" + "</think>" + "\n\nanswer"
p = OutputParser(thinking=True)
ev = p.feed(text) + p.finish()
j = lambda k: "".join(e.text for e in ev if e.kind == k)
print("reasoning=", repr(j("reasoning")))
print("content  =", repr(j("content")))
```

```text
reasoning= 'the marker `\n'
content  = '\n\n` appears once\nanswer'
```

The turn contains **both** tags: the mentioned one first, the real end second. The thinking is cut
after `` `\n `` — three characters of reasoning — and everything from there is delivered as the
answer. The real end *is* recognised correctly; the bug is only that the first tag is already taken
for it. (The mentioned tag is what `#537`'s gate exists for; it only fires when the quote sits
directly before the tag, and here the model typed `\n` between them.)

### Reproduction status

Reproduced locally, deterministic at parser level, and seen end to end with a live model (logs below).

### Strata version

engine 0.1.60.1 (served from `v0.2.14.1` + local patches). All three rules measured against
`serve/frontend.py` from `main` (`c77572806`).

### Graphics card, memory and system

2x RTX 5070 Ti 16 GB · 91 GB RAM · Linux, served in Docker with the source mounted

### Model, size and settings

qwen3.8-flash-next nvfp4 · 262144 context · thinking on · `/v1/messages` (Anthropic form) and
`/v1/chat/completions` from agentic clients

### Logs

`STRATA_DEBUG=1` dumps the whole generated text per turn. This is the live turn, verbatim — the
parser's decision point, then what the client received:

```text
--- reasoning, last 90 characters ---
'the raw shows `</tool_call><|im_end|>` at the end, and marker count 1. Hmm, the marker `\n'
--- the tag the parser took for the end of the thinking ---
'</think>'
--- "answer" after it, complete ---
'\n\n` appears once. But where? The raw starts with reasoning text... Actually the raw dump shows
the *whole* generation including tool calls in `<|im_end|>'
```

A healthy turn for comparison, same session:

```text
thinking until the marker : 484 characters
answer after the marker   : 360 characters
ends with                 : '\n</tool_call><|im_end|>'
```

The engine log of a broken turn runs `thinking:` straight into `done:` with no `answering:` block —
from the parser's point of view there was never an answer — and reports `stop, cancel=False`. So the
turn is not aborted, not truncated by `max_tokens`, and not a client problem: the client receives a
well-formed reply whose "answer" is the second half of a chain of thought, then goes idle.

### Two things this report is NOT about

**1. It is not the second half of that turn.** The same raw dump shows the model emitting a real
`<|im_end|>` *mid-answer* while writing about control tokens (`… including tool calls in ` + EOS).
That is the model ending its own turn; no parser can undo it. This report is only about the cut the
parser makes.

**2. It is not specific to this repository.** The sibling project Basalt (`jesdga95/basalt`) carries
the same function with the same gate, and the same four shapes behave the same there. One difference
worth flagging for anyone keeping the two parsers in step: Basalt's `_reasoning()` carries only the
last character across feed boundaries (`self.rtail = text[-1:] or self.rtail`), so there a rule that
looks further back than one character needs a rolling window
(`self.rtail = (self.rtail + text)[-4:] if text else self.rtail`) or it silently does nothing for
one-character deltas.

### What would actually settle this

Everything the tickets need is on the **other side** of the tag: a real end is followed by the
answer, a mention is followed by more reasoning. `after[0] not in "\r\n"` is the first and weakest
form of that (`\n\n` per the template), and it is the only part of the current rule that never
disappoints. A rule that reads the *after* side more thoroughly — more than one character, whether
what follows reads like an answer or like continued reasoning — could reach all four shapes; a rule
that keeps reading the character before the tag cannot, because the four tickets above are four
different answers to a question that character does not answer.

I am reporting this rather than only carrying a local patch, and I am deliberately not opening a PR:
the two obvious patches cancel each other out, and choosing between them is exactly the call the
maintainers can make and I cannot. Related: #1526 (closed, the gate's origin), #4077 and #5605 (open,
opposite directions).

Related on this site

Quick links to related pages.