Issues / #1178

#1178 serve: Special tokens (<|im_end|>) emitted during model thinking cause tasks to terminate early

open · @TwoAnts · 5 comentários · No GitHub

Server & APIModels & quants

Descrição

Recently, I’ve been using a model (Qwen3.8 Flash Next IQ3_S) to fix Strata’s code related to special tokens. The model tends to output special tokens in its thinking, which causes the task to terminate immediately.

Strata may need to add some fallback handling to mitigate this issue.

Minimal test input:

```
Qwen 聊天模板里,助手回合的结束标记是什么?请原样写出,例如 "标记"。
```

Please check whether it terminates early and whether it can still produce output normally.

I can reproduce this locally multiple times: it either terminates during thinking, or fails to output the answer normally.

No site

Links install, modelos, releases.