Pull requests / #1839
#1839 fix(serve): preserve literal closing thinking tags
closed · draft · @CC-David-CC · 0 commentaires · Sur GitHub
Server & APINVIDIA / CUDAWindowsLinux
Description
Follow-up to #537 and related to #1814. When ordinary text tokens spell `</think>` during reasoning, the parser must keep those bytes as reasoning text; only the tokenizer's dedicated end-of-thinking token should close the block. This change makes parser transitions depend on token identity. The bounded `literal_think_guard` handles dedicated `<think>` / `</think>` tokens in answer text or inside reasoning code spans by rendering the tag with ordinary text-token IDs and resuming from the preserved prefix. If end-of-turn arrives inside an unclosed reasoning code span before any answer or tool call, it similarly inserts literal `</think>` text and resumes. The guard is enabled by default to match #1814 and can be disabled with `"literal_think_guard": false`. It is capped at 64 repairs per reply, respects client stops, and accounts for inserted tokens against the output budget. ### Evidence - Controlled ordinary-text tag case: the original parser misrouted the tag in 12/12 requests; token-identity parsing handled all 12 with the same generated token IDs. - A live RTX PRO 6000 run naturally emitted dedicated closing-tag IDs inside reasoning code. The guard replaced them and continued from the verified prefix, but those replies reached their output cap while still reasoning; this verifies repair mechanics, not complete answers. - A Linux stress test with 4,106- and 8,370-token parser/tool histories produced one baseline stop after an opening backtick in 8 requests. No dedicated closing-tag ID appeared in any of 16 baseline/guard requests. Its synthetic history included the expected symptom, so it is not an independent reproduction of the reporter's agent session. - We could not test the reported two RTX 3090 GPUs, and the exact dedicated-token-then-stop sequence remains unobserved. ### Validation - Focused parser/service/HTTP tests: 30 passed on Windows and Linux. - Full Windows serving suite: 623 passed, 10 skipped (438.513s). - Full Linux serving suite from a native WSL filesystem with `regex` and `jsonschema` available: 623 passed, 12 skipped (339.043s). Related but separate work: [#1818](https://github.com/Niko1221/Strata/pull/1818) covers CPU IQ expert dispatch and draft-grouping independence; [#1813](https://github.com/Niko1221/Strata/pull/1813) covers speculative/TCP parity; [#1773](https://github.com/Niko1221/Strata/pull/1773) covers Q8 TCP parity/timings.
Sur le site
Liens install, modèles, releases.