Issues / #3465
#3465 [Engine] Deterministic all-logits NaN (#879) on a fresh ~1.7K-token prompt of percent-encoded URLs (IQ3_S, CUDA, Windows)
open · @angellfear · 1 comentarios · En GitHub
BenchmarksServer & APINVIDIA / CUDAModels & quantsWindows
Descripción
### Summary
A fresh request (no cache reuse, no parked conversations) whose prompt contains a Wikimedia API listing with long
percent-encoded Cyrillic URLs (`%D0%9F%D0%B0%D0%BD...`) ends with `non-finite logits (#879) in a prompt window`
every time, on the retry too (503). Same position on every run and across engine restarts. Half of the same text
passes. The repro `min-wiki.json` (5 KB, ~1,717 tokens, public Wikimedia data) is below, sent as-is to `/v1/chat/completions`
(`stream: false`, `max_tokens: 8`) - fails in ~11 s.
In real use it hits coding agents as soon as a tool result contains such URLs: every later turn of that
conversation fails, including the compaction request (positions 29420 / 30706 in my logs).
### Environment
- Engine: release **0.1.46** (`strata-engine-0.1.46-windows-x86_64-cuda-cuda13`, from v0.2.1); also reproduced on a local
build 0.1.45.3 (`f-020-int`, CUDA 13.0, `-DSTRATA_Q6K_EXPERTS=ON -DSTRATA_MMQ_KQUANTS=ON`)
- Models: Qwen3.8-Flash-Next GSQ-RCO **IQ3_S**; the same symptom earlier on Swift 1.5 **IQ2_XS**
- GPU: RTX 4070 SUPER 12 GB, driver 610.88, PCIe 3.0 x16 (passthrough)
- CPU: Xeon E5-2697 v4 (Broadwell, AVX2, no AVX-512/VNNI), 16 vCPUs; RAM 96 GB
- Windows 10 Pro 19045 in a Proxmox (QEMU q35) VM, the GPU passed through
- Args: `--expert-cache auto --prefill auto --spec 4 --spec-min-p 0.5 --mtp <rt> --max-context 262144 --kv int8
--kv-resident 32768 --vram-reserve-mib 1600 --pool-workers 8 --vision`; ~1.2 GiB VRAM free at READY, no LOW warnings
### What changes the outcome (one engine restart per row)
| Setting | `min-wiki.json` (1.7K) | other short prompts (1.7K / 5.4K, plain text) | the 11K / 30K agent prompts containing the same text |
| --- | --- | --- | --- |
| defaults | NaN | NaN | NaN |
| `STRATA_PREFILL_CPU_SHARE=0` | NaN | NaN | NaN |
| + `STRATA_ROUTE_TAIL_SKIP=0` | NaN | **pass** | NaN |
| + `--pcie-frac 1` | **pass** | pass | NaN (decode 33 -> 17.6 tok/s) |
So the CPU pool's share of the missed experts looks involved for the short prompt, but not for the long ones.
The NaN is always reported in the prompt's last window (`N-5 of N` read, then the tail window); a 1,949-token
prompt read in one pass (no tail window) passed.
Also seen during `--calibrate`: NaN in a decode window at position 105 on the 0.1.45.3 build, and on 0.1.46 at the
expert-tier candidate `--adapt-every 1 --adapt-swaps 80 --adapt-decay 0.97` (after ~10 clean measurements).
Possibly related but a different trigger: #3036 (parked-conversation copies at 262144). Here the conversation cache
is off (`budget_mib 0`) and every failing request starts at token 0.
### Log lines
```
strata serve: non-finite logits (#879) in a prompt window at position 11091 (row 0): its tokens are not emitted, the request ends (DONE nonfinite) and the next request reads its prompt from token 0
strata serve: prompt 11096 tokens = 0 reused + 11091 of 11096 read in 16624 ms (667.2 tok/s), 0 generated in 0 ms (0.0 tok/s)
[strata] the engine's logits became non-finite again on the retry (#879): the request ends with an error
```
Happy to run a debug build or env switches (e.g. a per-layer NaN trace) on this machine.
<details><summary>min-wiki.json</summary>
```json
{
"model": "x",
"messages": [
{
"role": "user",
"content": "What is this?\n\n{\"batchcomplete\":\"\"}\nFile:- Snow covered cobblestone pavement -.jpg | https://thumb.wikimedia.org/wikipedia/commons/thumb/6/64/-_Snow_covered_cobblestone_pavement_-.jpg/960px--_Snow_covered_cobblestone_pavement_-.jpg?utm_source=commons.wikimedia.org&utm_campaign=imageinfo&utm_content=thumbnail\r\nFile:02.Trinidad (59).JPG | https://thumb.wikimedia.org/wikipedia/commons/thumb/e/ed/02.Trinidad_%2859%29.JPG/960px-02.Trinidad_%2859%29.JPG?utm_source=commons.wikimedia.org&utm_campaign=imageinfo&utm_content=thumbnail\r\nFile:Ancient road surface.jpg | https://thumb.wikimedia.org/wikipedia/commons/thumb/b/be/Ancient_road_surface.jpg/960px-Ancient_road_surface.jpg?utm_source=commons.wikimedia.org&utm_campaign=imageinfo&utm_content=thumbnail\r\nFile:Cobblestone Pavement in Cres, Croatia.jpg | https://thumb.wikimedia.org/wikipedia/commons/thumb/1/10/Cobblestone_Pavement_in_Cres%2C_Croatia.jpg/960px-Cobblestone_Pavement_in_Cres%2C_Croatia.jpg?utm_source=commons.wikimedia.org&utm_campaign=imageinfo&utm_content=thumbnail\r\nFile:Cobblestone pavement of the Alexandru D. Xenopol Street.jpg | https://thumb.wikimedia.org/wikipedia/commons/thumb/1/13/Cobblestone_pavement_of_the_Alexandru_D._Xenopol_Street.jpg/960px-Cobblestone_pavement_of_the_Alexandru_D._Xenopol_Street.jpg?utm_source=commons.wikimedia.org&utm_campaign=imageinfo&utm_content=thumbnail\r\nFile:Cobblestone pavement of the Alexandru D. Xenopol Street 1.jpg | https://thumb.wikimedia.org/wikipedia/commons/thumb/d/df/Cobblestone_pavement_of_the_Alexandru_D._Xenopol_Street_1.jpg/960px-Cobblestone_pavement_of_the_Alexandru_D._Xenopol_Street_1.jpg?utm_source=commons.wikimedia.org&utm_campaign=imageinfo&utm_content=thumbnail\r\nFile:Cobblestones street Pompeii.jpg | https://thumb.wikimedia.org/wikipedia/commons/thumb/a/aa/Cobblestones_street_Pompeii.jpg/960px-Cobblestones_street_Pompeii.jpg?utm_source=commons.wikimedia.org&utm_campaign=imageinfo&utm_content=thumbnail\r\nFile:Experimental video - walkthrough the cobblestone passageway in the stand of Regattastrecke Oberschleißheim near Munich, Germany.webm | https://thumb.wikimedia.org/wikipedia/commons/thumb/a/ac/Experimental_video_-_walkthrough_the_cobblestone_passageway_in_the_stand_of_Regattastrecke_Oberschlei%C3%9Fheim_near_Munich%2C_Germany.webm/500px--Experimental_video_-_walkthrough_the_cobblestone_passageway_in_the_stand_of_Regattastrecke_Oberschlei%C3%9Fheim_near_Munich%2C_Germany.webm.jpg?utm_source=commons.wikimedia.org&utm_campaign=imageinfo\r\nFile:Krishna Pushkarani - Hampi Ruins.jpg | https://thumb.wikimedia.org/wikipedia/commons/thumb/8/8d/Krishna_Pushkarani_-_Hampi_Ruins.jpg/960px-Krishna_Pushkarani_-_Hampi_Ruins.jpg?utm_source=commons.wikimedia.org&utm_campaign=imageinfo&utm_content=thumbnail\r\nFile:Münster, Fürstbischöfliches Schloss -- 2018 -- 1930-31 (cropped).jpg | https://thumb.wikimedia.org/wikipedia/commons/thumb/d/d8/M%C3%BCnster%2C_F%C3%BCrstbisch%C3%B6fliches_Schloss_--_2018_--_1930-31_%28cropped%29.jpg/960px-M%C3%BCnster%2C_F%C3%BCrstbisch%C3%B6fliches_Schloss_--_2018_--_1930-31_%28cropped%29.jpg?utm_source=commons.wikimedia.org&utm_campaign=imageinfo&utm_content=thumbnail\r\nFile:Münster, Schloss -- 2014 -- 6769-71 (crop).jpg | https://thumb.wikimedia.org/wikipedia/commons/thumb/1/1a/M%C3%BCnster%2C_Schloss_--_2014_--_6769-71_%28crop%29.jpg/960px-M%C3%BCnster%2C_Schloss_--_2014_--_6769-71_%28crop%29.jpg?utm_source=commons.wikimedia.org&utm_campaign=imageinfo&utm_content=thumbnail\r\nFile:PSM V56 D0544 Cobblestone pavement on saratoga ave in brooklyn.png | https://thumb.wikimedia.org/wikipedia/commons/thumb/5/5f/PSM_V56_D0544_Cobblestone_pavement_on_saratoga_ave_in_brooklyn.png/960px-PSM_V56_D0544_Cobblestone_pavement_on_saratoga_ave_in_brooklyn.png?utm_source=commons.wikimedia.org&utm_campaign=imageinfo&utm_content=thumbnail\r\nFile:Pavement made of cobblestone of quartz porphyr.jpg | https://thumb.wikimedia.org/wikipedia/commons/thumb/b/bf/Pavement_made_of_cobblestone_of_quartz_porphyr.jpg/960px-Pavement_made_of_cobblestone_of_quartz_porphyr.jpg?utm_source=commons.wikimedia.org&utm_campaign=imageinfo&utm_content=thumbnail\r\nFile:Sett block pavement of Lipscani Street.jpg | https://thumb.wikimedia.org/wikipedia/commons/thumb/f/f5/Sett_block_pavement_of_Lipscani_Street.jpg/960px-Sett_block_pavement_of_Lipscani_Street.jpg?utm_source=commons.wikimedia.org&utm_campaign=imageinfo&utm_content=thumbnail\r\nFile:Пагорби над Дністром. Панорама.jpg | https://thumb.wikimedia.org/wikipedia/commons/thumb/4/4a/%D0%9F%D0%B0%D0%B3%D0%BE%D1%80%D0%B1%D0%B8_%D0%BD%D0%B0%D0%B4_%D0%94%D0%BD%D1%96%D1%81%D1%82%D1%80%D0%BE%D0%BC._%D0%9F%D0%B0%D0%BD%D0%BE%D1%80%D0%B0%D0%BC%D0%B0.jpg/960px-%D0%9F%D0%B0%D0%B3%D0%BE%D1%80%D0%B1%D0%B8_%D0%BD%D0%B0%D0%B4_%D0%94%D0%BD%D1%96%D1%81%D1%82%D1%80%D0%BE%D0%BC._%D0%9F%D0%B0%D0%BD%D0%BE%D1%80%D0%B0%D0%BC%D0%B0.jpg?utm_source=commons.wikimedia.org&utm_campaign=imageinfo&utm_content=thumbnail"
}
]
}
```
</details>
En el sitio
Enlaces a install, modelos, releases.