#6636 [Engine] strata-inkling 0.1.65 (gfx1101): non-finite logits for every prompt of 64+ tokens in serve mode
open · @giorgiorossi · 1 评论 · 去 GitHub 看
BenchmarksSetup & installServer & APIAMD / HIPNVIDIA / CUDAModels & quantsWindowsLinux
说明
**Problem:** With the Inkling-2-Mini SQ-15 `.strata` pack and the Windows `rdna3` shape engine on an RX 7800 XT (`gfx1101`), `strata-inkling --serve` returns `DONE 0 <n> ... nonfinite` for **every prompt of 64 tokens or more**, before the first token. 63 tokens works. Through `serve/server.py` the request ends with the `#879` error ("the model's output became non-finite (NaN) twice in a row"). Short prompts (17 tokens) answer normally.
The engine's own `--bench` path is fine at 2,048 prompt tokens and `--selfcheck` passes, so this is the serve/prefill path, not the weights.
**Environment:**
- Windows 11 Pro 10.0.26200 (build 26200), AMD Ryzen 9 7900, 32 GB RAM
- AMD Radeon RX 7800 XT 16 GB, `gfx1101`, driver/runtime 71726391 as reported by `strata-device --selftest`
- Strata 0.2.19 (git `c6f6427d9`), Control 0.1.65
- Shape engine `strata-inkling` 0.1.65, package `strata-inkling-0.1.65-windows-x86_64-hip-rdna3` (`BUILD.json`: archs `gfx1100;gfx1101;gfx1102;gfx1103;gfx1150`, ROCm 10.2.0a20260930, portable)
- Model: `Inkling-2-Mini-SQ-15.strata` (14,348,316,672 bytes, SHA-256 `2ea32eed533afba257e2cbee7b073dc2c3da80fe55d1d2d794779b5498e30737`), plus `Inkling-2-Mini-Vision.strata` (0.59 GB)
- Run config from setup: `--max-context 32768 --vision ... --vram-experts-gb 12.28 --ram-experts-gb 0.00`, `drafting: dspark`
**Reproduce (no server needed):**
```
strata-inkling.exe --serve --native <models>\inkling-2-mini\Inkling-2-Mini-SQ-15.strata \
--max-context 32768 --vram-experts-gb 12.28 --ram-experts-gb 0.00
```
Wait for `READY 32768 stop hold holdb`, then send a `GEN` line with comma-separated token ids (any ids in range):
```
GEN 8 <63 ids> -> DONE 8 63 425.7 371.5 length
GEN 8 <64 ids> -> DONE 0 64 93.5 0.0 nonfinite
GEN 8 <65 ids> -> DONE 0 65 100.0 0.0 nonfinite
```
63 passes; 64, 65, 70, 128, 200 and 300 all fail. Through `serve/server.py`, a chat request with ~250 tokens of prose (1,000 characters) also fails, while a 17-token prompt works.
**Expected:** the prompt is read and a reply is produced, as the 0.2.19 notes describe (1M context, 64 slots).
**Actual:** `DONE 0 <n> <ms> 0.0 nonfinite`; through the server, `HTTP` request ends with: `the model's output became non-finite (NaN) twice in a row (#879); the engine has dropped its cached state`.
**What I ruled out, all measured on this machine (2026-11-22):**
- Context: `--max-context` 32768, 65536 and 96120 — same 64-token threshold.
- VRAM/RAM split: 12.28/0.00 and 10.92/1.36 (the value `inkling_fit.place()` computes for 96K) — same.
- `--kv fp8`, `--parallel 2` — same.
- Drafter and vision: config with `drafting: off` and no `--vision` — same.
- `STRATA_HIPBLASLT=0` — same.
- Tokenizer: ids are valid (312 ids for 1,000 characters, max 198,531 of vocab 200,058; no ids >= 200,000). Random ids in the `GEN` line show the same threshold, so the tokenizer/template is not the trigger.
- The engine's other paths: `--bench --prompt-len 2048 --gen 16` → `prefill=1147.8 ppts`, `decode=51.1 tps`, no error; `--selfcheck --native ...` → `PASS backend=hip (AMD Radeon RX 7800 XT (gfx1101, 30 CUs, UMA=0, eff=15.98 GiB, chunk<=1024, blasLt=0, hybrid=0))`.
- The same card with the Flash-Next engine 0.1.65 handles long prompts (16,066-token prompt: 772 tok/s prompt, 45 tok/s decode), so the GPU/driver is not generally producing NaNs.
**Limits / not tested:**
- Not tested on gfx1100, gfx1102, gfx1151, gfx1201 or any CUDA/Linux build; not tested with SQ-10, SQ-20, official safetensors or Unsloth GGUF (they use the same serve path, so I did not spend the download).
- No source-level inspection of the engine: it is a separate component and its source is not in this checkout. The 64-token boundary is the only structural clue I have.
- The pack's SHA-256 verified and `--bench` uses the same weights, so the weights are unlikely to be the cause; that is an inference, not a proof.
**Related:** #6580 (0.1.65 HIP gfx1200, Inkling GGUF: illegal memory access in long prefill, short prompts fine) and #6442 (Inkling model support). This report is a different symptom and architecture (non-finite logits, `.strata` pack, gfx1101, exact 64-token boundary), so I opened it separately; feel free to fold it into #6580 if it is the same prefill defect.
**Reproduction status:** reproduced locally, 10+ times, including the minimal `GEN` sequence above.
本站相关内容
相关页面的快捷入口。