#2452 0.1.44/HIP: deterministic non-finite logits (#879) at a fixed prompt position on long-context request; retry-from-0 reproduces

本文

On 0.1.44 the engine emits non-finite logits at a fixed prompt position, deterministically: the server's automatic retry-from-token-0 reproduces it at the identical position. No private prompt content is attached; the signature alone should be actionable.

## Environment

| | |
|---|---|
| Strata | 0.1.44 (HIP source build, gfx1100+gfx1100) |
| Model | Qwen3.8-Flash-Next, UD-Q4_K_XL native pack |
| Engine args | `--max-context 200000 --kv int8 --kv-resident 32768 --spec 4 --spec-min-p 0.5 --mtp --expert-cache auto --prefill auto --vision --vram-reserve-mib 2048 --pool-workers 13 --conversation-cache-mib 2048 --conversation-cache-slots 8 --conversation-cache-min-tokens 2000 --layer-split auto` |
| GPUs | 2x AMD RX 7900 XTX (gfx1100), layer split 24/24, ROCm/HIP |
| OS | Linux, kernel 7.0.0-38-generic |

## What happened

Server log:

```
[strata] reading the prompt: 102,369 of 102,374 tokens, 8 s so far
[strata] the engine's logits became non-finite before the first token (#879); it dropped its cached state: sending the request again once
[strata] reading the prompt: 8,192 of 102,374 tokens, 18 s so far
... (full re-read) ...
[strata] reading the prompt: 102,251 of 102,374 tokens, 83 s so far
[strata] the engine's logits became non-finite again on the retry (#879): the request ends with an error
```

Engine log (both attempts, identical position):

```
strata serve: non-finite logits (#879) in a prompt window at position 102369 (row 0): its tokens are not emitted, the request ends (DONE nonfinite) and the next request reads its prompt from token 0
strata serve: prompt 102374 tokens = 98673 reused + 3696 of 3701 read in 8966 ms (412.2 tok/s), 0 generated in 0 ms (0.0 tok/s), drafts accepted 0 of 0, 6 checkpoints (cancelled)
...
strata serve: after non-finite logits (#879): dropped the live session, 6 checkpoints, 0 parked conversations and 0 idle slots (0 running slots not kept after their reply); reading this prompt from token 0
...
strata serve: non-finite logits (#879) in a prompt window at position 102369 (row 0): ...
strata serve: prompt 102374 tokens = 0 reused + 102369 of 102374 read in 75865 ms (1349.4 tok/s), 0 generated in 0 ms (0.0 tok/s), drafts accepted 0 of 0, 6 checkpoints (cancelled)
```

## Key properties

- **Deterministic in content, not transient.** First attempt (98,673 tokens reused from cache) and retry (full cold re-read from 0) fail at the identical position 102369 of 102374 — 5 tokens from the prompt tail. Resending is futile.
- **Healthy session otherwise.** Prior requests in the same session decoded at 44–53 tok/s with 87–94% expert-cache hit rates. Engine stayed loaded throughout; no restart needed.
- **Expensive retry.** The automatic retry-from-0 burned a ~76 s full prefill of a 102k prompt to reproduce the same failure. For a deterministic #879, that retry is pure waste — consider skipping the re-read when the first failure is at (near-)the same tail position, or capping auto-retry to transient-looking cases.

## Notes

- The triggering bytes are in the user's private session content, so no prompt is attached. The tail region (last few tokens: newest tool result / message + chat-template closing) is where to look; typical suspects are odd byte sequences from tool output, but that's unverified.
- Related: #2449 (verify-window watchdog on this same box), #267, #251.
- Happy to run a bisection or dump token IDs around the failing position if you describe a safe way to extract them (the raw session can't leave the machine).