#2449 0.1.44/HIP: verify-window watchdog still fires on long-context decode despite the 0.1.31 bounded-wait fix (residual #267)

Descripción

Follow-up to #267 (commented there too, filing separately since backend/OS/symptom differ). On 0.1.44 the verify-window watchdog still fires, with the post-fix message showing the 0.1.31 release path executed but wasn't enough.

## Environment

| | |
|---|---|
| Strata | 0.1.44 (HIP source build, gfx1100+gfx1100) |
| Model | Qwen3.8-Flash-Next, UD-Q4_K_XL native pack |
| Engine args | `--max-context 200000 --kv int8 --kv-resident 32768 --spec 4 --spec-min-p 0.5 --mtp --expert-cache auto --prefill auto --vision --vram-reserve-mib 2048 --pool-workers 13 --conversation-cache-mib 2048 --conversation-cache-slots 8 --conversation-cache-min-tokens 2000 --layer-split auto` |
| GPUs | 2x AMD RX 7900 XTX (Navi 31, gfx1100), layer split 24/24, ROCm/HIP |
| CPU | Core Ultra 7 265K |
| RAM | 96 GB DDR5 |
| OS | Linux, kernel 7.0.0-38-generic |
| Launch | `reasoning high`, vision on CPU |

## What happened

Mid-decode on a long-context request, decode rate decayed progressively, then the watchdog fired:

```
[strata] thinking: 482 of max 32768 tokens, 28.7 tok/s, 17 s
[strata] thinking: 516 of max 32768 tokens, 18.4 tok/s, 28 s
[strata] thinking: 516 of max 32768 tokens, 13.6 tok/s, 38 s
[strata] the engine stopped unexpectedly (exit code None). Unrecoverable native verification failure: verify: timed out at layer 29; its GPU waits were released but the GPU did not finish within 5 s (#267) The next request starts the engine again.
[strata] done: 516 tokens in 48 s (10.8 tok/s) (error, cancel=False)
```

## Engine log: the dying request

The request that died was a ~95k-token conversation doing heavy KV streaming (this looks more like #251's long-context trigger than #267's short prompt):

```
strata serve: prompt 95487 tokens = 95387 reused + 100 read in 2198 ms (45.5 tok/s), 1841 generated in 44006 ms (41.8 tok/s), drafts accepted 868 of 1471, 6 checkpoints
strata serve: decode expert cache hit rate: 94.5% (1107035 hits / 1171357 lookups); 2723 more read by the GPU over PCIe or from another GPU (0.2% of all 1174080 routed)
strata serve: KV streaming: 98.03% of 46963620 block reads hit VRAM, 3730.4 MiB read from RAM
strata serve: suffix drafts: 18 windows, 27 of 52 drafts accepted
strata serve: prompt 97349 tokens = 97329 reused + 20 read in 356 ms (56.2 tok/s), 107 generated in 2015 ms (53.1 tok/s), drafts accepted 78 of 85, 6 checkpoints
strata serve: decode expert cache hit rate: 87.3% (47655 hits / 54560 lookups); 640 more read by the GPU over PCIe or from another GPU (1.2% of all 55200 routed)
strata serve: KV streaming: 98.03% of 47376474 block reads hit VRAM, 3762.1 MiB read from RAM
strata serve: suffix drafts: 11 windows, 36 of 39 drafts accepted
```

(The next engine-log line is the auto-restart on the following request — no engine-side stall report or dump was emitted before death.)

Prior requests in the same session were healthy (44–53 tok/s decode, 87–94% expert hit rates).

## How this differs from #267

- **The 0.1.31 fix ran and didn't save it.** The message says GPU waits *were released* but the GPU still didn't finish within 5 s — so this isn't the unreleased-flag case, it's a genuinely stuck GPU-side wait.
- **No device loss.** Both cards stayed healthy; `server.py` restarted the engine on the next request and serving resumed. No reboot.
- **Silent at kernel level.** No amdgpu/GTT/OOM lines in dmesg or the journal around the stall.
- **Not the low-VRAM case.** 2048 MiB reserve per card; per-request stats show no VRAM distress (98% KV-streaming VRAM hit, PCIe expert share ~1%). I've since raised `--vram-reserve-mib` to 3072 as the documented mitigation — will report back if it recurs.
- **Long context, not short.** ~95k-token conversation vs #267's one-sentence prompt; closer to #251.

## Notes

- Related: #267 (closed, 0.1.31), #251 (long-prompt verify stall), #29 (stall → engine restart).
- Box also runs other GPU/CPU load (long-lived ComfyUI, desktop), so system memory pressure can't be fully ruled out as a contributor — but nothing spilled loudly.
- Full engine log around the stall and the server log are on hand; say what to attach or capture next time it fires (e.g. `/metrics` snapshot, thread stacks if the engine can emit them on this path).