Issues / #3177
#3177 v0.1.45 HIP/gfx1100: verify timeout during ~54K-token request with MTP enabled (single RX 7900 XTX)
open · @peterhon168 · 0 Kommentare · Auf GitHub
Beschreibung
## Summary On a single Radeon RX 7900 XTX, Strata v0.1.45 (Linux HIP) serves a short MTP request successfully, but a long agent request of about 54,316 input tokens stalls shortly after decode starts. The request eventually returns HTTP 503 and the engine exits with code -15. The backend reports a native verification timeout at layer 18. This appears related to the long-context HIP verify watchdog reported in #2449, though this case uses v0.1.45 and only one GPU. ## Environment - GPU: one AMD Radeon RX 7900 XTX (gfx1100) - Strata: v0.1.45, HIP backend - ROCm: 7.2.4 - Kernel: 6.8.0-139-generic - Model: Qwen3.8-Flash-Next GSQ-RCO IQ3_S - Context limit: 262144; int8 KV cache - Expert resident cache: 36 GiB - MTP: enabled with `--spec 2` - Prefill: `--prefill auto:32768` - `HSA_USERPTR_FOR_PAGED_MEM=0` was already set (the workaround discussed in #1705). ## Observed behavior 1. Start the server with the above configuration and MTP enabled. 2. A short test request succeeds (HTTP 200; 14 draft tokens offered, 10 accepted). 3. Send a long, multi-turn agent request with approximately 54,316 input tokens. Prefill completes, and about 444 output tokens are generated before progress stalls. 4. After roughly 126.8 seconds, the request fails with HTTP 503. The engine reports: ```text Unrecoverable native verification failure: verify: timed out at layer 18; its GPU waits were released and the GPU finished (#267) ``` The engine is then observed to stop with exit code -15. The client sees the engine-stopped error on its next request. ## Expected behavior The request should finish or fail with a recoverable error without terminating the inference engine. A watchdog should not misclassify a completed GPU operation as an unrecoverable verify failure. ## Control / limitations With MTP disabled, short requests pass, and one earlier long Pi agent call completed. This was not a strict A/B reproduction with identical prompt and load, so I cannot yet conclude that MTP is the sole cause. The short MTP success shows that enabling MTP itself does not immediately fail. No private prompts, local paths, addresses, or unredacted logs are included here. Could you advise whether this is the same verify-wait issue as #2449, and whether there is a fix or a specific diagnostic trace that would distinguish a HIP watchdog problem from an MTP/long-context interaction?
Mehr auf der Site
Links zu Install, Modellen, Releases.