Issues / #1103

#1103 hip: gfx1030 (RX 6800 XT) intermittent verify timeouts (#267): ROCm 7.14 gfx103X wheels regression, fixed by 7.13

open · @ChenCui26 · 2 commentaires · Sur GitHub

BenchmarksAMD / HIPModels & quantsWindowsLinux

Description

## Environment

- GPU: AMD Radeon RX 6800 XT 16 GB (gfx1030), i5-13400F, 63 GB RAM, Arch Linux (kernel 7.0.12-arch1-1)
- Engine 0.1.39, HIP build, ROCm TheRock wheels from the `gfx103X-all` index
  (the pinned 7.10.0a20251120 is no longer on the nightly index, so `STRATA_ROCM_VERSION` was overridden)
- Model: OrcaRouter Uncensored IQ3_XXS via the compat-bf16 pack, `--spec 4 --mtp --prefill 512 --max-context 16384 --kv int8`

## Symptom

Intermittent `verify: timed out at layer N (#267)` at random layers (observed 2, 8, 16, 30). Retrying the same
request often succeeds. With `STRATA_VERIFY_TRACE=1`, the host handshake is fully SERVED (flag / A / B all set)
when the timeout fires, and `RELEASE-NOT-DRAINED` is reported. The GPU breadcrumbs stop between arbitrary
kernels mid-window (e.g. after `out-proj`, between `VRAM hits` and `waitB`, between `z` and `rec`) — not one
specific kernel, and many windows complete fine before the stall (e.g. 65 windows OK, window 66 stalls at
layer 2).

## What did NOT fix it

- `STRATA_DOORBELL_STORE=1` + `STRATA_VERIFY_COHERENT=1`: still ~1 in 4 requests hung.
- Adding `HSA_ENABLE_SDMA=0`: short requests all passed, but a 12K-token generation still died at ~1K tokens
  (layer 32). `GPU_MAX_HW_QUEUES=1` + `HIP_HOST_COHERENT=1` on top: same result (layer 21 at ~1K tokens).

## Resolution (update)

Downgrading the wheels from **7.14.0a20260612 to 7.13.0a20260515** (same `gfx103X-all` index) and rebuilding
the engine fixed it. Two long generations (~9.7K and ~7.5K tokens, both `finish: stop`) ran clean at 43-46
tok/s with zero stalls, with `HSA_ENABLE_SDMA=0` + `STRATA_VERIFY_COHERENT=1` + `STRATA_DOORBELL_STORE=1`.
This points at a regression in the 7.14 gfx103X wheels rather than the SDMA path alone; 7.14.0a20260611 was
not tried.

Sur le site

Liens install, modèles, releases.