#6580 [Engine] 0.1.65 HIP gfx1200: illegal memory access in batched prefill (Inkling-2 GGUF, NVFP4→SF4 experts); short prompts are fine
open · @oakd2003 · 0 Kommentare · Auf GitHub
BenchmarksAMD / HIPModels & quants
Beschreibung
### Summary With the GGUF loading fix from #6579 applied, Inkling-2-Mini GGUF runs on gfx1200 for short prompts, but any longer prompt aborts with a HIP illegal memory access during prefill. ### Environment Same as #6579: v0.2.19 / engine 0.1.65 / gfx1200 / ROCm 7.10.0a20251120, `Inkling-2-Mini-JP-16GB-UD-Q3_K_XL.gguf` (loop model: 24 blocks, `num_loops=3` → 48 visits, SWA window 1024 on 40 of 48 layers, experts NVFP4 → SF4 `code_id=15`). ### Reproduction ``` # OK strata-inkling --bench --native model.gguf --gpus 1 --prompt-len 8 --gen 2 [bench] backend=hip format=gguf parallel=1 gpus=1 kv=fp16 prompt=8 gen=2 | prefill=183.7 ppts (0.04s) | decode=39.0 tps agg # FAILS at 24 tokens: "logits D2H: an illegal memory access was encountered" strata-inkling --bench --native model.gguf --gpus 1 --prompt-len 24 --gen 4 # FAILS at 32 tokens: "tdr visit flush: an illegal memory access was encountered" strata-inkling --bench --native model.gguf --gpus 1 --prompt-len 32 --gen 8 bench prefill err: [ink-hip] tdr visit flush: an illegal memory access was encountered Hip error: 'an illegal memory access was encountered'(700) at .../hipblaslt/library/src/amd_detail/hipblaslt.cpp:170 # FAILS strata-inkling --bench --native model.gguf --gpus 1 --prompt-len 128 --gen 32 ``` ### What is already ruled out - Not the loader: the same file runs on the CPU backend (with the fix applied). - Not the manifest/shapes: 8-token prefill + decode on the HIP backend is coherent. - Measured threshold: 8 and 16 prompt tokens pass; 24 and 32 fail. At 24 the failure surfaces at `logits D2H`, at 32 at `tdr visit flush` in `HipBackend::forward_batch` (a sticky error then shows up from hipBLASLt, so that line is probably a bystander). ### Next steps I can help with Happy to test the Q4_K_XL (non-NVFP4 expert) variant of the same model to separate the NVFP4→SF4 repack path from the generic batched-prefill path, or to run with `STRATA_HIP_PROF=1` / `STRATA_HIP_TREE_PROF=1` if those print anything useful on 0.1.65.
Mehr auf der Site
Links zu Install, Modellen, Releases.