#5560 [Engine] STRATA_HC_REQ8=1 with STRATA_HC_PERSIST=1 (HIP): the persistent hc read takes the int8 codes as bf16
closed · @Vop3p · 3 commentaires · Sur GitHub
Setup & installMulti-GPUAMD / HIPModels & quantsWindows
Description
### Area [Engine] (speed, kernels, models, the server and API) ### Desktop app, Terminal or source? Source (START-HERE.bat, setup.sh or a build) ### What happened With `STRATA_HC_REQ8=1`, `w_down` / `w_up` of each hc read hold int8 codes and `q8_scale` holds their scales (`weights.cpp`), and the verifier passes them as `a.s_down` / `a.s_up` (`verify.cpp:1601`). `fused_gr_read_persist` (`fused_gr.cu:3769`) only declines for `q8_down` / `q8_up` (`STRATA_HC_Q8`) and `q8_mixed` (`STRATA_QFUSE`). It does not check `s_down`, so with `STRATA_HC_PERSIST=1` it launches the bf16 persistent kernel on int8 bytes. The non-persistent path handles this (`q8w = m.a[0].s_down != nullptr`, `fused_gr.cu:1600`). Expected: the persistent read declines and the default read (which knows the int8 rows) runs, as it does for `STRATA_HC_Q8`. Suggested fix, one line next to the HC_Q8 check in `fused_gr_read_persist`: ```cpp if (a[0].s_down != nullptr || a[0].s_up != nullptr) return false; // STRATA_HC_REQ8: the default read ``` ### Smallest steps or command that reproduce it Not run. By the code: one HIP card (`STRATA_HC_REQ8` refuses a layer split), `STRATA_HC_REQ8=1 STRATA_HC_PERSIST=1`, with `STRATA_QFUSE` / `STRATA_QFUSE_HC` off (the default off gfx12, e.g. gfx103x). Nothing at start-up turns `HC_PERSIST` off when `HC_REQ8` is on (`generate.cpp:5116`). With QFUSE on, `q8_mixed` is set and the persistent read already declines, which hides it. ### Reproduction status Unconfirmed static analysis (I read code or ran a scanner; I did not see it fail) ### Strata version main 146df3c4 (source) ### Graphics card, memory and system Found while working on 2x RX 6900 XT 16 GB (gfx1030) · 128 GB RAM · Ubuntu 26.04, ROCm 10.0; not run on this combination (our cards run a layer split) ### Model, size and settings Any model whose hc projections are BF16 in the pack (seen on Qwen3.8-Flash-Next IQ3_S) ### Logs _No response_ --- Found with an AI coding assistant by reading the code; not reproduced at runtime.
Sur le site
Liens install, modèles, releases.