#5560 [Engine] STRATA_HC_REQ8=1 with STRATA_HC_PERSIST=1 (HIP): the persistent hc read takes the int8 codes as bf16

closed · @Vop3p · 3 comentarios · En GitHub

Setup & installMulti-GPUAMD / HIPModels & quantsWindows

Descripción

### Area

[Engine]  (speed, kernels, models, the server and API)

### Desktop app, Terminal or source?

Source (START-HERE.bat, setup.sh or a build)

### What happened

With `STRATA_HC_REQ8=1`, `w_down` / `w_up` of each hc read hold int8 codes and `q8_scale` holds their scales (`weights.cpp`), and the verifier passes them as `a.s_down` / `a.s_up` (`verify.cpp:1601`). `fused_gr_read_persist` (`fused_gr.cu:3769`) only declines for `q8_down` / `q8_up` (`STRATA_HC_Q8`) and `q8_mixed` (`STRATA_QFUSE`). It does not check `s_down`, so with `STRATA_HC_PERSIST=1` it launches the bf16 persistent kernel on int8 bytes. The non-persistent path handles this (`q8w = m.a[0].s_down != nullptr`, `fused_gr.cu:1600`).

Expected: the persistent read declines and the default read (which knows the int8 rows) runs, as it does for `STRATA_HC_Q8`.

Suggested fix, one line next to the HC_Q8 check in `fused_gr_read_persist`:

```cpp
if (a[0].s_down != nullptr || a[0].s_up != nullptr) return false;   // STRATA_HC_REQ8: the default read
```

### Smallest steps or command that reproduce it

Not run. By the code: one HIP card (`STRATA_HC_REQ8` refuses a layer split), `STRATA_HC_REQ8=1 STRATA_HC_PERSIST=1`, with `STRATA_QFUSE` / `STRATA_QFUSE_HC` off (the default off gfx12, e.g. gfx103x). Nothing at start-up turns `HC_PERSIST` off when `HC_REQ8` is on (`generate.cpp:5116`). With QFUSE on, `q8_mixed` is set and the persistent read already declines, which hides it.

### Reproduction status

Unconfirmed static analysis (I read code or ran a scanner; I did not see it fail)

### Strata version

main 146df3c4 (source)

### Graphics card, memory and system

Found while working on 2x RX 6900 XT 16 GB (gfx1030) · 128 GB RAM · Ubuntu 26.04, ROCm 10.0; not run on this combination (our cards run a layer split)

### Model, size and settings

Any model whose hc projections are BF16 in the pack (seen on Qwen3.8-Flash-Next IQ3_S)

### Logs

_No response_

---
Found with an AI coding assistant by reading the code; not reproduced at runtime.

En el sitio

Enlaces a install, modelos, releases.