Issues / #2809
#2809 [Bug]: STRATA_HC_Q8=1 does nothing on the SYCL (Intel) build: the Q8_0 copies are loaded but never read
open · @hotschmoe · 0 commentaires · Sur GitHub
Setup & installMulti-GPUAMD / HIPNVIDIA / CUDAModels & quantsDocumentation
Description
### What happened
`STRATA_HC_Q8=1` is documented (docs/QUANTS.md, docs/STRIX_HALO.md, docs/MAC.md) as the hyper-connection read from
the GGUF's own Q8_0 projections (or, for a BF16 pack, a Q8_0 copy quantized at load): the verify window's
hyper-connection down / up projections are read as Q8_0 instead of BF16 - an output-changing, measured speed-up on
CUDA, the default on AMD gfx12 and Mac. On the SYCL build (`sycl/`, Intel Arc) the flag never reaches a Q8_0 read: the hyper-connection
read only ever takes the BF16 weights, so the flag cannot change the output or the speed.
What I saw: on a development tree whose loader already makes the Q8_0 copies from the BF16 tensors (start-up line
`strata: STRATA_HC_Q8=1: 0.63 GiB of Q8_0 hyper-connection projections for the verify read`), Qwen3.8-Flash-Next ISTA
IQ3_XXS on two Arc Pro B70s, decode speed with `STRATA_HC_Q8=1` was the same as without it, and a GPU trace of the
decode window showed the same BF16 kernels (the sliced `gr_down`, `gr_down_reduce`, `gr_up_multi_kernel`) with the
flag on and off: the 0.63 GiB was never read. Reading `main` explains it (below). I did not run `main` itself with
the flag; from the source, on an ISTA (BF16) file it uploads nothing, and on a file with Q8_0 projections it makes
the copies and never reads them.
What I expected: the Q8_0 read, as on CUDA - the projections read as Q8_0 (the file's own, or quantized at load from
BF16), decode faster, answers differing slightly.
Why, from the source on `main`:
- CUDA has the whole path: `src/core/native_dense.cpp:46-69` (`hc_q8_env`), `:384-409` (a GGUF's own Q8_0
projections), `:411-446` (a BF16 pack's projections quantized to Q8_0 at load), `:448-466` (the dequantization
scratches for the prompt path), `src/program/generate.cpp:3062-3065` (the BF16 copies dropped on a single GPU). The
read: `src/kernels/cuda/fused_gr.cu:1351-1358` (`gr_down_q8_kernel` / `gr_up_q8_kernel`), dispatched at
`:1981-1982` when `a[0].q8_down != nullptr && a[0].q8_up != nullptr`; the argument check at `:1965` accepts
`(w_down && w_up) || (q8_down && q8_up)`.
- The SYCL port has only the loader's first half: `sycl/src/core/native_dense.cpp:321-358` keeps a Q8_0 copy only for
a tensor of GGUF type 8 (`:327`), so a BF16 tensor (type 30) is skipped and nothing is uploaded; there is no
quantize-at-load and no `add_hc_skip` / `hc_bf16_view` (the comment at `:110`: the port "only needs the env switch").
- The SYCL read never looks at the Q8_0 pointers: `sycl/src/kernels/cuda/fused_gr.dp.cpp:1770-1786`
`fused_gr_read_multi` requires `x.w_down && x.w_up` and never reads `x.q8_down` / `x.q8_up`; every branch below it
takes the BF16 weights, and its last line says so (`:2051`: "the Q8_0 reads (S23 STRATA_HC_Q8) are not built").
The callers do set the pointers (`sycl/src/core/verify.cpp:982-983`, `:1549-1563`), so for a GGUF with Q8_0
projections the copies are allocated and then ignored.
Steps to reproduce: build `sycl/` (docs/INTEL.md), run a Flash-Next GGUF with and without `STRATA_HC_Q8=1` on a
fixed prompt at temperature 0: from the code above the outputs should be identical (the BF16 read runs either way).
A pull request that refers to this issue makes the flag take the Q8_0 read on SYCL; with it the output changes and the verify window gets 5% shorter
on the setup above.
### Strata version, GPU, OS
main 296b59d2 (0.1.44.1, the `sycl/` build) - 2x Intel Arc Pro B70 32 GB - Ubuntu, kernel
7.1, Compute Runtime 26.22.38646.4, oneAPI 2026.1
### Engine log
The development-tree run (loader already making the copies, read unchanged), the only line the flag adds:
strata: STRATA_HC_Q8=1: 0.63 GiB of Q8_0 hyper-connection projections for the verify read
Prepared with AI assistance (Claude Code): the analysis, the measurements and this text were produced by Claude on the submitter's two Arc Pro B70s, at the submitter's request.
Sur le site
Liens install, modèles, releases.