Issues / #2809
#2809 [Bug]: STRATA_HC_Q8=1 does nothing on the SYCL (Intel) build: the Q8_0 copies are loaded but never read
open · @hotschmoe · 0 comments · View on GitHub
Setup & installMulti-GPUAMD / HIPNVIDIA / CUDAModels & quantsDocumentation
Description
### What happened
`STRATA_HC_Q8=1` is documented (docs/QUANTS.md, docs/STRIX_HALO.md, docs/MAC.md) as the hyper-connection read from
the GGUF's own Q8_0 projections (or, for a BF16 pack, a Q8_0 copy quantized at load): the verify window's
hyper-connection down / up projections are read as Q8_0 instead of BF16 - an output-changing, measured speed-up on
CUDA, the default on AMD gfx12 and Mac. On the SYCL build (`sycl/`, Intel Arc) the flag never reaches a Q8_0 read: the hyper-connection
read only ever takes the BF16 weights, so the flag cannot change the output or the speed.
What I saw: on a development tree whose loader already makes the Q8_0 copies from the BF16 tensors (start-up line
`strata: STRATA_HC_Q8=1: 0.63 GiB of Q8_0 hyper-connection projections for the verify read`), Qwen3.8-Flash-Next ISTA
IQ3_XXS on two Arc Pro B70s, decode speed with `STRATA_HC_Q8=1` was the same as without it, and a GPU trace of the
decode window showed the same BF16 kernels (the sliced `gr_down`, `gr_down_reduce`, `gr_up_multi_kernel`) with the
flag on and off: the 0.63 GiB was never read. Reading `main` explains it (below). I did not run `main` itself with
the flag; from the source, on an ISTA (BF16) file it uploads nothing, and on a file with Q8_0 projections it makes
the copies and never reads them.
What I expected: the Q8_0 read, as on CUDA - the projections read as Q8_0 (the file's own, or quantized at load from
BF16), decode faster, answers differing slightly.
Why, from the source on `main`:
- CUDA has the whole path: `src/core/native_dense.cpp:46-69` (`hc_q8_env`), `:384-409` (a GGUF's own Q8_0
projections), `:411-446` (a BF16 pack's projections quantized to Q8_0 at load), `:448-466` (the dequantization
scratches for the prompt path), `src/program/generate.cpp:3062-3065` (the BF16 copies dropped on a single GPU). The
read: `src/kernels/cuda/fused_gr.cu:1351-1358` (`gr_down_q8_kernel` / `gr_up_q8_kernel`), dispatched at
`:1981-1982` when `a[0].q8_down != nullptr && a[0].q8_up != nullptr`; the argument check at `:1965` accepts
`(w_down && w_up) || (q8_down && q8_up)`.
- The SYCL port has only the loader's first half: `sycl/src/core/native_dense.cpp:321-358` keeps a Q8_0 copy only for
a tensor of GGUF type 8 (`:327`), so a BF16 tensor (type 30) is skipped and nothing is uploaded; there is no
quantize-at-load and no `add_hc_skip` / `hc_bf16_view` (the comment at `:110`: the port "only needs the env switch").
- The SYCL read never looks at the Q8_0 pointers: `sycl/src/kernels/cuda/fused_gr.dp.cpp:1770-1786`
`fused_gr_read_multi` requires `x.w_down && x.w_up` and never reads `x.q8_down` / `x.q8_up`; every branch below it
takes the BF16 weights, and its last line says so (`:2051`: "the Q8_0 reads (S23 STRATA_HC_Q8) are not built").
The callers do set the pointers (`sycl/src/core/verify.cpp:982-983`, `:1549-1563`), so for a GGUF with Q8_0
projections the copies are allocated and then ignored.
Steps to reproduce: build `sycl/` (docs/INTEL.md), run a Flash-Next GGUF with and without `STRATA_HC_Q8=1` on a
fixed prompt at temperature 0: from the code above the outputs should be identical (the BF16 read runs either way).
A pull request that refers to this issue makes the flag take the Q8_0 read on SYCL; with it the output changes and the verify window gets 5% shorter
on the setup above.
### Strata version, GPU, OS
main 296b59d2 (0.1.44.1, the `sycl/` build) - 2x Intel Arc Pro B70 32 GB - Ubuntu, kernel
7.1, Compute Runtime 26.22.38646.4, oneAPI 2026.1
### Engine log
The development-tree run (loader already making the copies, read unchanged), the only line the flag adds:
strata: STRATA_HC_Q8=1: 0.63 GiB of Q8_0 hyper-connection projections for the verify read
Prepared with AI assistance (Claude Code): the analysis, the measurements and this text were produced by Claude on the submitter's two Arc Pro B70s, at the submitter's request.
Related on this site
Quick links to related pages.