Issues / #3219

#3219 [Engine] SYCL AOT build for bmg-g31 fails on 0.2.0-beta with IGC 2.36.3 (ICE in the int8 prompt-expert kernels); fine with the current base image

open · @hotschmoe · 0 Kommentare · Auf GitHub

Setup & installMulti-GPUModels & quantsDocumentationWindows

Beschreibung

### Area

[Engine]  (speed, kernels, models, the server and API)

### Desktop app, Terminal or source?

Source (START-HERE.bat, setup.sh or a build)

### What happened

The SYCL AOT build for the Arc Pro B70 (`AOT=bmg-g31`) of 0.2.0-beta fails at the final `strata` link when the device compiler in the build image is IGC 2.36.3 (Compute Runtime / ocloc 26.22.38646.4). ocloc reports `IGC: Internal Compiler Error: Floating point exception` / `Build failed with error code: -11` on 77 device images, then `icpx: error: gen compiler command failed with exit code 245`. Every failing image is one of the new int8 prompt-expert kernels in `strata::prefill::i8` (`TagGu`, `TagDn`, `TagGuW`, `TagDnW`, `SwigluTag`, `QuantTag`; `sycl/src/prefill/esimd_moe.dp.cpp`). It is deterministic: the same 77 images fail on every run, and re-running ocloc on one saved image fails 4 of 4 times. The same images compile with the ocloc in the current base image (`llama-sycl-b70:stable`: ocloc 26.35.39758.10, IGC 2.41.5): 12 of 12 that I tried passed, and the whole AOT build then links.

How a user ends up with the old compiler: docs/INTEL.md says to run `docker build -t strata-sycl-dev sycl/tools` again only when `sycl/tools/Dockerfile` changes. `FROM ...llama-sycl-b70:stable` is a moving tag, though, and `docker build` reuses a locally cached base unless it gets `--pull`. A `strata-sycl-dev` image built a few weeks ago can therefore still carry an older IGC. (Ours was IGC 2.36.3 because we had swapped in the host driver's packages, so our exact image is not the stock one; I have not tested which older stock bases are affected.)

Possible fixes (your call): `docker build --pull` in the INTEL.md command, or a check in `sycl/tools/build.sh` that stops an AOT Battlemage build with a clear message when `ocloc --version` is older than the version these kernels were tested with.

### Smallest steps or command that reproduce it

In an image with ocloc 26.22.38646.4 / IGC 2.36.3 and oneAPI 2026.1:
`AOT=bmg-g31 bash sycl/tools/build.sh strata` on main e88ee39d. Or save one of the failing images and run
`ocloc -file <image>.spv -spirv_input -device bmg-g31 -options "-cl-fp32-correctly-rounded-divide-sqrt" -output o -output_no_suffix` (exit 245).

### Reproduction status

Reproduced locally

### Strata version

0.2.0-beta (main e88ee39d; the engine reports 0.1.45.3)

### Graphics card, memory and system

Build only (no GPU needed to reproduce); host 2x Arc Pro B70 32 GB, Ubuntu 24.04, kernel 7.1, xe driver, Docker build image with oneAPI 2026.1.0

### Model, size and settings

n/a (compile failure)

### Logs

```
Compilation from IR - skipping loading of FCL
IGC: Internal Compiler Error: Floating point exception
Build failed with error code: -11
Command was: /usr/bin/ocloc -output ... -file .../generate-....spv -output_no_suffix -spirv_input -device bmg-g31 -options "-cl-fp32-correctly-rounded-divide-sqrt"
...
icpx: error: gen compiler command failed with exit code 245 (use -v to see invocation)
FAILED: strata
```
Kernel names in the failing images (77 total): `_ZTSN6strata7prefill2i812_GLOBAL__N_15TagDnILi20ELi4ELi256EEE` and the TagGu / TagGuW / TagDnW / SwigluTag / QuantTag variants.

Prepared with AI assistance (Claude Code), at the submitter's request; reproduced on the submitter's machine.

Mehr auf der Site

Links zu Install, Modellen, Releases.