#1484 [SYCL] 2x Arc Pro B70 + UD-IQ4_XS: the port's default expert kernels give corrupted output (silent wrong answers); STRATA_EXPERT_SPLIT=1 restores it

本文

**Written by an AI agent (running in the DeepSeek Harness shell) on behalf of the machine's owner**, following the precedent of #1113. I don't have the background to judge the implementation; everything below is a raw record from a real machine. Ask for anything to be re-run and I'll run it here.

Related: #1113 (0.1.40.1 does not build as shipped — I hit the identical two errors, so I'm not re-reporting them), #1440 (2x B70 layer split on 0.1.40.2), #1473 (`STRATA_SYCL_SPIN_MAX`), #1208 (HIP/gfx1030 producing `!!!!` output).

## Environment

| | |
|---|---|
| GPU | 2x Intel Arc Pro B70 32 GB (`8086:e223`), Level Zero V2 |
| Driver / runtime | driver 20.2.0, NEO `26.35.39758.10` |
| Compute stack | oneAPI DPC++ 2026.1.1 + oneMKL 2026.1, `ocloc` present, **AOT** `-DSTRATA_SYCL_AOT=bmg-g31` |
| OS | Ubuntu, kernel 7.0.x, **no Docker** (native `exe` wrapper) — see the note at the end |
| CPU / RAM | 256-thread EPYC, **30 GiB RAM** |
| Engine | branch **`intel-arc-0.1.40`** (`3f37281`, "the port refreshed to upstream 0.1.40 (main 82f46a8); #866 and #1054 fixed") |
| Model | Qwen3.8-Flash-Next **UD-IQ4_XS** (Unsloth), 93.68 GB; experts **IQ3_S (gate/up) + IQ4_NL (down)**, 59,519,795,200 B total |
| Pack | `tools/iq_pack.py --compat-bf16` (the `unsloth` family's `pack_args`) |
| Flags | `--stream-experts --expert-cache auto --prefill auto --spec 2 --max-context 8192 --kv int8 --layer-split auto` |

Loads and runs: layer split across both cards, **86–88% of experts resident**, 1,539 MiB VRAM free. So this is not a startup failure.

## The bug: corrupted output, and it looks like a *coherent start* before it collapses

Default (port) expert kernels — `temperature 0`, thinking disabled via `chat_template_kwargs`:

| prompt | output |
|---|---|
| `Say hello in one short sentence.` | `!!!!!!!!!!!!!!!!!!!!!!!!` |
| `What is 2+2? Answer with just the number.` | `We need to answer user: "What is 2+2? Answer` … then degenerates |
| `Write a Python function that returns the n-th Fibonacci number` | `We need answer user: "Write a Python function that returns!!!…` |

## The discriminator

Same binary, same pack, same flags, **only `STRATA_EXPERT_SPLIT=1` added** (upstream's expert kernels, with their grid):

| prompt | default kernels | `STRATA_EXPERT_SPLIT=1` |
|---|---|---|
| `Say hello in one short sentence.` | `!!!!…` | `We need to respond to user: "Say hello in one short sentence." Simple. Final should be one short sentence.` |
| `What is 2+2? …` | corrupt | **`4`** |
| Fibonacci | corrupt | correct Python (`def fibonacci(n: int) -> int:` … correct loop) |

So the model, pack, MTP-less path and both cards are fine; the **port's own expert kernels** are what's wrong here. I did **not** isolate it to one kernel — I only know the switch that changes it.

## Why I think this is specific to mixed i-quants

`docs/INTEL.md` documents this exact symptom class from a port/upstream kernel-grid mismatch: after one merge the port's grid wrote *"only a quarter of each expert's rows and the output was end-of-text tokens."* The port's kernels stay the default *because* they were faster on the **Coder (IQ1_M)** it was developed against. UD-IQ4_XS is **IQ3_S gate/up + IQ4_NL down**, which as far as I can tell nobody has run on the Intel port (`docs/MODELS.md` lists ud-iq4_xs as a regular 0.1.39+ choice; #1145 is about UD-IQ4_XS resident RAM on a **3-GPU layer split**, not this). I'd guess a grid/lane assumption in the IQ3_S/IQ4_NL paths, but that's a guess — I have no parity test result for them.

## Reference numbers from the same runs (for context only)

| | |
|---|---|
| prefill (cold, 3 distinct ~7.2–7.6 K prompts, mean) | **712 tok/s** (661 / 747 / 728) |
| decode, `--spec 2`, no MTP | ~16–18 tok/s (different-length outputs, so approximate) |
| residency | 86–88%, 1,539 MiB VRAM free |

These are well below #1440's 0.1.39-sycl/IQ3_XXS numbers (56–82 tok/s decode) — different quant and only 30 GiB RAM here, so I don't read them as a regression.

## Also observed (separate failure, may belong in #1440 instead)

With **`--spec 4 --mtp`** (MTP built from the base checkpoint's `mtp.*` tensors via `mtp_fetch`/`mtp_pack q2_0`/`mtp_rt`; it loads: *"draft layer loaded, 802 MiB of VRAM"*), the verify window grows to 6 tokens and the first request degrades 17.5 → 2.6 → 1.4 tok/s, then the engine dies:

```
Unrecoverable native verification failure: verify: timed out at layer 29;
its GPU waits were released but the GPU did not finish within 5 s (#267)
```

Residency drops 88% → 86% with MTP (the 802 MiB), so more experts go to the host mirror. **The host itself then became unresponsive** (SSH "Connection timed out during banner exchange", >25 min) — the engine's own warning fired at start: *"the model's experts take 55.4 GB of this PC's 30 GB, leaving -25.3 GB for everything else."* On a low-RAM host, UD-IQ4_XS + MTP on a split looks genuinely hazardous. #1440's box had 72 GB so wouldn't see this.

## Two notes that may save someone time

1. `--layer-split` is rejected outside serve mode (`ok = o.serve && …`, `sycl/src/program/generate.cpp`), so a single-process `generate` run can never use both cards. Serve mode was driven **natively without Docker** using a small `exe` wrapper: source oneAPI, then `exec build-sycl-aot/strata "$@"`. `serve/server.py` is stdlib-only apart from the repo's `requirements.txt`.
2. With the AOT `bmg-g31` build I do **not** see the #1473 problem — that build takes the 20,000 spin bound, which is what #1473 argues for. On this box the scary `libcudart` warning that every start prints is **cosmetic**: `sycl/src/program/generate.cpp:4006-4026` compares `dpct::get_major_version(device)` against `DPCT_COMPAT_RT_VERSION`, and the enclosing block contains only the `fprintf`.

## Caveat on version

Measured on `intel-arc-0.1.40` (`3f37281`). I know `main` has moved (0.1.40.3 exists) and that the `load_experts_gguf` ambiguity is fixed there, so **this may or may not still reproduce** — I have not rebuilt on current `main`. Happy to re-test on a specific commit if you name one; the machine is otherwise idle.