Pull requests / #2319
#2319 SYCL verify: batch windows keep the doorbell nodes under the lean window ("parallel" crashed on a B70)
open · @aangelinsf · 0 comentarios · En GitHub
BenchmarksSetup & installServer & APIModels & quantsWindows
Descripción
## Problem
On v0.1.43.1 (a0116b27) and on current main / v0.1.44 (ed4773eb), SYCL port, one Arc Pro B70, `"parallel": 2` crashes the engine the first time two requests share a window:
strata verify: captured the batch window over slots 0,1
terminate called after throwing an instance of 'sycl::_V1::exception'
what(): level_zero backend failed with error: 37 (UR_RESULT_ERROR_UNINITIALIZED)
[strata] the engine stopped unexpectedly (exit code 139) ... verify batch: layer 0 never rang (graph finished)
It reproduced 6 times out of 6, with `--spec 4`, `--spec 2`, and with `--batch-mtp`. A single slot was fine, and so was `"parallel": 2` with requests sent one after another.
## Cause
0.1.43 made the decode window lean under `STRATA_VERIFY_NO_HOST` with the device plan (46a77907). The lean window drops the doorbell publish, the flag waits and the plan copy, because no host answers them (`sycl/src/core/verify.cpp`, the `lean` condition in `record_window`).
`capture_batch()` records batch windows through the same `record_window()` with `batch_rec_` set, so batch graphs were recorded lean too. The host batch path (`run_slot_rows` / `batch_poll`) still waits for each layer's doorbell unless the stage is all-resident. On a stage with mirrored experts the graph finishes with the ring silent, giving "layer 0 never rang (graph finished)".
`STRATA_VERIFY_LEAN=00` on the stock build makes the crash go away, which confirms this.
## Fix
Leave lean off while a batch window is being recorded: add `&& !batch_rec_` to the `lean` condition, plus a comment. Solo windows stay lean.
I kept it conservative: batch windows keep their doorbells even on an all-resident stage, where lean would be harmless, because a captured batch graph is reused across windows. A narrower condition is your call.
## Testing
Setup:
- Arc Pro B70 32 GB (8086:e223), xe driver, Ubuntu 26.04, kernel 7.0.
- `strata-sycl-dev` image: oneAPI DPC++ 2026.1.0, compute-runtime 26.35.39758.10, Level Zero 1.32.0. AOT `bmg-g31`.
- Qwen3.8-Flash-Next GSQ-RCO IQ2_XS native pack on one GPU, with 17,234 of 24,576 expert slots in VRAM and the rest mirrored.
- Engine args: `--stream-experts --spec 4 --spec-min-p 0.5 --mtp ... --kv int8 --kv-resident 32768 --max-context 110000 --prefill 4096 --vram-reserve-mib 2048`.
- Launcher defaults `STRATA_VERIFY_NO_HOST=1 STRATA_VERIFY_DEVICE_PLAN=1`.
This PR is against main (v0.1.44). On main I checked the essentials. Stock main still crashes the same way with `"parallel": 2`. With this patch: no crash, the `--dump` A/B against stock main with `STRATA_VERIFY_LEAN=00` is byte-identical, and `--batch-mtp` runs (two streams at 63–65 tok/s). The longer runs below were on v0.1.43.1 with the same one-line change.
Results:
- **Crash:** with `"parallel": 2` and no extra env, I saw zero crashes across about 20 concurrent toy requests and 10 runs of two real coding agents (opencode, one tool call each) against the server. The engine log shows batch windows averaging 1.99 rows.
- **Equivalence:** `tools/batch_test.py --batch 2 --n 2 --max-new 64 --extra "--pcie-frac 0 --adapt-every 1000000" --dump` gives **byte-identical** output, solo and batch, between this build and the stock build with `STRATA_VERIFY_LEAN=00`. That holds with and without `--batch-mtp`.
- **Solo vs batch parity (pre-existing, not changed by this PR):** on this card batch tokens still differ from solo at the same positions, slot 0 at token 6 and slot 1 at token 38. They differ in exactly the same way on the stock build with `STRATA_VERIFY_LEAN=00`, so this is the SYCL batch path's existing divergence, like the B65 report in #1721. The text stays coherent.
- **`--batch-mtp`** now runs: two slot drafters load, and there were no crashes in 5 agent runs.
- **Speed:** a single stream decodes ~73 tok/s. Two batched streams decode 66 tok/s from the first batch token, or 82 tok/s with `--batch-mtp`.
- **Two agents at once, wall time per run:**
| Config | Wall time |
|---|---|
| Single slot | 52–57 s (4 runs) |
| `parallel: 2` | 26–57 s, mean 38 s (5 runs) |
| `parallel: 2` + `--batch-mtp` | 33–55 s, mean 38 s (5 runs) |
Most of the gain comes from each slot keeping its own conversation cache instead of re-reading ~12k tokens on every switch.
- **Single slot after the patch:** unchanged. Solo windows are still lean; the solo runs were ~4% faster than with `STRATA_VERIFY_LEAN=00`.
## Side note: the launcher drops `STRATA_VERIFY_LEAN=0`
`sycl/serve/strata-sycl.sh` treats a value of exactly `0` as "not set" and doesn't pass it into the container. `STRATA_VERIFY_LEAN` is on by default and turned off by a leading `0`, so `0` never reaches the engine; I had to use `00` to A/B. It's not needed after this fix, but it's worth documenting or special-casing for switches that default on.
Related: #1762, #1721, #776 / #845 (the all-resident counterpart in the batch path), #1118.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
En el sitio
Enlaces a install, modelos, releases.