Pull requests / #2436
#2436 SYCL: defer pipelined one-token commits to the host verdict (port of c81f9984)
open · @deriknel · 0 コメント · GitHub で見る
BenchmarksMulti-GPUNVIDIA / CUDAModels & quantsWindows
本文
## Title
Issue: none filed. Needs #2433 for `--pipeline-windows` to run on SYCL at all.
## Summary
c81f9984 fixed this in the CUDA verifier on 2026-10-09; the SYCL verifier still has the old condition. A pipelined verifier (`set_always_publish`) captures its one-token window with the in-graph state advance, and `pl_commit_async` then runs the commit graph on top of it, so every one-token window advances the first stage's GDN state twice. With `--pipeline-windows 2`, greedy text forks from the serial loop's within the first few dozen tokens. This change ports c81f9984 line for line.
## What changed
`sycl/src/core/verify.cpp`, the same two conditions c81f9984 changed in `src/core/verify.cpp`:
- `Verifier::record_window`: `self_commit` is off when `always_publish_` is set.
- `Verifier::commit`: the branch that skips the commit graph for a one-token window ("the window advanced the state itself") is not taken when `always_publish_` is set.
Only `--pipeline-windows` calls `set_always_publish(true)` (`sycl/src/program/generate.cpp:7159-7166`), so the serial loop does not change.
## Extra Notes
- Measured on 0.1.43.1 with #2433 applied: 2x Arc Pro B70, Swift 1.5 Flash-Next IQ3_S, layer split 26, every expert in VRAM, greedy, 5 prompts, two rounds. The table shows the first character where the pipelined text differs from the serial loop's:
| | prose | code | maths | edit | story |
|---|---|---|---|---|---|
| without this change | 72 | 146 | 134 | same | 118 |
| with this change | same | same | same | same | same |
Both rounds gave the same result. With the pipeline off, every run matched the serial baseline.
- `STRATA_ONE_TOKEN_COMMIT=0`, which turns the one-token self-commit off in both loops, also gave identical text and identical state fingerprints (`STRATA_STATE_HASH`) on all 5 prompts. That is how we narrowed it down to this path.
- On 0.1.44.1 (296b59d2) with this change and #2433: identical text on the 5 prompts in each of 3 comparisons; decode 79.6 / 113.8 / 89.0 tok/s (prose / code / a long answer) with the pipeline against 75.0 / 99.3 / 79.1 without; a ~148K-token prompt followed by 20 prompts of ~13K tokens, 4 runs on a fresh engine, and a 109-request soak, all without errors.
- Built and tested the SYCL target only; the change is in `sycl/`.
We investigated (myself and Claude Opus 5.5 in Claude Code) and I reviewed the findings and ran every test on my two Arc Pro B70s on my CachyOS machine.
関連リンク
インストール・モデル・リリースへの站内リンク。