#6023 [Engine] peer_stream_ring: fix >= 2K peer_i8 ring wraparound + V100 docs
open · @midhatn · 0 comentários · No GitHub
BenchmarksMulti-GPUNVIDIA / CUDADocumentationWindowsLinux
Descrição
## Summary
Rebased cleanly onto **`origin/main` (`v0.2.17`, `3142b66b9`)**. Since most of PR `#6023`'s original changes are now upstream in `v0.2.17` (`240aeb985`), this PR now focuses cleanly on the remaining items:
1. **Fix `>= 2K` `peer_i8` + `STRATA_PF_PEER_STREAM` ring wraparound bug (`engine/src/prefill/peer_stream_ring.hpp`, `engine/src/prefill/prefill.cpp`, `engine/tests/prefill/peer_stream_ring_test.cpp`, `engine/CMakeLists.txt`):**
- Root cause (two interacting bugs when `peer_i8 || pgg_now` holds ring slots in `pgg_slots` until the group kernel runs):
1. Inside a compact group, `p_release_to(l, e, ps)` called `p_issue_until(P.pk + P.RP)` across skipped unrouted streamed experts while earlier experts in the same group still held `pgg_slots` (whose `P.pused` events are not recorded until *after* the group kernel launches), wrapping `P.s_cp` around and overwriting active `pstage` blobs.
2. Compact group construction checked `j - g0 < MMQ_GROUP` and `gr + c <= P.G`, but could span `>= P.RP` entries of `P.pseq` when unrouted streamed experts lay between active experts.
- Fix:
- Defer `p_issue_until(P.pk + P.RP)` inside `p_release_to` whenever `peer_i8 || pgg_now` has `pgg_nslots > 0` (`peer_ring_issue_allowed`), issuing only after the group kernel records `P.pused`.
- Split compact groups cleanly whenever `ps_cur - grp_ps0 >= P.RP` (`build_peer_ring_groups`), guarding `P.pseq_start` indexing with `ps_on && l + 1 < P.pseq_start.size()`.
- Leaves `STRATA_PF_PEER_STREAM` default unchanged (`0` on `peer_i8` layers); opt-in `STRATA_PF_PEER_STREAM > 0` now runs with zero non-finite logits and bit-exact greedy output matching `PS=0`.
2. **Documentation & locked-clock `v0.2.17` (`STRATA_ROUTE_TAIL_SKIP=0`) Dual V100-SXM2-16GB benchmark report (`docs/NVIDIA_V100.md`, `docs/bench-results/2026-11-18-dual-v100-sxm2-256k-mmq/README.md`, `docs/bench-results/README.md`):**
- Updates `docs/NVIDIA_V100.md` to reflect `v0.2.17` `-DSTRATA_MMQ=OFF` defaults (`native_expert_i8` / `i8h_volta.cuh`), scope `-DSTRATA_MMQ=ON` vs `100%` Tier-0 residency per `@noahark`'s `3 × V100-PCIE-32GB` A/B, and cite `@xrip`'s `STRATA_P2P_HANDOFF=1` PCIe Gen2 PHB finding.
- Adds controlled locked-clock (`nvidia-smi -pm 1 -pl 200 -lgc 1020,1020 -ac 877,1020`, `sm_min=sm_max=1020 MHz`, `mem_clk=877 MHz`, `STRATA_ROUTE_TAIL_SKIP=0`, `--minimal-env`) `2K` + `64K` verification tables alongside clearly labeled historical `v0.2.15`/`v0.2.16` `TAIL_SKIP=7` tables.
- Peer route tail-skip changes remain excluded as requested in review.
## Validation
### Passed
* **CPU unit tests (`peer_stream_ring_test` & `mmq_resident_sort_test`) on Linux (`g++-14`, CUDA `12.9`) and Windows `cmake-cpu`:**
* `peer_stream_ring_test: ok` (reproduces unfixed `RP=16` ring overwrite corruption `> 0`, then verifies `0` corruptions across `RP={1,2,4,8,16,48}` and seeds `1..5`)
* `mmq_resident_sort_test: ok`
* **Repo checks:** `python tools/bench_results_index.py --check`, `python tools/settings_register.py --check`, `python -m unittest tools.test_settings_register` (`8/8 OK`), `python -m unittest tools.test_check_no_mmq` (`7/7 OK`), `python tools/check --only docs,cmake-cpu` (`docs: 6 passed, 0 failed`; `cmake-cpu: 137 passed, 0 failed, 3 skipped` — includes `peer_stream_ring_test` & `mmq_resident_sort_test`).
* **2K Peer-Stream Repro & Bit-Exact Parity on `2 × Tesla V100-SXM2-16GB` (`NV6` NVLink, `4,171` prompt tokens, `--peer-device 1`, `STRATA_PF_WMMA=0`, `STRATA_PREFILL_CPU_SHARE=0`, `STRATA_ROUTE_TAIL_SKIP=0`, locked `200W / 1020 MHz / 877 MHz`):**
* Unpatched `v0.2.17` `base` (`PS=0.35`, `RING=16`): reproduced `DONE nonfinite` at prompt pos `4164` and decode pos `4170`.
* Unpatched `v0.2.17` `base` (`PS=0.35`, `RING=48` default): silent ring corruption diverging output (`sha=1a2b533e73ea`, `63.2 tok/s` decode vs reference `sha=632fe447bfa0`, `70.8 tok/s`).
* Patched (`PS=0.35`, `RING=16`): `0` non-finite, `1,006.2 tok/s` prefill, `71.7 tok/s` decode, bit-exact `sha=632fe447bfa0`.
* Patched (`PS=0.35`, `RING=16`, `STRATA_PEER_NO_P2P=1`): `0` non-finite, `863.0 tok/s` prefill, `70.1 tok/s` decode, bit-exact `sha=632fe447bfa0`.
* Patched (`PS=0.35`, `RING=48` default): `0` non-finite, **`1,180.1 tok/s`** prefill (`+17.5%` vs `PS=0` `1,004.5`), `71.6 tok/s` decode, bit-exact `sha=632fe447bfa0`.
* **64K Locked-Clock Verification (`61,694` prompt tokens, `STRATA_ROUTE_TAIL_SKIP=0`, `codes=True 3/3`, `ecc=0/0`):**
* `--peer-device 1` (`PF_WMMA=0`, `P2P_DECODE_SCATTER=1`): `PS=0` `1,435.7 tok/s` prefill -> `PS=0.28` **`1,532.6 tok/s`** (`+6.7%`, `67.9 tok/s` decode, identical `sha=affdf99307c6` across `PS={0, 0.22, 0.28, 0.35}`, `0` non-finite).
* `--layer-split auto`: `PF_WMMA=0` `1,814.7 tok/s` prefill (`65.7` decode) -> `PF_WMMA=1 STRATA_MMQ_TIER_SORT=1 STRATA_P2P_HANDOFF=1` **`1,993.2 tok/s`** prefill (`+9.8%`, `67.0 tok/s` decode).
### Failed
* None (after guarding `P.pseq_start[l + 1]` on `!ps_on`).
### Not run
* `256Ki` re-runs under `200W / 1020 MHz` lock on `v0.2.17` (the `256Ki` rows in the report remain labeled `v0.2.15` / `v0.2.16` `TAIL_SKIP=7`).
No site
Links install, modelos, releases.