#6639 [Engine] STRATA_VERIFY_QDEDUP: cover the forked shared-expert stream (event-ordered, bitwise-identical image)

open · @xyzzing · 0 commentaires · Sur GitHub

Server & APIAMD / HIPNVIDIA / CUDAModels & quantsDocumentationWindows

Description

## Problem and result

`STRATA_VERIFY_QDEDUP=1` lets the shared expert's gate/up read the experts' q8_1 image of `xm` instead of re-quantizing it — but the gate excludes the forked shared-expert stream (`!sh_fork`), so when a workload forks the shared stream *and* runs qdedup, the fork quantizes `xm` into its own buffer anyway: one redundant quantize pass per window and two q8_1 images of the same tensor.

After this change the image is made first on `cs` and the fork waits on the (re-recorded) `ev_fork_` event before its own kernels launch, so the forked stream reads the same image — the redundant pass and the duplicate image go away. `ev_join_` ownership is unchanged (recording it twice would break the join's ordering). The interleaved MMVQ image (`nsw.q8_1_il`, new in 0.2.19) keeps its own fork exclusion untouched — the fork's gate/up takes the non-IL `x_q8_1` path this covers.

Exactness is by construction: `quantize_q8_1_rows` and `native_quantize_q8_1` write the same bytes — the premise the non-forked qdedup path already relies on.

## Related discussion

none (no open issue; small, opt-in-path change).

Evidence base: this change was baked on our gfx1100 fork line and is part of its deployed production stack — the measurement ledger for that line, its methodology and the surrounding campaign records live at **https://github.com/xyzzing/qwen-rdna3-optimization-ledger** (earlier reference of evidence; ~129 recorded paired experiments, the adopted-stack table and the rejected-with-reasons archive). The QDEDUP family screened at +2.3% median ms_round on that line (n=2 screen, `record-r42-envflags-20261103`); the honest claim for this hunk itself is *removed redundant work, no regression*, not a headline win.

## Approach

Widen the qdedup gate to cover the forked stream and order it with the fork's existing event mechanism (`ev_fork_` re-recorded after the image is made on `cs`; `cudaStreamWaitEvent` on the fork stream). No arithmetic change, no new env knob (no settings-register change), no new state: the image already exists whenever qdedup is on; the fork simply consumes it. 11 insertions / 4 deletions in `engine/src/core/verify.cpp`.

## Validation

**Passed:**
- HIP build, gfx1100, this exact commit (`fde24ef73`): cmake 4.3 + ninja, vendored ROCm 10.2.0a nightly SDK (`CMAKE_PREFIX_PATH` + pinned `CMAKE_HIP_COMPILER`), `STRATA_ENABLE_HIP=ON STRATA_NATIVE_EXPERTS=ON`, Release — full build rc=0 (762/762 targets).
- The same hunk (identical code lines, the fork tree additionally carries unrelated fork commits) compiles in the fork's production build and has soaked in its production serve **since 2026-11-21** — two 24-cell paired gate windows (each stop-prod, frozen-gate, B-vs-A over real 32K/128K workloads) and a 28-cell attribution screen since, with no regression attributable to this change.
- Register/docs checks: adds no `STRATA_*` knob; `tools/check --only docs` 6/6 on the fork line carrying the same code.

**Failed:**
- none observed.

**Not run:**
- CUDA / SYCL / Metal builds (HIP-validated here; the hunk is backend-neutral host code in `verify.cpp` and upstream CI covers the other backends).
- A device bitwise A/B of qdedup-on-fork vs off: cross-restart greedy output comparison is unsound on this stack (identical prompts produce different token counts across engine instances — a measured drift finding in the ledger above), so exactness rests on the by-construction argument plus the production soak. KL/logit dumps are infeasible on native packs (0-byte dumps).
- An isolated per-window timing of the removed pass (the +2.3% screen row above is n=2; not promoted as a claim).

## Limits

The saving only materializes when both `STRATA_VERIFY_QDEDUP=1` and the forked shared-expert stream are active — both opt-in; the default path is untouched (the gate's widened form is identical for `sh_fork == false`). Perf evidence is soak + screen, not a gated campaign.

*Prepared with AI assistance (fork-maintenance agent); reviewed and posted by the operator.*

Sur le site

Liens install, modèles, releases.