Pull requests / #462
#462 diagnostics: dump every verify window's logits, so greedy decode can be bisected
closed · @constantindjonkam · 0 评论 · 在 GitHub 查看
BenchmarksNVIDIA / CUDAModels & quantsWindows
描述
## What
A diagnostic that makes greedy decode bisectable, plus the traces that localised a real bug.
Greedy output on a two-GPU split was **not reproducible**: repeated identical requests diverged from
about the 28th generated character. Expert routing was byte-identical across runs and the first
verify window's logits were byte-identical, so both were ruled out by measurement rather than by
argument. What remained needed a way to see **which verify window** stopped reproducing.
`--dump-logits` cannot do that. It is written from the per-token loop, which a native pack leaves at
if (native_pack) { spec_pos = pos; break; }
before the dump site. (Fixing that is a separate, small PR.)
## What this adds
- **`STRATA_DUMP_WINDOW_LOGITS=<path>`** records row 0 of every verify window - the row that chose the
next token - with the position, the window length `T` and the accepted count `a`.
Records are streamed and **no count is declared up front**, so a run killed mid-write leaves a short
file rather than one that lies about its contents.
- **Env-gated traces, off by default**: `STRATA_TRACE_ADAPT` (whether an adaptive round considered,
skipped or swapped, and whether the previous round's copies had landed) and `STRATA_TRACE_RES` (a
digest of the expert residency table per request).
- The four probe scripts used to run all of this, under `bench-probes/`.
The file format lives in its own header so it is testable **without a GPU**:
`window_logits_dump_test` round-trips synthetic windows, and checks that a foreign or missing file is
rejected rather than misread.
## Cost
Default off: one function-local `static` getenv, and nothing opened, copied or allocated on the
default path.
## Evidence it produced
With the per-window dump, on an IQ3_XXS / RTX 3090 run with `--adapt-every 4`:
windows 0-15 bit-identical across 5 runs
window 16 (pos=86) runs fork: 3 of 5 agreed, 2 differed
8 of 24 windows differing afterwards; max |delta logit| 0.266
which pointed at the adaptive tier and became the fix in
[#463](https://github.com/Niko1221/Strata/pull/463), **merged in 0.1.38**. That release also waits
for the swap by default (`STRATA_ADAPT_NOWAIT=1` opts out), so the property this probe depends on is
on for everyone - not only for people who read the note.
站内延伸阅读
链到安装、模型与版本说明,便于 SEO/GEO,非官方 issue 正文。