Pull requests / #2831
#2831 prefill: STRATA_PREFILL_STAGES (opt-in) prints where a prompt chunk's wall went
open · @gopinath87607 · 0 commentaires · Sur GitHub
BenchmarksAMD / HIPNVIDIA / CUDA
Description
## What this changes `STRATA_PREFILL_STAGES=1` (opt-in; unset is off) turns on the chunk's stage line: **one line a stage a chunk** with where the chunk's wall went — the chunk's wall, the copy lane's own span split into what ran before it (lead), the lane itself and what ran behind it (tail), the bytes it moved and the bytes the chunk's consumers waited on (the difference is the ring's read-ahead), the hand-off a later stage of a layer split paid, the GPU's own phases over the chunk, and the experts and host time it spent. It also turns the phase timer on. Without it a prefill change can only be judged by the prompt's ms/token, which mixes the chunk size, the copy lane, the host and the phases. The line is what separates them. From the Maya checkout (`project-maya`), local commit `a34f57c`, with its fixes `04b8d65` (the line's events are destroyed with the prompt path) and `63b3690` (the line reads only the events its chunk recorded). Its spelling `STRATA_GLM_PREFILL_STAGES` is accepted as the same knob, so a config written for that checkout runs unchanged. ## What it leaves alone Unset (the default), `stages_on()` is false: no line is printed, the phase timer is not read for it, and no extra sync is taken for it. The syncs the line does add when on wait only on copies the compute stream has already waited for (the issuer is joined when the chunk ends), so `STRATA_PREFILL_STAGES=0` beside a `--bench` A/B is how you check it costs nothing. The default path is unchanged by construction. ## Measurements This is a diagnostic; it makes no claim to speed anything up. Its own cost is what the set-vs-unset A/B under `--bench` measures, and the design keeps that at zero by waiting only on work already joined. ## Checks - **Default path:** unset = off by construction (`stages_on()` reads the environment once and returns false). - **Build:** `src/prefill/prefill.cpp` compiles clean against this branch's headers with the build's own flags (`-fsyntax-only`). The development branch that also carries the change built fully with ninja (CUDA 13.2). - **HIP / SYCL:** not compiled here (no toolchain on the machine). The change adds no backend-specific branch; it uses the same stream/event API the file already used, and the events are only created when the knob is on. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sur le site
Liens install, modèles, releases.