Pull requests / #3308
#3308 [Engine] an optional line showing where each prompt chunk's time goes
open · @gopinath87607 · 0 comentarios · En GitHub
BenchmarksAMD / HIPNVIDIA / CUDALinux
Descripción
> **Why this PR was recreated.** The original (#2831) was opened from the `gopinath87607/Strata` fork. Deleting that fork deleted its branches, and GitHub closes every pull request whose branch lived in a deleted fork - a closed PR whose source branch is gone cannot be reopened. This is the same work from a new fork, based on `477ac13c`, and GitHub reports that it merges cleanly with current `main` (`e88ee39d`). It replaces the closed #2831, which was closed by the fork's deletion, not by a maintainer. ## Problem and result Without a stage line, a prefill change can only be judged by the prompt's ms/token, which mixes the chunk size, the copy lane, the host and the GPU's phases. After this change, `STRATA_PREFILL_STAGES=1` (opt-in; unset is off) prints one line a stage a chunk showing where the chunk's wall went, so those parts can be told apart. It also turns the phase timer on. It is a diagnostic and claims no speedup. ## Related discussion The original pull request and its discussion: #2831 (closed when its fork was deleted; this PR replaces it). The work comes from the Maya checkout (`project-maya`), local commit `a34f57c`, with its fixes `04b8d65` (the line's events are destroyed with the prompt path) and `63b3690` (the line reads only the events its chunk recorded). Its spelling `STRATA_GLM_PREFILL_STAGES` is accepted as the same knob, so a config written for that checkout runs unchanged. ## Approach `STRATA_PREFILL_STAGES=1` turns on the chunk's stage line: **one line a stage a chunk** with where the chunk's wall went - the chunk's wall, the copy lane's own span split into what ran before it (lead), the lane itself and what ran behind it (tail), the bytes it moved and the bytes the chunk's consumers waited on (the difference is the ring's read-ahead), the hand-off a later stage of a layer split paid, the GPU's own phases over the chunk, and the experts and host time it spent. It also turns the phase timer on. Unset (the default), `stages_on()` is false: no line is printed, the phase timer is not read for it, and no extra sync is taken for it. The syncs the line does add when on wait only on copies the compute stream has already waited for (the issuer is joined when the chunk ends), so `STRATA_PREFILL_STAGES=0` beside a `--bench` A/B is how you check it costs nothing. The default path is unchanged by construction. ## Validation Tested revision: `782433f6` (one commit on `477ac13c`; not rebased, and GitHub reports it merges cleanly with current `main` `e88ee39d`). Environment: Linux, CUDA 13.2; the development branch carries `sm_86` + `sm_120` builds. **Passed:** - `-fsyntax-only` on `src/prefill/prefill.cpp` with the build's own flags - clean. - The development branch that also carries the change built fully with ninja (CUDA 13.2). **Failed:** - none observed. **Not run:** - A full `ninja -C build strata` at this exact head: only the syntax check above was run at `782433f6`; the full build was of the development branch carrying the same change. - A served run with `STRATA_PREFILL_STAGES=1` at this head: the line has not been observed here, so its output and its cost are unverified. - The set-vs-unset `--bench` A/B that prices the line. The design keeps the cost at zero by waiting only on work already joined, but that is reasoning, not a measurement. - HIP and SYCL builds: no toolchain on this machine. The change adds no backend-specific branch; it uses the same stream/event API the file already used, and the events are only created when the knob is on. ## Limits - This is a diagnostic; it makes no claim to speed anything up. - Its own cost is unmeasured here. The set-vs-unset A/B under `--bench` is the check. - The three Maya fixes are carried, but no Maya measurement is quoted for this engine. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
En el sitio
Enlaces a install, modelos, releases.