Pull requests / #3245
#3245 [Engine] log which batch-row switches are on at start-up
open · @yi-sudo · 0 comentários · No GitHub
AMD / HIPNVIDIA / CUDAModels & quantsDocumentationWindows
Descrição
## Problem and result `STRATA_BATCH_ATTN_ROWS` and `STRATA_BATCH_GDN_GROUPS` (merged from #2234 in f629fb0) are silent: nothing in the engine log says whether they were picked up, and attention quietly keeps its per-row launches when a slot's KV streams. In #2234 you mentioned a start-up line showing which switches are on and when they fall back would be welcome; this adds it. Off (the default), nothing is printed and the engine runs exactly as before. With either switch on, the verifier prints one line after its existing `batch windows of up to N sequences` line (once per stage of a layer split): ``` strata verify: batch rows in one launch: attention on (STRATA_BATCH_ATTN_ROWS), GDN on (STRATA_BATCH_GDN_GROUPS); the KV streams (--kv-resident), so attention keeps one launch per row ``` The last clause appears only when attention is on and a slot streams its KV (`kv_mode == 1`). ## Related discussion https://github.com/Niko1221/Strata/pull/2234#issuecomment-6231123506 ## Approach - The two env reads moved from three function-local statics in `src/core/verify.cpp` to two file-scope constants next to the other batch switches (`g_batch_self_commit`, `g_qsa_slots`). The parsing is the same as before: set and `atoi != 0`. - `Verifier::init_slots` prints the line, so it sits with the existing batch-window line and can check each slot's QSA `kv_mode`. - `docs/BATCHING.md`: one paragraph with the example line. - The SYCL tree has its own `verify.cpp` without these switches, so it is untouched. ## Validation **Passed:** - CUDA build: `cmake --build build` at 0f31ba6e (on upstream e88ee39d), Ubuntu, CUDA 13.0, sm_120. 462/462 targets, no errors and no new warnings in `verify.cpp`. - `./build/batch_rows_parity`: all bitwise. - `ctest` in `build`: everything passed except `expert_parity`, which needs `pack/full/experts.bin`, a pack I don't have locally. - HIP build of the `strata` target: ROCm 7.10 pip SDK (`rocm[libraries,devel]==7.10.0a20251120`), `-DCMAKE_HIP_ARCHITECTURES=gfx1201`. 157/157 targets with no errors. Compile only; I have no AMD GPU. - Engine start with both switches unset, on an RTX PRO 6000 Blackwell (IQ3_S, `--parallel 4`): the log shows only the existing `batch windows of up to 4 sequences` line and no new line. **Failed:** - none observed **Not run:** - Engine start with a switch set. The line is not checked on hardware, but the arithmetic paths are unchanged from #2234. - HIP and SYCL execution (no hardware). SYCL build not run because its sources are not touched. ## Limits - The message only reports what was configured. It does not count how many windows actually used the batched path.
No site
Links install, modelos, releases.