Pull requests / #3245

#3245 [Engine] log which batch-row switches are on at start-up

open · @yi-sudo · 0 comentários · No GitHub

AMD / HIPNVIDIA / CUDAModels & quantsDocumentationWindows

Descrição

## Problem and result

`STRATA_BATCH_ATTN_ROWS` and `STRATA_BATCH_GDN_GROUPS` (merged from #2234 in f629fb0) are silent: nothing in the engine log says whether they were picked up, and attention quietly keeps its per-row launches when a slot's KV streams. In #2234 you mentioned a start-up line showing which switches are on and when they fall back would be welcome; this adds it.

Off (the default), nothing is printed and the engine runs exactly as before. With either switch on, the verifier prints one line after its existing `batch windows of up to N sequences` line (once per stage of a layer split):

```
strata verify: batch rows in one launch: attention on (STRATA_BATCH_ATTN_ROWS), GDN on (STRATA_BATCH_GDN_GROUPS); the KV streams (--kv-resident), so attention keeps one launch per row
```

The last clause appears only when attention is on and a slot streams its KV (`kv_mode == 1`).

## Related discussion

https://github.com/Niko1221/Strata/pull/2234#issuecomment-6231123506

## Approach

- The two env reads moved from three function-local statics in `src/core/verify.cpp` to two file-scope constants next to the other batch switches (`g_batch_self_commit`, `g_qsa_slots`). The parsing is the same as before: set and `atoi != 0`.
- `Verifier::init_slots` prints the line, so it sits with the existing batch-window line and can check each slot's QSA `kv_mode`.
- `docs/BATCHING.md`: one paragraph with the example line.
- The SYCL tree has its own `verify.cpp` without these switches, so it is untouched.

## Validation

**Passed:**
- CUDA build: `cmake --build build` at 0f31ba6e (on upstream e88ee39d), Ubuntu, CUDA 13.0, sm_120. 462/462 targets, no errors and no new warnings in `verify.cpp`.
- `./build/batch_rows_parity`: all bitwise.
- `ctest` in `build`: everything passed except `expert_parity`, which needs `pack/full/experts.bin`, a pack I don't have locally.
- HIP build of the `strata` target: ROCm 7.10 pip SDK (`rocm[libraries,devel]==7.10.0a20251120`), `-DCMAKE_HIP_ARCHITECTURES=gfx1201`. 157/157 targets with no errors. Compile only; I have no AMD GPU.
- Engine start with both switches unset, on an RTX PRO 6000 Blackwell (IQ3_S, `--parallel 4`): the log shows only the existing `batch windows of up to 4 sequences` line and no new line.

**Failed:**
- none observed

**Not run:**
- Engine start with a switch set. The line is not checked on hardware, but the arithmetic paths are unchanged from #2234.
- HIP and SYCL execution (no hardware). SYCL build not run because its sources are not touched.

## Limits

- The message only reports what was configured. It does not count how many windows actually used the batched path.

No site

Links install, modelos, releases.