#2357 sm_70: the multi-block greedy pick (argmax_rows) as an opt-in, STRATA_ARGMAX_ROWS_SM70=1: decode +0.7-0.8% on a V100 + PG500 layer split

Description

## Title

sm_70: the multi-block greedy pick (argmax_rows) as an opt-in, STRATA_ARGMAX_ROWS_SM70=1: decode +0.7-0.8% on a V100 + PG500 layer split

Issue: none (on `main`, independent of other PRs; four lines in `argmax_rows_wanted()` and two comments)

## Summary

`argmax_rows_wanted()` turns on `argmax_rows`, the greedy pick that spreads a row over several blocks (one per 4,096 vocabulary entries, at most 128),
for compute capability 8.x only. On 7.x the verify's and the drafter's greedy pick is `sampler_greedy_kernel` (one block of 1,024 threads a row).
`argmax_rows` returns the same token as the one-block pick bit for bit; `verify_parity` checks that.

`STRATA_ARGMAX_ROWS_SM70=1` lets a compute capability 7.x card take that path. Without the variable nothing changes (the 7.x branch reads the variable
and returns false). The check is `major == 7`, so sm_72 and sm_75 (Turing) could opt in too; we measured sm_70 only.
As before, the answer is decided once per process, on the device that first asks (both cards here are sm_70).

## What changed

- `src/kernels/cuda/verify_kernels.cu`: in `argmax_rows_wanted()`, for major 7 return true when `STRATA_ARGMAX_ROWS_SM70` is set and not 0.
  The drafter (`src/core/mtp.cpp`) calls the same function and follows the same switch (unchanged).
- The comments in `argmax_rows_wanted()` and at its call in `src/core/verify.cpp` that said "sm_80 to sm_89" now mention the 7.x opt-in (second commit, comments only).
- Other backends: the change is in the `#else` branch of `#if defined(__HIPCC__)`. Both HIP builds (`STRATA_ENABLE_HIP` and `STRATA_HIP_GFX906`)
  compile the `.cu` files as HIP and take the other branch (`return false`, unchanged). The SYCL build uses its own copy
  (`sycl/src/kernels/cuda/verify_kernels.dp.cpp`), which this does not touch.
  HIP builds, without a GPU in ROCm 7.14 containers: `-DSTRATA_HIP_GFX906=ON -DCMAKE_HIP_ARCHITECTURES=gfx906` (the `mixa3607/rocm-gfx906:7.14-complete`
  image docs/AMD_HIP.md uses; `strata`) and `-DSTRATA_ENABLE_HIP=ON -DSTRATA_ENABLE_CUDA=OFF -DCMAKE_HIP_ARCHITECTURES="gfx1100;gfx1201"` (`rocm/dev-ubuntu-24.04:7.14.1-full`;
  `strata`) both compile with 0 errors. Not run (no AMD card here). SYCL not built: it does not compile the changed files (it builds its own copies of both).

## Checks (Tesla V100-PCIE-32GB + Tesla PG500-216 (sm_70, 32 GB), layer split K=24, Linux, CUDA 12.9, `-DSTRATA_EXPERIMENTAL_SM60=ON`)

How we checked: a build of this branch's first commit (`main` `a0116b2` + `9812a44`; the second commit changes comments only), the parity test on
each card, then the engine with and without the variable in ABBA order at two power limits.

- `verify_parity` on the V100 and on the PG500 prints `argmax_rows, row_top_prob_split: bitwise equal to the one-block kernels (ties, NaN, -inf, 1-8 rows on one scratch)`
  and PASS. It calls both kernels directly and does not read the variable.
- Engine: the GSQ-RCO IQ3_XXS model and pack that `setup.py --model IQ3_XXS` installs (`ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF`), with setup's
  launch settings (`--mmap-experts`, image input on, `--spec 4 --spec-min-p 0.5`), 256K context, int8 KV, layer split K=24, `STRATA_SM70_TABLE=1`,
  `STRATA_PLE_PROBE=0` (skips the start-up PLE read check and keeps direct reads). Every expert resident on both cards.
  Temperature 0, no thinking, three short questions (English, Japanese, code; generated to their caps of 400 / 600 / 400 tokens).
  A fresh engine start per run, one warm-up pass, then two passes (9 requests per start). On, off, off, on (ABBA) at 170 W and at 250 W.
  Decode tok/s (measured on the client from the stream; mean of two starts):

  | | English | Japanese | code |
  |---|---:|---:|---:|
  | 170 W off | 99.95 | 111.10 | 122.20 |
  | 170 W `STRATA_ARGMAX_ROWS_SM70=1` | 100.75 (+0.8%) | 112.00 (+0.8%) | 123.20 (+0.8%) |
  | 250 W off | 99.90 | 111.05 | 122.15 |
  | 250 W `STRATA_ARGMAX_ROWS_SM70=1` | 100.70 (+0.8%) | 111.90 (+0.8%) | 123.05 (+0.7%) |

  Per start (English / Japanese / code): 170 W on 100.6 / 111.8 / 123.0 and 100.9 / 112.2 / 123.4, off 100.0 / 111.2 / 122.3 and 99.9 / 111.0 / 122.1;
  250 W on 100.7 / 111.9 / 123.1 and 100.7 / 111.9 / 123.0, off 99.9 / 111.0 / 122.1 and 99.9 / 111.1 / 122.2.
  The two starts of a setting are within 0.4 tok/s, and every start with the variable is above every start without it.
- Replies: identical bit for bit per question in all 72 requests (8 starts x 9), on and off alike.
  MTP drafts accepted / drafted (the 9 requests of one start): 2,712 / 3,448 in 7 of 8 starts, 2,713 / 3,448 in one (250 W, on, second start).

Rechecked on main at `ed4773e` (271 commits after `a0116b2`) with the same two commits cherry-picked onto it: it builds, `verify_parity` passes on both cards, and decode (one run each way, 170 W, `--spec-min-p 0.5`, mean of the three short questions over two repetitions) is 112.9 tok/s with the variable and 112.2 without (+0.6%).

## Extra Notes

- It could be on by default for sm_70 like 8.x; we kept it opt-in because we could measure Volta only, and the same switch would also reach Turing.
- Disclosure: the change and this text were prepared with an AI assistant (Claude); the numbers are from runs on the machine above.

🤖 Generated with [Claude Code](https://claude.com/claude-code)