#2464 Qwen3.6-35B-A3B on a V100 (sm_70): decode falls from 154 to 56 tok/s between 256 and 16K tokens; llama.cpp holds ~90 on the same file

说明

Tested v0.1.44 (ed4773e) with Qwen3.6-35B-A3B UD-Q4_K_XL (setup's pinned file from unsloth/Qwen3.6-35B-A3B-MTP-GGUF @ 5bc3e238, SHA-256 checked) on one Tesla V100-PCIe-32GB (i7-13700KF, 16 GB RAM), CUDA 12.9, built with `-DSTRATA_EXPERIMENTAL_SM60=ON` and `libstrata_kq.so` next to the engine. Config as setup writes it: `--native <gguf> --max-context 131072 --prefill auto --mtp 2`. Every routed expert fit in VRAM (18.75 GiB, after "the packed Q8_0 decode copies (1.78 GiB) given back"). Greedy, 256-token answers, the same benchmark.py prompts as our other V100 reports, 3 runs per size, medians. Decode tok/s, with time to first token in brackets:

| prompt | `--mtp 2` | `--mtp 0` (decode) | llama.cpp, same file |
|---|---|---|---|
| 256 | 153.8 (0.23 s) | 111.3 | 92.2 (0.55 s) |
| 1,000 | 154.2 (0.60 s) | 106.3 | 92.1 (1.54 s) |
| 4,096 | 114.8 (2.38 s) | 86.3 | 91.3 (6.01 s) |
| 16,384 | 56.5 (10.56 s) | 59.2 | 87.3 (25.38 s) |

llama.cpp = mainline 5e03bdd built for sm_70, `-ngl 99 -fa on`, the identical request bodies (thinking off), streamed.

Short prompts are great on this card: MTP drafts are accepted 69% of the time and decode is ~1.7x llama.cpp's. But decode falls with context with and without MTP, while llama.cpp's barely moves, so at 16K Strata decodes ~35% slower than llama.cpp on the same card (it still reads the prompt 2.4x faster). Our guess is the full-attention layers' decode on sm_70, but we haven't profiled it, and we know Volta is experimental. HumanEval (thinking off) was 155/164 in 4.0 min, the same score as our earlier llama.cpp test of this model.

Happy to run variants (`STRATA_Q36_KEEP_PACKED=1`, longer contexts) if that helps.