#2981 Community benchmark: RTX PRO 4500 Blackwell, Qwen3.6-35B-A3B UD-Q4_K_XL, engine 0.1.45 (official suite, --long, MTP 2)

Descripción

One folder under `bench/results/`, produced by `tools/strata_bench.py --long` on v0.1.45 (three runs per test, medians).

Machine: one RTX PRO 4500 Blackwell (32 GB, 200 W, GPU memory 14,000 MHz; the second card of the box was idle), Threadripper 7960X, 61 GiB RAM, Linux, driver 595.91.07, CUDA 13.2.
Model: `Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf` (Unsloth, with the MTP block), own `qwen35` path, every main-layer expert resident in VRAM, 65,536 context, `--mtp 2`, vision on.

| Test | Result (median) |
| --- | --- |
| short chat decode | 259.5 tok/s |
| 4K prompt | 6,968 tok/s, first token 0.58 s |
| 16K prompt | 6,123 tok/s, first token 2.63 s |
| 32K prompt | 4,855 tok/s, first token 6.62 s |
| agent session, follow-up turns | decode 161.7 tok/s, first token 1.50 s, cache hit 0 % |
| `--clients 4` (separate run, 32K context, 4 slots) | 403.9 tok/s total |

Things worth knowing, all measured on this machine:
- **MTP drafts only run with explicit zero penalties.** `Sampling::needs_logits()` sends any request with a repetition, frequency or presence penalty to the one-token path, and a request next to others also gets no drafts. With the server's default sampling the first run of this suite had `drafts accepted 0 of 0` everywhere (short chat 183.6 tok/s, agent 137.3 tok/s). This entry has `temperature 0, presence_penalty 0, repetition_penalty 1.0, frequency_penalty 0` in the config's `sampling` block, and the suite shows `drafts accepted 1207 of 1786` (68 %). Decode over context (3 runs, medians): `--mtp 0` / `2` / `4` = 166.5 / 246.9 / 240.2 tok/s at 1K and 113.1 / 125.0 / 109.9 tok/s at 16K. Prompt speed is unchanged by drafts.
- **`--batch-mtp` with 4 sessions aborts** (`--max-context 32768`, `--mtp 2`, `--batch-mtp`, parallel 4; the MTP block had 163 of 256 experts in VRAM, the rest in the CPU pool): the first multi-session step ends with `strata-q36: cudaMemcpyAsync(ecpu, cpu_out, ...) failed: operation failed due to a previous error during capture (src/qwen35/q36_engine.cu:2284)`, then `terminate called ... native MMVQ launch: operation failed due to a previous error during capture`. With `STRATA_Q36_NO_MTP_CPU=1` it runs (65 % of drafts accepted), but that switches every layer to the hybrid mode (prompt speed halves: 4K 3,173 instead of 5,487 tok/s) and `--clients 4` gave 364.9 tok/s against 403.9 without `--batch-mtp`, so I cannot tell whether `--batch-mtp` itself helps. I did not look further into the cause.
- For comparison, the same suite without drafts on a RTX PRO 4000 Blackwell SFF (24 GB, 70 W; 96 of 256 experts per layer in VRAM, the rest in host memory): short chat 82.3 tok/s, 4K prompt 1,770 tok/s, 32K first token 22.1 s.
- Speed only; no correctness check. The agent cache hit is 0 % (each turn re-reads the earlier turns).

*Written with AI assistance (Claude); numbers reviewed by me.*

🤖 Generated with [Claude Code](https://claude.com/claude-code)