Pull requests / #2852
#2852 Add RTX PRO 4000 Blackwell benchmark reports: UD-IQ4_XS tuning A/B and UD-Q4_K_XL
open · @Echokovo · 0 comments · View on GitHub
BenchmarksSetup & installNVIDIA / CUDAModels & quantsDocumentationLinux
Description
Results-only submission of four folders under `bench/results/`, all measured on one machine with `tools/strata_bench.py` (suite 1, 3 runs per test) plus an interleaved A/B driver for the tuning part. No engine changes. ## Hardware and software - GPU: NVIDIA RTX PRO 4000 Blackwell, 24 GiB VRAM - CPU: Intel Core Ultra 9 285K (24 cores / 24 threads, no AVX-512), 125.3 GiB RAM - OS: Linux 6.17.0-1033-oem, driver 595.99.02, CUDA 13.2 - Strata: project 0.1.44.1, commit 296b59d2, engine 0.1.44.1 ## What is in each folder | Folder | Model | What it is | | --- | --- | --- | | `2026-10-15-...-unsloth-ud-iq4-xs` | UD-IQ4_XS | baseline before the tuning switches: `STRATA_ROUTE_TAIL_SKIP=0`, no `STRATA_ADAPT_OVERLAP`, `--prefill auto` | | `2026-10-16-...-unsloth-ud-iq4-xs` | UD-IQ4_XS | same machine after the switches: tail-skip at its 0.1.42+ CUDA default, `STRATA_ADAPT_OVERLAP=1`, `--prefill auto:16384` | | `2026-10-16-tuning-tail-skip-adapt-overlap-rtx-pro-4000` | UD-IQ4_XS | the interleaved A/B that picked the switches (`ab*.py`, `arms/` configs and logs, raw rows), plus the two reports above as its before/after | | `2026-10-17-...-unsloth-ud-q4-k-xl` | UD-Q4_K_XL | the larger quantization at the same settings | The two UD-IQ4_XS community reports are not duplicates: the first is the "before", the second the "after", and the tuning folder is what links them. ## Headline numbers (median, ranges in the reports) UD-IQ4_XS, before -> after the switches (decode tok/s pooled over 4 requests): 77.9 -> 89.1 (**x1.144** from `STRATA_ROUTE_TAIL_SKIP` alone), `STRATA_ADAPT_OVERLAP=1` adds x1.015. Full suite after the change: short chat 89.1 tok/s, 4K prompt 2465.8 tok/s, 16K prompt 2874.4 tok/s, agent decode 96.2 tok/s, cache hit 68.8 %. UD-Q4_K_XL at the same settings: short chat 77.8 tok/s, 4K prompt 2142.1 tok/s, 16K prompt 2916.5 tok/s, agent decode 84.4 tok/s, cache hit 68.4 %. So the bigger quantization costs about 13 % decode and 13 % on a 4K prompt, while 16K prefill is the same within noise. All runs: 131072 context, `--kv int8 --kv-resident 32768`, `--spec 4 --spec-min-p 0.5`, `--mtp rt`, `--pool-workers 15`, thinking off, temperature 0, one slot. Configs in the folders are the install's own, unchanged. ## Limitations Speed only: no correctness check, no KL check, no needle test. One machine, one GPU, two rounds per tuning arm (greedy), background load and power limits not recorded. `STRATA_ROUTE_TAIL_SKIP` changes the output slightly by design (teacher-forced KL 0.003-0.006, top-1 agreement 97.7-98.2 % per `docs/DETAILS.md`). Home folder paths in the tuning folder are replaced by `/home/user`; no API keys, host names or IP addresses are included.
Related on this site
Quick links to related pages.