#1981 bench: community report, NVIDIA TITAN RTX 24 GB, IQ3_S at 262K, engine 0.1.43 (BF16_TC + SELECT_SIMT)

本文

Single-card TITAN RTX community report, supersedes #1531 (engine 0.1.40.2 → 0.1.43, produced by the shipped `tools/strata_bench.py` this time). One NVIDIA TITAN RTX 24 GB, Xeon E5-2660 v3, 126 GiB RAM, NVMe, Linux 6.1.183, driver 580.178.04.

## Suite 1 medians (IQ3_S, 262K context, one slot)

| Metric | Value |
| --- | ---: |
| Decode, short chat | 65.3 t/s (59.2-69.6) |
| Decode, agent turns | 67.5 t/s (55.2-87.9) |
| Prompt, 4K fresh | 1,140 tok/s, TTFT 3.55 s |
| Prompt, 16K fresh | 1,615 tok/s, TTFT 9.98 s |
| Agent conversation-cache hit | 68.6% |
| Expert cache | 7,946 slots resident; 94.6-98.1% decode hit during the run |

## Opt-in Turing switches

The run's config carries `STRATA_BF16_TC=1` + `STRATA_SELECT_SIMT=1`. A clean A/B was run on this machine the same day with the same engine and the same suite, one restart per arm (switches off vs on):

- 16K prompt: 1,319 -> 1,615 tok/s (+22%), TTFT 12.2 -> 10.0 s
- 4K prompt: 1,088 -> 1,140 tok/s (+4.9%)
- agent-turn first token: 4.09 -> 3.96 s
- decode: within the run-to-run spread
- replies can differ from the default path (the documented FP16/scorer rounding change)

Against the old #1531 row (0.1.40.2): ~4K prompt 1,006 -> 1,140, decode 61.4 -> 65.3 - the engine update (0.1.41-0.1.43, `ROUTE_TAIL_SKIP=7`, prefill CPU share) accounts for most of the decode part, the switches for the prompt part.

## Limitations

- Speed suite only: no needle/recall run this time (#1531 had 9/9 to 250K on the same card).
- One session with a warm expert cache; cold-box rows would sit lower.
- Desktop compositor on the same GPU; power limit 280 W (default), 251-272 W observed under decode at 1965 MHz, memory utilization 30-52%.
- The tested tree is `d7953d8f`, a working merge of `upstream/main`; its diff to `main` touches setup/serve/tests only, no engine sources.

Files: `README.md` (script template), `results.json` (per-request data), `engine-log-excerpt.txt` (the run's engine session).