#3616 [Benchmark] 2x Tesla P40, Flash-Next IQ3_XXS at 262K: #1469 on a second P40 pair, 0.1.44.1, strata_tune sweep, --batch 2

Descripción

Problem and result: results-only report, no code changes. Adds `bench/results/2026-10-16-community-2x-p40-iq3_xxs-262k/`.
On 2x Tesla P40 (sm_61) with an EPYC 7302, Flash-Next IQ3_XXS at 262,144 context, layer split 26, single-stream
decode on 1,024-token answers at a 4K prompt went 17.5 / 18.0 / 18.1 tok/s on 0.1.40.3, to 35.4 / 32.6 / 33.2 on
0.1.40.4, to 36.0 / 35.7 / 40.3 on 0.1.44.1 with `STRATA_ROUTE_TAIL_SKIP=0` and `STRATA_PREFILL_CPU_SHARE=0` held
off per `docs/OLDER_GPUS.md`. Same machine, model, config and prompts throughout; placement unchanged
(22,961 to 22,964 resident experts, 543 MiB free).

Also in the report:
- container vs bare metal on 0.1.40.3, same config: 17.8 / 17.8 / 18.0 vs 16.6 / 17.0 / 16.8 tok/s
- a per-request `strata_tune` sweep on the live server: `spec_min_p` 0.7 raised acceptance (0.61 to 0.79) and won the
  sweep (median 40.2 vs 36.5), but deployed as the running value it measured 36.5 / 41.3 / 35.3 against 36.0 / 35.7 / 40.3,
  so it was reverted; a fixed `pcie_frac` 0.34 cost about 4 tok/s against auto (0.36) in every round
- two direct clients with `--batch 2 --batch-groups 2`: a pair of 4K requests finishes in 33.4 s vs 53.4 s on the FIFO

Related discussion: #1469 (the regression and fix this confirms), #1799 (the 2x P40 IQ3_S report whose
`--pcie-frac 0.34 --spec-min-p 0.70` pair is measured here).

Validation:
  Passed: engine 0.1.40.3 (`d5ea713`), 0.1.40.4 (`6674a00`), 0.1.44.1 (`296b59d2`), each a Docker image built from
  Strata's Dockerfile with the Pascal changes listed in `Dockerfile.p40`; Ubuntu 22.04.5, driver 580.178.04, CUDA
  12.9.1 base; model revision `ed59f92`, SHA-256 verified. Every request and its engine timings are in `raw/*/runs.jsonl`.
  Greedy 20-prompt compares between builds: final answer identical 16/20 (0.1.40.3 to 0.1.40.4), 14/20 (to 0.1.44.1),
  13/20 (gate 0.7), no answer wrong.
  Failed: one request, HTTP 502 when the local proxy swapped models mid-sweep (`raw/tune/20261016T181038Z`); that run
  is included as a partial round.
  Not run: the standard protocol (`strata bench run`), any 0.2.x engine, needle or quality suites, vision.

Limits: own harness, not the standard protocol. Strata single-stream runs are sampled with the model card's
thinking-mode values (temperature 1.0) through a local proxy, so the absolute numbers are not comparable with
standard results; the comparisons between versions are within one harness. One or two runs per standard-pass cell.
The engine log excerpt is bounded by engine starts because only start lines carry timestamps. The report was
prepared with an AI assistant (Claude Code) from the raw files, and every number was checked against them.

Thank you to Niko for Strata. It is the reason a 125B model runs on two 2016 cards at all, and the #1469 fix landed
about 75 minutes after the report. Thanks also to @paulhothersall, whose bisection in #1469 found the commit and the
line that was costing Pascal half its decode rate, and to @gun-powder, whose #1799 numbers gave me a second P40 pair
to measure against. I wrote up the week this took here, with full credit to the project:
https://michaelbrewer.me/2026/10/21/the-number-was-a-version/

🤖 Generated with [Claude Code](https://claude.com/claude-code)