#3189 [Benchmark] RTX 5090: engine 0.1.45 vs 0.1.45.3, prompt reading +2.7 to +9.8 %

Description

## Problem and result

Two folders with measurements from one RTX 5090, taken with the standard bench suite
(`tools/strata_bench.py`): the install's own report for the current engine, and an **engine A/B**
between the 0.1.45 release build and a 0.1.45.3 local build of the same checkout.

The result: **prompt reading is faster with 0.1.45.3, and the gain grows with the prompt length** —
+2.7…+4.7 % at 4K, +7.6…+8.5 % at 16K and +8.4…+9.8 % at 32K — with time to first token down 2-9 %
everywhere (chat, 4K, 16K, 32K and the agent turns). The conversation cache is unchanged (69.3 %).

**Decode is not established.** The two new runs disagree with each other by 8.5 % (chat decode) and
30 tok/s (agent decode), and the old value lies between them, so the report claims no decode change
in either direction.

## Related discussion

none (a results-only submission; no engine change is proposed)

## Approach

The runs were done in the order **old, new, new** on purpose. This machine moves 8-28 % between
engine starts at an unchanged setting (#1332), so one old-vs-new pair cannot separate drift from a
real difference; the two new runs bracket the old one, and a real effect shows up in both of them
while drift shows up as a disagreement between them. That is what the bracket says: the prompt
figures agree in both new runs, the decode figures do not.

The two sides differ only in the engine binary: the same model, the same run configuration
(byte-identical `args`), the same six `STRATA_*` switches (all six exist in the 0.1.45 tree as
well), the same machine. The configs are in the folder, together with the script that started the
server, ran the suite and stopped it, and with the engine logs of both sides.

## Validation

**Passed:**

- `python tools/strata_bench.py --url http://127.0.0.1:<port> --api-key <key> --runs 3 --long`
  against each of the three servers, on commit `3fa1b524` (v0.2.0-beta-7), Windows 11, driver
  617.42, RTX 5090 32 GiB, i7-12700KF, 95.8 GiB RAM, Python 3.13.12. The tool's own reports are
  in the folders (`README-engine-*.md`) with every request in the `results*.json`.
- the comparison the tool prints for the two pairs:
  `python tools/strata_bench.py --compare <old> --compare-only <new>`.

**Failed:**

- none observed

**Not run:**

- **A control that separates the code from the compiler.** The old side is the official 0.1.45
  release build (CUDA 13.0, sm_75/86/89/120 + PTX); the new side is a local build of the same
  source with **CUDA 13.1** for sm_120 only. A part of the prompt-path gain could therefore be the
  toolkit rather than the code. The same source built with 13.0, or the 0.1.45 source built with
  13.1, would settle it; neither was run.
- **No correctness check.** No needle test, no HumanEval, no tool-call check, and the answers of
  the two engines were not compared token by token. 0.1.45.3 changes the default prompt path on
  sm_120 (fused int8 prompt experts, fused gate/up SwiGLU), which the project's own notes describe
  as slightly different answers at the same quality; this report does not measure that difference.
- **One machine, one model, one configuration** — a 24 GiB card, or another quantisation, may land
  elsewhere.
- Background load and power limits were not recorded; the drive model is not in the report.

## Limits

The A/B folder is a report, not a change: no engine, tool or documentation file is touched, and the
README's headline performance claims are left alone. The `docs/COMMUNITY_BENCHMARKS.md` index is
not edited here — that entry is the maintainer's to add if the report is accepted.

The bench tool's own "start the server" path raises a `NameError` (`port` is undefined at
`tools/strata_bench.py:990`), so the server was started by hand and the bench pointed at it with
`--url`; that is issue #3188, reported separately. With `--url` the tool does not write
`engine-log-excerpt.txt`, so the engine logs are included here in full instead.