#2854 bench: community report — 2x Tesla P40 (PCIe 3.0 x4), Flash-Next IQ2_XS, 262K

Beschreibung

Results-only submission (no engine change): a `tools/strata_bench.py --long` report on two Tesla P40s.

**Hardware.** 2× Tesla P40 24 GB, each on a **PCIe 3.0 x4** link (one on a mini PC's OCuLink port, one through an M.2-to-OCuLink adapter, external enclosures), AMD Ryzen 7 8845HS, 59.7 GiB RAM, Linux 7.0, driver 580.178.04.

**Software.** Strata 0.1.44.1, engine built from source with CUDA 12.8 for sm_61/sm_70 (Docker, `GPUS=0,1`, `LAYER_SPLIT=auto`), Flash-Next IQ2_XS, 262,144 context, KV int8 (32,768 resident), vision on.

**Results (medians of 3).**

| Test | Prompt tok/s | Decode tok/s | TTFT |
|---|---:|---:|---:|
| short chat | — | 50.3 | 0.68 s |
| 4K fresh | 350.9 | — | 11.5 s |
| 16K fresh | 503.0 | — | 32.0 s |
| 32K fresh | 638.4 | — | 50.3 s |
| agent session (5 turns) | cache hit 68.8 % | 50.7 | 9.0 s |

**Notes.**
- The layer split puts 12,288 experts on each card: **24,576 / 24,576 experts resident in VRAM** (33.0 GiB) despite the x4 links.
- Same model on one Tesla V100 32 GB in the same box (our own runbook measurements, not this tool): decode ~68 tok/s, prompt ~1,255 tok/s at 31K. On a real agent task the P40 pair gave the same answers about 26 % slower, the prompt path being the gap.
- Limitations: speed only, no quality suite; background load and power limits are described by hand in the README (default 250 W per card).

A separate issue follows about the vision encoder, which setup did not rebuild for sm_61 after the engine was.

🤖 Generated with [Claude Code](https://claude.com/claude-code)