#2328 Community benchmark: Tesla P40 (Windows Server 2022 VM), four models at 256K context

说明

Community benchmark: one Tesla P40 (sm_61, experimental CUDA 12 engine) in a Windows Server 2022 VM, four models at a 262,144-token context, measured on 2026-10-13 with `BENCH.bat` (`tools/strata_bench.py`, suite 1, 3 runs per test).

Results only: four folders under `bench/results/` and their four rows in `bench/results/COMMUNITY.md`, no code changes.

## Machine

- GPU: 1x Tesla P40 24 GB (22 GiB usable), passed through to the VM, TCC mode, driver 582.78; power limit raised to 250 W (`nvidia-smi -pl 250`) for all four runs
- CPU: Xeon E5-2630 v3 @ 2.40 GHz (8 cores / 16 threads, AVX2, no AVX-512, DDR4-1866); the VM has 16 vCPUs on 1 socket and 98 GB RAM
- Hypervisor and OS: Hyper-V; the guest is Windows Server 2022 (10.0.20348)
- Storage: model files on a VHDX attached to the VM as an extra drive, on a Samsung 990 PRO 2 TB NVMe in a PCIe-to-M.2 adapter card (host-side Samsung Magician: 3,408 MB/s sequential read, 469K random-read IOPS). The VM had a checkpoint, so the VHDX was a differencing disk (reads go through the checkpoint chain)
- Strata 0.1.43 from the main zip (not a git checkout), ready-made CUDA 12 engine
- Background on the VM during the runs: Task Manager, a PowerShell window with `nvidia-smi`, and a browser with Strata's Monitor page open; the Ollama service on the VM was stopped

## Configuration

Every model was set up with `--context 262144 --vision yes`, 8-bit KV with KV streaming, then tuned with `START-HERE.bat --calibrate` at 250 W before its benchmark. The calibrated values differ per model and are in each report's launch line (all chose `--pcie-frac 1.00` and `--adapt-every 1`; `--spec-min-p` 0.50-0.70, `--adapt-swaps` 80-160).

## Results (medians; ranges and per-request data in each folder)

| Model | Chat decode tok/s | Agent decode tok/s | 4K prompt tok/s | 16K prompt tok/s | Agent turn TTFT (s) | Suite time |
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
| Coder IQ1_M | 26.8 | 31.3 | 358.7 | 405.5 | 8.91 | 373 s |
| IQ3_S (ISTA-DASLab) | 21.5 | 26.2 | 344.7 | 396.4 | 9.91 | 420 s |
| UD-IQ4_XS (Unsloth) | 11.9 | 15.8 | 274.2 | 355.8 | 12.83 | 838 s |
| UD-Q4_K_XL (Unsloth) | 9.4 | 11.4 | 244.5 | 333.1 | 14.44 | 949 s |

## Observations

- The two native packs (Coder, IQ3_S) hold all their experts in RAM; the two Unsloth GGUF-in-place packs run with the RAM budget (55 / 70 GiB) and are about half as fast at decode on this CPU.
- Unsloth packs, agent session: the follow-up turns read their new tokens at only 38-73 tok/s (run 1 of each), against 274-356 tok/s for fresh 4K/16K prompts in the same run; the native packs read the same agent turns at 226-335 tok/s. Their first fresh 4K/16K prompt was also slow (UD-IQ4_XS 86.5 / 224.3, UD-Q4_K_XL 84.2 / 195.9 tok/s) before the later runs reached full speed. Not investigated further.
- The UD-IQ4_XS config carries `STRATA_WATCHDOG_S=600` (shown in its report) from an earlier 512K attempt; it only changes the stall watchdog's limit, and no stall happened in these runs.

## Edits to the generated files

`results.json` kept the absolute paths of the model files and the image encoder under the data folder next to the checkout (`<drive>\...\Strata-data\models\...`): the redactor replaces the checkout (`<strata>`), home, temp, user and host, but not that folder. I replaced those five paths with `<strata>\..\Strata-data\...` by hand; nothing else was changed.

## Not tested

Correctness (speed suite only), `--long`, `--clients`, thinking on, contexts above 256K (a 512K start with YaRN stalled at the first prompt chunk on this card; that goes into a separate issue).

🤖 Generated with [Claude Code](https://claude.com/claude-code)