Pull requests / #1948
#1948 Add files via upload
open · @kxyct · 0 comentarios · En GitHub
BenchmarksNVIDIA / CUDAModels & quantsWindows
Descripción
```markdown Results-only submission (no engine changes): `bench/results/2026-10-11-community-rtx-5070ti-laptop-12gb/`. The two 5070 Ti rows already in the index are desktop cards. This is the **12 GB laptop** part, and I believe it is the smallest-VRAM row in the table running a 262K context. - **Hardware**: RTX 5070 Ti Laptop GPU 12,227 MiB (driver 616.64, 140 W limit, AC), Ryzen 9 9955HX (Windows reports 8C/16T for this part), 61.7 GB RAM, YMTC PC411 1 TB NVMe, Windows 11 build 26100. - **Software/model**: engine 0.1.41 release (CUDA 13.0); Flash-Next IQ3_XXS, 262,144 context, `--kv int8 --kv-resident 32768 --vram-reserve-mib 700 --expert-cache auto --spec 4`, plus this machine's own `--calibrate` values. Expert cache filled 1585/1585 slots (2.61 GiB) at startup. - **Method**: 3 fresh runs per tier (unique nonce first, server reports `0 reused`), 256-token cap. Prefill, decode and the expert-cache hit rate are read back out of the engine's own log lines, not timed by the harness. Script and per-run JSON in the folder. - **Results (medians)**: 4K: prefill 788.5 / decode 63.8 tok/s; 32K: 901.4 / 64.1; 128K: 886.8 / 64.4; 192K: 869.4 / 61.2. Expert cache hit 56-58 % throughout. - **The point**: decode is **flat** from a 4K prompt to a 192K prompt (63.8 → 61.2 tok/s). Context length on this card costs prefill *time* (231 s for a 197K prompt at ~870 tok/s), not decode speed. I put this down to `--kv-resident`: the same card at 204800 **without** it held only 207 expert slots and decoded at 2.3 tok/s. - **Limitations**: single laptop, English prose prompts only (prefill is content-sensitive), no clock or power capture, non-streaming so no separate TTFT. The 204800-without-streaming figure is 2 replies from an earlier session — a direction, not a measurement. ```
En el sitio
Enlaces a install, modelos, releases.