Pull requests / #1948

#1948 Add files via upload

open · @kxyct · 0 comentarios · En GitHub

BenchmarksNVIDIA / CUDAModels & quantsWindows

Descripción

```markdown
Results-only submission (no engine changes): `bench/results/2026-10-11-community-rtx-5070ti-laptop-12gb/`.

The two 5070 Ti rows already in the index are desktop cards. This is the **12 GB laptop** part, and I
believe it is the smallest-VRAM row in the table running a 262K context.

- **Hardware**: RTX 5070 Ti Laptop GPU 12,227 MiB (driver 616.64, 140 W limit, AC), Ryzen 9 9955HX
  (Windows reports 8C/16T for this part), 61.7 GB RAM, YMTC PC411 1 TB NVMe, Windows 11 build 26100.
- **Software/model**: engine 0.1.41 release (CUDA 13.0); Flash-Next IQ3_XXS, 262,144 context,
  `--kv int8 --kv-resident 32768 --vram-reserve-mib 700 --expert-cache auto --spec 4`, plus this
  machine's own `--calibrate` values. Expert cache filled 1585/1585 slots (2.61 GiB) at startup.
- **Method**: 3 fresh runs per tier (unique nonce first, server reports `0 reused`), 256-token cap.
  Prefill, decode and the expert-cache hit rate are read back out of the engine's own log lines, not
  timed by the harness. Script and per-run JSON in the folder.
- **Results (medians)**: 4K: prefill 788.5 / decode 63.8 tok/s; 32K: 901.4 / 64.1; 128K: 886.8 / 64.4;
  192K: 869.4 / 61.2. Expert cache hit 56-58 % throughout.
- **The point**: decode is **flat** from a 4K prompt to a 192K prompt (63.8 → 61.2 tok/s). Context
  length on this card costs prefill *time* (231 s for a 197K prompt at ~870 tok/s), not decode speed.
  I put this down to `--kv-resident`: the same card at 204800 **without** it held only 207 expert slots
  and decoded at 2.3 tok/s.
- **Limitations**: single laptop, English prose prompts only (prefill is content-sensitive), no clock
  or power capture, non-streaming so no separate TTFT. The 204800-without-streaming figure is 2 replies
  from an earlier session — a direction, not a measurement.
```

En el sitio

Enlaces a install, modelos, releases.