Pull requests / #2004
#2004 Community benchmark: RTX 4060 8 GB + 32 GB RAM (Coder IQ1_M, Flash-Next IQ2_XS low-RAM)
open · @a5372935 · 0 comments · View on GitHub
BenchmarksNVIDIA / CUDAModels & quantsDocumentationWindows
Description
Results-only community report in `bench/results/2026-10-11-community-rtx-4060-8gb/`, following `docs/COMMUNITY_BENCHMARKS.md`. **Hardware:** RTX 4060 8 GB (also drives the desktop, PCIe 3.0 x8), Core i7-8700 (AVX2, no AVX-512), 31.9 GB RAM, NVMe SSD, Windows 10 22H2, driver 610.88. **Software:** Strata `61b3fb5` (engine 0.1.42), official prebuilt Windows engine. **Configurations:** Coder IQ1_M with `--resident-experts`, and the original Flash-Next IQ2_XS with `--mmap-experts` (low-RAM mode, since this machine has less than the suggested 48 GB). Both at 32,768 context, `--kv int8`, MTP, vision on the CPU, `--vram-reserve-mib 300`. | | Coder IQ1_M | Flash-Next IQ2_XS | |---|---|---| | Decode, ~82-token prompt (median of 3) | 15.5 tok/s | 20.7 tok/s | | Prompt, ~9.7K fresh tokens | 134.9 tok/s (median of 3) | 31.3 / 35.7 tok/s (2 of 3; one stalled) | | Needles 8K / 28K | 3/3, 3/3 | 3/3, 3/3 | | 60-minute soak | 168/179 OK (11 JSON answers looped to the cap → 502) | 74/74 OK | **Two observations that may be useful:** - IQ2_XS stalled once while reading a ~9.7K-token prompt (357 MiB RAM available, heavy paging of expert file pages). The watchdog restarted the engine and the run continued. The startup log had warned "0 MiB of VRAM free … add --vram-reserve-mib 812". - On this 8 GB display GPU, a Coder restart with ~70 MB less free VRAM made the automatic prompt chunk drop from 512 to 256 tokens, and ~9.7K-token prompt throughput fell by 43% (131–136 → 76–78 tok/s), with decode unchanged. **Limitations:** one machine, one engine version, 32K context only (no 128K), and IQ2_XS only in low-RAM mode. The stall dump can be shared on request. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Related on strata.com
Editorial links to help you install, pick models, or read release notes — not part of the upstream thread.