#3082 Community benchmark: Strata 0.1.41 vs llama.cpp's new MoE expert cache (--moe-cache-mib), same Coder IQ1_M files, V100-PCIe 32 GB + 16 GB RAM

Description

llama.cpp added a VRAM expert cache in October (`--moe-cache-mib`, PR #29887), which is close to what Strata does, so I ran both on the same card with the **same two Coder IQ1_M GGUF files** and the same client.

**Hardware:** Tesla V100-PCIe 32 GB (sm_70), PCIe Gen3 x4 (3.3 GB/s), i7-13700KF, 16 GB RAM, model on NVMe. Strata 0.1.41 (resident RAM mode, `STRATA_SM70_TABLE=1`, `--spec 4` MTP). llama.cpp b11718 (`547208dca`), sm_70, CUDA 12.9.

**Method:** one streaming client for both. Temperature 0, presence_penalty 0, thinking off, 256 tokens out. Prompts of 256/1000/4096 tokens, each with its own first line so neither engine can reuse a prompt. A cold start per setup, 3 runs per size, and the repeats agree within 1%.

| setup | experts on the card | write tok/s (256/1000/4096) | first token, 4096-token prompt |
|---|---|---|---|
| llama.cpp, whole model on the GPU (no cache) | 100% | 42.9 / 42.4 / 41.7 | 9.8 s |
| llama.cpp, `--cpu-moe --moe-cache-mib 24000` | 100% (cache) | 35.2 / 37.5 / 37.3 | 13.2 s |
| Strata, no vision, `--spec 2` without the MTP drafter | 100% | 70.9 / 66.7 / 60.4 | 3.4 s |
| Strata, everyday (vision on the GPU) | 95% + 5% RAM | 92.4 / 88.5 / 84.8 | 3.4 s |
| Strata, no vision | 100% | 101.2 / 98.5 / 98.0 | 3.4 s |

- Like for like, Strata writes about 2.4× faster and reads prompts about 2.9× faster. Even without the MTP drafter it writes about 1.5× faster. llama.cpp has no drafter for this model: no MTP head ships in llama.cpp format.
- llama.cpp's cache costs it about 10% against loading normally: 50 graph splits per token against 2.
- llama.cpp cache-size sweep: flat down to 20 GB (83% of the experts, 99.1% hit rate), then a cliff. At 16 GB the hit rate is 96.4% but the traffic is 4×, and writing drops to about 23 tok/s. At 8 GB it falls to about 5 tok/s.
- One model, one card. llama.cpp does better elsewhere (e.g. #2464).

Full data, scripts and per-run replies: https://github.com/christopherrobertbrooks-tech/strata-vs-llamacpp