#6279 [Engine] RTX 4090 (Windows, CUDA 13): decode about 10-20% slower on 0.1.62 vs 0.1.60 with UD-IQ4_XS-v31 (prompt reading unchanged)
closed · @m4rantza · 5 Kommentare · Auf GitHub
BenchmarksMulti-GPUAMD / HIPNVIDIA / CUDAModels & quantsDocumentationWindows
Beschreibung
### Area
Engine
### Desktop app, Terminal or source?
Terminal
### What happened
After updating from 0.2.15 to 0.2.16, decode on Qwen3.8-Flash-Next UD-IQ4_XS-v31 is about 10-20% slower. Prompt reading is unchanged, output is fine, and the sanity checks pass 12/12.
0.1.62 is the first engine with decode fully on Strata's own kernels, and the release notes show no RTX 4090 numbers, so this may just need 4090 tuning.
Results (strata bench run --suite standard-v1, medians in tok/s):
```
0.1.60 0.1.62 run 1 0.1.62 run 2
Decode short 81.6 71.8 72.9
Decode 4k 78.0 73.1 72.8
Decode 16k 87.8 74.7 68.7
Prompt read 4k 966 950 954
Prompt read 16k 1837 1825 1765
```
The 0.1.60 result is a single run with a wide range at 16k (72-93), so the real gap may be smaller. Both 0.1.62 runs agree with each other.
### Smallest steps or command that reproduce it
1. strata launch --family unsloth --model ud-iq4_xs-v31 --data-dir "G:\Strata\Strata-data" --context 128k --kv int8 --vision gpu --mtp on --no-tui
2. strata bench run --url http://127.0.0.1:8080 --suite standard-v1 --skip-model-hash
3. Compare decode tok/s on engine 0.1.60 vs 0.1.62. Restart the model between runs, because a second run on the same server reuses cached prompts.
### Reproduction status
Reproduced: 2 clean runs on 0.1.62 (fresh model start each), compared with 1 run on 0.1.60.
### Strata version
0.2.16 (engine + Control 0.1.62), compared with 0.2.15 (engine 0.1.60, Control 0.1.61)
### Graphics card, memory and system
RTX 4090 24 GB (driver 621.14, windows-x86_64-cuda-cuda13 engine, PCIe 4.0 x8), AMD Ryzen 9 7950X3D, 64 GB DDR5, Windows 11 Pro. Model files on a RAID of 2x Intel Optane P5800X.
### Model, size and settings
Qwen3.8-Flash-Next UD-IQ4_XS-v31 (Unsloth Dynamic v3.1), 128k context, int8 KV, vision on GPU. The expert cache is auto (5412 slots); the rest of the experts stream from the SSD. KV streaming is on.
Drafting is the same in both versions: DFlash2 (q4_0-selq4 pack, up to 8 tokens, no MTP). Acceptance on 0.1.62 looks normal, about 75-80% (for example 299/369 at 4k and 289/385 at 16k).
### Logs
No errors in the engine log. I'm happy to attach the full bench reports (README.md / results.json) and the engine log, run a back-to-back rollback test, or try a build or flag.
Thanks for Strata. On this PC the 4-bit model went from about 25 tok/s (31 Oct) to 70-88 tok/s, which is amazing work.Mehr auf der Site
Links zu Install, Modellen, Releases.