#6279 [Engine] RTX 4090 (Windows, CUDA 13): decode about 10-20% slower on 0.1.62 vs 0.1.60 with UD-IQ4_XS-v31 (prompt reading unchanged)

closed · @m4rantza · 5 评论 · 去 GitHub 看

BenchmarksMulti-GPUAMD / HIPNVIDIA / CUDAModels & quantsDocumentationWindows

说明

### Area
Engine

### Desktop app, Terminal or source?
Terminal

### What happened
After updating from 0.2.15 to 0.2.16, decode on Qwen3.8-Flash-Next UD-IQ4_XS-v31 is about 10-20% slower. Prompt reading is unchanged, output is fine, and the sanity checks pass 12/12.
0.1.62 is the first engine with decode fully on Strata's own kernels, and the release notes show no RTX 4090 numbers, so this may just need 4090 tuning.

Results (strata bench run --suite standard-v1, medians in tok/s):

```
                    0.1.60    0.1.62 run 1    0.1.62 run 2
Decode short         81.6         71.8            72.9
Decode 4k            78.0         73.1            72.8
Decode 16k           87.8         74.7            68.7
Prompt read 4k        966          950             954
Prompt read 16k      1837         1825            1765
```

The 0.1.60 result is a single run with a wide range at 16k (72-93), so the real gap may be smaller. Both 0.1.62 runs agree with each other.

### Smallest steps or command that reproduce it
1. strata launch --family unsloth --model ud-iq4_xs-v31 --data-dir "G:\Strata\Strata-data" --context 128k --kv int8 --vision gpu --mtp on --no-tui
2. strata bench run --url http://127.0.0.1:8080 --suite standard-v1 --skip-model-hash
3. Compare decode tok/s on engine 0.1.60 vs 0.1.62. Restart the model between runs, because a second run on the same server reuses cached prompts.

### Reproduction status
Reproduced: 2 clean runs on 0.1.62 (fresh model start each), compared with 1 run on 0.1.60.

### Strata version
0.2.16 (engine + Control 0.1.62), compared with 0.2.15 (engine 0.1.60, Control 0.1.61)

### Graphics card, memory and system
RTX 4090 24 GB (driver 621.14, windows-x86_64-cuda-cuda13 engine, PCIe 4.0 x8), AMD Ryzen 9 7950X3D, 64 GB DDR5, Windows 11 Pro. Model files on a RAID of 2x Intel Optane P5800X.

### Model, size and settings
Qwen3.8-Flash-Next UD-IQ4_XS-v31 (Unsloth Dynamic v3.1), 128k context, int8 KV, vision on GPU. The expert cache is auto (5412 slots); the rest of the experts stream from the SSD. KV streaming is on.
Drafting is the same in both versions: DFlash2 (q4_0-selq4 pack, up to 8 tokens, no MTP). Acceptance on 0.1.62 looks normal, about 75-80% (for example 299/369 at 4k and 289/385 at 16k).

### Logs
No errors in the engine log. I'm happy to attach the full bench reports (README.md / results.json) and the engine log, run a back-to-back rollback test, or try a build or flag.

Thanks for Strata. On this PC the 4-bit model went from about 25 tok/s (31 Oct) to 70-88 tok/s, which is amazing work.

本站相关内容

相关页面的快捷入口。