#6466 [Engine] 0.2.14 was faster than 0.2.17 rx 9060 xt (engine 0.1.61 vs 0.1.63)

closed · @Gerporgl · 4 评论 · 去 GitHub 看

BenchmarksModels & quants

说明

The token generation went from 45-47 to around 32-35 mostly during thinking.
Peak token generation can reach similar numbers (50-70)

Same prompt with both versions.

Sorry if I don't have more formal numbers or benchmarks, but the benchmark (bench.sh) actually looked great at first, prompt processing rate seemed better, but in practice for coding this new version is slower for me.

Same kind of config, uses the same amount of RAM on my server (64GB ryzen 9700 machine with 16gb vram gfx1200)

qwen iq3-s with --resident-experts in both cases (so that it fits better on my ram)

The new version 0.2.17 also forced a migration under the "/terminal" folder, which required some data files to be regenerated for some reasons, and for me to remap my launch configs.

Both used dflash2 as far as I can tell.

Anyway, since I run in a lxc container on proxmox, I did a snapshot rollback to go back to version 0.2.14 so I guess this is fine.

Anyway, I wanted to see if others have seen similar things with gfx1200

本站相关内容

相关页面的快捷入口。