问题 / #406
#406 128K hard limit on 64GB RAM, 32GB VRAM is too conservative
closed · @khalidejaz · 3 评论 · 去 GitHub 看
Setup & installModels & quantsWindows
说明
I am using IQ3_S quantization of the Qwen3.8-Flash-Next on Ubuntu. The setup reverts my 256K context length choice down to 128K. With full context used, the system shows >6GB of free RAM. 1.8GB more was needed for 256K instead of 128K. A manual change to the run script allowed me to run with full native context of 256K with zero issues. This may be different for windows systems or if other RAM-heavy programs are in use, but for my case there was no reason to force it to 128K. Loving the about 2.5x speed in token gen and >6x on pp. 70%+ expert cache hit. Thanks!
本站相关内容
相关页面的快捷入口。