Issues / #406
#406 128K hard limit on 64GB RAM, 32GB VRAM is too conservative
closed · @khalidejaz · 3 Kommentare · Auf GitHub
Setup & installModels & quantsWindows
Beschreibung
I am using IQ3_S quantization of the Qwen3.8-Flash-Next on Ubuntu. The setup reverts my 256K context length choice down to 128K. With full context used, the system shows >6GB of free RAM. 1.8GB more was needed for 256K instead of 128K. A manual change to the run script allowed me to run with full native context of 256K with zero issues. This may be different for windows systems or if other RAM-heavy programs are in use, but for my case there was no reason to force it to 128K. Loving the about 2.5x speed in token gen and >6x on pp. 70%+ expert cache hit. Thanks!
Mehr auf der Site
Links zu Install, Modellen, Releases.