Issues / #2616
#2616 Feature Addition For Strata: have Vision encoder loadable/unloadable at runtime via a web button
open · @Schumi08 · 2 comments · View on GitHub
Description
Qwen 3.8 flash next and other models vision encoder can eat 0.9 gb of VRAM equals to 700 experts for Q3 quants... Is it possible for vision encoders to be loaded on request and required experts shifting/kv cache compression be done on run time? That way same model can run at long context or fast and when using images act accordingly
Related on this site
Quick links to related pages.