#2925 Run Qwen3.6 A3B Q4 on 6 GB VRAM and 16 GB RAM

open · @CC-David-CC · 0 评论 · 去 GitHub 看

Benchmarks AMD / HIP NVIDIA / CUDA Models & quants Linux

说明

Adds opt-in `--low-memory` with a bounded GGUF-backed expert cache and unused MTP weights unloaded. Qwen3.6-35B-A3B UD-Q4_K_XL generated 128 tokens at ~4.5 tokens/s on a GTX 1060 6 GB with 16 GB system RAM.

Used `--low-memory --expert-ram-gib 6 --mtp 0 --max-context 2048 --prefill 256`. Linux/CUDA validated; HIP untested. CUDA 12.0 needs the separate compiler fix in #2913.

本站相关内容

Install Strata
Model sizes & RAM
Release notes