Pull requests / #2917

#2917 Fix Qwen3.6 hybrid staging allocation on small GPUs

open · @CC-David-CC · 0 コメント · GitHub で見る

AMD / HIPNVIDIA / CUDA

本文

Release optional packed Q8 copies when hybrid staging and CPU-output scratch cannot fit within the VRAM reserve. This fixes Qwen3.6 Q4 startup on the tested Tesla P4. Explicit expert budgets, slots and KEEP_PACKED remain respected.

One source file only. Validated on CUDA 12.0 with the separate compiler fix in #2913: startup and generation passed; seven responses matched the manual workaround. HIP was not compiled.

関連リンク

インストール・モデル・リリースへの站内リンク。