Pull requests / #3388
#3388 [Engine] Rebase Qwen3.6 bounded low-memory expert cache onto main
open · @CC-David-CC · 0 commentaires · Sur GitHub
AMD / HIPNVIDIA / CUDAModels & quants
Description
## Problem and result Qwen3.6 A3B low-memory expert caching from #2925 was based on older main. This branch rebases its unchanged, opt-in patch onto e88ee39d1. ## Related discussion #2925, including https://github.com/Niko1221/Strata/pull/2925#issuecomment-6274280223. ## Approach Keep --low-memory and --expert-ram-gib for bounded GGUF-backed expert caching and skip unused MTP weights. One rebased commit; two changed files. ## Validation Passed: - Clean rebase; range-diff confirms the patch is unchanged; git diff --check. - Built strata and strata-q36 on middle-child: GTX 1050 Ti, CUDA 12.9, sm_61, AVX CPU floor. - Existing bounded-cache regression passed with AddressSanitizer and UBSan; CLI startup checked. Failed: - No build or cache regression failures observed. Not run: - Model inference, token parity, performance benchmarks, HIP and other backend builds. ## Limits Build and cache checks do not establish inference correctness or performance on this GPU. No new speed or memory measurements are claimed. Follow up in the original discussion before resubmitting the closed proposal.
Sur le site
Liens install, modèles, releases.