#6173 [Other] Source setup accepts --context 262144 for Qwen3.6-35B-A3B on a 12 GB card, and the engine then refuses to start
open · @m-wells · 1 comentários · No GitHub
BenchmarksSetup & installNVIDIA / CUDAModels & quantsWindowsLinux
Descrição
### Area [Other] (anything else, including setup and installation) ### Desktop app, Terminal or source? Source (START-HERE.bat, setup.sh or a build) ### What happened Source setup accepted a 256K context for Qwen3.6-35B-A3B on a 12 GB card without a warning, and the engine then refused to start. `./setup.sh --yes --family qwen36 --model UD-Q6_K_XL --vision no --context 262144 --no-start --no-browser` printed `[ok] context: 262144 tokens` and wrote the config. At start, the server warned and the engine exited: ``` [strata] At 256K, this model's conversation memory needs about 11.5 GB of graphics memory. With the model itself that is more than this card's 12 GB, so it will probably not start. 32K fits on this card: choose 32K or less. strata-q36: .../Qwen3.6-35B-A3B-UD-Q6_K_XL.gguf: 753 tensors (layout checked), 4.40 GiB on the GPU in 0.9 s (the routed experts are placed after the engine's buffers) strata-q36: a context of 262144 tokens does not fit on this graphics card: its conversation memory needs 11.56 GiB and only 2.28 GiB is free after the model. Start it again with a shorter context (50176 tokens fit here). ``` Expected: setup to warn at the choice (or cap the context), as it does for Qwen3.8-Flash-Next, where it warned that IQ3_XXS at 256K needs about 70 GB of RAM. The same setup at `--context 32768` started and served (11K-token prompt: 776-777 tok/s prompt, 32.7-33.1 tok/s decode, greedy, thinking off). #6156 reports the same engine limit with two 12 GB cards; this one is about setup accepting the context on a single 12 GB card. ### Smallest steps or command that reproduce it `./setup.sh --yes --family qwen36 --model UD-Q6_K_XL --vision no --context 262144 --no-start --no-browser`, then start the written `run-qwen36-ud-q6_k_xl.sh`. ### Reproduction status Reproduced locally ### Strata version v0.2.15 (commit c775728), engine 0.1.61, built from source ### Graphics card, memory and system RTX 4080 Laptop GPU 12 GB (sm_89, PCIe x8), 64 GB DDR5, Arch Linux, CUDA 13.4 ### Model, size and settings Qwen3.6-35B-A3B UD-Q6_K_XL (unsloth/Qwen3.6-35B-A3B-MTP-GGUF at the pinned revision), context 262144, pictures off; setup's defaults otherwise (`--mtp 2 --pool-workers 9`).
No site
Links install, modelos, releases.