Issues / #3116
#3116 MTP block loaded on Qwen3.6-35B-A3B-UD-Q4_K_XL made output slowly
open · @zhangiser · 0 comentários · No GitHub
BenchmarksServer & APINVIDIA / CUDAModels & quants
Descrição
I used win11 and has 12G VRAM + 64G RAM,strata 0.1.45 log: ```txt [strata] 2026-10-19 10:30:02 engine started: D:\code5\Strata-main\engine\strata.exe --serve --native D:\code5\Strata-data\models\qwen36-UD-Q4_K_XL\Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf --max-context 32768 --prefill auto --expert-cache 8000 --mtp 2 --pool-workers 15 strata-q36: ignoring --expert-cache 8000 (not an option of this engine) strata-q36: D:\code5\Strata-data\models\qwen36-UD-Q4_K_XL\Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf: 753 tensors (layout checked), 4.40 GiB on the GPU in 1.6 s (the routed experts are placed after the engine's buffers) strata: K-quant prompt kernels from D:\code5\Strata-main\engine\strata_kq.dll (STRATA_MMQ_KQ=1) strata-q36: prompt experts on MMQ (int8) strata-q36: context 32768 cells, prompt chunk 2048, ### MTP block loaded, 2.63 GiB of VRAM free strata-q36: hybrid experts: 10 of 256 per layer in VRAM (0.73 GiB), all 18.75 GiB in pinned host memory, the rest of a batch of <= 128 rows on 16 CPU threads, bigger prompt chunks streamed (1.10 GiB staging); 9.8 s strata-q36: warmed up in 1.1 s (one prompt chunk, a decode step, a speculative step) strata serve: session 0: prompt 18 tokens = 0 reused + 18 read in 447 ms (40.3 tok/s), 4940 generated in 114289 ms (43.2 tok/s), drafts accepted 0 of 0 (stop) ``` speed: <img width="1849" height="1197" alt="Image" src="https://github.com/user-attachments/assets/1244f7c7-46de-49fa-b940-53b834989885" /> I've compared Qwen3.8-Flash-Next_IQ2_XS before, got 70 tok/s of output speed Maybe the problem is MTP block loaded?
No site
Links install, modelos, releases.