Issues / #3116

#3116 MTP block loaded on Qwen3.6-35B-A3B-UD-Q4_K_XL made output slowly

open · @zhangiser · 0 comments · View on GitHub

BenchmarksServer & APINVIDIA / CUDAModels & quants

Description

I used win11 and has 12G VRAM + 64G RAM,strata 0.1.45
log:
```txt
[strata] 2026-10-19 10:30:02 engine started: D:\code5\Strata-main\engine\strata.exe --serve --native D:\code5\Strata-data\models\qwen36-UD-Q4_K_XL\Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf --max-context 32768 --prefill auto --expert-cache 8000 --mtp 2 --pool-workers 15
strata-q36: ignoring --expert-cache 8000 (not an option of this engine)
strata-q36: D:\code5\Strata-data\models\qwen36-UD-Q4_K_XL\Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf: 753 tensors (layout checked), 4.40 GiB on the GPU in 1.6 s (the routed experts are placed after the engine's buffers)
strata: K-quant prompt kernels from D:\code5\Strata-main\engine\strata_kq.dll (STRATA_MMQ_KQ=1)
strata-q36: prompt experts on MMQ (int8)
strata-q36: context 32768 cells, prompt chunk 2048, ### MTP block loaded, 2.63 GiB of VRAM free
strata-q36: hybrid experts: 10 of 256 per layer in VRAM (0.73 GiB), all 18.75 GiB in pinned host memory, the rest of a batch of <= 128 rows on 16 CPU threads, bigger prompt chunks streamed (1.10 GiB staging); 9.8 s
strata-q36: warmed up in 1.1 s (one prompt chunk, a decode step, a speculative step)
strata serve: session 0: prompt 18 tokens = 0 reused + 18 read in 447 ms (40.3 tok/s), 4940 generated in 114289 ms (43.2 tok/s), drafts accepted 0 of 0 (stop)
```
speed:

<img width="1849" height="1197" alt="Image" src="https://github.com/user-attachments/assets/1244f7c7-46de-49fa-b940-53b834989885" />

I've compared Qwen3.8-Flash-Next_IQ2_XS before, got 70 tok/s of output speed
Maybe the problem is MTP block loaded?

Related on this site

Quick links to related pages.