问题 / #3066
#3066 Decode 2x slower under agent frontends (Windows, RTX 3060 12GB): GPU downclocks to P0 during MoE expert misses — locking clocks recovers ~60%
open · @ahsvel · 0 评论 · 去 GitHub 看
BenchmarksSetup & installMulti-GPUNVIDIA / CUDAModels & quantsDocumentationWindows
说明
## What do you want to do?
Report a performance finding + workaround for agent-frontend users, and suggest a docs note. Related to #1910 (same family, disk side).
## Setup
- RTX 3060 12GB, Windows 10, 48GB RAM, NVMe
- Qwen3.8-Flash-Next IQ3_xxs (27GB MoE + PLE), engine 0.1.42 (0.1.45 also tested)
- Frontend: Hermes Agent (~12K-token system prompt, tool-heavy multi-turn)
## Symptom
- Web panel decode: ~31 tok/s. Same engine under the agent: 15-16 tok/s.
- MSI Afterburner: core clock jumps to ~2000 MHz during prefill ("reading prompt"), but drops to 850-900 MHz (P0 base clock) during decode, ~15W.
- `--ple-io direct / ram / mmap`: no difference on this NVMe rig. 0.1.45 behaves same as 0.1.42.
## Diagnosis
With a long system prompt, decode is dominated by MoE expert misses: the engine pulls experts from RAM/disk over PCIe, SMs starve, the driver sees low utilization and downclocks to P0 — then every kernel runs ~2x slower. The clock drop is an amplifier of the starvation, not the root cause.
## Workaround
Before launching the engine (admin shell):
nvidia-smi -lgc 2000,2000
Result: decode 15 -> 25 tok/s (+60%), clocks stay at 2000 MHz during decode, power ~35-80W.
Reset: `nvidia-smi -rgc`; the lock also resets on reboot.
## Suggestion
Worth a note in the Windows setup docs (or an optional launcher flag) for agent-frontend users: if decode is much slower than prefill implies, watch the core clock during decode — locking it may recover most of the gap.
## Caveats
- Single-rig data point.
- This treats the symptom; the root cause is expert-fetch bandwidth (see #1910 for the disk side of the same problem).
- Not recommended for laptops / hot chassis.
本站相关内容
相关页面的快捷入口。