#6083 [Engine] Single-socket dual-CCD Zen 5: the 12 pool workers pin to CPUs 0-12, the second CCD stays idle, and the loaded die hits 95 °C where an all-core load runs 72 °C
open · @MarcoM-M · 1 comments · View on GitHub
BenchmarksServer & APIAMD / HIPNVIDIA / CUDAModels & quantsDocumentationLinux
Description
### Area [Engine] (speed, kernels, models, the server and API) ### What do you want to do? Place the CPU expert pool across the physical cores of **both** CCDs on a single-socket multi-chiplet CPU, or expose a supported way to name the worker CPUs. Today the workers land on one CCD, which costs nothing in throughput but concentrates the whole package power in half the silicon: the loaded die sits at 95 °C while the other idles at 45-75 °C. An all-core synthetic load drawing more package power than the engine runs 23 °C cooler because it uses both dies. `--pool-affinity all` (documented as "one worker per physical core") changes nothing here. A container `cpuset` makes it worse rather than better — details under "What I saw". ### What I measured **Strata version:** strata 0.2.13.1, engine reporting 0.1.59.1, built from `v0.2.13.1` for sm_86 on this machine **OS:** Linux Mint (Ubuntu 24.04 base), kernel 7.0.0-43-generic **GPU / CPU / RAM:** RTX 3090 24 GB (PCIe 4.0 x16) / Ryzen 9 9950X, 16 cores, 32 threads, two CCDs of 8 cores (CCD0 = logical 0-15, CCD1 = logical 16-31), ECO mode 105 W (PPT 142 W measured as ~129 W under an all-core burn) / 64 GB DDR5-5600 Model `qwen3.8-flash-next-ud-iq4_xs`, `--kv k8v4`, `--kv-resident 262144`, `--pool-tasks 18`, `--pcie-frac 0.9`. **What I saw** Thread affinity of the `--serve` process while a request runs (`Cpus_allowed_list` per thread): | threads | allowed CPUs | | --- | --- | | 12 (pool workers) | one each: `0`, `1`, `2` … `12` | | 11 (aux: prefill helpers, CUDA driver, adaptive tier) | `13-16,29-31` | Per-CPU utilisation during the same request: ``` cpu0..cpu12 97-100 % cpu13..cpu31 2-10 % ``` So the pool maps to logical CPUs 0-12, which is CCD0 only (cores 0-6 with their SMT siblings). CCD1 never runs a worker. Temperatures, same machine, same session: the loaded die is the hotspot. | load | package power | Tctl | CCD1 | CCD2 | CPU fan | | --- | --- | --- | --- | --- | --- | | strata, 12 workers on CCD0 | ~120-150 W bursts | **95.6 °C** | **95.9 °C** | 93.4 °C | 2135 RPM | | 32-thread busy loop, 61 s | 129 W average | 72.2 °C | 72.4 °C | 71.9 °C | 1252 RPM | The synthetic load moves more total power and runs 23 °C cooler, purely because it uses both dies. Nothing in the cooling changed. **What I tried, and what happened** 1. `--pool-affinity all` (explicitly set): worker masks stayed `0..12`, no change in placement or speed. 2. Container `cpuset=0,2,4,...,30` (one thread per physical core across both CCDs, as the flag's doc describes): the engine re-pinned the 12 workers to `0,2,4,...,14` and put **11 aux threads on CPU 16 alone**. Decode fell from ~135 to **27.4 tok/s**. 3. Container `cpuset=0-7,16-23`: same collapse, 27.4 tok/s. 4. The above with `STRATA_AUX_CPUS=off`: 7 threads ended up on CPU 0, decode **35.1 tok/s**. All four variants were reverted; with no cpuset the engine returns to ~135 tok/s decode and one-CCD placement. **Pool size sweep** (100k-token cold prefill, decode at short context): | workers | prefill tok/s | decode tok/s | | --- | --- | --- | | 8 | 1369 / 1205 / 1091 | 135 | | 10 | 1042 | 139 | | 12 | 1167 / 958 / 898 | 139 | | 20 | — | 138 (and warmer dies: 85.2 / 78.9 °C) | 8 workers is the best prefill on this CPU; adding workers does not buy decode and does raise die temperature. **Why it matters beyond this box:** every dual-CCD Ryzen (7900X/7950X/9900X/9950X) has this shape, and so does every Threadripper with more than one CCD. Spreading workers costs a little cross-CCD latency but removes a 20-25 °C hot spot and the fan noise that follows it. #5333 asked for the same thing on two sockets. **What would help:** map workers to distinct **physical** cores across the package by default (as `--pool-affinity all` already promises), or accept a worker CPU list from the caller. A `--pool-cpus LIST` symmetric with `--aux-cpus` would let us place them without fighting the engine. ### Current state Running with no cpuset: healthy, decode 135-140 tok/s short context, 59 tok/s at 90k, 33-37 tok/s at 150-200k, prefill 1665 tok/s at 150k. The only cost is the single-die hotspot.
Related on this site
Quick links to related pages.