#6083 [Engine] Single-socket dual-CCD Zen 5: the 12 pool workers pin to CPUs 0-12, the second CCD stays idle, and the loaded die hits 95 °C where an all-core load runs 72 °C

open · @MarcoM-M · 1 评论 · 去 GitHub 看

BenchmarksServer & APIAMD / HIPNVIDIA / CUDAModels & quantsDocumentationLinux

说明

### Area

[Engine] (speed, kernels, models, the server and API)

### What do you want to do?

Place the CPU expert pool across the physical cores of **both** CCDs on a
single-socket multi-chiplet CPU, or expose a supported way to name the worker
CPUs. Today the workers land on one CCD, which costs nothing in throughput but
concentrates the whole package power in half the silicon: the loaded die sits at
95 °C while the other idles at 45-75 °C. An all-core synthetic load drawing more
package power than the engine runs 23 °C cooler because it uses both dies.

`--pool-affinity all` (documented as "one worker per physical core") changes
nothing here. A container `cpuset` makes it worse rather than better — details
under "What I saw".

### What I measured

**Strata version:** strata 0.2.13.1, engine reporting 0.1.59.1, built from
`v0.2.13.1` for sm_86 on this machine
**OS:** Linux Mint (Ubuntu 24.04 base), kernel 7.0.0-43-generic
**GPU / CPU / RAM:** RTX 3090 24 GB (PCIe 4.0 x16) / Ryzen 9 9950X, 16 cores,
32 threads, two CCDs of 8 cores (CCD0 = logical 0-15, CCD1 = logical 16-31),
ECO mode 105 W (PPT 142 W measured as ~129 W under an all-core burn) /
64 GB DDR5-5600

Model `qwen3.8-flash-next-ud-iq4_xs`, `--kv k8v4`, `--kv-resident 262144`,
`--pool-tasks 18`, `--pcie-frac 0.9`.

**What I saw**

Thread affinity of the `--serve` process while a request runs
(`Cpus_allowed_list` per thread):

| threads | allowed CPUs |
| --- | --- |
| 12 (pool workers) | one each: `0`, `1`, `2` … `12` |
| 11 (aux: prefill helpers, CUDA driver, adaptive tier) | `13-16,29-31` |

Per-CPU utilisation during the same request:

```
cpu0..cpu12   97-100 %
cpu13..cpu31   2-10 %
```

So the pool maps to logical CPUs 0-12, which is CCD0 only (cores 0-6 with their
SMT siblings). CCD1 never runs a worker.

Temperatures, same machine, same session: the loaded die is the hotspot.

| load | package power | Tctl | CCD1 | CCD2 | CPU fan |
| --- | --- | --- | --- | --- | --- |
| strata, 12 workers on CCD0 | ~120-150 W bursts | **95.6 °C** | **95.9 °C** | 93.4 °C | 2135 RPM |
| 32-thread busy loop, 61 s | 129 W average | 72.2 °C | 72.4 °C | 71.9 °C | 1252 RPM |

The synthetic load moves more total power and runs 23 °C cooler, purely because
it uses both dies. Nothing in the cooling changed.

**What I tried, and what happened**

1. `--pool-affinity all` (explicitly set): worker masks stayed `0..12`, no change
   in placement or speed.
2. Container `cpuset=0,2,4,...,30` (one thread per physical core across both
   CCDs, as the flag's doc describes): the engine re-pinned the 12 workers to
   `0,2,4,...,14` and put **11 aux threads on CPU 16 alone**. Decode fell from
   ~135 to **27.4 tok/s**.
3. Container `cpuset=0-7,16-23`: same collapse, 27.4 tok/s.
4. The above with `STRATA_AUX_CPUS=off`: 7 threads ended up on CPU 0, decode
   **35.1 tok/s**.

All four variants were reverted; with no cpuset the engine returns to ~135 tok/s
decode and one-CCD placement.

**Pool size sweep** (100k-token cold prefill, decode at short context):

| workers | prefill tok/s | decode tok/s |
| --- | --- | --- |
| 8 | 1369 / 1205 / 1091 | 135 |
| 10 | 1042 | 139 |
| 12 | 1167 / 958 / 898 | 139 |
| 20 | — | 138 (and warmer dies: 85.2 / 78.9 °C) |

8 workers is the best prefill on this CPU; adding workers does not buy decode and
does raise die temperature.

**Why it matters beyond this box:** every dual-CCD Ryzen (7900X/7950X/9900X/9950X)
has this shape, and so does every Threadripper with more than one CCD. Spreading
workers costs a little cross-CCD latency but removes a 20-25 °C hot spot and the
fan noise that follows it. #5333 asked for the same thing on two sockets.

**What would help:** map workers to distinct **physical** cores across the
package by default (as `--pool-affinity all` already promises), or accept a
worker CPU list from the caller. A `--pool-cpus LIST` symmetric with
`--aux-cpus` would let us place them without fighting the engine.

### Current state

Running with no cpuset: healthy, decode 135-140 tok/s short context, 59 tok/s at
90k, 33-37 tok/s at 150-200k, prefill 1665 tok/s at 150k. The only cost is the
single-die hotspot.

本站相关内容

相关页面的快捷入口。