问题 / #3262

#3262 [Engine] Linux hybrid CPU: LP-E cores (cpu_lowpower) counted as P-cores, and 0.2.0's 15-worker default is ~35 % slower than 9 on a Core Ultra 7 255H

open · @faf-net · 0 评论 · 去 GitHub 看

BenchmarksAMD / HIPNVIDIA / CUDAModels & quantsWindowsLinux

说明

## Problem and result

On an Intel Core Ultra 7 255H (Arrow Lake-H: 6 P + 8 E + 2 LP-E, no SMT) under Linux, Strata's pool planning has two effects I could measure:

1. **The two LP-E cores are treated as P-cores.** `pool.cpp` (Linux branch) decides P/E from Intel's hybrid PMU lists `/sys/devices/cpu_core/cpus` and `/sys/devices/cpu_atom/cpus`. On Meteor Lake / Arrow Lake-H the LP-E cores are in a **third** list, `/sys/devices/cpu_lowpower/cpus`, so `is_e()` returns false for them. They sort after the real P-cores (lower capacity) but before every E-core, and get workers early: with `--pool-workers 9` the engine reports `9 pool workers on logical processors 1,2,3,4,5,14,15,6,7`. `--pool-affinity measure` (0.1.45) reported "8 P-cores / 8 threads, 8 E-cores".
2. **The new 0.2.0 default (one worker per physical core but the host's) picks 15 workers here**, and that is much slower than 9 on this CPU (single `bench.py` runs, same model and settings otherwise):

| Engine | Workers | Short answer | Long answer |
|---|---|---:|---:|
| 0.2.0-beta prebuilt `hip-rx7900` (0.1.45.3) | `--pool-workers 9` | 59.7 tok/s | 64.0 tok/s |
| 0.2.0-beta prebuilt `hip-rx7900` (0.1.45.3) | default → 15 (`1,2,3,4,5,14,15,6,...,13`) | 39.8 tok/s | 41.2 tok/s |

This matches an earlier sweep on 0.1.44.1 (IQ2_XS, long answer, median of 3): automatic 56.1, 5 workers 76.6, 9 workers 76.8, 13 workers 74.0 tok/s.

## Topology (sysfs, kernel 6.18.15)

```
cpu_core/cpus     0-5     cpu_capacity 1024, max 5.1 GHz, private L2, L3 24 MiB
cpu_atom/cpus     6-13    cpu_capacity  774, max 4.4 GHz, L2 shared by 4 (6-9, 10-13), L3
cpu_lowpower/cpus 14-15   cpu_capacity  312, max 2.5 GHz, L2 shared by 2, no L3 (SoC tile)
```

## What the LP-E cores cost (A/B, 0.1.44.1)

Same engine and config (IQ3_XXS, MTP, `--pool-workers 9`), container limited with `--cpuset-cpus 0-13` (B) or not (A). Three interleaved pairs, each a fresh engine, one warm-up answer, then two measured answers of 900 tokens:

| | A: CPUs 0-15 (workers on 14, 15) | B: CPUs 0-13 |
|---|---:|---:|
| Decode tok/s, 6 answers | 61.7–64.4, median 63.3 | 59.9–63.8, median 62.2 |
| 63-token prompt, prompt tok/s | 60.6–69.0, median 66.1 | 91.4–95.4, median 93.9 |

Decode is unchanged; short prompts are read about 40 % faster without the LP-E cores (first token for a short chat turn about 0.95 s → 0.67 s). Longer prompts were not part of this A/B.

## Suggested change (not tested as a patch)

Read `/sys/devices/cpu_lowpower/cpus` beside `cpu_core` and `cpu_atom`, treat those CPUs as the lowest tier (after E-cores, or excluded by default), and keep the capacity fallback (`cap * 10 < max_cap * 9`) for CPUs that are in no list. For the default worker count on a hybrid CPU, this box suggests "P-cores + a few E-cores" rather than "every physical core". I can build and measure a patch on this machine (HIP only; I cannot build CUDA or SYCL here).

## Validation

**Passed:** sysfs reads above; A/B with `--cpuset-cpus` (`folgetests.py cpu`, raw JSON lines); 0.2.0 prebuilt engine with and without `--pool-workers 9` (`bench.py --tests schreiben`); placement lines copied from the engine log.
**Failed:** none.
**Not run:** a patched engine; Windows (the Windows branch has its own detection); other Meteor/Arrow/Lunar Lake CPUs; more than one run per configuration for the 0.2.0 worker comparison.

Machine: UGREEN iDX6011 Pro NAS, Core Ultra 7 255H, 64 GB LPDDR5X, RX 7900 XTX over OCuLink x4, UGOS Pro 1.19 (Debian 12 base), ROCm 7.10 (TheRock), model Qwen3.8-Flash-Next IQ3_XXS. Benchmark report: #2904. Scripts: https://github.com/faf-net/ugreen-idx6011-egpu-llm

本站相关内容

相关页面的快捷入口。