Pull requests / #2174

#2174 pool: STRATA_POOL_UMWAIT - parked workers sleep with UMONITOR/UMWAIT instead of spinning (opt-in, -3.1% ms/window on a 12 GB laptop, bitwise)

open · @zsariboga · 0 comentarios · En GitHub

Server & APIAMD / HIPNVIDIA / CUDAModels & quantsWindows

Descripción

## What

`STRATA_POOL_UMWAIT=<TSC cycles>` (opt-in, x86 with WAITPKG, MSVC x64 builds): a parked CPU pool worker no longer spins on
`_mm_pause`. It arms `UMONITOR` on the epoch's cache line and sleeps in `UMWAIT` (C0.1) until the host publishes the next
phase (the write to that line wakes it in hardware) or the cycles run out; then it checks the clock and the sleep rule as
before.

- `STRATA_POOL_UMWAIT_C02=1`: the deeper C0.2 state instead of C0.1.
- `STRATA_POOL_UMWAIT_ESC=<cycles>`: C0.1 for the first cycles of a park, C0.2 after (a long park is a GPU phase).
- Unset, `0`, a CPU without WAITPKG (CPUID.7.0:ECX[5], one line says so) or a non-MSVC build: the spin, unchanged.

## Why

On a laptop the CPU and the GPU share one power and thermal budget. During the GPU's part of a decode window the parked
workers spin, and that power is taken from the GPU (here `sw_power_cap` was active in ~100% of the samples). Sleeping in
UMWAIT gives it back; the wake on the publish is a hardware monitor, so a worker still starts the next phase at once.

## Measured

RTX 5070 Ti Laptop 12 GB, Core Ultra 7 255HX (16 P+E cores, WAITPKG), 64 GB DDR5-5600, Windows 11, Swift 1.5 IQ3_XXS
(GSQ-RCO), the same config in every arm, mirrored order, engine ms per decode window, GPU power from `nvidia-smi` every 500 ms:

| arm | ms/window (two runs) | mean | vs spin | GPU power | SM clock |
| --- | --- | --- | --- | --- | --- |
| 13 workers, spin (before) | 29.42 / 29.79 | 29.61 | - | 122.0 / 122.7 W | 2400 / 2361 MHz |
| 13 workers, `STRATA_POOL_UMWAIT=100000` | 29.02 / 29.04 | 29.03 | -2.0% | 128.2 / 125.3 W | 2416 / 2400 MHz |
| 16 workers, `STRATA_POOL_UMWAIT=100000` | 28.96 / 28.44 | 28.70 | **-3.1%** | 128.4 / 128.0 W | 2444 / 2422 MHz |
| 19 workers, `STRATA_POOL_UMWAIT=100000` | 31.31 / 31.15 | 31.23 | +5.5% | 126.6 / 124.9 W | 2365 / 2391 MHz |

The sleep hands the GPU ~6 W, and it also lets more workers pay off (16 spinning workers had been slower than 13). C0.2
and the escalating wait were within the noise of C0.1 (28.57 / 28.68 vs 28.90 ms, inconsistent across mirrored pairs), so
C0.1 is the default state. Output is bitwise the same: only how a parked thread waits changes.

Running daily on this laptop since 10 October with `STRATA_POOL_UMWAIT=100000 --pool-workers 16`. Built and smoke-tested
on today's `main` (a0116b27): the engine prints `STRATA_POOL_UMWAIT: parked workers wait with UMWAIT (100000 cycles)`,
chat and tool calls answer as before.

## Not done

GCC/Clang builds (they need `-mwaitpkg` or a target attribute on the worker), and `TPAUSE` for parks without a monitored
line. AMD CPUs have `MONITORX/MWAITX` instead; not tried.

En el sitio

Enlaces a install, modelos, releases.