Pull requests / #2672

#2672 pinned arena (Windows): touch the pages from 16 threads before cudaHostRegister (engine start 5.1 s sooner)

open · @page1 · 0 comentários · No GitHub

BenchmarksAMD / HIPNVIDIA / CUDAModels & quantsWindowsLinux

Descrição

## Title
Issue: none

## Summary
On Windows with 4 KB pages, `cudaHostRegister` did the first touch (zero-fill) of the expert arena itself, on one thread: 4.98 s for 46.8 GiB here. Now min(16, cores) threads touch the pages first (0.5-1.2 s), and the registration then takes 0.08 s. The engine starts 5.1 s sooner; decode and prompt speed do not change.

## What changed
- `src/core/pinned.cu`: under `#ifdef _WIN32`, for `PageBacking::NormalPages` with no shared arena file, the pages are touched by min(16, hardware threads) threads before `cudaHostRegister`. The whole arena is still registered (`cudaHostRegister PORTABLE ok`). The time goes into the arena's log line.
- `STRATA_ARENA_PREFAULT=0` keeps the old order; `=N` sets the thread count.
- The old comment (a region faulted in page by page is far more expensive to register and may fail) did not hold here, and is reworded.

What it leaves alone: large pages, the shared arena file, Linux and macOS, and everything after loading.

## Extra Notes
Measured on an RTX 3090 24 GB, Core Ultra 9 285K, 95 GB DDR5, WD SN8100, Windows 11 26300, IQ3_S at 262K context, against a source build of v0.1.44.1 (`296b59d2`) the same day. 6 + 6 server starts in a rotated order, `/health` polled every 0.25 s:

| | v0.1.44.1 | this branch |
|---|---|---|
| server start to loaded, median (range) | 17.82 s (16.91-18.23) | **12.68 s** (12.38-12.93) |
| decode at 1.7K / 61K tokens (3 + 3 starts) | 89.2 / 85.4 tok/s | 90.0 / 84.8 tok/s |
| prompt at 61K tokens | 2718 tok/s | 2705 tok/s |

Built: CUDA 13.4 (sm_86) only. **Not built for HIP or SYCL**: there is no ROCm or oneAPI on this machine. `pinned.cu` is part of the HIP build, so it needs a HIP build check; the new code is all inside `#ifdef _WIN32`.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

No site

Links install, modelos, releases.