Pull requests / #2757
#2757 sycl: STRATA_RESIDENT_TOUCH=1 - re-touch the pinned resident experts after reclaim (xe shrinker backs them up to shmem)
open · @aslater3 · 0 comentarios · En GitHub
Descripción
## sycl: STRATA_RESIDENT_TOUCH=1 - re-touch the pinned resident experts after reclaim (xe shrinker backs them up to shmem) On an A770 under the xe driver with swap on, the decode CPU pool ran at 20-22 GB/s in some runs and ~30 in others (decode ~10% apart). Root cause: the 28 GB load drives memory reclaim through to the prompt; reclaim runs the xe shrinker (xe_shrinker_scan -> xe_bo_shrink -> ttm_backup, which only acts while free swap exists), and it backs the idle pinned host chunks of the resident expert copy up to shmem. The pool then restores them one major fault at a time during decode: 17-65K major faults / 2 s, on /dev/dri/renderD128 (perf -e major-faults). Opt-in keeper thread: it watches /proc/vmstat (pgscan_kswapd + pgscan_direct); once reclaim has been quiet for 2 s after a burst, it reads one byte of every 4 KiB page of the pinned chunks on 4 threads, so the restore is paid then rather than inside the pool. Joined in close(). Default off: unchanged behaviour. Builds on the chunked pinned copy (#2485). A770, Ryzen 7 3800X, 30 GB RAM + swap, Coder IQ1_M, 8K code prompt, 3 pairs: decode-phase major faults 17-65K -> 8-1.4K / 2 s, pool 20.4 -> 27.0 GB/s, decode 34.68 -> 37.59 tok/s (+8.4%), VRAM-tier refill unchanged, reference tokens. Prefill 785 -> 743 tok/s (-5.4%) in that bench, where the prompt starts right after the load and the touch (2-4 s) overlaps it; a server's prompts come later. Alternatives without it: no swap, or STRATA_RESIDENT_CHUNKED=0. Stacked on #2485 (the chunked pinned copy this touches); the diff to review is the top commit. If you prefer, this can be an issue first: the root cause is a driver / swap interaction, and turning swap off also fixes it. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
En el sitio
Enlaces a install, modelos, releases.