改动 / #2501
#2501 resident: fix budget clamp double-counting the pre-claimed arena
open · @G0-0000 · 0 评论 · 去 GitHub 看
说明
## Bug With the NUMA pre-claim path (`--numa-hot-node`), `reserve_resident_arena()` faults the whole `--resident-budget-gib` arena into the cgroup's `current` **before** `pin_cache_complement()` reads available memory. The clamp (`clamp_resident_budget`) then charges the budget for its own arena — the same RAM counted once as "used" and again as "to allocate". Measured on a Tesla V100-SXM2-32GB (UD-Q4_K_XL, 71.73 GiB of experts, 48.22 GiB RAM complement, cgroup MemoryMax 100 GiB, 125.76 GiB host): ``` --resident-budget-gib 49.00 exceeds available RAM (47.40 GiB minus 4 GiB headroom and a 0.25 GiB margin): 42.75 GiB ``` 42.75 < 48.22, so 5.08 GiB of experts stayed file-backed although the host had ~50 GiB free. ## Fix Add the not-yet-used claim back to the available-memory reading — the reading the clamp would have taken had it run before the claim. The credit is zero when no arena was claimed, so the non-arena path is byte-identical. A new log line prints the accounting (`reading + claim = available − headroom − margin → granted`). ## Measured, after the fix - 49.00 GiB requested → 49.00 GiB granted; the 48.22 GiB complement is fully resident (0 experts file-backed) - from_files == 0 now holds, so `--drop-file-cache` fires at last (see #2243 — it was gated off in every partial-residency run) - post-load: file cache 4.75 GiB (was 43.2), cgroup current 59.1 GiB (was 88.9), load 52 s - no claim → zero credit → arithmetic unchanged (default path untouched) ## Discussion (not in this PR) Once the budget covers the whole complement, the file tier is never read at runtime, so its page cache is dead weight. We load with `STRATA_UNBUFFERED_LOAD=1` (O_DIRECT), which removed the remaining ~43 GiB of load-time cache pollution for us. Arguably, unbuffered load should be the default when `--resident-budget-gib` covers the full complement — happy to follow up with a measured PR if you agree. ## Dependency and validation Depends on #2242: arena_preclaim_ is introduced there, so this PR cannot build standalone against main until #2242 merges. Please do not merge this PR first. The fix plus credit regression tests were integrated LOCALLY on #2242 head f2ab2310; CUDA sm_70 strata and file_expert_source_test built, and CPU-only test passed with active NUMA credit assertions. These tests are NOT YET in this PR branch; I will update it once #2242 merges. HIP/SYCL and GPU runtime not tested.
本站相关内容
相关页面的快捷入口。