Pull requests / #2413
#2413 docs/AMD_HIP.md: short GPU stalls while the kernel compacts memory, and how to keep them out of a speed comparison
open · @libratechw · 0 comentários · No GitHub
AMD / HIPModels & quantsDocumentationLinux
Descrição
## Title Issue: none (related: #750, #920 and #1705, the reclaim stalls of the section before it; #2126 is a different compaction stall) ## Summary On a Ryzen AI Max+ 395 (gfx1151), a prompt that met heavy page migration was usually 0.1-1.6 s slower, mostly the first prompt after a start. That can hide a 1-2% change: in one 11-pair 4K A/B, the off / on ratio of `prompt_ms` had a 95% interval of [0.979, 1.078] on each engine's first prompt and [1.016, 1.020] on the second. This adds a short section after the reclaim section: what was seen, and how to measure when the difference is small enough for these stalls to hide it. ## What changed - `docs/AMD_HIP.md`: one new section between the reclaim section and "RDNA4 (gfx1201)". No code changes. Measured on gfx1151, 128 GB, ROCm 7.14.1, Linux 7.0, UD-IQ4_XS with the experts read from the GGUF in place (engine 7695859a; the later prompts on a0116b27): | prompt | met more than 50,000 migrated pages | of those, 0.1 s or more slower | |---|---:|---:| | first 4K prompt after a fresh start | 11 of 50 | 10, by 0.17-1.60 s (than the second prompt) | | second 4K prompt | 0 of 50 | - | | later 4K-128K prompts of an A/B | 3 of 138 | 2, by 0.12 and 0.27 s (than the median of the same kind) | - KFD queue evictions, read as root through `AMDKFD_IOC_SMI_EVENTS` over 20 starts: in all 6 prompts that met any page migration, in none of the 34 that did not. - `GPU_PINNED_MIN_XFER_SIZE=1048576`, `HSA_USERPTR_FOR_PAGED_MEM=0` and the two compaction sysctls each left slow first prompts (worst 0.39, 1.27 and 0.25 s in 10 starts each). - Still seen on 296b59d2. The runs, the KFD event log and a script that recomputes these numbers: https://github.com/libratechw/Strata/tree/a13f5285516c3f39090a8705103084d94a9fecb7 ## Extra Notes - One machine, one kernel and ROCm version. Whether the restore goes through the `-EAGAIN` retries of the section above was not traced. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
No site
Links install, modelos, releases.