Pull requests / #2823
#2823 Q36: release the GGUF mapping and stop zeroing the overwritten arena
open · @ggeorgovassilis · 0 comentários · No GitHub
BenchmarksAMD / HIPNVIDIA / CUDAModels & quantsDocumentation
Descrição
## Summary
Implements the Q36 memory-compaction plan (`docs/Q36_MEMORY_COMPACTION.md`): the engine reports its own memory/compaction counters, releases the GGUF mapping after load, and stops zeroing the 18.75 GiB expert arena it immediately overwrites. The original goal - quiet the `kcompactd` churn - was already met by host settings; the measured engine win is ~18.5 GiB of RAM returned to the OS at no decode cost.
## What changed
- `include/strata/platform/mem_report.{hpp,cpp}` (new) + `src/platform/mem_report_test.cpp`: read `/proc/meminfo`, `/proc/vmstat` and the THP mode into the engine's own log, behind `STRATA_MEM_REPORT` (off by default). A row the kernel does not export prints `-`, not 0.
- `include/strata/artifact/gguf_reader.hpp`: `GgufFile::release_pages()`/`released()` and `GgufModel::release_pages()`/`released()` — `madvise(MADV_DONTNEED)` then `posix_fadvise(POSIX_FADV_DONTNEED)` (fadvise cannot drop pages that are still mapped).
- `src/qwen35/q36_engine.cu`: `release_model_mapping()` after `place_experts()`, on by default; `STRATA_Q36_RELEASE_MAPPING=0` reverts. A guard makes `expert_blob()` fail loudly if it is ever called after the release. The arena memset now zeroes only the stride padding and the `kMmqTail` tail (plus an NVFP4 scale tail) instead of the whole 18.75 GiB.
- `tests/core/expert_blob_padding_test.cpp` (new): the padding-only zeroing is byte-identical to the whole memset.
## Extra Notes
Measured on an RTX 4060 Max-Q 8 GiB, Ryzen 7 7735HS, 62 GB, 32K, `--kv int8 --mtp 2`:
| | Cached | MemFree | decode tok/s |
|---|---|---|---|
| before | 47.27 GiB | 3.47 GiB | ~10 |
| after | 28.75 GiB | 22.03 GiB | ~10 |
`Mlocked` stayed 104 KiB throughout - `cudaHostAlloc` on this driver (580.178.04) is memfd/shm-backed, not mlocked, so the plan's "un-pin the arena" phase and its dependent were dropped.
Built: CPU suite 44/44, CUDA build, and the native-expert parity suite on the card (14/14). HIP and SYCL builds were not run here (no ROCm/SYCL on this box).
No site
Links install, modelos, releases.