#5937 [Engine] Windows/gfx1030: working set doubles after ordinary serving (19.5 -> 36 GB) and never returns to baseline
open · @mircolike · 2 comments · View on GitHub
AMD / HIPNVIDIA / CUDAModels & quantsWindowsLinux
Description
### Area [Engine] ### Desktop app, Terminal or source? Desktop app (Strata 0.2.13 GUI) launching the bundled engine. Not built from source. ### What happened On Windows, a **single uninterrupted serving session** roughly doubles the engine's working set, and the memory never comes back after the requests finish. The machine ends up with ~2 GB of usable physical RAM and stays there until I restart the engine. Fresh start, before any request: - `strata.exe` working set **19.48 GB**, private 27.84 GB - physical available **18.03 GB**, load 61 % After serving a handful of short chat requests plus one agent-style session (largest single prompt ~7.6k tokens, everything well inside the 32k window): - `strata.exe` working set **36.02 GB**, private 28.30 GB - physical available **1.98 GB**, load 95 % Note `WS (36.02) > Private (28.30)` by ~7.7 GB, so the retained part looks mapped/locked rather than privately allocated. It does not recover. I sampled `GlobalMemoryStatusEx` every 3 s for 24 minutes with **zero requests in flight**: ``` 8.58 -> 5.00 -> 2.80 -> 2.50 -> 2.00 -> 2.1 -> 2.1 -> 2.1 -> 2.0 GB available (then locked) ``` Expected: the working set should return to something close to the post-start baseline once the last request completes (the automatic resident budget was computed specifically to keep 8 GiB of headroom on this host). On a 47 GB box, 2 GB of usable physical memory means ordinary development work (IDE + browser) starts paging heavily. The only workaround I have is restarting the engine, which costs a full weight reload. ### Smallest steps or command that reproduce it 1. Launch the engine with the flags below on a 47 GB Windows host and note the working set (19.5 GB here). 2. Send ordinary traffic: a few short chat completions plus a handful of ~5-8k-token prompts. Nothing that requires a large KV allocation. 3. Stop sending requests and wait. Sample available physical memory. 4. Working set sits at ~36 GB and available stays ~2 GB indefinitely. ### Reproduction status Consistently reproducible — 3 clean restarts today, same shape every time. I have not tried to narrow it to one specific request shape yet. ### Strata version Strata 0.2.13 · engine 0.1.59 · `windows-x86_64-hip-rdna2` ### Graphics card, memory and system RX 6800 XT 16 GB (gfx1030) · 47.4 GB RAM · Windows 11 · pagefile 7 GB (commit limit ≈ 54.4 GB) · pagefile use stayed under 100 MB throughout, so this is physical RAM, not commit exhaustion ### Model, size and settings Qwen3.8-Flash-Next-Gyro-M **TQ2_0** (native pack) · 32K context · `--kv q4_0` · `--spec 6` · MTP on · `--expert-cache auto` · vision on · `--vram-reserve-mib 1536` · `--pool-workers 7` `--resident-budget-gib` was **not** passed; the engine self-derived it: ``` FileExpertSource: --resident-budget-gib 26.11 exceeds available commit (26.23 GiB minus 8 GiB headroom and a 0.25 GiB margin): 17.98 GiB ``` ### Logs Startup lines that look related — note the arena pins **26 GiB** while the self-derived resident budget was **17.98 GiB**: ``` strata generate: expert arena: locked 3826 MiB via working-set minimum + VirtualLock; cudaHostRegister of the whole arena FAILED (invalid argument); 42 slices pinned (26 GiB); large pages refused ... using 4 KB pages strata generate: loaded 29.88 GiB at 1.25 GiB/s FileExpertSource: WARNING: 2114 of 2114 lendable slots (100%) will read their experts from the pack during a long prompt ``` ### Why I think this is not #5020 / #5411 / #4605 - **#5020** asks to stop keeping the host copy of experts the VRAM cache already holds (Linux, 125 GB box). That is a footprint-at-start request, not a "grows during use and never returns" report. - **#5411** is memory left behind after engine **start/stop cycles** (Linux, host eventually freezes). My session never restarts; it is one process that grows once and stays. - **#4605** is Windows **commit** exhaustion with a 101 GB RAM store. Here commit headroom stayed at ~5 GB and the pagefile was essentially untouched — the physical memory is what disappears. ### Question Is the 26 GiB pin / the ~17 GB growth expected to be released once the last request completes, or is the expert arena deliberately kept resident for the life of the process? If it is deliberate, is there a flag to bound it on small-RAM Windows hosts, or is `--mmap-experts` the intended mode for a 47 GB machine?
Related on this site
Quick links to related pages.