#5937 [Engine] Windows/gfx1030: working set doubles after ordinary serving (19.5 -> 36 GB) and never returns to baseline

open · @mircolike · 2 comentarios · En GitHub

AMD / HIPNVIDIA / CUDAModels & quantsWindowsLinux

Descripción

### Area

[Engine]

### Desktop app, Terminal or source?

Desktop app (Strata 0.2.13 GUI) launching the bundled engine. Not built from source.

### What happened

On Windows, a **single uninterrupted serving session** roughly doubles the engine's working set, and the memory never comes back after the requests finish. The machine ends up with ~2 GB of usable physical RAM and stays there until I restart the engine.

Fresh start, before any request:

- `strata.exe` working set **19.48 GB**, private 27.84 GB
- physical available **18.03 GB**, load 61 %

After serving a handful of short chat requests plus one agent-style session (largest single prompt ~7.6k tokens, everything well inside the 32k window):

- `strata.exe` working set **36.02 GB**, private 28.30 GB
- physical available **1.98 GB**, load 95 %

Note `WS (36.02) > Private (28.30)` by ~7.7 GB, so the retained part looks mapped/locked rather than privately allocated.

It does not recover. I sampled `GlobalMemoryStatusEx` every 3 s for 24 minutes with **zero requests in flight**:

```
8.58 -> 5.00 -> 2.80 -> 2.50 -> 2.00 -> 2.1 -> 2.1 -> 2.1 -> 2.0 GB available (then locked)
```

Expected: the working set should return to something close to the post-start baseline once the last request completes (the automatic resident budget was computed specifically to keep 8 GiB of headroom on this host).

On a 47 GB box, 2 GB of usable physical memory means ordinary development work (IDE + browser) starts paging heavily. The only workaround I have is restarting the engine, which costs a full weight reload.

### Smallest steps or command that reproduce it

1. Launch the engine with the flags below on a 47 GB Windows host and note the working set (19.5 GB here).
2. Send ordinary traffic: a few short chat completions plus a handful of ~5-8k-token prompts. Nothing that requires a large KV allocation.
3. Stop sending requests and wait. Sample available physical memory.
4. Working set sits at ~36 GB and available stays ~2 GB indefinitely.

### Reproduction status

Consistently reproducible — 3 clean restarts today, same shape every time. I have not tried to narrow it to one specific request shape yet.

### Strata version

Strata 0.2.13 · engine 0.1.59 · `windows-x86_64-hip-rdna2`

### Graphics card, memory and system

RX 6800 XT 16 GB (gfx1030) · 47.4 GB RAM · Windows 11 · pagefile 7 GB (commit limit ≈ 54.4 GB) · pagefile use stayed under 100 MB throughout, so this is physical RAM, not commit exhaustion

### Model, size and settings

Qwen3.8-Flash-Next-Gyro-M **TQ2_0** (native pack) · 32K context · `--kv q4_0` · `--spec 6` · MTP on · `--expert-cache auto` · vision on · `--vram-reserve-mib 1536` · `--pool-workers 7`

`--resident-budget-gib` was **not** passed; the engine self-derived it:

```
FileExpertSource: --resident-budget-gib 26.11 exceeds available commit
(26.23 GiB minus 8 GiB headroom and a 0.25 GiB margin): 17.98 GiB
```

### Logs

Startup lines that look related — note the arena pins **26 GiB** while the self-derived resident budget was **17.98 GiB**:

```
strata generate: expert arena: locked 3826 MiB via working-set minimum + VirtualLock;
  cudaHostRegister of the whole arena FAILED (invalid argument);
  42 slices pinned (26 GiB); large pages refused ... using 4 KB pages
strata generate: loaded 29.88 GiB at 1.25 GiB/s
FileExpertSource: WARNING: 2114 of 2114 lendable slots (100%) will read their
  experts from the pack during a long prompt
```

### Why I think this is not #5020 / #5411 / #4605

- **#5020** asks to stop keeping the host copy of experts the VRAM cache already holds (Linux, 125 GB box). That is a footprint-at-start request, not a "grows during use and never returns" report.
- **#5411** is memory left behind after engine **start/stop cycles** (Linux, host eventually freezes). My session never restarts; it is one process that grows once and stays.
- **#4605** is Windows **commit** exhaustion with a 101 GB RAM store. Here commit headroom stayed at ~5 GB and the pagefile was essentially untouched — the physical memory is what disappears.

### Question

Is the 26 GiB pin / the ~17 GB growth expected to be released once the last request completes, or is the expert arena deliberately kept resident for the life of the process? If it is deliberate, is there a flag to bound it on small-RAM Windows hosts, or is `--mmap-experts` the intended mode for a 47 GB machine?

En el sitio

Enlaces a install, modelos, releases.