Issues / #467
#467 Windows + 32 GB RAM: the low-RAM resident mode never engages - the fit check runs after the engine's own cache warm-up
closed · @icocoon · 1 commentaires · Sur GitHub
BenchmarksSetup & installServer & APIAMD / HIPModels & quantsDocumentationWindows
Description
**Version:** Strata 0.1.34 (ready-made `strata-windows-x64-hip.zip` engine), Windows
**OS:** Windows 11 Pro (build 26300), AMD Software: Adrenalin driver 32.0.31035.1003
**GPU:** AMD Radeon RX 7900 XTX 24 GB (gfx1100)
**CPU:** Ryzen 7 7800X3D (AVX-512) | **RAM:** 32 GB (31.1 GiB usable) | **Disk:** NVMe SSD
**Model:** `ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF` IQ2_XS (revision `ed59f92082b1e93c0e96d60a8b11aab089b52f09`), installed by setup itself (`--family qwen --model IQ2_XS`)
## What happens
The low-RAM resident mode (`--resident-experts`, chosen by setup) never engages on this PC; every start falls back to `--mmap-experts` with:
```
strata generate: WARNING: the resident RAM mode does not fit (FileExpertSource: resident complement 16.20 GiB
exceeds available RAM (0.44 GiB) minus the 4 GiB safety headroom); the experts the GPU does not hold are read
from the model folder through the OS file cache (--mmap-experts), which is slower when the RAM cannot keep them
```
`setup.py --check` itself says this configuration fits ("IQ2_XS needs ~48 GB RAM: fits in the low-RAM mode (the GPU holds ~53% of its experts, the rest stays in RAM)"), and MODELS.md states "a 32 GB PC with a 24 GB GPU runs Q2_0, IQ2_XS and the Coder this way".
## Measurements
I probed memory every 2 s from just before starting the server (setup-written config; `STRATA_RESIDENT_PIN=0` as setup wrote it):
```
T-0 available 20.7 GiB (clean state, right before start)
T+2s available 19.9 GiB strata.exe WS 0.6 GiB
T+10s available 14.5 GiB strata.exe WS 5.9 GiB
T+20s available 7.6 GiB strata.exe WS 12.7 GiB
T+36s available 0.5 GiB strata.exe WS 19.9 GiB
then stable at ~0.5-0.7 GiB
```
The WARNING's position in the engine log shows when the fit check runs - after the engine has fully initialized:
```
strata generate: pre-filled 12551 of 12551 slots from the profile; slot 0 verified
strata generate: R4 hit path ON - resident experts are computed on the GPU
strata generate: 7 expert-pool workers + the host thread
strata generate: session is up (engine 0.1.34)
strata generate: token graph hit path: 12551 resident experts, decided on the device
strata generate: WARNING: the resident RAM mode does not fit (...) <-- here
strata serve: prompt chunk auto: 8192 tokens
```
## The math
- Needed for the resident complement: 16.20 GiB + 4 GiB headroom = **20.2 GiB**
- Available in the clean state: **20.7 GiB** - only 0.5 GiB of margin, before any of the engine's own allocations
- Available once the engine has warmed its own caches: **~0.5 GiB**
So the check cannot pass at the moment it runs, and no value of `STRATA_RESIDENT_HEADROOM_GIB` helps (for any headroom >= 0: 0.5 GiB - headroom < 16.2 GiB). On this machine the docs' "32 GB + a 24 GB card runs IQ2_XS in low-RAM mode" is not reachable on Windows.
## Impact
Not a crash: the mmap fallback works (server healthy, correct answers; an informal test measured roughly 47 tok/s including reasoning tokens). But the resident protection against memory pressure is not available - anything that shrinks the OS file cache sends the missing experts back to the SSD - and the documented 32 GB configuration silently degrades.
## Possible fix direction
Read the available memory for the fit check **before** the engine warms its own caches (or reuse the reading taken at the start of initialization), similar in spirit to the #403 fix ("the safety check uses the same RAM reading"). The file-cache pages the engine itself has just created may be counted against available memory here; excluding the engine's own working set from the reading would also fix it.
Related: #403 (a clamped budget fails its own safety check), #250 (miscalculated limits despite theoretically available space), #286 (Windows low-RAM tier).
Sur le site
Liens install, modèles, releases.