#2478 CPU prefill share stays unarmed on a 32 GB machine; forcing it gives 1.8-2.2× prompt speed with no failures

本文

RTX 3080 Laptop 8 GB, 32 GB RAM, Linux, Strata 0.1.44.1, Coder IQ1_M, setup's config (`--prefill auto`, resident low-RAM mode), two starts from a cool GPU on the same day:

| | setup's config | + `STRATA_PREFILL_CPU_SHARE=auto` |
|---|---:|---:|
| 4K prompt tok/s (3 runs) | 294 (282-295) | 541 (538-571) |
| 16K prompt tok/s (3 runs) | 283 (282-289) | 628 (611-633) |
| 16K first token | 56.8 s | 25.6 s |
| chat decode tok/s (9) | 24.0 (22.7-29.0) | 25.3 (22.3-35.0) |
| failed requests | 0 | 0 |

Without the env, the log says `the CPU prefill share is not armed by default: the experts it would read are 2.9 GiB of file pages and only 4.3 GiB of RAM is free to keep them`. With it, the engine warns that the share `can slow prompts or fail on a machine this short of memory` (1.0 GiB of file pages, 3.6 GiB free), then runs at the speeds above.

The resident set is decided the same way. Over three starts with the same config, `resident RAM mode` kept 19.43, 21.30 and 23.10 GiB of experts (the last with `STRATA_RESIDENT_HEADROOM_GIB=3`). The first start warned `the whole resident RAM mode does not fit (... resident complement 22.33 GiB exceeds available RAM (23.68 GiB) minus the 4 GiB safety headroom; 24.4 GiB of the RAM is file cache ...)`. We could not separate a decode effect of the resident size (one start each, overlapping ranges).

On a 32 GB machine, prompt speed therefore depends on what ran before the start. Is the free-RAM threshold for the share conservative for the resident mode? Could setup measure the share once and record `STRATA_PREFILL_CPU_SHARE=auto` when it helps?