#3237 [Engine] Linux file tier: the room rule still picks the file cache on 0.2.0-beta, 2x slower prefill and 3-4x slower decode than unbuffered reads

Description

### Area

[Engine]  (speed, kernels, models, the server and API)

### Desktop app, Terminal or source?

Source (START-HERE.bat, setup.sh or a build)

### What happened

On 0.2.0-beta, with a model whose file tier is much larger than the free RAM, the automatic choice (`file_tier_unbuffered`, the #1194 room rule) reads the experts through the page cache. On this machine that path reads prompts at about half the speed of `STRATA_UNBUFFERED_LOAD=1` and decodes at a third to a quarter of its speed, and the OS reads up to 3x the expert bytes from the drive. #1682 and #2344 reported this on 0.1.4x (#2344: +43% prefill unbuffered on an RTX 4070 SUPER with 32 GB). Both were closed in the 0.2.0 reset, so this is the re-test on 0.2.0-beta.

What the engine chose by default:

```
strata generate: the file tier reads through the file cache (46.2 GiB available, 38.0 GiB of it still to be taken by the RAM copy, 63.7 GiB of experts read from the files: the file cache can keep the ones that come back)
strata generate: the file tier reads through the file cache (re-checked with the RAM copy built (38.00 GiB): 6.4 GiB available, 0.0 GiB of it still to be taken by the RAM copy, 63.7 GiB of experts read from the files: the file cache can keep the ones that come back)
```

**Results**, round 1 / round 2 (each arm a fresh server, page cache dropped before it, round 2 in reversed arm order; 2026-10-19 12:03-12:19):

| arm | prefill ~4.5K tok/s | decode tok/s | prefill ~13K tok/s | decode tok/s |
|---|---|---|---|---|
| default (file cache) | 184.9 / 179.0 | 5.0 / 6.0 | 274.4 / 258.4 | 3.4 / 5.8 |
| `STRATA_UNBUFFERED_LOAD=1` | 372.8 / 365.6 | 19.2 / 19.2 | 601.0 / 622.8 | 18.3 / 19.1 |

Per-request lines from the engine log:

- default: `the OS read 180282.2 MB from storage (222370 major faults) for 60804.5 MB of expert reads; page cache held 38535.4 MB at the read` (round 2: `91715.3 MB ... (68704 major faults) for 31969.8 MB`). Every logged request had 10,615 to 222,370 major faults.
- unbuffered: the OS read matches the expert reads within 0.5%, for example `124989.5 MB ... (0 major faults) for 124529.8 MB`, with 0 to 3 major faults per request.

**Expected:** with 6 GiB free after the RAM copy and a 63.7 GiB read set, the cache holds about a tenth of what is read. #1194 measured the cache winning when it holds a twentieth or more of a skewed read set; on this drive and model it loses at about a tenth. (The opposite case, #577 on a 96 GB PC where the cache holds the whole read set, should keep the cache.)

**A possible fix, offered, not assumed.** I carry a local, opt-in check, `STRATA_FILE_TIER_PROBE=1`. It runs only where the room rule picks the cache but the cache cannot hold the whole read set. It times the drive's direct reads (depth 8) and cached-path reads (depth 3) once at load, then reads unbuffered when the cache holds under 30% of the read set, or 30-50% with direct reads at least 1.5x faster. Here it logged `the cache holds 13% of the read set, direct reads 8.18 GB/s at depth 8, cached-path reads 4.93 GB/s at depth 3 (1.66x)`, chose unbuffered, and measured:

| arm | prefill ~4.5K | decode | prefill ~13K | decode |
|---|---|---|---|---|
| `STRATA_FILE_TIER_PROBE=1` | 377.7 / 366.4 | 20.9 / 17.8 | 605.3 / 573.8 | 15.0 / 17.5 |

The thresholds come from runs on this one machine with UD-Q4_K_XL and UD-Q6_K_XL, unthrottled and with the drive capped by systemd's `IOReadBandwidthMax` down to 250 MB/s (on 0.1.45). If it is wanted, I can open it as a draft PR. It builds for CUDA; I have no HIP or SYCL toolkit, so those builds are not run. If a different rule fits better, the numbers are yours to use.

**Not tested:** AMD/HIP, Windows, ext4, a slower drive on 0.2.0 (the throttled runs were on 0.1.45), a page cache warm from an earlier run.

### Smallest steps or command that reproduce it

1. Start UD-Q6_K_XL from source with `--resident-budget-gib 38` on a 62 GB PC, with no file-tier variable set. The start log says `the file tier reads through the file cache`.
2. Send a fresh ~4.5K-token prompt and then a fresh ~13K-token prompt, 256 output tokens each.
3. Repeat with `STRATA_UNBUFFERED_LOAD=1`.

The script, the config and the raw output of both rounds: https://gist.github.com/vthomasino/aef6f8e2413733cbe149a91178e889d8

### Reproduction status

Reproduced locally

### Strata version

Strata 0.2.0-beta · engine 0.1.45.3 · tag v0.2.0-beta (b73dbce3) plus one local commit of opt-in code (STRATA_FILE_TIER_PROBE, STRATA_FILL_SPANS, a debug trace), not active in the default and unbuffered arms

### Graphics card, memory and system

RTX 5090 Laptop 24 GB (driver 610.57.04, CUDA 13.2) · 62 GB RAM · Core Ultra 9 275HX · Samsung 9100 PRO 2 TB PCIe 5.0, btrfs · Linux 7.2.5 (Omarchy/Arch)

### Model, size and settings

unsloth UD-Q6_K_XL (158 GB GGUF read in place) · 256K context · pictures on · `--resident-budget-gib 38 --kv int8 --kv-resident 131072 --spec 4 --mtp --pool-workers 15 --vram-reserve-mib 1700`, env `STRATA_MMQ_KQ=1 STRATA_HC_Q8=1`

### Logs

The engine's choice and per-request lines for every arm are in `ab-q6-020.out` in the gist above.