#2101 [Linux, HIP gfx1100] 0.1.42+ PLE probe auto-switches to `--ple-io mmap` on a loop-device disk: decode -20-25%, prefill unaffected; explicit `--ple-io direct` restores 0.1.41 speed

Beschreibung

### Summary

On a single AMD Radeon PRO W7900D, 0.1.43 decodes 20-25% slower than 0.1.41 with the same config. The cause is the start-up PLE probe added in 0.1.42. It measures ~13-15K rows/s on this disk, which is under the 15K threshold, so it switches the default `--ple-io direct` to `mmap`. Passing `--ple-io direct` explicitly brings decode back to 0.1.41's speed.

This is the same mechanism as #2040, #1910 and #1984. Those reports show prompt reads slowing down; here the prompt path is unaffected and the loss is in decode.

### Environment

- GPU: AMD Radeon PRO W7900D (gfx1100, 48 GB), one card, ROCm 7.2.1, HIP build from source.
- Host: a cloud container (Ubuntu 24.04) on a 2-socket EPYC 9334, cgroup limits of 55 GiB of memory and 16 CPUs.
- Disk: the model files sit on an ext4 filesystem on a **loop device** backed by a host file. The probe measured 13,137-14,809 rows/s over four starts (~54-61 MB/s of random 4 KiB reads).
- Model: Qwen3.8-Flash-Next IQ3_S (GSQ-RCO). The PLE table is 320,001,536 rows in shard 2.
- Engine: 0.1.43 (7695859), compared with 0.1.41 (fb58e0d).
- Config:
  - `--resident-experts --max-context 393216 --rope-scaling yarn --rope-scale 1.5 --kv int8 --mtp ... --spec 4 --prefill auto --vision`;
  - `parallel` 1;
  - **no `--ple-io` given**;
  - expert cache 18,202 slots, VRAM hit rate ~99.7% in decode.
- We carry a few small local patches (host-thread CPU affinity, a server-side reasoning-loop mode). None touches PLE or the decode path, and every arm below uses the same build except the 0.1.41 one.

Boot log, 0.1.43 default:

```
strata generate: --ple-io direct measures slow here (13192 rows/s, about 54 MB/s of random 4 KiB reads): using --ple-io mmap instead (the OS file cache holds the pages; --ple-io direct forces the old mode, --ple-io ram keeps the table in RAM)
```

### Method

All three arms ran on the same instance and the same GPU. Each arm went through the same steps:
1. stop the server;
2. drop the model files' page cache (`posix_fadvise(DONTNEED)`);
3. start the server;
4. run 2 rounds of the same requests.

The requests, in order:
- a warm-up decode;
- a fixed ~8.9K-token prompt with 256 greedy tokens;
- fresh ~8.8K and ~32.6K-token prompts;
- three turns appended to the 32K conversation (~0.3K, ~1K and ~3K tokens);
- three sampled decodes (temperature 1.0 / top_p 0.95 / top_k 20, seeds 1-3) and one greedy decode, 768 tokens each.

Timing comes from `STRATA_DECODE_TIMING=1`. The table shows the mean of the 2 rounds.

### Results

| | 0.1.41 | 0.1.43 default (auto -> mmap) | 0.1.43 `--ple-io direct` |
|---|---|---|---|
| fresh 8.8K prompt, tok/s | 1,890 | 1,865 | 1,887 |
| fresh 32K prompt, tok/s | 2,433 | 2,443 | 2,437 |
| appended 0.3K / 1K / 3K turns, wall s | 2.05 / 1.67 / 2.12 | 2.27 / 1.82 / 2.50 | 2.05 / 1.65 / 2.15 |
| sampled decode, tok/s (mean of 3 seeds) | 62.0 | **47.4** | 59.0 |
| greedy decode, tok/s | 67.2 | **54.7** | 67.0 |
| ms per verify window (sampled) | 30.2 | **39.1** | 31.2 |
| of which GPU-reach wait, ms | 23.3 | 23.2 | 23.1 |
| first decode after start (warm-up, round 1), tok/s | 55.5 | 33.2 | 48.5 |

- **The extra time is on the host.** The GPU wait per window is unchanged, while the window grows by ~9 ms with mmap. It is worst right after start (55 ms per window on the first decode) and shrinks as the page cache warms (39 ms in round 2), which looks like the host thread blocking on PLE page faults.
- **Answers are identical.** The fixed prompt gives the same output hash in all three arms, and so does the first decode after start.
- **With direct the gap closes:** greedy decode matches 0.1.41, and the sampled -5% is inside the 59-62 tok/s spread we saw across restarts of 0.1.41. Only 2 rounds per arm, so small differences are noise.
- One more observation from the direct arm: the first fixed-prompt decode after start was slow (48 tok/s against 76 on 0.1.41), and round 2 was normal (79 vs 82). We saw it once and did not look into it.

### Notes

- The probe's readings on this disk (13.1K-14.8K rows/s) sit just under the 15K line, so the mode can differ from one restart to the next.
- The probe's cold, low-depth read predicts neither the prompt path (#2040) nor decode (here). Either of the proposed fixes would avoid this:
  - keeping direct unless the user opts in (PR #2073);
  - measuring both modes and picking the faster one (#2040).
- The warning printed with an explicit `--ple-io direct` ("a prompt can spend minutes reading layer 1 and trip the stall watchdog") did not apply here: prompts read at 0.1.41 speed.

### Workaround

Put `--ple-io direct` in the config's `args`.

Related: #2040, #1910, #1984, PR #2073.