#2344 [Bug]: Linux/HIP prefill regression from file-cache heuristic 805088d5, still present in v0.1.44

说明

## Summary

Linux/HIP prefill regressed after **v0.1.40.1** and remains affected in **v0.1.44 (`ed4773eb`)**. Testing [805088d5](https://github.com/Niko1221/Strata/commit/805088d5b61a95912c5d19797053339b9aa979f7) against its immediate parent isolates the slowdown to Linux expert file-cache selection. `STRATA_UNBUFFERED_LOAD=1` restores performance on the unmodified latest build.

Related: #1682 and #1407, closed when v0.1.44 shipped. This adds current-version reproduction and a consecutive-commit comparison on one GPU.

## Environment

RX 9060 XT 16 GiB (gfx1200), Ryzen 5 9600X, 32 GiB RAM, approximately 30 GiB swap; CachyOS kernel 7.2.9-2-cachyos; ROCm 7.2.4. Model: Qwen3.8-Flash-Next-GSQ-RCO **IQ3_XXS**. Identical pack, GGUF shards, expert profile, MTP and control vector across arms.

## Measurements

**4096 input / 64 output tokens, five fresh-process runs per commit:**

| Commit | Prefill median (range), tok/s | Decode median tok/s | Request major faults, median |
|---|---:|---:|---:|
| Parent `58b1ea221aad1950597b40462be19daae24bc45a` | 361.15 (357.23–363.40) | 31.41 | 0 |
| `805088d5b61a95912c5d19797053339b9aa979f7` | 154.16 (55.48–194.34) | 11.34 | 61,104 |

**57.31% lower median prefill**, with the same 4901 expert-cache slots / 7.95 GiB and no intervening HIP kernel change. Median peak RSS rises from 18,580 to 27,342 MiB.

At **4096 tokens, five warm repetitions**, v0.1.40.1 defaults give **367.42 tok/s**; v0.1.44 with direct I/O gives **343.51 tok/s**, or **361.83** with `--aux-cpus auto`.

On v0.1.44 defaults, repeated **1024-token** fresh requests deteriorate through **71.25, 81.30, 33.43, 17.73, 7.96 tok/s**, then the next request hits the 60-second watchdog at layer 33. Fresh restarts can initially exceed 100 tok/s; the deterioration depends on cache/memory history.

## Cause and workaround

The changed Linux heuristic requires cache room for only **5% of file-tier reads**, at least **1.5 GiB**, with a 1 GiB margin. Logs show it selecting buffered reads with about **9.7 GiB available for a 24 GiB read set** after the 16 GiB resident copy. The parent instead selects unbuffered reads.

Broad prefill appears to defeat the decode-oriented hot-set assumption. During a stall, `perf stat` records **88,625 major faults in five seconds**, copy threads wait in `folio_wait_bit_common`, and swap/reclaim counters rise sharply. The mode switch also changes staging concurrency, so its separate contribution remains uncertain.

## Reproduction

- Build both commits with identical Release/HIP/gfx1200 settings and the same ggml archive (`3cf03257f219afbe7334045ff7c6a06ac68c627d`): PREFILL_MMQ/Q6K_EXPERTS on, MMQ_KQUANTS/CUDA off.
- Reuse identical model assets and fixed token IDs. Core settings:

```text
--resident-budget-gib 16 --expert-cache auto --prefill auto
--max-context 131072 --kv int8 --vram-reserve-mib 1000
--spec 4 --spec-min-p 0.70 --pcie-frac 0.75 --pool-workers 2
--prompt-cache 0 --prompt-cache-root 0
--conversation-cache-mib 0 --conversation-cache-slots 0
```

- Use native `--serve`, sending `GEN 64 ckpt=0 temperature=0 <token-IDs>` after READY. Check zero reused tokens and exact fresh counts. Restart five times for cold comparisons; use first request plus five repetitions for warm comparisons. Loading is excluded; OS caches are not globally flushed. Run sequentially without competing inference/builds.
- Repeat v0.1.44 with **`STRATA_UNBUFFERED_LOAD=1`**, keeping a resident budget. Unset timing/IO diagnostics for throughput.

A fixed-cache check (`--expert-cache 3000 --adapt-every 0`) gives **5/5 identical buffered/direct token-output hashes** at 1024 input / 64 output tokens.

Separately, direct I/O + aux affinity + the gfx1200 hipBLASLt 100202 tuning table reaches **604.16 tok/s prefill / 45.06 tok/s decode** at 4096 tokens. Tuning changes some logits/output wording and is not required for the conservative workaround.

Could automatic expert I/O selection account for broad prefill working sets or measured I/O behavior? Full raw measurements/scripts are retained locally. Investigation was assisted by Codex; all performance numbers are from actual local runs.