#2918 ple: no per-row MADV_WILLNEED on a locked table (#2281)

Description

## Title

Issue: #2281 (closed). There, `--no-ple-prefetch`, the second toggle you suggested, brought ben0112's speed back, and @ben0112 suggested skipping the hint once `--ple-io ram` has locked the table. ben0112 also offered to test a patch build. This is that change.

## Summary

With `--ple-io ram`, the PLE table is locked in RAM when it opens. On Linux, `PleTable::gather_batch` still calls `madvise(MADV_WILLNEED)` for every row it reads: 16 calls per token, on the prompt path and in the decode windows. Those pages are already resident. This change makes the POSIX branch of the hint return when the table is locked.

On ZFS, each of those calls becomes an ARC prefetch of the record that holds the row. When the record is not in the ARC, it is read from disk. #2281 has the trace and the source links.

Measured on 4x Tesla P100 with Qwen3.8-Flash-Next IQ3_XXS, the root file system on ZFS and an 8 GiB ARC (details below):

| `--ple-io ram`, new ~16K-token prompts | prefill tok/s | disk reads per prompt |
|---|---:|---:|
| 0.1.45, default | 466-479 | 17.2-18.4 GB |
| 0.1.45, `--no-ple-prefetch` | 685-705 | 0.00 GB |
| this branch, default | 685-706 | 0.00-0.01 GB |

Each row is two fresh server starts with the same three prompts. Per prompt, this branch is within 0.3% of `--no-ple-prefetch`, and the answers were byte-identical in every run. Decode was 2.3% slower with the hint. This branch decoded 0.2-0.3% below 0.1.45 with `--no-ple-prefetch`, and so did this branch with the flag (run 7), which never reaches the new line.

I also copied the table to ext4 on the same machine. There the hint made little difference: 0.1.45 prefilled at 684-707 tok/s with and without `--no-ple-prefetch`, within 0.5% per prompt, and decoded within 0.4%. If your 2x A4000 machine is not on ZFS, this fits what you saw there.

ben0112's machine also has the model on ZFS (#2571). On POSIX, `--no-ple-prefetch` turns off only this hint, so on a locked table this branch skips the same calls as the flag that fixed it there. I have not run the branch on that machine.

## What changed

- `src/kernels/ngram.cpp`: the POSIX branch of `hint` in `gather_batch` returns when `impl_->locked` is set. On POSIX, `locked` is only set after `mlock` of the whole table succeeds, so a table whose lock failed keeps its hints. Besides configs that ask for `--ple-io ram`, setup writes it on Linux when the model is on a rotational disk and the RAM has room for the table on top of the model (#605).
- `src/ngram/ple_reader_test.cpp`: `format_round_trip` gets a third leg, "ram" (mmap with `lock`). It checks `read_row`, `gather` and `gather_batch` on a locked table against each format's dequantizer. If `mlock` is refused, the leg reads the table unlocked and prints a note.

Not changed: the whole-table hint when the table opens, `--ple-io mmap`, `--ple-io direct`, `--no-ple-prefetch`, and Windows. On Windows the hint in `gather_batch` is one `PrefetchVirtualMemory` call per window of up to 4096 rows, and `gather`, `issue` and `prefetch_rows` hint there too; I have no Windows machine to measure a change on. With this branch and `--ple-io mmap`, the engine made the same 767,936 `madvise` calls in three prompts as 0.1.45 with `--ple-io ram`.

## Extra Notes

- `ctest -R ple_reader_selftest` passes. The selftest also passes under `prlimit --memlock=0`, where the "ram" leg reads the table unlocked.
- The selftest checks values, so it cannot see a hint. A local harness linked with `-Wl,--wrap=madvise` counted the calls inside `gather_batch` for all seven formats, with one row per batch past the table's end, which gets no hint. On a locked table it counted 15, 255 and 11,199 calls for 1, 16 and 700 tokens before this change, and 0 after. Unlocked tables and tables whose lock failed kept every call. I can add it as a Linux-only test if you want one.
- Built: the CUDA engine (CUDA 12.9.1, sm_60, the flags setup.py uses for a CUDA 12 build) and `ple_reader_test`. I could not build HIP, SYCL or the Mac engine here. All three compile this file. The change is one `if` on a member they already have, with no new include or call, and HIP and SYCL on Linux and the Mac build all take this POSIX branch.
- Not tested: Windows, macOS, the HIP and SYCL builds, this branch on ext4, a larger ARC, bare metal, and ben0112's machine.

<details><summary>Where the time goes on ZFS</summary>

ben0112's last comment in #2281 puts the cost on the ZFS ARC, competing for disk I/O and memory bandwidth. The disk reads are ARC prefetch misses: 17-18 GB per prompt read into an 8 GiB ARC. Inside the hint, though, the threads almost never blocked. In a pass with kprobes on the engine's threads, the threads that issue the hints entered `zpl_fadvise` 768,192 times, slept there 3 times and were preempted 8,553 times. `arc_wait_for_eviction` never slept and took 1.0% of their time in `zpl_fadvise`.

The cost was CPU time. System time on the whole machine (8 vCPUs) averaged 3-5 s per prompt without the hint and 37 s with it. At least 15.8 s of that was in ZFS's read-completion threads (`z_rd_int`). Under ftrace, `zpl_fadvise` averaged 39 µs a call counting only time on a CPU, and 95-96 µs counting time off the CPU too. I did not measure memory bandwidth.

</details>

<details><summary>Environment and method</summary>

- 4x Tesla P100 PCIe 16 GB (driver 535.230.02), VMware VM, 8 vCPUs (Xeon E5-2696 v4), 62.8 GiB of RAM.
- Ubuntu 22.04.5, kernel 6.8.0-138. Root on ZFS: OpenZFS kernel module 2.2.2-0ubuntu9.4, recordsize 128K, lz4. `zfs_arc_max` is 8 GiB, to leave RAM for the locked table and the experts.
- Qwen3.8-Flash-Next GSQ-RCO IQ3_XXS. The PLE table is in shard 2: 320,001,536 rows x 90 B = 26.8 GiB. `VmLck` was 28,125,136 kB in every `ram` run.
- 0.1.45 (`477ac13`) and this branch on top of it, built from two checkouts with the same flags.
- Server: 4 slots, layer split 13,26,38. Engine flags (abridged): `--max-context 32768 --kv int8 --expert-cache auto --prefill 2048 --pool-workers 6 --batch-groups auto --vram-reserve-mib 1536 --ple-io ram --spec 4 --spec-min-p 0.5 --resident-experts --trim-stage-weights --no-kv-grow --conversation-cache-mib 8192`.
- Each run is a fresh server start under systemd with `LimitMEMLOCK=infinity`, one request at a time. Prompts are slices of Strata's own docs with a fixed nonce in front, so every run gets the same bytes. Nothing was reused: the engine logged 0 reused tokens for all 141 prompt reads. Temperature 0, thinking off. Three prompts of ~16K tokens with 32 tokens out, and three decodes of up to 512 tokens.
- Speeds are the engine's own figures. Disk reads come from `/sys/block/sda/sda4/stat` (the pool's partition in the VM), read before and after each request. System time is the `system` column of `/proc/stat`. No run that recorded steal time had any.
- The traced passes ran on other prompts, after the untraced ones: `perf stat -p <engine pid>` for the `madvise` counts, ftrace's function profiler (once more in run 3, counting only time on a CPU), and the kprobe pass in run 8. Tracing slows the hint, so none of the speeds in this description comes from a traced request.
- ext4: the PLE shard was copied to an ext4 file system on a loop file, itself on the ZFS pool. Shard 1 is a symlink back to ZFS, so that `--native` and `--ple-gguf` name the same directory. The locked table then comes from ext4's page cache.

</details>

<details><summary>Every run, in order</summary>

Prefill and decode are ratios to the first `--no-ple-prefetch` run on the same file system (run 1 on ZFS, run 9 on ext4). Each value is the mean ratio over the same three requests.

| # | file system | build | flags | prefill | decode | disk reads per prompt | engine `madvise`, 3 prompts |
|---:|---|---|---|---:|---:|---:|---:|
| 1 | ZFS | 0.1.45 | `--no-ple-prefetch` | 1.000 | 1.000 | 0.00 GB | 192 |
| 2 | ZFS | branch | default | 1.000 | 0.997 | 0.01 GB | 192 |
| 3 | ZFS | 0.1.45 | default | 0.678 | 0.977 | 17.2-18.1 GB | 767,936 |
| 4 | ZFS | branch | default | 1.001 | 0.997 | 0.00-0.01 GB | 192 |
| 5 | ZFS | 0.1.45 | `--no-ple-prefetch` | 1.002 | 0.999 | 0.00 GB | 192 |
| 6 | ZFS | branch | `--ple-io mmap` | 0.293 | 0.812 | 17.2-17.8 GB | 767,936 |
| 7 | ZFS | branch | `--no-ple-prefetch` | 1.000 | 0.997 | 0.00-0.01 GB | 192 |
| 8 | ZFS | 0.1.45 | default | 0.675 | 0.977 | 17.8-18.4 GB | 767,936 |
| 9 | ext4 | 0.1.45 | `--no-ple-prefetch` | 1.000 | 1.000 | 0.00-0.03 GB | 192 |
| 10 | ext4 | 0.1.45 | default | 1.000 | 1.002 | 0.00-0.01 GB | 767,936 |
| 11 | ext4 | 0.1.45 | `--no-ple-prefetch` | 1.002 | 1.000 | 0.00-0.02 GB | 192 |
| 12 | ext4 | 0.1.45 | default | 0.998 | 1.002 | 0.00-0.03 GB | 767,936 |

Run 6 only checks that the hint is still sent without a lock. Its page cache was not controlled, so its speed is not comparable.

</details>