#3059 Windows file tier: reading experts through the file cache after load gave +26% decode on RX 7800 XT (opt-in patch)

Beschreibung

### What do you want to do?

## Summary

When the Windows file tier reads unbuffered (chosen at load when the file cache cannot keep the experts, or forced with `STRATA_UNBUFFERED_LOAD=1`, which both arms below set), every expert outside RAM and VRAM is read with `FILE_FLAG_NO_BUFFERING` once `drop_mapping()` closes the views. On my machine (RX 7800 XT 16 GB, 64 GB RAM, Windows 11) I tried an opt-in change
that keeps the unbuffered handles for the initial load and then reopens `experts.bin` (or the GGUF shards) with
`FILE_FLAG_OVERLAPPED` only, so later expert reads can be served by the Windows file cache.

Same binary, same day, only this switch changed, `STRATA_UNBUFFERED_LOAD=1` in both arms (the unset and `=0` cases are not part of this A/B; with `=0` an earlier build of the same tree at 19 GiB stopped on my 4 GiB free-RAM guard during the first 60K request), 60,047-token fresh prompt (0 reused), three runs each:

| | prompt tok/s | decode tok/s | TTFT s | total s |
| --- | ---: | ---: | ---: | ---: |
| `STRATA_R6_FILE_READ_CACHE=0` | 1,399.8 (1,383.3–1,403.8) | 28.5 (24.6–28.9) | 43.08 | 56.19 |
| `STRATA_R6_FILE_READ_CACHE=1` | 1,476.4 (1,462.1–1,484.4) | 36.0 (32.2–36.3) | 40.74 | 51.20 |
| change | +5.5% | **+26.3%** | −5.4% | −8.9% |

The generated text was identical pair by pair (345 / 381 / 340 tokens). Physical SSD reads fell from 947.9 to
306.7 MiB/s while reading the prompt and from 678.4 to 24.7 MiB/s while decoding, while the engine's requested read rate did not drop (947 → 987 and 678 → 818 MiB/s). A real coding-agent run (6 tasks × 3) went from a median
of 61.45 s to 50.31 s over the 18 runs, 18/18 functional in both.

Full report, per-run data and launch configs: the community benchmark folder
`bench/results/2026-10-18-community-rx-7800-xt-windows/` (PR #3057).

This looks like the Windows counterpart of #1194 (Linux: cache vs `O_DIRECT` on the file tier, fixed for Linux in
0.1.40.2). It also touches the question in #2344, where the buffered path made prompt reading slower on some Linux
machines; here the prompt got slightly faster, but this is overlapped ReadFile through the Windows cache on one Windows machine, and I did not measure page faults.

## What the patch does

`r6-windows-cached-readfile.patch`, against `v0.1.44.1`, 57 added lines, one function
(`FileExpertSource::drop_mapping`, `_WIN32` branch). `drop_mapping` on today's `main` is the same as in 0.1.44.1.

- Off unless `STRATA_R6_FILE_READ_CACHE=1` (the name is from my experiment; any name is fine). With it unset the code
  path is unchanged.
- Runs only after the final view, section and cached handle are closed, so the initial RAM/GPU fill still reads
  unbuffered. `open_direct()` and the startup unbuffered path are not touched.
- Before switching, it reads three blobs (first, middle and last layer/expert) with the original unbuffered handles,
  reopens every file with `FILE_FLAG_OVERLAPPED` only, reads the same three blobs twice and compares bytes. If any open
  or compare fails, it closes the new handles and restores the original ones. The result is written to the startup log
  line (`R6 Windows cached ReadFile active; ... byte-probe PASS (3 blobs x 2 reads)`).
- The existing `read_direct()` overlapped reads are reused unchanged.

## Caveats

- **The measured binary is a private build**: 0.1.44.1 plus other local changes (a compact MMQ buffer, a persistent fetch team, CPU/IO overlap, @vizdatom's layer-major prefill from #1670 ported to this tree, and libratechw's opt-in HIP prompt-attention patches, not enabled in these runs; listed in the report). The A/B isolates this switch inside that
  build, but I have **not** measured the patch alone on top of a clean 0.1.44.1 or `main`.
- I built and ran it with HIP on Windows only. The change is inside `#if defined(_WIN32)` and has no GPU-backend code,
  but I did not build CUDA or SYCL.
- One machine: RX 7800 XT 16 GB, Ryzen 5 9600X, 64 GB DDR5, NVMe, expert RAM budget 19 GiB, Flash-Next IQ3_XXS,
  64K context, int8 KV. With 64 GB of RAM there is room for the file cache; a machine with less RAM may behave
  differently (the file cache competes with the resident arena and the commit limit).
- OFF ran before ON and the OS file cache was not flushed between arms, so this is not a cold-SSD comparison.
- MTP GPU fence waits still happen with the change (1 in the ON coding runs, 3 in the OFF runs); I do not claim it fixes
  them.

## Question

Would an opt-in switch like this be acceptable for Windows, or would you rather fold it into the existing
cache-vs-unbuffered decision (as #1194 did for Linux)? If useful, I can open a PR against current `main` with the patch
alone and re-measure it on a clean build first.

## Patch

```diff
diff --git a/src/core/expert_source.cpp b/src/core/expert_source.cpp
index 4878c11..1634a65 100644
--- a/src/core/expert_source.cpp
+++ b/src/core/expert_source.cpp
@@ -1723,6 +1723,61 @@ bool FileExpertSource::recheck_unbuffered(std::string& why) {
 bool FileExpertSource::drop_mapping(std::string& why) {
 #if defined(_WIN32)
     if (direct_.empty()) { why = "the reads are not unbuffered"; return false; }
+    // R6 opt-in, called only AFTER the final view/section/handle is closed.
+    // open_direct and begin/end_startup_unbuffered stay completely unchanged.
+    // The current native pack uses experts.bin; the GGUF fallback is also
+    // supported, without enabling cached reads during initial RAM/GPU fill.
+    auto r6_reopen_cached = [&] {
+        const char* r6_cache_env = std::getenv("STRATA_R6_FILE_READ_CACHE");
+        if (r6_cache_env != nullptr && std::atoi(r6_cache_env) == 1) {
+            std::vector<void*> original_handles = std::move(direct_);
+            direct_.clear();
+            std::vector<std::vector<uint8_t>> reference;
+            const int64_t layers[3] = {0, n_layers_ / 2, n_layers_ - 1};
+            const int64_t experts[3] = {0, n_expert_ / 2, n_expert_ - 1};
+            bool ok = n_layers_ > 0 && n_expert_ > 0;
+            // Read the reference with the original unbuffered handles.
+            direct_.swap(original_handles);
+            for (int i = 0; ok && i < 3; ++i) {
+                reference.emplace_back((size_t) layer_blob_bytes_[(size_t) layers[i]]);
+                const Fill f{0, layers[i], experts[i], reference.back().data()};
+                ok = !reference.back().empty() && read_direct(&f, 1);
+            }
+            direct_.swap(original_handles);
+            std::string reopen_why;
+            for (const std::string& path : paths_) {
+                if (!ok) break;
+                const int wide = MultiByteToWideChar(CP_UTF8, 0, path.c_str(), -1, nullptr, 0);
+                std::vector<wchar_t> w((size_t) (wide > 0 ? wide : 1), L'\0');
+                if (wide > 0) MultiByteToWideChar(CP_UTF8, 0, path.c_str(), -1, w.data(), wide);
+                HANDLE h = CreateFileW(w.data(), GENERIC_READ, FILE_SHARE_READ, nullptr, OPEN_EXISTING,
+                                       FILE_FLAG_OVERLAPPED, nullptr);
+                if (h == INVALID_HANDLE_VALUE) {
+                    reopen_why = "cached CreateFileW error " + std::to_string((unsigned long long) GetLastError());
+                    ok = false;
+                    break;
+                }
+                direct_.push_back(h);
+            }
+            for (int i = 0; ok && i < 3; ++i) {
+                std::vector<uint8_t> check(reference[(size_t) i].size());
+                const Fill f{0, layers[i], experts[i], check.data()};
+                for (int repeat = 0; ok && repeat < 2; ++repeat) {
+                    ok = read_direct(&f, 1) &&
+                         std::memcmp(check.data(), reference[(size_t) i].data(), check.size()) == 0;
+                }
+            }
+            if (ok) {
+                for (void* h : original_handles) close_direct(h);
+                why += "; R6 Windows cached ReadFile active; flags=0x40000000; byte-probe PASS (3 blobs x 2 reads); source=";
+                why += maps_.empty() ? "experts.bin" : "GGUF shards";
+            } else {
+                for (void* h : direct_) close_direct(h);
+                direct_ = std::move(original_handles);
+                why += "; R6 cached ReadFile NOT activated; original handles restored: " + reopen_why;
+            }
+        }
+    };
     if (!role_ptr_.empty() || !maps_.empty()) {
         // the GGUF in place: nothing in the engine reads the shards through these views any more (the token embedding is
         // copied into pinned host memory at load, the dense weights and the head into VRAM, the PLE table has its own
@@ -1742,6 +1797,7 @@ bool FileExpertSource::drop_mapping(std::string& why) {
         std::snprintf(note, sizeof note, "%zu GGUF shard%s unmapped; every expert outside RAM and VRAM is read unbuffered",
                       maps_.size(), maps_.size() == 1 ? "" : "s");
         why = note;
+        r6_reopen_cached();
         return true;
     }
     if (base_ == nullptr || unmapped_) { why = base_ == nullptr ? "not open" : "already closed"; return unmapped_; }
@@ -1753,6 +1809,7 @@ bool FileExpertSource::drop_mapping(std::string& why) {
     if (file_ != nullptr) CloseHandle((HANDLE) file_);
     file_ = nullptr;
     why = "every expert outside RAM and VRAM is read unbuffered";
+    r6_reopen_cached();
     return true;
 #else
     // Linux's page cache has no such lock: only the tests close the GGUF views (STRATA_TEST_UNMAP_GGUF=1), to run the
```

Full experiment branch: https://github.com/durikim1-cpu/Strata/tree/rx7800xt-experiments