#6551 [Engine] STRATA_ARENA_MMAP on Windows (mapped experts.bin, streamed first fill)

open · @mirifiuto135-debug · 0 comments · View on GitHub

BenchmarksSetup & installServer & APINVIDIA / CUDAModels & quantsDocumentationWindowsLinux

Description

_(Written by Claude, Anthropic's AI, on Ale's behalf. She isn't a developer and has me do the code and the write-ups.)_

This is #1002 again, re-cut on the new `main` (6e38f588) as you suggested after the history cleanup. Sorry it took us six weeks to follow up.

## What

`STRATA_ARENA_MMAP=1` (the mapped expert arena from #640) is still POSIX-only on `main` (`arena_mmap = false` under `_WIN32`). This ports it to Windows, in `engine/src/core/expert_source.cpp` only:

- **open:** `experts.bin` mapped read-only with `CreateFileMapping` / `MapViewOfFile`
- **prefetch:** `PrefetchVirtualMemory`; **release:** `VirtualUnlock` on the unlocked pages (drops them from the working set, they stay on the standby list)
- **close:** `UnmapViewOfFile`
- **first start on Windows** fills `experts.bin` through a **writable mapping** with `load_experts_gguf`, the same loader that fills the RAM arena, so the arena is never held in RAM (written to `.tmp`, then renamed; refused with a message when the drive lacks the space). The POSIX first start is unchanged.

One line in `docs/DETAILS.md` says "Linux and Windows" instead of "Linux".

## Measured on this branch

Windows 11, 3x RTX 5060 Ti 16 GB, 128 GB RAM, Qwen3.8 Flash-Next UD-Q6_K_XL native pack. `strata generate`, a 2,047-token prompt, 128 tokens, `--spec 4` with MTP, `--kv int8`, `--expert-cache auto`, `--prefill auto`, layer split over the three cards:

| | RAM arena (no env) | `STRATA_ARENA_MMAP=1` |
|---|---|---|
| Lowest free RAM during the run | 12 GiB | 57–65 GiB |
| Decode | 25.3 tok/s | 21.9–22.2 tok/s |
| Prefill | 813 tok/s | 333–372 tok/s |
| First start | – | writes `experts.bin` (101.8 GiB) in 60 s, 1.71 GiB/s |

The log says `expert arena: mapped read-only from …/experts.bin (STRATA_ARENA_MMAP: not locked, not pinned)`. As on Linux, the GPUs get no mapped alias, so no expert goes by DMA (0 against ~4,900 with the RAM arena), and prefill pays for that. That's the trade the option is for.

## Not shown

- **Token identity.** The generated tokens differ between the two modes, but two plain RAM-arena runs on this setup already differ from the 2nd token on (the prompt is random token ids, so greedy picks are near ties, and the adaptive tier moves experts during the run). So this test can't show bit-exactness either way. The arena bytes come from the same `load_experts_gguf` as the RAM path.
- A 32 GB RAM Windows machine. I built for `sm_120` only (MSVC 14.44, CUDA 13.3), with no tests run.

AI assistance: Claude (Anthropic) wrote the port, ran the checks and wrote this description; Ale directed it and owns the hardware it was tested on.

Related on this site

Quick links to related pages.