#6446 [Engine] Release Metal streaming resources when the window closes

open · @Maryarty · 0 コメント · GitHub で見る

Server & APIModels & quants

本文

## Problem and result

The Metal streaming prompt fallback mapped anonymous memory when borrowing scratch from the expert arena was impossible. Its Metal views used a no-copy wrap with no deallocator, and neither the mapping nor the `PromptGemm` wrappers had a release path when the window closed. The same streaming reader also left the optional `.experts` file descriptor open. In a four-cycle reproducer, the mapping survived each reader teardown and the number of open descriptors grew from zero to four.

## Related discussion

None. This is a focused teardown fix for the existing streaming reader and its scratch.

## Approach

The expert arena returns a region only when it can spare the slots. The ordinary shared Metal allocator supplies the fallback scratch. `PromptGemm` owns the resulting buffers or arena wraps and releases them before the arena and context are destroyed. Scratch is still reused for the lifetime of the window. The reader closes both file descriptors after its worker threads join. GPU operations keep their normal Metal buffer lifetimes.

## Validation

**Passed**

- Release arm64 Metal build at `9fb9a39ddeb35c8279500084022ecdca434117d7` from main `3142b66`. `ctest --test-dir engine/build-metal -R '^runtime_hello$' --output-on-failure` passed on the M1 Max, 32 GPU cores, 64 GiB, macOS 26.6.2 (25G83).
- A four-cycle reader reproducer opened the actual GGUF and verified STRX sidecar, forced the empty-arena fallback, and checked memory with `mincore` and descriptors with `fcntl(F_GETFD)` after the destructor. Main left one descriptor per cycle and a surviving mapping in each cycle. The patched reader left zero in both checks. The reproducer explicitly unmapped each observed baseline mapping after measuring it.
- A Metal reproducer instantiated the patched `PromptGemm`, used the GPU to write into one of two 8 MiB scratch buffers each cycle, then destroyed the owner. `MTLDevice.currentAllocatedSize` returned from 17,235,968 to 458,752 bytes in all four cycles.
- Stock and patched full-model `strata` CLI runs used 192 fixed code prompt IDs, 128 greedy output IDs, Qwen3.8-Flash-Next GSQ-RCO Q2_0 with PLE, MTP width 3, context 4096 and `STRATA_METAL_PROMPT_CHUNK=512`. Both exercised standalone scratch with `STRATA_METAL_ARENA_GB=1` and borrowed scratch with `STRATA_METAL_ARENA_GB=12`. Both pairs matched all 128 output IDs and MTP acceptance (72/114) with zero new Swapouts.
- Patched `strata --serve` with the same 1 GB arena completed two linked requests (192 and 448 input IDs, 128 generated each) with one scratch allocation and zero new Swapouts.

**Failed**

The old reader reproduced the mapping and descriptor leaks described above. The corrected revision passed the lifecycle checks.

**Not run**

Additional Apple generations, and a repeated full engine unload/reload assay that accounts for every allocation in the model. The teardown probes isolate the prompt scratch and the reader descriptors. The changed files are Metal-only.

## Limits

The fix only affects teardown and how fallback scratch is allocated. The resident footprint of a long-running window still includes its live scratch. The single-run CLI timings vary with SSD wait.

関連リンク

インストール・モデル・リリースへのサイト内リンク。