Pull requests / #3231
#3231 macOS: conversation cache parks again (Metal checkpoint validation, memory guard)
open · @mnott · 0 コメント · GitHub で見る
本文
On macOS (Metal) the conversation cache never parks a conversation, so every turn of an agent conversation re-reads its whole prompt. Two separate causes, one commit each.
## 1. Metal checkpoints never validate
`Window::checkpoint_save` (metal/src/engine/window.cpp) ends with `c.block_pos.clear()`, but `conversation_checkpoint_validate` (src/core/conversation_state.cpp) requires `layers*4` bytes of `block_pos`. Every save is rejected with "invalid checkpoint running-state payload".
The Metal restore (`Window::checkpoint_restore`) never reads `block_pos`, because the window keeps the indexer positions itself. So the validator now expects 0 bytes under `STRATA_GPU_METAL`. The error also names the field with its actual and expected sizes ("invalid checkpoint running-state payload: <field> is N bytes, expected M").
## 2. The memory guard cannot read RAM on macOS
`sample_host_memory()` (src/core/conversation_memory.cpp) has no macOS branch. At start the engine logs "memory guard: the available RAM cannot be read; parking and session save/restore are refused".
The new macOS branch reads free + inactive + purgeable pages through `host_statistics64(HOST_VM_INFO64)` and gets the total from `hw.memsize`. Its value matched `vm_stat` to within 6 MB (24,222 against 24,216 MiB available of 131,072 MiB).
## Measured
On an M-series Mac with 128 GB, 0.2.0-beta engine built from source with `--conversation-cache-mib 8192 --conversation-cache-slots 2` and two Claude Code agents:
- **Before:** 76 of 78 requests had "0 reused", and the log showed "skip parking" 71 times and "parked" 0 times. Each turn re-read 28k–37k tokens, taking 35–50 s.
- **After fix 1, with one agent:** 94% of the prompt tokens were reused (616k of 653k over 37 requests). A two-turn test with a 6.5k-token prompt dropped from 15.9 s to 0.2 s for the second turn (6,520 reused, 17 read).
- **Fix 2:** built, and its RAM reading checked against `vm_stat` with a standalone program. The live run with two alternating conversations is still to come.
## Not covered
The conversation unit tests are CUDA-built and not part of metal/CMakeLists.txt, so none ran on Mac. Whether a restored conversation gives the same output as a full re-read is not separately tested.
## Question
Purgeable pages may already be counted among the inactive pages, so adding them can count them twice. On this machine that is about 40 MB. psutil uses only free + inactive. Which do you want the guard to mean by "available"?
関連リンク
インストール・モデル・リリースへの站内リンク。