Pull requests / #2503

#2503 macOS: read available memory for the conversation memory guard

open · @joesinvestments · 0 コメント · GitHub で見る

BenchmarksSetup & installAMD / HIPNVIDIA / CUDAModels & quantsDocumentationWindowsLinux

本文

## Title
Issue: part of #2502 (cause 1 of 2; the Metal snapshot check is the other and is not touched here)

## Summary
On every Mac the engine logs `memory guard: the available RAM cannot be read; parking and session save/restore are
refused`, because `sample_host_memory()` has readings for `_WIN32` and `__linux__` only. This adds a macOS reading:
free + inactive + purgeable pages, the rule `src/core/expert_source.cpp` already uses for macOS, and `hw.memsize` for
the total.

## What changed
- `src/core/conversation_memory.cpp` only (+16 lines): an `#elif defined(__APPLE__)` include of `<mach/mach.h>` and
  `<sys/sysctl.h>`, and an `#elif defined(__APPLE__)` branch in `sample_host_memory()` that feeds `combine()` as the
  other platforms do. The source label stays `meminfo`, to keep the change small.
- Left alone: Windows, Linux (cgroups included), the guard's rules, defaults, setup, docs.

## How it was checked
- Built and run on a Mac Studio M5 Max 64 GB, macOS 27.0.1 (Metal engine; this file is byte-identical in v0.1.44 and
  `main` 296b59d2). The start-up line changes from `the available RAM cannot be read; parking and session save/restore
  are refused` to `memory guard: limit 64.0 GiB (source: meminfo), current 5.1 GiB, reclaimable cache credited 0.0 GiB;
  available 58.9 GiB (MemAvailable 58.9 GiB), floor 2560 MiB`.
- Default path: answers byte-identical to the release engine. Greedy, 3 prompts x 5 runs, 256 tokens, on IQ3_XXS and
  UD-Q4_K_XL (both streaming from the SSD), same speed (UD-Q4_K_XL 56.7 / 51.9 / 52.5 tok/s before and after).
- Not built: CUDA, HIP, SYCL (no such hardware here). With `__APPLE__` undefined the file reduces to exactly the
  current source: `unifdef -U__APPLE__ -D__linux__ -U_WIN32` gives the same 284 lines before and after, and
  `unifdef -U__APPLE__ -D_WIN32 -U__linux__` the same 292 lines.

## Extra Notes
With this change and `--conversation-cache-mib 4096 --conversation-cache-slots 4`, parking on Metal is attempted but
then refused by the snapshot check (`invalid checkpoint running-state payload`), described in #2502. So this PR alone
does not make a side request keep the main conversation on a Mac; it removes the first of the two blocks.

関連リンク

インストール・モデル・リリースへの站内リンク。