#6454 [Engine] serve: an integrated Intel GPU's memory is the system RAM in the Monitor and the free-memory checks

open · draft · @criskurtin · 0 コメント · GitHub で見る

Setup & installServer & APIModels & quantsDocumentationWindows

本文

## Problem and result

On an Intel GPU built into the processor (here an Arc Pro B390, Panther Lake, `8086:b086`, on `xe`), Level Zero sysman answers MemAvailable as the GPU's size and MemFree as its free memory. Both move with every program on the computer, and neither shows what the engine holds. Measured on main 3142b66b9 with `telemetry.gpu_reader(0).read()` and `telemetry.free_vram_mib(0)`, next to `/proc/meminfo`:

| | main: size | main: in use | main: `free_vram_mib` | this PR: size | this PR: in use | this PR: `free_vram_mib` |
|---|---|---|---|---|---|---|
| no engine running | 31.24 GiB (= MemAvailable) | 5.39 GiB | 26,470 MiB (= MemFree) | 62.24 GiB (MemTotal) | 0.18 GiB | 57,382 MiB |
| model loaded, 40 GiB of experts on the GPU | 6.53 GiB (= MemAvailable) | 5.81 GiB | 739 MiB (= MemFree) | 62.24 GiB | 47.82 GiB | 6,686 MiB |

With the model loaded, main's Monitor showed a 6.5 GiB GPU and the server's free-memory figure was 739 MiB, while the RAM still had 6.5 GiB available. The server's checks read the same figure: the `min_free_vram_mib` load gate and the room check for a GPU picture reader.

With this PR such a GPU has the RAM as its size (MemTotal), and as its use the memory its GPU clients hold (fdinfo's `drm-resident-gtt` + `drm-resident-system`; `xe` has no `vram` region here). Free memory is MemAvailable plus the pages the GPU driver keeps pooled after an unload (`GPUReclaim`; 24.8 GiB in the idle reading above), which the next load can take.

## Related discussion

#6448 (scope for integrated Intel GPUs). Scope is not agreed yet, so this stays a draft. #2412 reported the same pooled pages ("TTM page pool"). This PR changes only the server side: it does not depend on #6453 or #6455 and merges cleanly with both.

## Approach

- `_Intel` reads Level Zero's integrated flag through `_Zes.integrated()`, as setup does. Only when it is set does `read()` take the size from MemTotal and the use from fdinfo. A discrete card, and a GPU sysman cannot answer for, read as before.
- `telemetry.free_vram_mib()` answers MemAvailable + `GPUReclaim` for such a GPU, or MemAvailable alone when the kernel has no `GPUReclaim` line.
- `Service.free_vram_mib()` and `card_free_mib()` in `server.py` each kept their own copy of "size less use". They now call `telemetry.free_vram_mib()`, which gives every other card (NVML, amdgpu, Metal, discrete Arc) the same figure as before. Without this the load gate would have counted every other program's RAM as free.
- Panther Lake's GPU ids get their names from hwdata's `pci.ids` (B390, Pro B390, B370, Pro B370).
- The tests' sysman stand-ins answer `integrated()`: `NoZes` gives None, `FakeZes` gives False.

## Validation

**Passed:**
- On the Arc Pro B390 (Core Ultra X9 388H, 62 GiB LPDDR5X shared, Fedora 44, kernel 7.2.11, compute runtime 26.35.39758): the readings in the table, at main 3142b66b9 and at this branch, once with no engine and once with the Swift IQ3_XXS files of `huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-v2-GGUF` loaded on #6453's engine (main's sizes the cache as #6448 describes), each next to `/proc/meminfo`. main's figures match MemAvailable and MemFree to the MiB.
- `uv run --python 3.12 --with-requirements terminal/requirements.txt --with-requirements terminal/requirements-dev.txt --with-requirements terminal/requirements-build.txt python tools/check --strict --only py-serve,py-tools,docs` at 07cda5745: py-serve 2059 passed, py-tools 1851 passed, docs 6 passed; 0 failed. `python terminal/tools/minimal_server_smoke.py --report smoke.json` with only `terminal/requirements.txt` installed: 63 tests, OK, 4 skipped.
- The new tests fail on main's `telemetry.py` (3 failed) and `server.py`, and pass here.
- Logs: https://github.com/xenonhr/Strata/tree/b390-evidence (`monitor-idle.txt`, `monitor-loaded.txt`).
- Fork CI at the same head, run by hand: [Config snapshot](https://github.com/xenonhr/Strata/actions/runs/44840599968), and every job of [Checks](https://github.com/xenonhr/Strata/actions/runs/44840596981) and [Minimal serving runtime](https://github.com/xenonhr/Strata/actions/runs/44840603307) but the tests under Failed.

**Failed:**
- In CI, two tests that fail the same way on main 3142b66b9 ([Checks](https://github.com/Niko1221/Strata/actions/runs/44699739037), [Minimal serving runtime](https://github.com/Niko1221/Strata/actions/runs/44699738802)): `terminal.tools.test_strata_mcp.Install::test_a_start_that_came_in_before_a_cancel_does_not_launch_the_job` on Windows, and `test_abort_sync` in Minimal serving runtime (`from serve import server`: no module named `serve`). macOS `host-test` failed one UI test on the first run ("a reply the agent began while the event stream was down is asked for when the stream comes back") and passed when rerun; this PR changes no host file.

**Not run:**
- A discrete Intel card: none here. By code reading its path is unchanged (`integrated` is False on a card).
- A machine with an `xe` integrated GPU and a discrete `xe` card: `_Intel`'s card search can take the integrated one as index 0, as main already does.

## Limits

- When the root sampler's `/run/gpustat.json` is fresh, `read()` still uses it first, and it shows sysman's figures on such a GPU.
- "In use" counts every GPU client this user can see on that device; on a laptop that includes the desktop (the 0.18 GiB with no engine).
- Only the B390 was measured.

Prepared with an AI coding assistant (Claude Code); every number above was measured on the machine named in Validation.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

関連リンク

インストール・モデル・リリースへのサイト内リンク。