Pull requests / #2704

#2704 Feature/windows amd vision telemetry

open · @D4ve0s · 0 comentarios · En GitHub

Setup & installServer & APIMulti-GPUAMD / HIPNVIDIA / CUDAModels & quantsDocumentationWindowsLinux

Descripción

## Title

Windows AMD: ADL2 telemetry, one Monitor tile per GPU, images on Windows, and a setup that keeps a two-card split

Issue: Refs #2614, #1380, #112, #2442

## Summary

A Windows AMD card had no telemetry: the Monitor's GPU tiles were empty and a PC with two cards had no way to
see them apart. This reads every AMD card through ADL2 PMLog (`atiadlxx.dll` over ctypes — the same interface
HWInfo and LibreHardwareMonitor use, no new dependency), gives the Monitor one tile per card, and carries the
same numbers into `/metrics`.

Alongside it, three things that stopped a Windows AMD install working: SETUP renumbered or dropped cards when
`HIP_VISIBLE_DEVICES` was already in the environment, re-running SETUP collapsed a two-card split back to one
card (the engine then died at start, reported as "the embedding buffer failed"), and Windows had no image
encoder at all.

One commit, `d0dcf489` plus the review fixes listed under Telemetry and Tests: 35 files, +1501/−145 against
`main` (v0.1.44.1). Everything is opt-in — the CUDA path is untouched, `--yes` never asks the new questions,
AMD images stay off by default, and the default install path writes the same config as v0.1.44.1. No engine
(C++/CUDA/HIP kernel) changes.

## What changed

### Telemetry (`serve/telemetry.py`, +366)

- `_Adl` — ctypes bindings for ADL2 PMLog: adapter enumeration, `PmLogStart` with the sensor set (edge/mem
  temp, board power, gfx/mem activity, bus lanes, power limit), and the malloc callback through
  `ucrtbase.dll` (the heap `atiadlxx.dll` frees on).
- `_AmdWin` — util = max(gfx, mem activity), temp = edge else mem, power = `BOARD_POWER` (already watts, no
  scaling), VRAM used = `DedicatedVRAMUsage`, total = `MemoryInfoX4`. Enumerates present discrete cards,
  deduped by the UDID's `DEV_xxxx` code (each card repeats once per display), integrated GPU skipped.
- Every discrete card is sampled (`amd_card_count()`), not just the display-driving one — `Adapter_Get` hid a
  second idle card. HIP indices and ADL indices are different namespaces and are never mixed.
- PCIe per card — PMLog `BUS_SPEED` is the GPU's internal bus and power-saves to Gen4 at idle, so the slot's
  generation comes from Device Manager (`DEVPKEY_PciDevice_MaxLinkSpeed`, one cached PowerShell query keyed by
  DEV code) and the negotiated width from PMLog `BUS_LANES`. Matches HWInfo on this PC: 9070 XT Gen5 x8,
  R9700 Gen5 x4.
- A reader gives back what the driver lent it: `close()` releases the PMLog device handle and the ADL context,
  a finaliser covers readers that go away with the server, and the one-off call sites (`free_vram_mib()`,
  `server.gpu_vram_mib()`) release theirs. Numbers under Measured.
- `sample()` always emits `gpus[]` with name, `pcie_gen_max` and `pcie_width` per card, even for one card.
- Small NVML fix: `power_limit` returns `float(lim)` instead of a raw value that could be 0.

### SETUP behaviour (`setup.py`, +200/−33)

- **AMD asks "Do you want images?" everywhere, Windows included** (default off; `--yes` never asks; the
  Unsloth Linux default-on is unchanged). `hip_vision()` is now platform-neutral: `yes`/`gpu` → gpu,
  `cpu` → cpu, else none.
- **`previous_amd_split()`** — a re-run with no `--gpu`/`--gpus` keeps the installed config's split (accepts
  `"gpu"` as a list or a `"0,2"` string, newest config wins, keeps the split only while every card is still
  present and usable).
- **`hip_devices()` clears `HIP_VISIBLE_DEVICES` for the probe.** A user-set value hid cards and renumbered
  the rest, so a saved split "0,2" named a card that was no longer at 2.
- **Windows `--gpus` works** — the old "multi-GPU is Linux only" refusal is gone. `engine_step()` renumbers
  the split's cards to HIP numbers after the probe (an integrated Radeon takes HIP 0) and prints what it
  settled on.
- **`hip_vision_windows()`** — for the ready-made zip: run the zip's encoder when `BUILD.json` says
  `vision`/`vision_src`/`vision_archs` match; `gpu` asked without an encoder built for this arch warns and
  falls back to cpu; a stale or missing encoder is recompiled as CPU with MSVC; no compiler offers the winget
  Build Tools (~2 GB), and declining leaves images off.
- **`install_msvc_build_tools()`** — winget `Microsoft.VisualStudio.2022.BuildTools` with the VCTools workload,
  asked first. `cmake_build()` pip-installs cmake and ninja when they are missing (old installs predate
  `requirements.txt`).
- **VRAM reserve rule** — Windows AMD with the CPU encoder writes no `--vram-reserve-mib` at all, so the engine
  keeps its 2560 MiB WDDM floor (#1376, #1709); a written 700 would have defeated it. With a GPU encoder the
  reserve is `max(reserve, 3260)` = the 2560 floor + the encoder's 700. The 12 GB tip only shows where it
  applies.

### Multi-GPU enumeration and JSON config

- The config is re-read and `gpu` normalised whether it is a list or a `"0,2"` string, so an old config survives.
- `tools/hip/package_windows.py` records `vision: "cpu"`, `vision_src` (source hash) and `vision_archs` in
  `BUILD.json`, so SETUP can tell a matching encoder from a stale one.
- `.gitignore` adds `/strata-*.bak` (SETUP's config backups) and `build-vision-cpu.bat`.

### Vision build and packaging

- `tools/hip/build_windows.bat` builds a portable CPU `strata-vision.exe` (AVX2, static CRT, llama.cpp pinned
  by `setup.get_llama_cpp()`) into the zip; `package_windows.py --vision` ships it.
- `tools/vision/CMakeLists.txt` gains `STRATA_VISION_HIP` (GGML_HIP) for the Linux HIP encoder path in
  `setup.build_vision_hip()` — **not yet run on hardware**, flagged in TEST_REQUESTS.
- `tools/vision/strata_vision.cpp` also sets `HIP_VISIBLE_DEVICES=-1` in CPU mode — the AMD runtime ignores
  `CUDA_VISIBLE_DEVICES`.
- `tools/strata_mcp.py` drops the "hip on Windows" and vision refusals; the MCP `strata_install` accepts the
  same vision answers as SETUP (gpu is downgraded to cpu by setup itself when the arch has no encoder).

### Monitor UI (`serve/web/`)

- **Image encoder card**: encoder cache bar, encode-time sparkline, facts (device + threads, last picture with
  tokens/grid/cache-hit, since-start totals, mmproj, picture cap), and a "Reading N pictures" pill and status
  line while an encode runs.
- **Per-GPU tiles** (`renderGpuCards`) when the PC has ≥2 cards; summary Power and PCIe tiles get a `GPU:0 …` /
  `GPU:1 …` line each; the PCIe tile shows each card's own link ("Gen5 x8", "Gen5 x4").
- `app.css`/`app.js` are cache-busted (`?v=3`) so an open tab can't keep a stale script.

### Metrics (`serve/server.py`, `serve/prometheus.py`)

- `Vision` records starts/stops, per-encode times, cache hits, failures and image tokens; `/metrics` gains a
  `vision` block, `encoding_images`, and per-request history rows (images / image tokens / encode ms).
- Prometheus gains 7 series: `strata:vision_images_total`, `image_tokens_total`, `encode_seconds_total`,
  `encode_failures_total`, `cache_images`, `encoder_up`, `encode_ms_last` (absent on a cache hit). Nothing is
  emitted when the model has no vision.
- `engine_args()` adds `--vram-reserve-mib 3260` for hip on Windows with a GPU encoder (2560 WDDM floor + 700
  for the encoder). `gpu_vram_mib()` releases the reader it builds per card.

### Docs

- `AMD_HIP.md`: the ADL2 telemetry paragraph with the B650E measurements, the two-cards-one-model Windows
  section, and the Images-on-Windows section (zip encoder, 2.7–2.9 s per 512×512 picture on a 7800X3D, Build
  Tools ~2 GB, the 3260 reserve).
- `TROUBLESHOOTING.md`: new entry "SETUP.bat forgets a two-card split" with the `HIP_VISIBLE_DEVICES` root
  cause, how to check it, and the measured before/after probe results.
- `DETAILS.md`: Monitor sources (ADL2/NVML/sysfs, per-slot PCIe), the `strata:vision_*` series, the AMD
  image-encoder table row, and the Image encoder card paragraph.
- `MULTI_GPU.md`: Windows AMD `--gpus` line. `AI_SETUP.md`: the AMD images question. `TEST_REQUESTS.md`: two
  rows — Windows AMD `--vision cpu` reports wanted, `--vision gpu` never run on hardware.
- README ×7: "AMD cards read pictures through the processor, on Linux and on Windows alike."

### Tests

- New `serve/test_telemetry_adl.py` (+118): the ADL reader never raises when ADL is missing, readings are sane
  (temp 0–120 °C, watts), every card index below `card_count` reads, per-card PCIe present, and
  `AdlReaderCleanup` — `close()` is safe and idempotent, and 40 one-off `free_vram_mib()` reads do not grow
  the process handle count (Windows only, skipped where there is no `atiadlxx.dll`).
- `tools/test_setup_amd.py`: probe ignores a set `HIP_VISIBLE_DEVICES`; the vision question is
  platform-neutral.
- `tools/test_setup_choices.py`: 5 new `hip_vision_windows` cases — zip encoder used, gpu from zip, gpu falls
  back to cpu when the arch is missing, stale `vision_src` recompiles, no compiler asks then declines.
- `tools/test_setup_unsloth.py`: `HipImages` (Windows asks, yes → cpu encoder with no `--vram-reserve-mib` in
  the config args, default off, `--yes` never asks, Linux Unsloth default stays on) and `HipSplit` (Windows
  `--gpus 0,2` renumbers to [1,2] when the iGPU takes 0, setup-again keeps a list or a "0,2" string, a fresh
  PC stays one card, Linux keeps its split).
- `serve/test_server.py`: the vision instrumentation, and the three `AmdTelemetry` tests pinned to the Linux
  sysfs reader — they fake an amdgpu tree, so building a real ADL reader here made them depend on the machine
  (#1380). `WebApp.test_page_and_files` now matches `web/app.js` with its `?v=3` cache-bust.
- `serve/test_monitor.py`, `test_prometheus.py`, `test_vision_elastic.py`: the Monitor card's DOM and the
  Prometheus series. `tools/test_strata_mcp.py`: hip vision cpu/yes/gpu/no accepted on both platforms.

## Extra Notes

### Test machine

- Windows 11, Ryzen 7 7800X3D, B650E motherboard, Adrenalin 26.9.2
- AMD Radeon AI PRO R9700 32 GB — HIP 0, gfx1201, Gen5 x4 (Device Manager and HWInfo agree: "PCIe v5.0 x16 @ x4")
- AMD Radeon RX 9070 XT 16 GB — HIP 2, gfx1201, Gen5 x8
- 7800X3D integrated graphics (2 CU RDNA2, gfx1036; rocminfo's marketing name is "AMD Ryzen 7 7800X3D 8-Core
  Processor") — HIP 1, skipped by the split and by the ADL2 enumeration
- Model split across the two cards: R9700 layers 0–30, 9070 XT layers 31–47 + head, 131K context
- CPU image encoder running on the 7800X3D; a hand-built HIP encoder also ran on the R9700 (below)

### Measured on that machine

- **Handle growth (the finding reported on #1434).** 40 one-off `free_vram_mib(0, amd=True)` calls hold the
  process handle count at 197 → 197 now that the reader is released. Keeping the readers gave 197 → 278, about
  +2 per call — the same signature crkva measured on a 7900 XTX in #1434 (267 → 347). The server's sampler
  keeps one reader per card, so the growth showed up on the one-off calls: `free_vram_mib()` in the startup
  retry loop and `gpu_vram_mib()` once per card.
- **PCIe per card:** 9070 XT Gen5 x8, R9700 Gen5 x4 — Device Manager and HWInfo agree, and the B650E board
  really does bifurcate, so one aggregate link number would be wrong.
- **Images:** 2.7–2.9 s per 512×512 picture (256 tokens) with the CPU encoder; the screenshot run measured
  3,647 ms for a 23×12 (276-token) picture, cache hit on repeat.
- **Prometheus scrape after 4 image requests:** `strata:vision_images_total 4`, `vision_image_tokens_total
  1073`, `vision_encode_seconds_total 12.767`, `vision_encode_failures_total 0`, `vision_cache_images 4`,
  `vision_encoder_up 1`. `/metrics` also shows both `gpus[]` rows with correct ADL2 values.
- **SETUP probe with `HIP_VISIBLE_DEVICES="0,2"` set:** the old probe returned 2 cards numbered 0,1; the fixed
  probe returns all 3 (R9700 = HIP 0, 7800X3D iGPU = HIP 1, 9070 XT = HIP 2).

### Test suite

- `python -m unittest discover 

En el sitio

Enlaces a install, modelos, releases.