Issues / #516

#516 desktop (KDE/Wayland): compositor VRAM exhaustion when expert-cache auto fills a 24 GB card - pin framebuffer -12, KWin graphics reset

closed · @xyzzing · 4 comentários · No GitHub

Server & APIAMD / HIPDocumentation

Descrição

## Desktop compositor crashes when the serve engine's expert cache fills VRAM (24 GB cards)

**Observed on**: RX 7900 XTX 24 GB, Fedora/KDE (Wayland), engine 0.1.35 + our HIP line, `--expert-cache auto` (the serve config we run).

**Symptom**: KWin (the Wayland compositor) fails repeatedly once the engine's cache fills the card:
```
kwin_wayland: atomic commit failed: Cannot allocate memory
kwin_wayland: Waiting for glGetGraphicsResetStatus to return GL_NO_ERROR timed out!
kernel: amdgpu 0000:03:00.0: [drm] *ERROR* Not enough memory for command submission!
kernel: [drm:amdgpu_dm_plane_helper_prepare_fb [amdgpu]] *ERROR* Failed to pin framebuffer with error -12
kernel: A graphics reset of an unknown cause occurred
```
plasmashell SIGABRT'd twice. rocm-smi at the time: the engine held **23.33 GiB of 24 GiB**.

**Cause**: `--expert-cache auto` sizes the cache from free VRAM minus a small internal reserve (~700 MiB plus draft headroom), which leaves a desktop compositor almost nothing. A cache sized seconds before a browser/video window appears can also age into the same state — the compositor's allocations then fail with ENOMEM instead of the engine yielding.

**Mitigation that fixed it for us**: `--vram-reserve-mib 4000` on the serve config (engine VRAM 23.3 → 21.8 GiB, zero compositor failures since; the cache loses ~1k slots ≈ a few percent of hit rate). The knob exists and works — the gap is that `auto`'s default reserve is sized for headless/dedicated setups, and nothing warns or adapts when a desktop compositor is present.

**Suggestion**: one of (a) raise the effective desktop reserve under `auto` when a DRM render node has an active compositor (detectable via which processes hold the other KFD/DRI devices), (b) document `--vram-reserve-mib` as required on desktop 24 GB setups in docs/AMD_HIP.md, or (c) at minimum warn at startup when free VRAM after the cache is below a threshold. Happy to send a patch for whichever shape you prefer.

Data: full kernel/KWin log excerpts and before/after rocm-smi available on request. (Filed from the xyzzing HIP integration line; same report cross-posted to our rollout records.)

No site

Links install, modelos, releases.