#6648 [Engine] Hy4-Flash SQ-128 at 128K on Strix Halo (hip-halo, 128 GiB unified): OOM-killed after [load]; --vram-experts-gb and HY4_VRAM_CAP_GB ignored

open · @bunnie · 0 comments · View on GitHub

BenchmarksSetup & installServer & APIAMD / HIPModels & quantsLinux

Description

**Area:** [Engine]
**Desktop app, Terminal or source?** Terminal (`strata launch` 0.2.18)

## Problem

Hy4-Flash **SQ-128 at the default 128K context will not start on a 128 GiB Ryzen AI Max+ 395 (Strix Halo, gfx1151) Linux box** — exactly the "128 GiB+ systems and unified-memory workstations" class the model card names SQ-128 for. The engine is SIGKILLed by the OOM killer seconds after `[load] total`. The adaptive VRAM expert-slot pool is sized from `[gpu] free` (the GTT pool = system RAM) without subtracting the host expert arena the same runner allocates next to it; on unified memory the two land on the same physical 124 GiB.

Related: #6556 (same `[args] ... not used by this runner yet` lines, hip-rx7900 discrete-GPU host, thrashing instead of an OOM kill). Precedent: #1715 (closed) — the same accounting bug in the Qwen engine on this same machine class, fixed there by making `--expert-cache auto` subtract the host/resident draw from the unified pool; the Hy4 runner has no equivalent.

## Environment

- Strata 0.2.18 Terminal (control 0.1.64); engine component `strata-hy4-0.1.64-linux-x86_64-hip-halo`, installed by `strata update` (signature checked, test-started)
- GPU: Radeon 8060S Graphics (gfx1151), unified LPDDR5X
- RAM: 128 GiB (MemTotal 124.0 GiB); kernel boot `ttm.pages_limit=29360128 ttm.page_pool_size=29360128` (112 GiB GTT)
- OS: Ubuntu 24.04.5, kernel 7.0; user session with systemd user-slice OOM (systemd-oomd active)
- Model: SQ-128 + Engram-Q8 + Vision, all three SHA-256 verified against the model card; `--max-context 131072`, drafter dfly2, `--session-reuse` default

## Reproduce (reproduced locally, 5×)

```
strata launch --family hy4-flash --model SQ-128 --yes --no-tui   # 128K default, vision on
```

The same happens hand-running the engine binary with the launch's args (exit 137). Pack integrity is not the cause: `strata doctor` clean; SQ-96 + the same Engram/Vision files serve 128K fine on this machine (~94 GiB resident, ~30 GiB available); SQ-128 at `--context 32k` also serves fine (~114 GiB resident, ~10 GiB left, 26 tok/s decode).

## Actual

Engine log (trimmed):

```
[gpu] Radeon 8060S Graphics gfx1151 free 112.0 / 112.0 GiB
[plan] ctx 131072 ... trunk 7.82 GB on a 120.3 GB card -> host mask 0; reserve 6.52 GB
[load] experts tiered: 0 static VRAM (0.00 GB) + 16054 adaptive VRAM slots (97.45 GB) = 97.45 GB VRAM,
       20992 in host RAM computed by the CPU (105.76 GB), 0.1 s
[load] total 10.3 s
```

→ SIGKILL (137) right after `[load] total`; systemd user-slice journal: "A process of this unit has been killed by the OOM killer." 97.45 GB of slots + 105.76 GB of host arena > 124 GiB physical.

What each knob does on this runner (SQ-128, 128K, hand-run with the launch args):

| Setting | Slots after `[load]` | Result |
| --- | --- | --- |
| (default) | 97–104 GB adaptive | SIGKILL |
| `--vram-experts-gb 88` | 104.32 GB — `[args] --vram-experts-gb 88: not used by this runner yet` | SIGKILL |
| `HY4_RESERVE_GB=30` (plan line: `reserve 30 GB`) | 79.08 GB adaptive; host arena unchanged 105.76 GB | SIGKILL |
| `HY4_VRAM_CAP_GB=8` | 103.26 GB adaptive — not honored | SIGKILL |
| `HY4_EXPERT_VRAM_GB=8` | 8.59 GB adaptive — honored; host arena still 105.76 GB | SIGKILL (~0.5 GiB short) |
| `HY4_EXPERT_VRAM_GB=8 HY4_RAM_BUDGET_GB=85` | 8.59 GB; `[load] RAM budget 85.0 GB ... 3573 experts (14.31 GB, the coldest ranks) stay on the SSD` | **starts, `READY 131072`** |

`STRATA_ARENA_PIN_GIB=0` does not change the outcome.

## Expected

- On unified memory, sizing the adaptive VRAM slots should subtract the host expert arena (same physical pool): SQ-128's 113 GB footprint fits the model card's own "128 GiB" tier, and the engine's `[gpu] free 112.0 GiB` figure invites that reading.
- `--vram-experts-gb` / `--ram-experts-gb`, which setup writes (per #6556), should be honored or the runner should not print them as accepted; `HY4_VRAM_CAP_GB` looks intended to be honored and is not on this backend.

## Workaround (works today)

Engine env `"HY4_EXPERT_VRAM_GB": "8", "HY4_RAM_BUDGET_GB": "85"` (run-config `env`) — SQ-128 reaches `READY 131072` with ~14.3 GB of coldest experts on the SSD. Or: SQ-96 at 128K / SQ-128 at 32K, both no-configuration starts.

## Logs

Terminal `strata doctor`:

```
ok  terminal_cli
ok  monitor_tui  version 8.2.8
info: PROFILE_NOT_SELECTED: select a run config or saved profile for model-specific checks
```

Engine log lines above are verbatim from `launch/sq-128-128k.engine.log` and the hand-run captures; full files available on request.

Reproduction status: **Reproduced locally** (5 engine deaths, same point; 3 knob combinations additionally measured as above).

Related on this site

Quick links to related pages.