#2846 5700xt use IQ3_S mistake

Descrição

# RDNA1 (gfx1010, RX 5700 XT): GPU page-fault floods, card resets and host freezes — three findings, one fixed

**Engine**: Strata 0.1.44.1, locally built HIP engine (`source: local-hip`, `archs: ["gfx1010"]`, `src: 7dfd34378bd2d32a`)
**Card**: AMD Radeon RX 5700 XT, Navi 10, gfx1010, 8 GB, 40 CU — the card `docs/OLDER_GPUS.md` lists as "built by hand"
**Status**: usable, with a hard context ceiling. One of the three findings has a one-line fix that I believe other RDNA1 users need.

I am reporting this because `AGENTS.md` says a report from hardware the maintainers do not have is welcome. All
numbers below are measured on this machine; nothing is estimated. Everything I tried and measured — including what
did **not** help — is listed in §6 so you do not have to re-run it.

---

## 1. Summary

| # | Finding | State |
|---|---|---|
| **A** | The expert arena is larger than the kernel's default amdgpu **GTT window**, so the GPU is handed host addresses it cannot translate → page-fault flood → `GPU reset begin!` → **`VRAM is lost`, the desktop dies with it** | **Fixed** by `amdgpu.gttsize=57344` (one line + `update-initramfs -u` + reboot) |
| **B** | The **batched prompt path** stalls at "waiting for the GPU (attention, router)" and resets the card | Unfixed; worked around with `--short-read` |
| **C** | A prompt **longer than ~5,000 tokens** completes reading, then the first generation step fails with `verify: ... never rang` while the GPU accesses **fabricated addresses in the 92–124 GiB range** on a machine with 70 GiB of memory total | Unfixed; worked around with a small `--max-context` |

Finding A is the important one for other users, and I think it is a *documentation* bug at least, because its
arithmetic covers a large group of machines (§3.4).

---

## 2. What to record — machine, model, settings

**Hardware**

- GPU: AMD Radeon RX 5700 XT (Navi 10, gfx1010), 8 GB VRAM (7.98 GiB), 40 CU, PCIe **4.0 x16 at full speed**
  (16.0 GT/s, width 16), power cap 180 W, 53 °C at idle and 54–56 °C under load. `gfx1010:xnack-`.
- CPU: Intel Xeon E5-2680 v4 (28 threads, AVX2, **no AVX-512**)
- RAM: 62 GiB (DDR4 ECC; per-module speed/type not captured — `dmidecode` needs root)
- Motherboard: X99-D8I, Intel VT-d / IOMMU active
- Storage: KIOXIA KXG70PNV2T04 2 TB NVMe (models on it)

**Software**

- Ubuntu 24.04.5 LTS, kernel **7.0.0-38-generic**, amdgpu (open source, no amdgpu-pro)
- Strata 0.1.44.1, release tarball, engine compiled locally for gfx1010
- ROCm **7.14.0a20260612** from TheRock's `https://rocm.nightlies.amd.com/v2/gfx101X-dgpu/` index, installed **inside the project venv** (no system ROCm)
- llama.cpp pinned commit `3cf03257f219afbe7334045ff7c6a06ac68c627d` (the `LLAMA_CPP_COMMIT` of 0.1.44.1)
- Build options: `STRATA_ENABLE_HIP=ON`, `STRATA_PREFILL_MMQ=ON`, `CMAKE_BUILD_TYPE=Release`, `CMAKE_HIP_ARCHITECTURES=gfx1010`

**Model**

- `ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF`, **IQ3_S**, 2 shards, 83,617,662,656 bytes, both SHA-256 verified (fetched from ModelScope — HuggingFace is unreachable from this network)
- MTP draft layer: `mtp/rt/dense.bin` 116,099,072 bytes, `mtp/rt/experts.bin` 707,788,800 bytes, `mtp-q2_0.gguf` 889,017,216 bytes
- Engine reports the expert region as **46.84 GiB**, loaded at 2.48–2.55 GiB/s
- `docs/MODELS.md` / `setup.py` list the IQ3_S arena as **50.3 GB**

**Settings (the configuration all measurements below use)**

```
--pack <IQ3_S pack> --native <shard1> --ple-gguf <shard2>
--expert-profile data/expert-profile.bin --expert-cache auto --prefill auto
--spec 2 --spec-min-p 0.5 --mtp <mtp/rt>
--max-context 5120 --short-read 5120 --kv fp16 --vram-reserve-mib 1024
env: HSA_USERPTR_FOR_PAGED_MEM=0, GPU_PINNED_MIN_XFER_SIZE=1048576
```

Serving with `host: 0.0.0.0`, `port: 8080`, API key set. `vm.min_free_kbytes` raised from 66 MiB to 1 GiB.

**Getting the gate open.** I mirrored your own `STRATA_EXPERIMENTAL_GFX1103` opt-in rather than editing defaults —
4 additions in `setup.py` (`ROCM_INDEXES`, `ROCM_FAMILY_PINS`, `AMD_ARCHS`, `AMD_NAMES`) plus
`GFX1010_OPT_IN = os.environ.get("STRATA_EXPERIMENTAL_GFX1010") == "1"`. With it set, `./setup.sh --backend hip
--check` reports `[ok] GPU: AMD Radeon RX 5700 XT ... 8.0 GB VRAM` / `IQ3_S needs ~62 GB RAM: fits`.
If you want this upstream I am happy to open it as a one-change PR; it leaves the default path byte-identical.

---

## 3. Finding A — the expert arena does not fit the default GTT window (**fixed**)

### 3.1 Symptom

A request that sustains work for more than ~20 s produced this, every time, and took the desktop down with it:

```
amdgpu 0000:04:00.0: [gfxhub] page fault (src_id:0 ring:24 vmid:8 pasid:19)
amdgpu 0000:04:00.0:   in page starting at address 0x0000000000000000
amdgpu 0000:04:00.0: GCVM_L2_PROTECTION_FAULT_STATUS:0x00801031
amdgpu 0000:04:00.0:          Faulty UTCL2 client ID: TCP (0x8)
amdgpu 0000:04:00.0:          PERMISSION_FAULTS: 0x3
amdgpu 0000:04:00.0: gmc_v10_0_process_interrupt: 188 callbacks suppressed
amdgpu 0000:04:00.0: GPU reset begin!. Source:  1
amdgpu 0000:04:00.0: VRAM is lost due to GPU reset!
```

The `pasid` in the fault header shows a **host/userptr** address space — the faults are against memory the engine
pinned in RAM and handed to the GPU, not against VRAM.

### 3.2 The arithmetic

```
amdgpu's default gttsize = half of system RAM
                         = 31.31 GiB   (on 62 GiB;  confirmed via
                                        /sys/class/drm/card*/device/mem_info_gtt_total)
IQ3_S expert arena       = 50.3 GB     (46.84 GiB as the engine reports it)
```

The arena is **1.6× the window**. Anything the GPU touches past the window cannot be translated.

### 3.3 The fix and its measured effect

```sh
echo 'options amdgpu gttsize=57344' | sudo tee /etc/modprobe.d/amdgpu-gtt.conf   # MB => 56 GiB
sudo update-initramfs -u          # amdgpu loads from the initramfs on Ubuntu; the file alone did not change it
sudo reboot
```

After the reboot `mem_info_gtt_total` reads **56.00 GiB** and system RAM is unchanged (so `gttsize` here is a limit,
not a preallocation — worth confirming on your side).

Same request, before and after (short prompt + 1,200 generated tokens, identical settings otherwise):

| | before | after |
|---|---|---|
| result | **GPU reset at 23.7 s, desktop frozen** | **completes in 62.8 s** |
| decode | — (crashed) | **20.0 tok/s** |
| GPU page faults | every request | **none** |
| card access beyond the window | yes | none |

A 4,554-token prompt that previously died with `verify: layer 44 never rang` also then completed
(`prompt=4554 tokens at 26.4 tok/s`).

### 3.4 Why I think this matters beyond my card

The rule "default GTT = RAM/2" means the threshold is `RAM >= 2 × arena`. Against your own `MODELS` table:

| model | arena | RAM needed for the default GTT to fit |
|---|---|---|
| IQ1_M (Coder) | 23.4 GB | 47 GB |
| Q2_0 | 34.0 GB | 68 GB |
| IQ2_XS | 35.5 GB | 71 GB |
| IQ3_XXS | 42.9 GB | 86 GB |
| **IQ3_S** | **50.3 GB** | **~101 GB** |

So **any Linux AMD machine under ~101 GB of RAM running IQ3_S has the default window too small** — including every
64 GB machine, which is what `docs/MODELS.md` recommends as the IQ3_S minimum ("needs a 64 GB PC"). It does not bite
on CUDA (no GTT) or on Windows, which may be why it has not shown up.

**Asks:**
1. Can you confirm on an AMD Linux box with a known-small `gttsize` whether the arena exceeding the window is the
   same root cause behind other "device wedged"/reset reports? A one-line check is `mem_info_gtt_total` vs the
   arena size.
2. `docs/OLDER_GPUS.md` asks RDNA1 users to build by hand. I would suggest adding one line there: **check
   `mem_info_gtt_total` against the model's arena before the first run, and raise `amdgpu.gttsize` if it is
   smaller.** It cost me several hours and one hard power cycle to find.
3. If the engine can read `/sys/class/drm/card*/device/mem_info_gtt_total` at startup and warn (or refuse) when
   the arena exceeds it, that would catch this class of machine before the freeze.

---

## 4. Finding B — the batched prompt path stalls and resets the card (**unfixed**)

Your own watchdog reports it (issue #29), verbatim:

```
no progress for 60 s during a request (reading the prompt (batched): waiting for the GPU
(attention, router) at layer 32 of the prompt chunk from token 0) - stopping the engine so
the server starts it again (issue #29) - please report it at github.com/Niko1221/Strata/issues.
```

Kernel side, at the same moment:

```
amdgpu 0000:04:00.0: [gfxhub] page fault (src_id:0 ring:24 vmid:8 pasid:19)
amdgpu 0000:04:00.0:   in page starting at address 0x0000000300000000 from client 0x1b (UTCL2)
amdgpu 0000:04:00.0:   in page starting at address 0x0000000000000000
amdgpu 0000:04:00.0:   in page starting at address 0x000000003f78a000
amdgpu 0000:04:00.0:   in page starting at address 0x000000003f787000
amdgpu 0000:04:00.0:   in page starting at address 0x00007f0935df5000
amdgpu 0000:04:00.0: GPU reset begin!. Source:  1
```

**It is not fixed by finding A's change** — it still stalls and still resets the card with the 56 GiB window.
`--short-read <ctx>` routes prompts through the decode path and is a clean workaround (measured: a 2,000-character
prompt went from "must crash" to success in 46.9 s, `prompt=1182 tokens at 27.3 tok/s`).

Only a handful of attempts; I stopped once the workaround held, so I cannot say how reproducible the batched path
is on other layer numbers (I saw it at layer 32, and the same class of failure at layers 1, 3, 7, 13, 35, 39, 44, 45).

---

## 5. Finding C — prompts above ~5,000 tokens fail with fabricated high addresses (**unfixed**)

### 5.1 Measured boundary

Fresh prompts, one run each, `--max-context` raised to 24576 for the first two so they were not truncated:

| prompt tokens | result |
|---|---|
| 1,182 | ✅ `prompt=1182 tok at 27.3 tok/s`, correct answer |
| **4,554** | ✅ `prompt=4554 tok at 26.4 tok/s` |
| 6,801 | ❌ `verify: unspecified launch failure` |
| 20,846 | ❌ `verify: layer 7 never rang (unspecified launch failure)` |

The engine **reads the whole prompt** (`reading the prompt: 20,846 tokens, 270 s so far`) and then fails on the
**first generation step**. So the prompt path itself completes; the failure is at the prompt→decode handover.

### 5.2 The addresses are not real memory

First failure in this class:

```
amdgpu 0000:04:00.0:   in page starting at address 0x0000001700000000 from client 0x1b (UTCL2)
amdgpu 0000:04:00.0:   in page starting at address 0x0000001800000000
amdgpu 0000:04:00.0:   in page starting at address 0x0000001900000000
... 0x0000001a00000000, 0x0000001b00000000, 0x0000001c00000000, 0x0000001d00000000,
    0x0000001e00000000, 0x0000001f00000000
amdgpu 0000:04:00.0: GCVM_L2_PROTECTION_FAULT_STATUS:0x00801031
amdgpu 0000:04:00.0:          Faulty UTCL2 client ID: TCP (0x8)
amdgpu 0000:04:00.0: GPU reset begin!. Source:  1
amdgpu 0000:04:00.0: VRAM is lost due to GPU reset!
```

**Nine consecutive addresses, exactly 4 GiB (`0x1_00000000`) apart, spanning `0x17_00000000` – `0x1f_00000000`
= 92 – 124 GiB.** This machine has 62 GiB of RAM and 8 GB of VRAM — 70 GiB of addressable memory in total. A
regular 4 GiB stride, and a range that is suspiciously close to **2× the 46.84 GiB arena**, is what I would expect
from a base-plus-offset computation that goes wrong rather than a permission problem.

The IOMMU sees the same thing from the host side:

```
DMAR: [DMA Write NO_PASID] Request device [04:00.0] fault addr 0x1fda40000
      [fault reason 0x05] PTE Write access is not set
```

I want to be explicit about one thing: **I did not "fix" this by disabling VT-d, and I would ask other users not
to.** Turning the IOMMU off does not repair a bad pointer, it removes the check and lets the GPU write into
arbitrary physical RAM — on a machine holding a 47 GiB model, that converts a diagnosable crash into silent
corruption. I mention it only because it is the obvious-looking knob in the BIOS f