#4883 [Other] Setup: --gpus on Windows AMD: RDNA 2, 3 and 4 cards sharing one model

open · @lwYeo · 0 comments · View on GitHub

BenchmarksSetup & installServer & APIMulti-GPUAMD / HIPNVIDIA / CUDAModels & quantsWindowsLinux

Description

[Other] Setup: --gpus on Windows AMD: RDNA 2, 3 and 4 cards sharing one model

## Problem and result

On Windows, `setup.py` refused `--gpus` outright: "several AMD cards sharing one model (--gpus) is Linux-only for now". The engine itself runs multi-GPU on Windows (HIP), so a PC with, say, an RX 6800 XT (RDNA 2) and an AI PRO R9700 (RDNA 4) could not split a model across its cards through setup.

After: `--gpus` (and `--gpus all`) is accepted for AMD on Windows. Setup names the cards in the HIP order the engine uses, a re-run keeps an installed model's card pair, the engine step renumbers the split as the HIP runtime numbers it (an iGPU taking HIP 0 no longer breaks a saved `"0,2"`), and the engine acquisition checks **every** architecture in the split, not just the main card's. Verified end to end: a 16 GB RDNA 2 + 32 GB RDNA 4 split serves a 50 GB-quant model in low-RAM mode on Windows.

## Related discussion

Resolves #4991 - the setup.py block ("several AMD cards sharing one model (--gpus) is Linux-only for now") is exactly what this removes.

## Approach

The Linux-only refusal predates the Windows HIP engine; nothing in the engine path is Linux-specific. The parts that needed work:

- Windows card numbering is display-adapter order, the HIP runtime numbers devices differently (an iGPU takes HIP 0). Setup now probes the installed engine's `strata-device --list-devices` and renumbers, so the saved config matches what the engine sees. `HIP_VISIBLE_DEVICES` is cleared for that probe so a leftover env var (Ollama, LM Studio) cannot compact the view.
- The signed feed ships one card family per package (`rdna2`, `rdna3`, `rx9070`). A cross-family split (gfx1030 + gfx1201) is in no single package, so the keep/zip/feed paths now check all archs in the split, and the feed result is checked **before** it is installed — a one-family package can never become the installed engine of a cross-family split. A split the feed cannot serve names the missing family and points at `tools/hip/build_windows.bat` (one fat build covers RDNA 2, 3 and 4).
- A run with `--prebuilt` is no longer vetoed in the card check by the **installed** engine's "not an architecture" verdict — that engine is what the run is about to replace. Other `cannot run` verdicts still stop.
- `start_gpus` (start an installed model with `--gpus`) takes the AMD card list on Windows instead of falling into the NVIDIA list, where `--gpus all` answered "this PC does not have two GPUs".
- `tools/strata_bench.py` labels runs by HIP numbering (`hip_ordinal`), so bench rows match the config.

Compatibility: the default path (one card, `--gpu`) is untouched; NVIDIA and Linux paths untouched; DeepSeek's Windows one-card rule keeps its own reason (per-card packages), no longer cites the removed Linux-only rule. New behaviour is opt-in via `--gpus` on Windows AMD.

## Validation

Branch `feature/windows-amd-multi-gpu` @ `7a966442` on main `146df3c4`. Environment: Windows 11, Gigabyte X870E, AMD RX 6800 XT (gfx1030, 16 GB) + AI PRO R9700 (gfx1201, 32 GB), Ryzen 7 7700X, 48 GB RAM.

**Passed:**
- `python setup.py --check --gpus 1,0` and `--check --gpus all` (no GPU work): step 1 accepts the pair — "GPU 1 (AMD Radeon AI PRO R9700, 32 GB) + GPU 0 (AMD Radeon RX 6800 XT, 16 GB) together (the model's layers are split across them)".
- Full run `START-HERE.bat --backend hip --prebuilt dist --gpus 1,0` on this PC: setup completed, engine started, model (Swift 1.5 IQ3_S, 262K context, 8-bit KV, images on, low-RAM mode ~57% of experts on the two GPUs) serving on port 11436.
- HIP build for the split: `tools/hip/build_windows.bat` → `dist/strata-windows-x64-hip.zip` (753 MiB, engine 0.1.58 built from this base, archs gfx1030/31, gfx1100/01/02, gfx1151, gfx1200/01; GPU vision reader for all of them). Installed and run on the gfx1030 + gfx1201 pair — hardware execution, not just compilation; the live instance now serves the Unsloth UD IQ4_XS pack with the split active ("layer split: layers 0-14 (CUDA0), 15-47 (CUDA1)").
- `python -m pytest tools/test_setup_amd.py` → 57 passed (includes new tests: split across families accepted/rejected by arch, feed checked before install, probe ignores `HIP_VISIBLE_DEVICES`, permissive `--gpus` parse on Windows).
- `python -m pytest tools/test_setup_start_gpus_amd.py` → 5 passed (includes the new Windows `--gpus all` case).
- `tools/test_setup_amd_cards.py`, `test_setup_cards_029.py`, `test_setup_choices.py`, `test_setup_dsv41.py`, `test_setup_gpu_fit.py`, `test_setup_configs.py`, `test_setup_config.py`, `test_setup_rerun_size.py`, `test_setup_menu.py`, `test_setup_hybrid.py`, `test_setup_feed_engine.py`, `test_setup_engine_keep.py` → all passed, no GPU.

**Failed:**
- `tools/test_setup_dsv41.py` → 1 failed (`test_the_pci_ids_name_the_card_when_kfd_and_rocminfo_say_nothing`) and `tools/test_setup_menu.py` → 1 failed (`test_tail_skip_stays_on_for_linux_gfx12`) in the 12-suite sweep. Both reproduce identically on plain main `146df3c4` without this change; not touched by this PR.

**Not run:**
- `tests/terminal` suite (the Terminal code paths are untouched; the run in this environment hit missing test dependencies and did not complete).
- Linux `--gpus` (no Linux environment here) — the Linux code path is not modified.
- An RDNA 3 card inside a split (no RDNA 3 hardware here); the fat build carries gfx1100/01/02 code, and the split-arch logic is arch-agnostic, but only the gfx1030 + gfx1201 pair is hardware-verified.
- CUDA/SYCL/Metal builds: no files in those backends are touched.

## Limits

- A cross-family split needs a self-built engine (`tools/hip/build_windows.bat`, one build covers all three RDNA generations): the signed feed publishes one family per package, so no feed package can serve e.g. RDNA 2 + RDNA 4. Setup says this by name when it happens.
- A local `--prebuilt` install is unverified by SHA-256 (existing behaviour for local prebuilts; setup prints the notice).
- hipBLASLt tuning tables: none published for gfx1030, so plain hipBLAS there (same as the release rdna2 package).
- The Windows HIP engine numbers devices by HIP ordinal; a saved split assumes the same physical cards in the same slots.

Related on this site

Quick links to related pages.