Pull requests / #2765

#2765 Experimental gfx803 (RX 580) Q36 Linux path

open · draft · @minkiboo · 0 comments · View on GitHub

BenchmarksSetup & installServer & APIAMD / HIPNVIDIA / CUDAModels & quantsDocumentationLinux

Description

## Title
Experimental gfx803 (RX 580) Q36 Linux path

Issue: Related to #2768 (experimental native RX580/gfx803 support).

## Summary

Opt-in Linux RX580/native gfx803 enablement for `strata-q36`, not general Polaris support.

```text
STRATA_HIP_GFX803=ON
  native gfx803 ISA + existing wave64 compatibility path
  model/expert uploads → per-thread 1 MiB pinned staging → GPU
  bulk CPU experts → pageable arena → scoped source-page reclaim
  prompt batches → hybrid path only (effective rows 8..32)
```

Leaves Q8 KV/tiled attention, CPU batching optimizations, startup caching, setup auto-detection and dependency installation out of this PR.

## What changed

- `CMakeLists.txt`: explicit experimental gfx803 selection and three focused host tests.
- HIP compatibility header: missing hipBLAS aliases.
- `src/prefill/kernels.cu`: gfx803 fallback for an unavailable packed-FP16 instruction.
- Q36 engine/helpers: bounded synchronized uploads, pageable Linux expert arena, reclaim advice restricted to fully copied page-aligned source regions, and an 8–32 hybrid-row cap preventing entry into the faulting streamed expert/MMQ path.
- `tests/qwen35/`: transfer chunking, CPU FP16 conversion, batch-limit regression tests and configurable API smoke harness.
- `docs/AMD_HIP.md`: experimental build prerequisites, usage and limitations.

## Evidence

- **Before:** Unclamped candidate API/tool prompts entered streamed expert/MMQ and produced GPU VM protection faults. The unchanged safety guard stopped those runs.
  **After:** The bounded hybrid implementation passed the in-repository API harness: tool request/result round-trip, stream disconnect during generation, recovery, and fresh-process restart. This avoids the failing path; it does not fix or qualify streamed MMQ.
- Native gfx803 `strata-q36` build passed. All **3/3 host tests** and `git diff --check` passed. The FP16 host test does **not** exercise the changed GPU fallback.
- Hardware run: RX580 8 GiB, Linux, 32 GiB RAM; isolated ROCm 6.3.4 and separately built gfx803 rocBLAS/Tensile. Acceptance took **138.75 s**, peak GPU **58°C**, swap increase **4 KiB**, empty cursor-bounded kernel log and no guard stop. Thresholds: GPU 75°C, CPU 85°C, 2 GiB available-memory reserve, 64 MiB maximum swap growth.
- Model/config: Qwen3.6-35B-A3B UD-Q4_K_XL; context 2,048; requested chunk 128 / hybrid rows 8 → effective chunk 8; 32 expert slots/layer; three CPU workers plus host; no graphs/warmup; `STRATA_Q8_PACKED=0`; native FP32 KV. This is a tested configuration, not a hardware context limit.
- Base: `296b59d2f73c8f04a805df29c2fa97655adace78`. Hardware-tested implementation: `452eb85a42eb177b195e44e5cb6d9d28fde2d202`. Head: `f82214d627f5fd4318de9a30f48d3c1d60674028`, differing only by a docs correction. Fresh independent final-head review verified unchanged executable/test/build tree identities and found no material findings.

Reproduce host checks after configuring/building the focused test targets:

```sh
ctest --test-dir <build> --output-on-failure \
  -R '^q36_(pageable_transfer|fp16_conversion|gfx803_batch_limits)_test$'
```

Hardware harness (run under an external thermal/memory/kernel-error guard; it is not itself a resource monitor):

```sh
<python> tests/qwen35/gfx803_hardware_acceptance.py \
  --cwd <checkout> --python <python> --config <engine-config.json> \
  --model <configured-model-name>
```

Build prerequisites are documented in `docs/AMD_HIP.md`. Guard receipts remain local; this is not bundled hardware CI or a same-day comparative performance benchmark.

## Merge Danger

**Door:** two-way

Opt-in source/build changes can be reverted; no data migration or host/driver changes.

**Blast Radius:** backend

The gfx803 path is opt-in, but shared CMake/HIP compatibility/prefill files are touched. Non-gfx803 runtime semantics were checked by source review, **not** demonstrated byte-identical to the last release. CUDA, existing gfx906/RDNA HIP and SYCL build/regression checks remain outstanding. Do not treat this draft as satisfying the repository's all-affected-backends build gate.

## Extra Notes

### Contribution-guide status — incomplete

- [x] Searched open issues/PRs for overlapping RX580/gfx803 work before submission.
- [x] One platform-enablement change; independent optimizations excluded; feature opt-in.
- [x] Changed behavior and untested areas described with measured hardware scope.
- [x] Opened and linked feature request #2768 as requested by `AGENTS.md`.
- [ ] Build every affected backend before requesting maintainer review; only native gfx803 was built here.
- [ ] Demonstrate the default path remains byte-identical to the last release, as requested by `AGENTS.md`.

### Why draft

Unresolved historical model-logit differences prevent a numerical-equivalence claim. GPU fallback numerical qualification, full populated 262K context, sustained long-context thermals, MTP/speculative decoding, vision and other models/quantizations/cards are unverified. Synchronous staging sacrifices overlap. No performance gain is claimed for this patch, and results from the separately optimized local runtime are not attributed to it.

This is an experimental draft with contribution gates outstanding, not a request to merge as supported hardware.

Related on this site

Quick links to related pages.