Pull requests / #3146

#3146 report: RX 6700 XT (gfx1031 running as gfx1030) - built-in suite in two arms; STRATA_HIP_PROMPT_F16=1 reads prompts 1.76x-1.99x faster, decode unchanged

open · @ljmbot · 0 commentaires · Sur GitHub

BenchmarksAMD / HIPModels & quantsDocumentationWindows

Description

Issue: Resolves #

## Summary
A community report for the card the maintainers do not have: RX 6700 XT, 12,272 MiB, `gfx1031`, run as `gfx1030`
under `HSA_OVERRIDE_GFX_VERSION=10.3.0`, which is the only way long prompts run on ROCm 7.1.1 at all (rocBLAS ships
31 `gfx1030` library files and 0 `gfx1031` ones - see the paired fix PR #3145). Swift 1.5 IQ3_S, 65,536-token context, KV
int8, one parallel slot, expert cache 2,645 slots / 5.07 GiB of VRAM.

`tools/strata_bench.py --long` (chat, 4K, 16K, agent session, 32K), three runs per test, run twice - the default
prompt GEMMs and `STRATA_HIP_PROMPT_F16=1` - one engine session per arm, model reloaded from cold between arms,
nothing else changed:

| prompt | default | with `STRATA_HIP_PROMPT_F16=1` | |
| --- | ---: | ---: | ---: |
| 4,019 tokens | 339.2 (294.1-339.5) tok/s | 595.9 (448.2-596.1) | 1.76x |
| 16,058 | 352.5 (343.2-355.7) | 692.9 (674.9-714.0) | 1.97x |
| 32,049 | 356.2 (353.8-357.4) | 707.5 (701.7-711.1) | 1.99x |

TTFT 11.90 / 45.61 / 90.05 s becoming 6.80 / 23.23 / 45.39 s. Decode is not touched: chat decode 21.7 (20.2-26.8)
against 21.0 (19.7-24.9) tok/s over 9 runs each, agent-turn decode 23.6 against 22.3. The agent session caches the
same in both arms (68.8% of the follow-up-turn tokens reused, about 2 tokens of earlier turns read again), so the
difference is the reading: turn TTFT 9.59 -> 6.16 s, and the turns that re-read about 2,900 uncached tokens ran at
207-263 against 519-552 tok/s. The whole suite took 731 s and 464 s.

## What changed
- `bench/results/2026-10-19-community-rx-6700-xt/` - README.md (hardware, config, both arms, every table, correctness
  and what was not tested), METHODS.md (how the arms were separated, the suite command, the probe build line,
  sanitizing, what was not measured), `results-default.json` and `results-f16.json` (the tool's own per-request
  data, field for field), `engine-log-default.txt` and `engine-log-f16.txt` (each arm's startup lines: the arch the
  card reported, the GEMM route the engine took, the cache sizes), and `scripts/` with the two extra measurements -
  `rocblas_gemm_ex` at the prompt shapes (7.0 TFLOPS f16 in / f32 out, 21.3 f16 in / f16 out, 6.6 bf16, 11.8 f32,
  each checked against a CPU reference) and a matched 11,103-token prompt read in the same session (34.4 s and 33.9 s
  against 17.9 s, 1.90x).
- `docs/COMMUNITY_BENCHMARKS.md`: one bullet in the report list, in date order.
- `bench/results/COMMUNITY.md`: one row in the index table, PR column "PR pending".

## Extra Notes
The README's arch links point at `docs/AMD_GFX1031.md`, which is in the paired fix PR #3145; the link 404s on `main` until
that one merges.

This is a speed run, not a parity run: every arm answered coherently (the chat questions, the agent turns against a
project file, the 11,103-token summary), and there is no needle test, no golden-answer parity and no HumanEval here.
The FP16 route is not bit-identical to the default (an FP16 rounding of the prompt GEMMs, which the engine says on
its startup line), so answers can differ slightly on borderline prompts.

Not tested: the 4K / 32K token-id parity between the two arms, Windows, TheRock's venv ROCm, a layer split with a
6700 XT in it, the vision encoder (`--vision` off in this config), concurrency (`--seqs 1`, no `--batch`), a long
soak, other quantisations than Swift 1.5 IQ3_S, the other `gfx1031` cards (6750 XT, 6800M), ROCm older than 7.1.1,
PCIe width (no passwordless sudo for `lspci -vv`), RAM speed, temperatures and clocks under load. One machine, one
engine session per arm, one order (default first, FP16 second), so drift between sessions is not separated out; the
chat control arm and the identical cache hit rates argue against a large drift. The engine is a source build for
`gfx1030;gfx1031` (`engine/BUILD.json`), not a release binary. The model's GGUF sits on a FUSE-mounted SATA SSD
(`fuseblk`), not compared against a native mount.

Sur le site

Liens install, modèles, releases.