Pull requests / #3210

#3210 [Benchmark] RX 7900 GRE (gfx1100, 16 GB): IQ2_XS production serving measurements at 262K context

open · @lsunay · 0 comments · View on GitHub

BenchmarksAMD / HIPModels & quants

Description

[Benchmark] RX 7900 GRE (gfx1100, 16 GB): IQ2_XS production serving measurements at 262K context

Community benchmark report, no code change. Adds a long-context gfx1100 data point: the 0.2.0 release notes list RX 7000 cards as untested on the 0.2.0 builds, and the earlier RX 7900 GRE report (#740) measured at 65,536 context.

**Method, stated plainly:** these are production serving measurements from the engine's own per-request log over six days of real agent traffic with reasoning enabled — not a controlled bench. 1,004 request lines, 168 with `reused = 0` (fully fresh prompt). Only the fresh lines are aggregated. Output lengths vary per request and `generated` includes thinking tokens, so the decode medians are not comparable to #740's greedy 128-token arm.

Median over the 168 fresh-prompt lines:

| Prompt size | n | Prompt tok/s | Decode tok/s |
| --- | ---: | ---: | ---: |
| < 1,000 | 61 | 198.2 | 52.1 |
| 1,000 - 8,000 | 25 | 777.8 | 53.5 |
| >= 8,000 | 82 | 965.9 | 52.5 |

Prefill holds ~950-990 tok/s up to 114K fresh prompt tokens on this card at 262K context.

Two settings decided on this machine, confirmed in use:
- `--vram-reserve-mib 832` - decode crashed when VRAM filled at long context before it was added (2026-10-13); no crash in six days of continuous serving after.
- `fit_max_tokens` - removed the 400 "request exceeds the context window" error when a client sent `max_output_tokens: 32768` at long context.

Environment: Strata 0.1.43.1 source build, commit `a0116b2`, HIP/gfx1100, hipBLASLt gfx1100 table 100200 (26 rows), Ryzen 5 5600X (no AVX-512), 64 GB RAM, `vm.nr_hugepages = 0`, Debian 13. Full config, per-request CSV, aggregates, sanitized log and SHA-256 manifest are in the report folder.

Limitations: one machine, one card, no cross-engine or cross-GPU baseline; no needle test, no TTFT, no greedy arm; ROCm runtime version is not printed in the engine log and could not be confirmed on this host, so only the tuning table version is recorded. The log is a snapshot taken 2026-10-19 while the engine was still serving.

Reproduction status: all numbers come from the log produced on this machine.

Related on this site

Quick links to related pages.