Issues / #1478
#1478 gfx1201 (R9700): the gfx1151 prompt switches in arch_defaults.cpp are exact and +4-8% prompt; suggest a gfx12 entry (decode half is slower)
open · @jkuepker · 0 评论 · 在 GitHub 查看
BenchmarksAMD / HIPModels & quantsSecurityLinux
描述
`src/core/arch_defaults.cpp` turns on a list of speed switches only when `gfx_arch_is_gfx1151()` is true. The comment calls them bitwise-exact on gfx1151. I tried them on an R9700 (gfx1201). The **prompt half is exact here too and gives +4 to +8% prompt speed**. The decode half is exact but slower on this card. **Prompt** (localeval, 3 reps, medians, same session): | | 4K | 16K | 32K | |---|---:|---:|---:| | baseline | 2,358 | 2,805 | 2,691 | | `STRATA_GDN_HEAD=1 STRATA_GDN_PP=2 STRATA_GDN_CONVL2=1 STRATA_GDN_NOY=1 STRATA_CVEC_FUSE=1 STRATA_HCD_EXACT=1` | **2,546 (+8%)** | **2,912 (+4%)** | **2,794 (+4%)** | | the whole gfx1151 list | 2,538 | 2,934 | 2,820 | **Decode** (3 fixed greedy prompts, 300 tokens, 3 rounds through the server; spread between server starts is 0.1-0.2%): | | tok/s | |---|---:| | baseline | 91.5 / 91.6 | | decode half (`Q8_PACKED Q6_PACKED MMVF_ROWS ATTN_LANECELL EXPERT_V2 TSUM LFUSE GDN_SPLIT QFUSE`) | 87.7 (−4%) | | `STRATA_SH_STREAM=1` | 89.5 (−2%) | **Suggestion:** add a gfx1201 (or gfx12) entry to `arch_default_env` with only the six prompt switches above. `STRATA_PF_PAD` can stay out because it does nothing without `STRATA_PF_GEMM`. Keep the decode switches gfx1151-only. I measured one card and one model (Coder IQ1_M). Q2_0/IQ3 packs on gfx1201 were not checked. **Environment:** Strata v0.1.40.3 (d5ea713), Linux, AMD Radeon AI PRO R9700 (gfx1201, 32 GB), Ryzen 9 9900X, 30 GB RAM, TheRock ROCm 7.14.0a20260612 (`STRATA_ROCM_VERSION`), Coder IQ1_M, `--resident-experts`, `STRATA_HIP_WMMA=1`, `--prefill auto:32768`, all 12,288 experts in VRAM. **Quality check used for every variant:** greedy needle retrieval at ~4K / ~16K / ~57K tokens plus a code task. All 4 pass, and the 4 answer hashes are identical to the baseline.
站内延伸阅读
链到安装、模型与版本说明,便于 SEO/GEO,非官方 issue 正文。