#2966 Gfx906 combined inference 20261017

Description

**Draft for review, not merge-ready.** This combines bounded expert, selector, prefill and serving optimizations for a qualified two-card gfx906 profile. CUDA, non-gfx906 HIP and SYCL remain untested; common HIP build prerequisites are still separate dependencies.

## Change

All new controls default off. The proposed single user-facing change is an opt-in hardware profile, organized into three reviewable code commits:

1. Exact Q2 GPU mode16 and AVX2 CPU-tail spreading; bounded FIT17 top-k plus current-main wide fallback; parity/policy tests and baseline-ISA test launcher.
2. Guarded primary-commit overlap and empty-PCIe-group omission, with publication, lifetime and cleanup checks.
3. Owned HC N4 and HC F16 exact-input prefill paths, resource limits, conservative fallbacks and comparison tests.

The final selector keeps FIT17 only in its useful live-context band. Earlier reference/reg66 variants regressed against newer main and were removed. Exact HC operand conversion is not asserted to prove general GEMM output identity; the comparison tests provide bounded evidence.

We recognize the repository's one-change-per-PR principle. Component commits make this proposed hardware-profile scope reviewable; they do not presume acceptance of that framing. No component-removal ablation establishes each component's individual contribution.

## Results

Base: main `477ac13cbe8e582c20484871ce459d20d82df09d`, unchanged at the October 17, 19:07 UTC audit. Both arms include identical build prerequisites equivalent to #2460 and #2020, excluded from optimization credit.

Two Radeon Pro VII 16 GiB gfx906 cards, Xeon AVX2, AMD Clang 23 (pinned toolchain; unified ROCm release unverified); Q8_0 PLE, fixed 65,536-token prompt and 1,024 generated tokens. Context 262144, prefill 4096, int8 KV, split 27, reserves 1200/1300 MiB. Both arms use full routed work (`STRATA_ROUTE_TAIL_SKIP=0`); this is not a comparison against upstream's approximate default.

Six fresh-process interleaved pairs, fixed order AB, BA, BA, AB, AB, BA:

| Phase | Pooled tokens/s, A → B | Geometric mean paired gain | Approximate 95% paired-log-t interval |
|---|---:|---:|---:|
| PP | 621.434 → 643.935 | +3.62505% | +2.24982% to +5.01878% |
| TG | 51.2641 → 55.1578 | +7.59548% | +7.27287% to +7.91906% |

All six pairs improved both phases. All twelve arms matched all 1,024 output IDs and non-timing work counters. The first, lower-gain pair is retained. Separate smoke/default-off and full canary checks passed and are excluded from these estimates.

The [maintainer’s criterion](https://github.com/Niko1221/Strata/pull/1535#issuecomment-6231053038) was >=3% over 5+ interleaved pairs. **TG is the stronger threshold evidence:** PP's point estimate exceeds 3%, but its interval extends below 3%. Neither interval guarantees performance on other workloads. Pooled throughput and paired geometric gain are different estimators; historical/component percentages are not added.

## Evidence and reproduction

[Benchmark evidence and reproduction kit](https://github.com/0FL01/Strata/blob/gfx906-combined-inference-20261017/bench/results/2026-10-17-gfx906-combined/README.md). Includes all twelve scored logs/receipts, frozen configuration, exact token fixture and offline replay:

`python3 protocol.py evidence/scored-six-pairs/manifest.json`

Source starts from the pinned main plus identical common prerequisite patch; apply the three optimization patches only to B. [Measured source/build commands](https://github.com/0FL01/Strata/blob/gfx906-combined-inference-20261017/bench/results/2026-10-17-gfx906-combined/BUILD.md) record the pinned compiler/dependency and exact baseline/candidate CMake invocations. The exact image is unpublished and identified only by local content ID: this is an explicit exact-image reproduction gap, not a downloadable image. Model assets are not bundled.

Final publication patch SHA256: `05ce05b722adc3ed61f0ec1fd11f8f4fb4d6a2b5d7f92cb916fae4516bd9df5d`. Its post-measurement ISA-launcher changes affect tests only; production code matches measured v2. Measured candidate binary SHA256: `891bfedf84e2eda3030b0f249f7add8d1ee406adc82192471e8fea082c55c280`.

## Attribution and remaining gates

Conceptually consolidates unmerged work from #1187, #1535 and #1743; those PRs have not been closed. Empty-stage omission credits #2223. Already-upstream query swizzle/reduce12 and parity infrastructure receive no new speedup credit. Existing async commit is extended, not newly invented.

Code commits: 185d9cca (experts/selector), 10651f8d (serving), 06ab098d (prefill). Keep this PR draft while build dependencies, other-backend validation and exact-image reproduction gaps remain unresolved.