问题 / #2498

#2498 calibrate's --pcie-frac 0.00 recommendation wedges the served engine at boot on AMD gfx1201 (ROCm)

open · @bhodgens · 0 评论 · 去 GitHub 看

Server & APIAMD / HIPModels & quants

说明

# Strata calibrate recommendation wedges the served engine at boot on AMD (gfx1201)

**Strata version:** v0.1.44 (`ed4773e`)
**Backend:** HIP/ROCm, built from source for **gfx1201** (Radeon AI PRO R9700)
**Host:** Ubuntu, ROCm 7.2.1, AMD Ryzen AI MAX+ 395 + R9700 (discrete)
**Model:** Qwen3.8-Flash-Next GSQ-RCO IQ3_S, served native, `--shared-expert-arena /dev/shm/strata-arena`, `--adapt-async 1`

## Problem

`tools/calibrate.py` recommended `--pcie-frac 0.00` and `--spec-min-p 0.70`, and its own interleaved measurement showed ~23% decode gain (65.8 vs 53.4 t/s). But applying those settings to the **served** config wedges the engine during startup.

## Repro

1. Start flashnext from a config whose args end with:
   ```
   ... --kv k8v4 --ple-io ram --shared-expert-arena /dev/shm/strata-arena --adapt-async 1 --pcie-frac 0.00 --spec-min-p 0.70
   ```
2. Engine prints up to `strata generate: GPU 0: AMD Radeon AI PRO R9700 (gfx1201)` then produces **no further log output**, indefinitely.
3. Process state is `D` (uninterruptible sleep) + `L` (memory-locked), CPU at 2-5%, for 6+ minutes (we killed it; may never complete).
4. Removing `--pcie-frac 0.00 --spec-min-p 0.70` (back to the pre-calibrate config, `--spec-min-p 0.5`, no explicit `--pcie-frac`) boots normally and serves ~70 t/s.

## Expected vs actual

- Expected: calibrate's recommended settings load and serve like any config.
- Actual: the engine hangs at/just after `GPU 0: ...` device enumeration when `--pcie-frac 0.00` is present.

## Additional notes

- `--spec-min-p 0.70` alone is likely safe (it is just the MTP draft-confidence floor and measured near-neutral); we isolated the hang to the full calibrate settings, not to `--spec-min-p` alone. We did not binary-search which flag causes it.
- `--pool-workers 15` (the default on this host) was already optimal per calibrate; no issue there.
- The calibrate harness measures the engine in-process (`StrataEngine.generate`) rather than the served path, so a setting that measures well can still fail the served bootstrap. It would help to have the served path covered too, or a note that calibrate results are engine-only.

This is on a discrete AMD card (gfx1201, ROCm) — not a Mac/Metal or NVIDIA path.

本站相关内容

相关页面的快捷入口。