Pull requests / #1992
#1992 calibrate: measure the shared-expert stream fork on AMD (STRATA_SH_STREAM) and keep it per PC (#1272)
open · @Vop3p · 0 Kommentare · Auf GitHub
BenchmarksSetup & installMulti-GPUAMD / HIPNVIDIA / CUDAModels & quantsDocumentationWindowsLinux
Beschreibung
## Summary `tools/calibrate.py` gets a sixth step on AMD: it measures the shared-expert stream fork (`STRATA_SH_STREAM`) off against on, a restart each, and writes `STRATA_SH_STREAM=1` into the config's `"env"` only when it wins by more than `MIN_GAIN`. This is the per-card choice left open in #1272. The HIP default stays off (#816, #826): an RX 6800 on Windows and 4x R9700 decode 12-45% faster without the fork. The same card family on another PC goes the other way (#1272: 2x RX 6900 XT on Linux), and the output is identical either way, so a measurement on the user's PC is the right place to decide it. ## What changed - `tools/calibrate.py`: step 6, HIP only (`cfg["backend"] == "hip"`), after the adaptive-tier step and with everything chosen before. Both values are set explicitly (`0` and `1`), so a value the config already carried does not decide the baseline; two rates each; the existing `pick()` / `MIN_GAIN` rule and the #1337 failed-start handling. `measure()` returns the choice as `"env"`; `start_engine` takes an optional env; `apply_env()` sets the chosen variables and removes the other `ENV_KEYS`, so an older calibration's choice does not linger. - `setup.py`: `--calibrate` writes `"env"` into the config and the settings file; a reinstall applies a saved calibration's `"env"` the same way (one saved before this step leaves `"env"` alone). Where setup offers calibration is unchanged (NVIDIA only; on AMD it is still `./setup.sh --calibrate` by hand, #566). - `docs/AMD_HIP.md`: the `STRATA_SH_STREAM` note says calibrate measures it. - `tools/test_calibrate.py`: four tests (the step runs on HIP only and keeps the fork only on a win; a failed start keeps the default; `apply_env`; setup writes and later removes the variable) and the `run()` test now checks that `hip` is passed. 21 of 21 pass. ## Measured Real engine (the 0.1.42 tag, built with setup's HIP flags), 2x RX 6900 XT (gfx1030, PCIe 4.0 x8 each), Ryzen 5 5600X, Ubuntu 26.04, ROCm 10.0, GSQ-RCO IQ3_S on a layer split, a setup-style config (`--spec 4 --spec-min-p 0.5`, empty `"env"`). The whole calibration ran in 982 s with no failed start and no stall: | step | chosen | measured | |---|---|---| | PCIe share | 0.00 | 62.0 against 59.2 tok/s at the probed 0.39 | | draft floor | 0.70 | 62.3 against 61.3 at 0.5 | | pool workers | 5 (the engine's) | 3: 60.3, 2: 56.5 | | expert tier | every 1, 80 swaps, decay 0.97 | 63.8 against 61.6 | | **shared-expert fork (new)** | **`STRATA_SH_STREAM=1`** | **off 62.2 / 61.3, on 67.2 / 66.6 tok/s (+8.3%)** | Identity, separately, with a fixed protocol (`--spec 4 --spec-min-p 0.5`, 4K and 32K prompts, two each, fresh starts; the baseline reproduces its text and draft counts across starts): fork on against off gives the same text on 4 of 4 requests on the split and on one card, decode +8.7 / +9.9% (split) and +5.8 / +6.0% (one card). Not tested: Windows HIP, other AMD cards (the step's purpose is that they decide for themselves), a CUDA build (the step does not run there). Developed with an AI coding assistant; every number above was measured on 2x RX 6900 XT (gfx1030, PCIe 4.0 x8 each) / Ryzen 5 5600X, ROCm 10.0. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Mehr auf der Site
Links zu Install, Modellen, Releases.