Pull requests / #2208
#2208 docs/STRIX_HALO.md: STRATA_PF_SWITCH_MIN_T compares the chunk, not the prompt (a 4,096-token prompt is a 4,089-token chunk); 2048 puts it on the switches
open · @libratechw · 0 comentários · No GitHub
BenchmarksAMD / HIPDocumentation
Descrição
## Title
Issue: #2021 (the note only; the recipe value is still open there)
docs/STRIX_HALO.md: `STRATA_PF_SWITCH_MIN_T` compares the chunk, not the prompt (a 4,096-token prompt is a 4,089-token chunk); 2048 puts it on the switches
## Summary
Section 5 of `docs/STRIX_HALO.md` sets `STRATA_PF_SWITCH_MIN_T=4096`, but the threshold is compared with the length of the prompt chunk. A 4,096-token prompt ("4K") is a chunk of 4,089 tokens with the prompt cache on (4,095 with `--prompt-cache 0`), so it stays under 4096 and the hyper-connection and padding switches are off for it (#2021: UD-Q4_K_XL 4K, 1,052-1,106 tok/s at 4096 against 1,262-1,298 at 2048). This adds the note the maintainers asked for in #2021, with the first-token KL for 2048 against 4096.
## What changed
- `docs/STRIX_HALO.md`, section 5, after the export block: one paragraph and a two-row table. Nothing in the code and no default changes.
The chunk length was measured by moving the threshold across a 4,096-token prompt (UD-IQ4_XS, `--spec 4`, `--prefill 16384`): on at 4089, off at 4090; with `--prompt-cache 0` on at 4095, off at 4096. The cause is in `generate.cpp`: the read is cut at the last `<|im_start|>` (`turn_at`, a 7-token header with reasoning off) and the last token is read alone.
First-token KL against the default engine, 8 prompts of 1.5K-8K tokens, mean (max), top token the same at both values in all 8:
| | `STRATA_PF_SWITCH_MIN_T=4096` | `STRATA_PF_SWITCH_MIN_T=2048` |
|---|---:|---:|
| UD-IQ4_XS | 0.0168 (0.047) | 0.0154 (0.047) |
| UD-Q4_K_XL | 0.0026 (0.0074) | 0.0023 (0.0074) |
## Extra Notes
Whether to change the recipe's 4096 is left to the maintainers (#2021); this only documents the behaviour. Measured on gfx1151 (Ryzen AI Max+ 395, ROCm 7.14.1). The header length depends on the template and the reasoning mode, so the 7-token offset can differ for other requests; the note says to set the value below the chunk of the shortest prompt wanted.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
No site
Links install, modelos, releases.