Pull requests / #2653

#2653 calibrate: the CPU prefill share's range measured per PC (STRATA_PREFILL_CPU_SHARE_MAX 3072 against the default, into the config's env)

open · @qiuzan2001 · 0 comentários · No GitHub

BenchmarksSetup & installAMD / HIPNVIDIA / CUDAModels & quantsDocumentationLinux

Descrição

For #2478's question ("Could setup measure the share once and record it when it helps?"), for the range rather than the arming.

## What it changes

- `tools/calibrate.py`: a sixth step, the CPU prefill share's range. The share is on by default for prompt chunks below 1,024 tokens; `STRATA_PREFILL_CPU_SHARE_MAX=3072` extends it, and that stays opt-in because it depends on the CPU: docs/DETAILS.md records -2% at 2,048 tokens on the Ryzen 5 7600 + RTX 5070, and on a Ryzen 7 5800X3D + RTX 5070 Ti it measured -24% / -14% / -3% at 1.5K / 2.2K / 2.9K (below). So it is measured per PC: one engine start with the variable absent and one with 3072, each reading a fresh ~2K-token prompt three times as a one-shot request (`strata_checkpoint: false`, so no checkpoint is kept and every read is from token 0; a read that reused one is not counted), the prompt tokens per second compared with the existing `pick` rule (more than 3% faster or nothing).
- The winner goes into the config's `env` (`apply_env`, the counterpart of `apply` for arguments: an older calibration's value goes when it is not chosen again), into the settings-file record, and is re-applied on a reinstall (`setup.py`).
- Every calibration start runs without the measured variables, so a value an earlier calibration wrote is not in the baseline of steps 1-5 either. A failed start or a failed read drops that arm only, as #1337 asked for the other steps.
- The step is skipped, with one line saying why, where the share is off (a HIP engine, a layer split, `--batch`) or already at 3,072 (a set `STRATA_PREFILL_CPU_SHARE`).
- docs/DETAILS.md: the fifth bullet in the `--calibrate` paragraph; the 0.1.41 paragraph's "stays opt-in" sentence gains the 5070 Ti numbers and the pointer to the calibration.

## What it leaves alone

- The engine: nothing under `src/` or `include/` changes; the default range stays 1,024 tokens. No backend to build.
- Steps 1-5 of the calibration and their tests; a config the calibration does not touch; a config's other `env` entries (setup's own, Unsloth's, AMD's) are kept as they are.
- The step adds two engine starts (about 36 s each here) to the 8-10 the calibration already does.

## How it was checked

- `tools/test_calibrate.py`: 25 of 25 (8 new: the range chosen at +15% and not at +2%, skipped without a prompt, measured with the chosen worker count, a failed start and a failed read drop that arm only, `apply_env`, `engine_env`, `share_step_applies`, the prompt builder, reads that reused a checkpoint). `tools/test_setup_choices`, `_amd`, `_hybrid`, `_qwen36`, `_unsloth` pass. `tools/test_setup_golden` has 8 failures (the unsloth UD-Q4_K_XL cases); they are the same on upstream/main 296b59d2 without this change.
- The step on the real engine, through the same functions the tool calls (RTX 5070 Ti 16 GB, Ryzen 7 5800X3D, 76 GB RAM, Linux, CUDA 13.1; Swift 1.5 IQ3_S, the setup's config, engine 0.1.44.1 from main): the prompt it builds is 2,375 tokens; two starts per arm, every read `reused=0`:

| arm | prompt tok/s, 3 reads per start | median |
| --- | --- | ---: |
| default (variable absent), start 1 / 2 | 1,292 1,379 1,380 / 1,291 1,379 1,378 | 1,378 |
| `STRATA_PREFILL_CPU_SHARE_MAX=3072`, start 1 / 2 | 1,404 1,553 1,555 / 1,321 1,546 1,551 | 1,548 |

  `pick` chooses 3072 (+12%).
- The range itself, measured the way a user feels it (fresh turns of real prose and code on a cached 57,102-token conversation, greedy, the engine restarted for every run, three alternating pairs, `prompt_ms` from the server; same box and model):

| fresh chunk | default (1,024) | `..._MAX=3072` | pairs |
| --- | ---: | ---: | --- |
| 591 tokens | 1,125 ms | 1,124 ms | +0.0 / +0.2 / -2.1% |
| 1,537 tokens | 1,979 ms | 1,501 ms | -24.2 / -23.7 / -24.0% |
| 2,142 tokens | 2,041 ms | 1,748 ms | -14.6 / -14.3 / -12.8% |
| 2,848 tokens | 2,173 ms | 2,113 ms | -2.0 / -2.8 / -3.1% |
| 4,631 tokens | 2,504 ms | 2,503 ms | +0.0 / -0.3 / -0.1% |

  The decode after each turn had the same expert-cache hit rates in both arms. A second set of two rounds per arm (a build whose default range was 3,072, against the 0.1.44.1 build) gave the same figures: -24.1 / -13.9 / -2.7% at 1,537 / 2,142 / 2,848 tokens, and `=1024` on that build equal to the 0.1.44.1 build.

An independent review pass read the diff; its points (a failed read must drop only its arm, the baseline must not carry an earlier calibration's value, skip where the share is off) are in.

The standalone companion, [Strata Profile Studio](https://github.com/qiuzan2001/strata-profile-studio) (Python standard library only), reads this tool's reports and compares runs against the noise floor of the PC, with a sign test over the agent turns.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

No site

Links install, modelos, releases.