#1928 Greedy output depends on wall-clock timing: the draft policy picks the verify window from measured round times, and the CPU expert rounding depends on the window

本文

## Summary

On a native IQ pack (IQ3_S, RTX 5090, v0.1.42 fork), the same prompt at `--greedy` produces different token streams from one fresh engine to the next. Two documented rules combine to cause this, and the documentation never mentions the combination:

1. `DraftPolicy::choose` decides, round by round, whether to verify a suffix (prompt lookup) window instead of the MTP window. It works from the EMA of measured round times (`src/spec/draft_policy.cpp:60-89`, cost EMA at `:91-97`, `kCostAlpha` at `:13`). It is called from `src/program/generate.cpp:12694` in generate, and from `:11122` and `:11540` in serve. It is also observed at `:12829` and `:11695`. The suffix drafter is on by default (`suffix_draft = 3`, `generate.cpp:652`).
2. A CPU expert is computed with ggml's dot product for one token, and with the multi-token kernels for two or more tokens. The two round differently (`src/kernels/cpu/native_expert.cpp:115-131`, used at `:158` and `:171`; down rows at `:204-213`; issue #152, `docs/DETAILS.md:97`). How many tokens in a window share an expert depends on the window, so the logits depend on the window shape.

Wall-clock noise (another process, clocks, a sampler) changes which windows get taken. That changes the rounding, which flips near-ties in the argmax. The header of `draft_policy.hpp` and two comments say the policy "only chooses which drafts to verify: the output is unchanged" (`include/strata/spec/draft_policy.hpp:15`, `draft_policy.cpp:83`, `generate.cpp:657`). That holds only with `STRATA_IQ_MT_MIN=1`.

## Repro

Host: Windows, RTX 5090, native IQ3_S pack, production flags (`--spec 4 --spec-min-p 0.5 --mtp <dir> --prefill auto:32768 --expert-cache auto --kv int8 ...`, `STRATA_PREFILL_CPU_SHARE=0`). Prompt: 2,520 tokens, run with `--max-new 256 --greedy --tokens-file p2.ids`. One fresh engine per run.

Default flags, 16 runs twice (32 runs): 12/16 and 15/16 identical. The minority runs differ from token 114. Each distinct output maps one to one to a distinct pattern in the engine's own `window sizes` and `suffix drafts` stats lines (for example `T5:1 T6:1, 7 windows, accepted 20 of 23` against `T5:0 T6:0, 3 windows, accepted 5 of 8`).

| Setting (one at a time, 16 runs each) | Identical runs |
| --- | --- |
| None (default) | 12, 15, 15 of 16 |
| `--suffix-draft 0` (twice) | 16 and 16 of 16 |
| `STRATA_IQ_MT_MIN=1` | 16 of 16, and the same output with `--suffix-draft 0` (16 of 16) |
| `--adapt-swaps 0` | 10 of 16 |
| `--pcie-frac 0.55` (the probed value) | 16, then 15 of 16 |
| `--pool-workers 8` | 14 of 16 |
| `STRATA_ROUTE_TAIL_SKIP=0` | 16 of 16 (generate mode never enables it: `generate.cpp:8068-8080` is serve-only) |

The decisive pair is `STRATA_IQ_MT_MIN=1` with and without `--suffix-draft 0`: the outputs are byte-identical even though the window patterns differ.

## Serve mode (the reported case) and 0.1.42's PcieAuto

In serve mode, `STRATA_IQ_MT_MIN=1`, the documented reproducibility switch, is no longer enough in 0.1.42. `PcieAuto` (`generate.cpp:1615-1665`, applied at `:11670-11681`) refines `pcie_frac` from the measured CPU pool time during the first decode windows (up to three updates). That changes which missed experts the GPU computes and which the CPU computes. The engine itself says the GPU rounds differently from the CPU (`generate.cpp:5019`). The trajectory differs between fresh servers (for example 0.55 to 0.45 in one step on two servers, and 0.55 to 0.50 to 0.45 on three), and the outputs split exactly along it.

Fresh servers, temperature 0, 256 tokens, p1/p2/p3 (distinct answers per prompt out of 5 servers, with the GPU and RAM sampler running):

| Setting | p1 | p2 | p3 |
| --- | --- | --- | --- |
| Default | 2 | 2 | 2 |
| `--pcie-frac 0.55` | 1 | 2 | 2 |
| `STRATA_IQ_MT_MIN=1` | 1 | 1 | 4 |
| `--suffix-draft 0` | 1 | 2 | 2 |
| `--pcie-frac 0.55` + `STRATA_IQ_MT_MIN=1` | 1 | 1 | 1 (10 servers) |
| `--pcie-frac 0.55` + `--suffix-draft 0` | 1 | 1 | 1 (10 servers) |

`--pcie-frac N` fixes the share (no refinement). `STRATA_PCIE_FRAC_DEFAULT=old` is the documented restore of the 0.1.41 rule (not tried here). `docs/DETAILS.md:1196` should name them next to `--adapt-every 100000`, which is not enough: `--adapt-swaps 0` still produced 3 variants in 16 runs and changed the answer itself.

## Proposal (any one)

- Document that greedy reproducibility needs `STRATA_IQ_MT_MIN=1` (or `--suffix-draft 0`) in `docs/DETAILS.md`, next to the serve section that currently points only at `--adapt-every 100000` (`docs/DETAILS.md:1196`), and fix the three "output unchanged" comments.
- Or add a deterministic mode (one switch) that pins the lookup choice and the multi-token threshold, so benchmarks and regression tests can compare outputs run to run.
- Or make the lookup choice independent of wall-clock, with a fixed margin on acceptance only.

## Notes

- Measured cost of `STRATA_IQ_MT_MIN=1` upstream: 1 to 3 % decode on IQ3_S with AVX-512 (`native_expert.cpp:118-120`). Not measured here.
- Serve mode adds the PCIe share refinement (`generate.cpp:1622-1665`, applied at `:11670`) and the tail skip (`expert_source.cpp:3592-3631`, which depends on the whole window). Both depend on timing or on the window.