#3318 [Benchmark] RTX 5090 desktop streaming regime: sm_120 gains land, but the identity gate has a large boot-to-boot floor

Descrição

We ran **0.2.0-beta** against our 0.1.45 daily on a configuration that appears to be new to the beta pool: RTX 5090 (sm_120), 128 GB host RAM, Ubuntu 26.04, engine self-compiled from tag `3d1fb480` with the CUDA 13.4 kit (following the docs' own warning that nvcc 13.2.0/13.2.1 miscompile sm_120 IQ products). Models are Qwen3.8-Flash-Next, 1M context via YaRN 4x, int8 KV, expert cache at 6,500 slots, daily settings throughout (`--spec 4 --spec-min-p 0.5`, solo MTP, batch 2, `STRATA_ADAPT_OVERLAP=1`, vram-reserve 2,200 MiB). We ran two pack geometries so the central question could be asked at two streaming depths: GSQ-RCO **IQ3_S** and Unsloth UD-**IQ4_XS**. Method: one fixed set of twelve prompts (ten short, plus 4K and 20K long-prompt dumps) at temperature 0, captured verbatim, plus one decode pair. The beta served measurement only; every pass included SHA-256 manifest checks of the model packs, expert profile and MTP files.

## 1. The sm_120 decode defaults transfer to a streaming rig

One interleaved pair, 300-token greedy story, fresh boot each side: 9.33 ms/token with the defaults against 10.51 ms/token with `STRATA_SM120_PERF=0` — **+11.2%**, against your claimed +13.8% on an all-resident 110-SM card. This is n=1 pair: consistent with the claim, not a tight replication. More boots on request.

## 2. On a streaming machine the identity gate has a large floor, and it grows as residency drops

The release notes quote each new default against "the answers are identical to 0.1.45 (the identity gate)." On this hardware, "identical" needs a noise floor before it means anything, because **two boots of the same binary with the same configuration at temperature 0 already diverge on a large fraction of short prompts**:

| Pair (same binary, same config, two boots) | Short prompts diverging | 4K dump | 20K dump |
| --- | --- | --- | --- |
| IQ3_S, 0.1.45 | 4 / 10 | identical | identical |
| IQ3_S, beta, both switches OFF | 6 / 10 | identical | diverges at token 36 |
| IQ4_XS, 0.1.45 | 6 / 10 | identical | identical |
| IQ4_XS, beta, defaults | 9 / 10 | diverges at token 1 | identical |

(Each cell is n=1 boot-pair.) The pattern is mechanistic: the floor grows as the expert cache streams more (IQ4_XS at ~25% residency flips more than IQ3_S at ~33%), and it concentrates in near-tie prompts — arithmetic and code probes never flipped in any pair; creative, multilingual and fact-ordering probes flip even at token 0.

The consequence reframes every cross-version number, including our own: **at one boot per version, byte comparisons on streaming machines measure mostly this floor, not the version change.** Our cross-version gates sit at or near the floor, so we report no quality regression — and symmetrically, we cannot confirm the "bitwise" claims of the sm_120 decode set on this machine, because a rig whose same-binary floor already covers 40–90% of probes cannot demonstrate byte-identity at all. For this regime we would suggest quoting the identity gate against a two-boot floor, or moving default-claims testing to teacher-forced KL, which the in-tree tooling already supports and whose floor is near zero by construction.

Related gap: the beta `serve` (engine pkg 0.1.45.3) accepts `"logprobs": true` on chat requests but returns no logprobs field, and `/v1/completions` answers "not found." If logprobs pass through by GA, distribution-level version diffs become measurable on streaming machines without any sampling floor at all.

## 3. #1121 (batch-MTP on native IQ3_S packs): the crash is gone — parity is not

On 0.1.40 through 0.1.45 this configuration died at the first concurrent batch admission with `mtp: unsupported native MMVQ GGML type` (our original report, #1121, since closed-stale). Under beta the same invocation starts cleanly and works: admissions succeed, batch windows are captured, and MTP proposals are accepted at 51/75 (68.0%). Repro: `python tools/batch_test.py --exe <beta strata> --config <iq3_s daily config, env STRATA_SM120_PERF=0 and STRATA_PF_FUSED=0> --batch 2 --n 2 --extra="--batch-mtp"` on an otherwise idle GPU.

The same run, however, reports a batch-vs-solo parity break: slot 0 diverged from its own solo decode at token 47 of 64 (slot 1 identical), with both sm_120 switch sets OFF, same boot, greedy. So batch-plus-MTP now runs on sm_120 streaming but is not yet equal to the solo path; we do not run batch-MTP daily, so we cannot say whether this is sm_120-specific, streaming-specific, or general. It seems worth a look before the policy settles at GA.

## 4. Robustness observations

The engine never wrote to any model artifact we could measure (hash manifests unchanged across four passes). When the GPU was still busy, the memory guard exited cleanly with a clear OOM message. And the new GPU vision reader loaded in 0.8 s inside our 2,200 MiB reserve and answered a single-pixel colour probe correctly on sm_120.

## 5. One note for source installs

0.2.0 moved the wrapper's shared modules to top-level packages (`strata_cli/`, `strata_control/`); a wrapper-checkout list that omits them boots the new engine with a broken wrapper, and a downgrade checkout does not remove files only the newer tag has.

Machine details, the full config line, and every identity/dump JSON are available, and a dashboard-format submission of the beta+IQ4_XS field-test lane will follow on brenoperucchi/strata-bench once the harness has run on a quiet card.