#3339 [Benchmark] RTX 4090 Laptop: EXL3 2-bit worker comparison

本文

## Problem and result

Community measurements for a 16 GB RTX 4090 Laptop + i7-13700HX running EXL3 `2.05bpw_h8`: ten versus seven CPU workers on the same modified fork binary.

- Fixed-prompt median decode: **+3.3–11.4%**; built-in agent follow-ups: **48.0 → 53.4 tok/s**.
- Short chat regressed: **30.1 → 28.9 tok/s**. Prefill changed little.

[Report and per-request evidence](bench/results/2026-10-20-community-rtx-4090-laptop-exl3-2bit/README.md).

## Related discussion

#1764 supplies the EXL3 context; it closed unmerged during the tracker reset. This is new benchmark evidence, not a request to merge that engine proposal.

## Approach

Adds one results folder only: a 903-word report, sanitized measurements/logs/configurations, provenance and reproduction inputs. Source patches are historical evidence inside that folder, not applied engine changes. No default, engine or headline-performance changes.

## Validation

**Passed:** Offline checks on Linux against the staged report: all 25 files match the reviewed draft; artifact hashes match; all 90 fixed-suite summary statistics recomputed from 30 measured requests; fixed/HTTP table values agree with saved data; links resolve; private-path/common credential-pattern scan clean. Evidence patch was also applied to a temporary archive of `fa21db8` and reproduced the modified source; temporary tree removed.

Recheck artifact hashes from the report directory:

```sh
python3 -c 'import hashlib,json,pathlib; p=pathlib.Path("."); m=json.loads((p/"provenance.json").read_text()); assert all(hashlib.sha256((p/n).read_bytes()).hexdigest()==v["shared_sha256"] for n,v in m["exports"].items()); print("PASS: artifact hashes")'
```

**Failed:** No report-consistency failures remain. Historical larger-cache/miss-sharing candidates failed the equivalence gate and are excluded from the selected results. Greedy generation matched only **2/15** control traces, despite matching the fixed-set quality checks.

**Not run:** No inference, GPU builds or VM/app tests during report preparation; this changes evidence/docs only. Original measurements are dated 2026-10-20 on fork `fa21db8` plus recorded patches, engine `0.1.45`—not current upstream.

## Limits

Q8 dense projections, INT8 KV and MTP; not all-EXL3. Three repeats per fixed length, uncontrolled background state, workload-dependent continuations; no significance or task-accuracy claim. Original MTP preparation details and full prompt-logit arrays are not included. Reproduction gaps are explicit. This is not a result under the draft standard benchmark protocol.