Issues / #485
#485 PCIe probe can read ~3x low and pin pcie_frac far below the calibrated value
closed · @yannickloth · 2 comments · View on GitHub
BenchmarksNVIDIA / CUDAModels & quants
Description
`probe_pcie_h2d_gbps()` (`src/program/generate.cpp`) measures one 256 MiB x4 pinned burst after a single warmup and derives `pcie_frac` from it. On the same machine the same binary read the link at very different rates across starts, and a low sample pins `pcie_frac` well below what `tools/calibrate.py` picks. ## Environment - NVIDIA RTX A3000 12 GB Laptop (sm_86), PCIe **x16 Gen4** (`current_link_width=16`, `16.0 GT/s`) - Intel Core i7-12850HX (8 P + 8 E, AVX2, no AVX-512), 128 GB RAM - Engine 0.1.34, native IQ3_XXS pack (39.97 GiB arena) ## Measurements (same host, across starts) ``` PCIe probe: 18.5 GB/s host->device -> pcie_frac 0.39 (default 0.55) PCIe probe: 6.9 GB/s host->device -> pcie_frac 0.15 (default 0.55) PCIe probe: 5.8 GB/s host->device -> pcie_frac 0.12 (default 0.55) ``` `tools/calibrate.py` on the same host (decode tok/s, `--spec-min-p` fixed): | `--pcie-frac` | 0.0 | 0.2 | 0.35 | 0.55 | 0.75 | | --- | ---: | ---: | ---: | ---: | ---: | | tok/s | 27.0 | 30.1 | **32.7** | 32.5 | 30.8 | So the auto value the probe produced (0.12-0.15) is ~5 tok/s below the calibrated 0.35, and 0.35 also edges out the 0.55 default. ## Likely cause One 256 MiB x4 burst (1 GiB) after one warmup, on the default stream, with no retry/median and no exclusivity check: concurrent copy-engine work, ASPM/low link power state, or a context that has not ramped can depress the single sample, and the result is then used as a hard tuning input. ## Suggested fix Prime the link and take the median of several bursts (a host-side fix is on the `fix/pcie-probe-median` branch of my fork), and/or read the trained link width/speed (NVML `nvmlDeviceGetCurrPcieLinkWidth` / `...LinkSpeed`) instead of one bandwidth sample. Note also the `bw / 26` formula's discontinuity at 20 GB/s (19.9 -> 0.42, 20.0 -> 0.55), which a noisy sample can straddle. A full write-up of this host (hugepages, probe, calibration) is in `bench/results/2026-10-02-rtx-a3000-i7-12850hx/`.
Related on this site
Quick links to related pages.