问题 / #485

#485 PCIe probe can read ~3x low and pin pcie_frac far below the calibrated value

closed · @yannickloth · 2 评论 · 去 GitHub 看

BenchmarksNVIDIA / CUDAModels & quants

说明

`probe_pcie_h2d_gbps()` (`src/program/generate.cpp`) measures one 256 MiB x4 pinned burst after a single warmup and derives `pcie_frac` from it. On the same machine the same binary read the link at very different rates across starts, and a low sample pins `pcie_frac` well below what `tools/calibrate.py` picks.

## Environment

- NVIDIA RTX A3000 12 GB Laptop (sm_86), PCIe **x16 Gen4** (`current_link_width=16`, `16.0 GT/s`)
- Intel Core i7-12850HX (8 P + 8 E, AVX2, no AVX-512), 128 GB RAM
- Engine 0.1.34, native IQ3_XXS pack (39.97 GiB arena)

## Measurements (same host, across starts)

```
PCIe probe: 18.5 GB/s host->device -> pcie_frac 0.39 (default 0.55)
PCIe probe:  6.9 GB/s host->device -> pcie_frac 0.15 (default 0.55)
PCIe probe:  5.8 GB/s host->device -> pcie_frac 0.12 (default 0.55)
```

`tools/calibrate.py` on the same host (decode tok/s, `--spec-min-p` fixed):

| `--pcie-frac` | 0.0 | 0.2 | 0.35 | 0.55 | 0.75 |
| --- | ---: | ---: | ---: | ---: | ---: |
| tok/s | 27.0 | 30.1 | **32.7** | 32.5 | 30.8 |

So the auto value the probe produced (0.12-0.15) is ~5 tok/s below the calibrated 0.35, and 0.35 also edges out the 0.55 default.

## Likely cause

One 256 MiB x4 burst (1 GiB) after one warmup, on the default stream, with no retry/median and no exclusivity check: concurrent copy-engine work, ASPM/low link power state, or a context that has not ramped can depress the single sample, and the result is then used as a hard tuning input.

## Suggested fix

Prime the link and take the median of several bursts (a host-side fix is on the `fix/pcie-probe-median` branch of my fork), and/or read the trained link width/speed (NVML `nvmlDeviceGetCurrPcieLinkWidth` / `...LinkSpeed`) instead of one bandwidth sample. Note also the `bw / 26` formula's discontinuity at 20 GB/s (19.9 -> 0.42, 20.0 -> 0.55), which a noisy sample can straddle.

A full write-up of this host (hugepages, probe, calibration) is in `bench/results/2026-10-02-rtx-a3000-i7-12850hx/`.

本站相关内容

相关页面的快捷入口。