问题 / #2696

#2696 GTX 1080 Ti (sm_61): STRATA_DENSE_F32=1 confirmed on cuBLAS 12.8.4.1 - gemm_f32_parity 5,201 sizes bitwise, 7K prompt 341 -> 375 tok/s

open · @MrKop-git · 0 评论 · 去 GitHub 看

BenchmarksMulti-GPUNVIDIA / CUDAModels & quantsWindows

说明

A second sm_61 card and a third cuBLAS for the opt-in `STRATA_DENSE_F32=1` route from #2178. Full report and per-run data in #2695 (`bench/results/2026-10-16-community-gtx-1080-ti-dense-f32-ab/`).

**Parity first, as OLDER_GPUS.md asks:** `gemm_f32_parity` (engine 0.1.44.1 build, the toolkit's own cuBLAS **12.8.4.1**) — **5,201 chunk sizes bitwise equal**, exit 0, the same count as the P40's 12.4.5.8 and the P100's 12.9 in #2178. The tool's kernel bench also shows the 7,040/7,041 cliff on this card (qsa query 121.7 -> 60.7 ms, gate 53.9 -> 27.9), while several sizes a 4,608-token chunk runs are already on the fast FP16 kernel (query T2048 0.98x).

**Engine A/B** (same binary both arms, `env.STRATA_DENSE_F32` 0 vs 1, arms alternated A B A B, one instance at a time, both NUMA-bound to the GPU's node per #2684, every arm started at 57-58 C, medians of 6 requests):

| | `=0` | `=1` |
| --- | ---: | ---: |
| ~565-token prompt | 240.4 tok/s (232-244) | 257.0 tok/s (249-260) |
| ~7,047-token prompt | 341.4 tok/s (331-356) | 375.2 tok/s (359-404) |
| decode | 27-34 tok/s | 28-34 tok/s (unchanged) |

Rig: GTX 1080 Ti 11 GB on NUMA node 1, 2x Xeon E5-2696 v4 (no AVX-512), driver 580.178.04, CUDA 12.8 toolkit, IQ3_S with MTP `--spec 4`, 262,144-token context, prompt chunk auto = **4,608** (11 GB card; the P40 report ran 8,192). Not tested: quality/answer identity (parity is the argument), prompts above 7K, other toolkits on this card, Windows.

本站相关内容

相关页面的快捷入口。