Pull requests / #2868

#2868 bench: add H20 Q8_0 256K capacity and serving report

open · @LeslieWylie · 0 comments · View on GitHub

BenchmarksModels & quants

Description

Adds a results-only community report for one NVIDIA H20 running Unsloth Q8_0 on the frozen Strata 0.1.44.1 build (`296b59d2`).

- Completes a 261,992-token uncached middle-phrase recall request at a configured 262,144-token window; retains the separate context-budget rejection and the cached-prefix continuous-output case.
- Records the three-run single-slot baseline and temporary four-slot/batch-MTP serving matrix, with every request and warm-up preserved. Eight clients share four slots; native eight-slot throughput was not tested.
- Includes a separately qualified four-H20 SGLang FP8 deployment reference, with different quantization, engine and stream chunking explicitly disclosed. No quality-equivalence or engine-speedup claim.
- Documents the observed cuBLAS 12.1 SIGFPE and successful task-private 12.8 runtime, without changing engine code, defaults or system libraries.

Includes hardware/configuration metadata, 47 reconstructable synthetic prompts, raw JSON/CSV, selected existing logs, and a portable extraction of the original HTTP timing function. Host/user paths and unrelated operational details are omitted.

Validation: reconstructed all 47 fixtures against their recorded SHA256/byte counts, checked that raw measurements are unchanged and the HTTP timing function matches the original, and checked report links. These were offline checks; no new GPU benchmark, current-main rerun, or backend build was performed for this documentation submission.

Related on this site

Quick links to related pages.