Pull requests / #2868
#2868 bench: add H20 Q8_0 256K capacity and serving report
open · @LeslieWylie · 0 commentaires · Sur GitHub
Description
Adds a results-only community report for one NVIDIA H20 running Unsloth Q8_0 on the frozen Strata 0.1.44.1 build (`296b59d2`). - Completes a 261,992-token uncached middle-phrase recall request at a configured 262,144-token window; retains the separate context-budget rejection and the cached-prefix continuous-output case. - Records the three-run single-slot baseline and temporary four-slot/batch-MTP serving matrix, with every request and warm-up preserved. Eight clients share four slots; native eight-slot throughput was not tested. - Includes a separately qualified four-H20 SGLang FP8 deployment reference, with different quantization, engine and stream chunking explicitly disclosed. No quality-equivalence or engine-speedup claim. - Documents the observed cuBLAS 12.1 SIGFPE and successful task-private 12.8 runtime, without changing engine code, defaults or system libraries. Includes hardware/configuration metadata, 47 reconstructable synthetic prompts, raw JSON/CSV, selected existing logs, and a portable extraction of the original HTTP timing function. Host/user paths and unrelated operational details are omitted. Validation: reconstructed all 47 fixtures against their recorded SHA256/byte counts, checked that raw measurements are unchanged and the HTTP timing function matches the original, and checked report links. These were offline checks; no new GPU benchmark, current-main rerun, or backend build was performed for this documentation submission.
Sur le site
Liens install, modèles, releases.