Issues / #1975
#1975 [Regression] v0.1.43 adaptive overlap reduces IQ3_S decode ~26% on RTX 5090
open · @oal · 1 Kommentare · Auf GitHub
Setup & installNVIDIA / CUDAModels & quants
Beschreibung
# [Regression] v0.1.43 adaptive overlap reduces IQ3_S decode ~26% on RTX 5090 ## Hardware / setup - RTX 5090, 32 GB VRAM; NVIDIA driver **610.57.04** - Ryzen 9 9950X, 16 cores; **123 GiB usable RAM** - Fedora host, Ubuntu 26.04 NVIDIA distrobox - Qwen3.8-Flash-Next GSQ-RCO **IQ3_S**, CPU vision - 409,600 max context, int8 KV, 32,768 resident KV cells - 2,048 MiB VRAM reserve - Existing calibration: `--adapt-every 1 --adapt-swaps 160 --adapt-decay 0.97` ## Results After upgrading from v0.1.42, typical decode fell from roughly 160–200 TPS to 100–130 TPS. Controlled tests **within v0.1.43**, changing only `STRATA_ADAPT_OVERLAP`: | Setting | Mean decode | |---|---:| | `STRATA_ADAPT_OVERLAP=1` | 126 TPS | | `STRATA_ADAPT_OVERLAP=0` | 170 TPS | Protocol: four engine starts in **on/off/off/on** order; calibration’s three prompts prefixed with synthetic documents at 4K and 100K tokens; temperature 0; 512-token output cap; one warm-up round followed by two measured rounds. No competing agent workload. Output-identical 512-token samples also showed **128 TPS enabled vs 189–195 TPS disabled**, so differing answers do not explain the entire gap. Disabling overlap restored live API code tests to **215–222 TPS at 100K**, without recalibration or downgrade. Could the new adaptive-overlap synchronization path regress on this hardware?
Mehr auf der Site
Links zu Install, Modellen, Releases.