Issues / #1975

#1975 [Regression] v0.1.43 adaptive overlap reduces IQ3_S decode ~26% on RTX 5090

open · @oal · 1 コメント · GitHub で見る

Setup & installNVIDIA / CUDAModels & quants

本文

# [Regression] v0.1.43 adaptive overlap reduces IQ3_S decode ~26% on RTX 5090

## Hardware / setup
- RTX 5090, 32 GB VRAM; NVIDIA driver **610.57.04**
- Ryzen 9 9950X, 16 cores; **123 GiB usable RAM**
- Fedora host, Ubuntu 26.04 NVIDIA distrobox
- Qwen3.8-Flash-Next GSQ-RCO **IQ3_S**, CPU vision
- 409,600 max context, int8 KV, 32,768 resident KV cells
- 2,048 MiB VRAM reserve
- Existing calibration: `--adapt-every 1 --adapt-swaps 160 --adapt-decay 0.97`

## Results
After upgrading from v0.1.42, typical decode fell from roughly 160–200 TPS to 100–130 TPS.

Controlled tests **within v0.1.43**, changing only `STRATA_ADAPT_OVERLAP`:

| Setting | Mean decode |
|---|---:|
| `STRATA_ADAPT_OVERLAP=1` | 126 TPS |
| `STRATA_ADAPT_OVERLAP=0` | 170 TPS |

Protocol: four engine starts in **on/off/off/on** order; calibration’s three prompts prefixed with synthetic documents at 4K and 100K tokens; temperature 0; 512-token output cap; one warm-up round followed by two measured rounds. No competing agent workload.

Output-identical 512-token samples also showed **128 TPS enabled vs 189–195 TPS disabled**, so differing answers do not explain the entire gap.

Disabling overlap restored live API code tests to **215–222 TPS at 100K**, without recalibration or downgrade.

Could the new adaptive-overlap synchronization path regress on this hardware?

関連リンク

インストール・モデル・リリースへの站内リンク。