反馈 / #2188
#2188 0.1.43.1, Windows: STRATA_ADAPT_OVERLAP=1 stops the first request with "the adaptive tier's copies for layer 45 never landed" (RTX 4070 SUPER); STRATA_ADAPT_SIGNAL=host works
open · @LeGeRyChEeSe · 0 评论 · 去 GitHub 看
Setup & installServer & APIAMD / HIPNVIDIA / CUDAModels & quantsDocumentationWindows
说明
**Summary** On Windows, `STRATA_ADAPT_OVERLAP=1` (opt-in there since 0.1.43) fails on the first request of every run with the default signal (stream-ordered copies): ``` ValueError: verify: the adaptive tier's copies for layer 45 never landed; its GPU waits were released and the GPU finished (#267) ``` With `STRATA_ADAPT_SIGNAL=host` added, the same runs finish and give the same speed as without the overlap. | setup | runs | result | |---|---|---| | `STRATA_ADAPT_OVERLAP=1`, `--vision`, `--vram-reserve-mib 450` | 2 | stopped at the first request | | `STRATA_ADAPT_OVERLAP=1`, no vision, `--vram-reserve-mib 1000` | 1 | stopped at the first request | | `STRATA_ADAPT_OVERLAP=1 STRATA_ADAPT_SIGNAL=host`, `--vision`, 450 | 3 | fine; x1.005 decode against no overlap (median of 23 turns, 12 of 23 faster) | **Setup** - RTX 4070 SUPER 12 GB (PCIe 4.0 x16, driver 617.42), Ryzen 5 5600X, 64 GB DDR4-3600, Windows 11 Pro build 26300. - Engine 0.1.43.1 built from the tag (CUDA 13.1, archs 86;89). Flash-Next IQ2_XS (GSQ-RCO), the shipped expert profile. - Args: `--expert-cache auto --prefill auto --spec 4 --spec-min-p 0.7 --mtp <rt> --max-context 131072 --kv int8 --kv-resident 32768 --pcie-frac 0.55 --ple-io direct`, plus the reserve and `--vision` above. - Load: the coding-agent replay from #1971 (public corpus), 256 tokens at T = 1. **Log before the stop** (no vision run): ``` strata serve: adaptive overlap signal: stream-ordered copies (STRATA_ADAPT_SIGNAL=host: a host function) strata serve: adaptive overlap (STRATA_ADAPT_OVERLAP=0 turns it off): the adaptive tier's copies land while the next window runs (same tokens, a layer waits for its own copies) strata serve: 753 MiB of VRAM free with everything loaded strata verify: captured the 4-token window (upload no error, sync no error) strata verify: captured the 1-token window (upload no error, sync no error) strata verify: captured the 2-token window (upload no error, sync no error) ``` Since it is opt-in on Windows this does not hit default users; maybe the Windows opt-in should pick the host signal, or the docs could mention it. The full engine logs are available.
本站相关内容
相关页面的快捷入口。