Issues / #2188

#2188 0.1.43.1, Windows: STRATA_ADAPT_OVERLAP=1 stops the first request with "the adaptive tier's copies for layer 45 never landed" (RTX 4070 SUPER); STRATA_ADAPT_SIGNAL=host works

open · @LeGeRyChEeSe · 0 comentarios · En GitHub

Setup & installServer & APIAMD / HIPNVIDIA / CUDAModels & quantsDocumentationWindows

Descripción

**Summary**

On Windows, `STRATA_ADAPT_OVERLAP=1` (opt-in there since 0.1.43) fails on the first request of every run with the
default signal (stream-ordered copies):

```
ValueError: verify: the adaptive tier's copies for layer 45 never landed; its GPU waits were released and the GPU finished (#267)
```

With `STRATA_ADAPT_SIGNAL=host` added, the same runs finish and give the same speed as without the overlap.

| setup | runs | result |
|---|---|---|
| `STRATA_ADAPT_OVERLAP=1`, `--vision`, `--vram-reserve-mib 450` | 2 | stopped at the first request |
| `STRATA_ADAPT_OVERLAP=1`, no vision, `--vram-reserve-mib 1000` | 1 | stopped at the first request |
| `STRATA_ADAPT_OVERLAP=1 STRATA_ADAPT_SIGNAL=host`, `--vision`, 450 | 3 | fine; x1.005 decode against no overlap (median of 23 turns, 12 of 23 faster) |

**Setup**
- RTX 4070 SUPER 12 GB (PCIe 4.0 x16, driver 617.42), Ryzen 5 5600X, 64 GB DDR4-3600, Windows 11 Pro build 26300.
- Engine 0.1.43.1 built from the tag (CUDA 13.1, archs 86;89). Flash-Next IQ2_XS (GSQ-RCO), the shipped expert profile.
- Args: `--expert-cache auto --prefill auto --spec 4 --spec-min-p 0.7 --mtp <rt> --max-context 131072 --kv int8
  --kv-resident 32768 --pcie-frac 0.55 --ple-io direct`, plus the reserve and `--vision` above.
- Load: the coding-agent replay from #1971 (public corpus), 256 tokens at T = 1.

**Log before the stop** (no vision run):
```
strata serve: adaptive overlap signal: stream-ordered copies (STRATA_ADAPT_SIGNAL=host: a host function)
strata serve: adaptive overlap (STRATA_ADAPT_OVERLAP=0 turns it off): the adaptive tier's copies land while the next window runs (same tokens, a layer waits for its own copies)
strata serve: 753 MiB of VRAM free with everything loaded
strata verify: captured the 4-token window (upload no error, sync no error)
strata verify: captured the 1-token window (upload no error, sync no error)
strata verify: captured the 2-token window (upload no error, sync no error)
```

Since it is opt-in on Windows this does not hit default users; maybe the Windows opt-in should pick the host signal,
or the docs could mention it. The full engine logs are available.

En el sitio

Enlaces a install, modelos, releases.