#6487 [Engine] 0.1.62: --vision cpu fails deterministically at startup with "kv_stream: reset: an illegal memory access" (sm_86, Swift IQ3_S) — --vision gpu on the same box is fine
open · @ccmspringming · 0 comentários · No GitHub
BenchmarksServer & APIAMD / HIPNVIDIA / CUDAModels & quantsLinux
Descrição
### Environment - GPU: RTX 3090 24 GB, sm_86, driver 615.78.08, CUDA 13 - CPU: i7-10700K, RAM 61.7 GiB, Linux (Unraid, Docker) - Model: Swift 1.5 fine-tune of Qwen3.8-Flash-Next, GSQ-RCO IQ3_S native pack (a community abliterated variant; the pack was built by the shipped `iq_pack` of the matching version line), 83,549,259,040 bytes - Image: the published `ghcr.io/niko1221/strata:0.2.16-cuda13`, engine 0.1.62 - Working baseline on this box: the same pack with `--vision gpu`, `--max-context 196608 --kv int8 --batch 1 --kv-resident 32768 --expert-cache 6800 --pcie-frac 0.25` — starts in ~90 s, serves at 89-91 tok/s decode with a 92.7% expert-cache hit rate. ### What happens With `--vision cpu` the engine dies during startup, every time, at ~90 s, exit code 6: ``` strata: The engine stopped while it was loading the model (exit code 1): kv_stream: reset: an illegal memory access was encountered. ``` The last engine-log lines before the fault — every verify window captures fine, then the fault: ``` strata verify: capturing the 1-token window (...) strata verify: capturing the 2-token window (...) strata verify: capturing the 3-token window (...) strata verify: capturing the 4-token window (...) strata verify: capturing the 5-token window (...) strata verify: capturing the 6-token window (604 MiB of VRAM free) kv_stream: reset: an illegal memory access was encountered ``` The picture reader never prints a line of its own. No image was ever sent: the fault is in the load/warmup path, not in image processing. ### Matrix (five attempts, one signature) | # | settings | result | |---|---|---| | 1 | vision cpu, kv-resident 32768, expert-cache 6800 | `kv_stream: reset`, exit 6 at ~90 s | | 2 | same, + `--batch 1`, + `--pcie-frac 0.25` | same | | 3 | vision cpu, **no** kv-resident | same | | 4 | vision cpu, expert-cache 7600 | same | | 5 | vision cpu, with STRATA_ADAPT_FETCH=2 / SELECT_TWO_PASS / PREFILL_ROUTED_STREAM | same | ### Notes on the error site `kv_stream: reset` is where the engine noticed the poisoned context, not necessarily the faulting kernel — the class of misattribution that #5636 (fixed in 0.2.17) describes. The fault lands right after the last verify-window capture, i.e. in the startup warmup, before the first request is ever served. The same *shape* of defect is documented in #4708 (Qwen3.6: warmup writes past the KV cache; the surfaced error was an unrelated `cudaMemcpyAsync`; compute-sanitizer traced the first invalid write to `k_attn_prep`). That engine's workaround `--set no-warmup=on` does not exist in this (Swift/Qwen) engine — passing it makes the engine print its `--help` and exit, so the warmup hypothesis could not be tested from a released image. ### It is a regression `--vision cpu` worked on this box on 0.1.52 (the v0.2.7.1 era): it started and served (I still have the slot and hit-rate logs). On 0.1.62 it never starts. ### Why upstream may not have seen it On a 24 GB card the GPU reader is the default, and the release-test matrix (RTX PRO 5000, RTX 5070, DGX Spark…) does not exercise `--vision cpu`. The CPU reader is mostly used below 24 GB, mostly with ISTA-DASLab packs. This box is 24 GB + a Swift-family pack + an explicit `--vision cpu` — a corner of the matrix. ### Asks 1. Can you reproduce `--vision cpu` on a 24 GB card with a Swift-family pack? 2. If a build or a flag can disable only the startup warmup in this engine (as #4708's `no-warmup` does for Qwen3.6), I will run it and report whether the fault disappears — that would confirm or deny the warmup cause. 3. Happy to run any instrumented build or a sanitizer session on this machine; it reproduces deterministically, 5/5. Full engine logs for all five attempts are available on request.
No site
Links install, modelos, releases.