#5822 [Engine] Flash-Next 0.1.58: native QSA indexer (batch) rejects arguments and exits on single-GPU CUDA
open · @Gpanazio · 1 Kommentare · Auf GitHub
BenchmarksServer & APIMulti-GPUAMD / HIPNVIDIA / CUDAModels & quantsLinux
Beschreibung
## Problem The released CUDA engine **0.1.58** exited three times today during normal agent use with: ```text strata serve: prefill indexer: native QSA indexer (batch): bad geometry, positions, parameters or scaling ``` The server reported zero generated tokens for each failed request. This is an observed failure in local logs, not just a source-code finding. There is **no deterministic reproducer yet**; the private conversations cannot be attached. ## Environment - Pop!_OS 24.04, Linux x86_64; 64 GB RAM. - One RTX 5070 Ti, 16 GB VRAM, compute capability 12.0; NVIDIA driver 615.71.09. - Strata **0.2.12**, tag commit `146df3c468739147f520ac3816de806f71232571`. - Released engine **0.1.58**, `BUILD.json` source `f-0212-rc 8a42a9457db6`, CUDA 13.0. - Engine SHA256: `a6ca30de01cf938764820a3fabe0725cb734e59933863a13f86c82c768be5692`. - Qwen3.8-Flash-Next, takanori-ishikawa Abliterated **UD-IQ3_XXS**, native experts pack and native MTP; vision enabled with the encoder on CPU. - The Python serving layer has a local reasoning-budget/wrap-up patch. The failing engine binary is the release binary, not a locally rebuilt engine. Requests normally pass through an OpenAI-compatible local proxy. Relevant engine arguments, with private paths replaced: ```text --serve --pack <PACK> --native <FIRST_GGUF> --ple-gguf <PLE_GGUF> --expert-profile <PROFILE> --expert-cache auto --prefill auto:16384 --spec 4 --spec-min-p 0.5 --mtp <MTP_DIR> --max-context 196608 --kv int8 --kv-resident 16384 --vram-reserve-mib 600 --vision ``` Environment: `STRATA_HC_Q8=1`. Startup sampling defaults: temperature 1.0, top_p 0.95, top_k 64, min_p 0.05, presence_penalty 0, repetition_penalty 1.0. Individual client requests may override these. No layer split or `--batch` argument is configured. ## Observed sequence All times below are **2026-11-11, UTC-03**. The engine's error lines are not timestamped; times are correlated with the serving/proxy logs. | Approximate failure time | Immediately preceding **successful** request | Failed request | | --- | --- | --- | | 18:01:14 | 19,503 prompt tokens = 18,875 reused + 628 read; 1,102 generated | 0 generated; engine exited with code 1; next request triggered a reload | | 19:05:10 | 101,219 prompt tokens = 100,817 reused + 402 read; 414 generated | 0 generated; engine exited | | 19:12:17 | 26,172 prompt tokens = 26,140 reused + 32 read; 75 generated | 0 generated; same exact error after a fresh server/model load around 19:09 | **These token counts describe the preceding successes, not the requests that failed.** The failing requests do not have completed prompt summaries, so their exact `p0`, chunk length and prompt size are unknown. Representative adjacent log lines for the second occurrence: ```text strata serve: prompt 101219 tokens = 100817 reused + 402 read in 637 ms (631.3 tok/s), 414 generated in 3592 ms (115.3 tok/s), drafts accepted 240 of 374, 6 checkpoints strata serve: prefill indexer: native QSA indexer (batch): bad geometry, positions, parameters or scaling [strata] the engine reported an error: prefill indexer: native QSA indexer (batch): bad geometry, positions, parameters or scaling [strata] done: 0 tokens in 1 s (no speed) (error, cancel=False) ``` Other engine statistics occur between the prompt-summary and error lines; this excerpt omits them. The logs show KV streaming with 16,384 of 196,608 logical cells resident in VRAM. There are low-free-VRAM warnings (about 403 MiB in one request), but no CUDA OOM adjacent to these failures; the kernel log checked around the first two failures had no corresponding OOM or NVIDIA Xid. ## Reproduction status / expected behavior 1. Serve the above configuration. 2. Use normal multi-turn agent/tool conversations, repeatedly reusing a large prefix and reading a new tail. 3. Intermittently the next prefill fails with the exact message and the engine exits. I cannot yet identify the minimum prompt, request sequence or flag that triggers it. Expected: a request admitted within the configured context should complete, or be rejected with a specific request-validation error rather than an opaque indexer error followed by engine exit. The Python server can remain alive with `/health` returning `loaded:false`. Its reload is demand-driven. A separate integration consequence is that a proxy which removes backends when `/v1/models` becomes empty can prevent the next request from reaching that recovery path; this is not evidence for the cause of the QSA error. ## Investigation and limits **Unconfirmed static analysis:** `native_qsa_indexer_append_batch()` in `src/kernels/cuda/native_qsa_indexer.cu` uses this same message for several different invalid arguments: stream, geometry, position/range/capacity, epsilon or RoPE scaling. The check happens before the indexer's kernel launches. The current message does not identify the failed predicate or its actual values. I cannot establish a cache-position bug or attribute it to VRAM pressure from these logs alone. The available 0.2.11/engine 0.1.57 historical log has roughly 1,493 request summaries and no matching QSA error, versus three matches in the available 0.2.12 log. This is not a controlled comparison. I rolled back to 0.2.11/0.1.57 and verified a short reply through the proxy, **not** that rollback fixes the intermittent failure or preserves quality/performance under the original workload. Searched open and closed issues and comments for the exact message. Related but not equivalent: #5564 (0.1.58 prefill hang), #5414 (batched-prefill watchdog stalls), and #5662 (proposed fixed-slot ownership). None establishes this particular invalid-argument failure or a fix for it. I have not reproduced this on 0.1.59/main. Would logging the failed predicate plus `p0`, `n`, `max_cells`, geometry, stream presence, scaling validity and the prefill layer/session be the right next diagnostic? Is there a known path that can pass an inconsistent range/session to this append after prefix reuse? AI-assisted investigation: local logs and running processes were inspected, source paths were checked, and the claims above were verified against actual host output. Private paths, credentials and conversation content have been omitted.
Mehr auf der Site
Links zu Install, Modellen, Releases.