#6646 [Engine] strata-inkling: --bench with --prompt-len >= 32768 without --max-context triggers CUDA error 700/715 and NVRM Xid 13 on RTX 3090 (sm_86)
open · @danbunnell · 0 comentários · No GitHub
BenchmarksNVIDIA / CUDAModels & quantsLinux
Descrição
### Area [Engine] (speed, kernels, models, the server and API) ### Desktop app, Terminal or source? Other ### What happened Running `strata-inkling --bench` on `Inkling-2-Mini-SQ-15.strata` with `--prompt-len 32768`, `131072`, or `245760` without passing `--max-context` fails with unguarded GPU faults rather than rejecting the prompt length or auto-sizing the KV cache: - At `--prompt-len 32768` (both `--draft off` and `--draft dspark`): exits with `CUDA error 700 (an illegal memory access was encountered)` at `ink_cuda_backend.cu:277`. - At `--prompt-len 131072` and `245760`: exits with `CUDA error 715 (an illegal instruction was encountered)` at `ink_cuda_backend.cu:401` and logs a kernel **NVRM Xid 13 (`Graphics SM Warp Exception: Illegal Instruction Encoding`)** in `dmesg`. When `--max-context 65536 --kv fp8` is passed explicitly, `--prompt-len 32768` runs cleanly with 0 Xids (`1,430.1` ppts prefill, `44.3` tps plain / `56.7` tps DSpark), which suggests `--bench` does not check or raise its default context allocation to cover `--prompt-len + --gen`. Expected: `--bench` should either auto-size context to at least `prompt_len + gen` (subject to VRAM checks) or reject `--prompt-len` exceeding `--max-context` cleanly before launching kernels. ### Smallest steps or command that reproduce it 1. Download `strata-inkling-0.1.65-linux-x86_64-cuda-cuda13.tar.gz` from `v0.2.19` and `Inkling-2-Mini-SQ-15.strata`. 2. Run without `--max-context`: ```sh LD_LIBRARY_PATH=./bin ./bin/strata-inkling --bench --native ./Inkling-2-Mini-SQ-15.strata --draft off --prompt-len 32768 --gen 64 --parallel 1 ``` 3. Run with `--prompt-len 131072` to observe `CUDA error 715` + `NVRM Xid 13`. ### Reproduction status Reproduced locally ### Strata version Strata 0.2.19 · strata-inkling 0.1.65 (commit c6f6427d9de9, linux-x86_64-cuda-cuda13) ### Graphics card, memory and system RTX 3090 24 GB (sm_86) · 32 GB RAM · Linux x86_64 ### Model, size and settings Inkling-2-Mini SQ-15 (14.35 GB) · default KV (fp16) · --prompt-len 32768 / 131072 / 245760 · --parallel 1 ### Logs At `--prompt-len 32768`: ```text [InkCUDA] Loaded Inkling-2-Mini-SQ-15.strata (SQ-15): trunk=2.07 GB (0.49s), experts=3072 VRAM / 0 host (2.35s, hyb=0, promo=0, pf_buf=0MB), sm120_tc=0, screen=0 strata-inkling CUDA error 700 (an illegal memory access was encountered) at /data/split-pkg/rc219-r2/src/engine/src/inkling2/cuda/ink_cuda_backend.cu:277 ``` At `--prompt-len 131072` (same at `245760`): ```text [InkCUDA] Loaded Inkling-2-Mini-SQ-15.strata (SQ-15): trunk=2.07 GB (0.60s), experts=3072 VRAM / 0 host (2.39s, hyb=0, promo=0, pf_buf=0MB), sm120_tc=0, screen=0 strata-inkling CUDA error 715 (an illegal instruction was encountered) at /data/split-pkg/rc219-r2/src/engine/src/inkling2/cuda/ink_cuda_backend.cu:401 ``` Kernel `dmesg`: ```text NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 3, SM 0): Illegal Instruction Encoding NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 3, SM 0): Multiple Warp Errors ```
No site
Links install, modelos, releases.