问题 / #2856
#2856 Significant regression after 0.1.45 on 32 GB RAM + 6 GB VRAM system (previously working config doesn't work anymore)
open · @Dampfinchen · 0 评论 · 去 GitHub 看
BenchmarksServer & APINVIDIA / CUDAModels & quantsWindows
说明
I have previously been using this configuration:
`{
"exe": "D:\\KI\\Strata\\engine\\strata.exe",
"args": [
"--pack",
"D:\\KI\\Strata-data\\packs\\iq3_xxs",
"--native",
"D:\\KI\\Strata-data\\models\\IQ3_XXS\\Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS-00001-of-00002.gguf",
"--ple-gguf",
"D:\\KI\\Strata-data\\models\\IQ3_XXS\\Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS-00002-of-00002.gguf",
"--expert-profile",
"D:\\KI\\Strata\\data\\expert-profile.bin",
"--expert-cache",
"auto",
"--prefill",
"auto",
"--spec",
"4",
"--max-context",
"81920",
"--mmap-experts",
"--kv",
"k8v4",
"--vram-reserve-mib",
"150",
"--spec-min-p",
"0.5",
"--adapt-every",
"1",
"--adapt-swaps",
"80",
"--adapt-decay",
"0.97"
],
"cwd": "D:\\KI\\Strata",
"tokenizer": "D:\\KI\\Strata-data\\packs\\iq3_xxs\\tokenizer",
"model_name": "qwen3.8-flash-next-iq3_xxs",
"log": "D:\\KI\\Strata\\strata-iq3_xxs.log",
"lib_dirs": [
"D:\\KI\\Strata\\.venv\\Lib\\site-packages\\nvidia\\cu13\\bin\\x86_64"
],
"port": 8080,
"gpu": 0,
"gpus_asked": true,
"draft_vocab": "en"
}`
With an earlier version this worked while having 41 experts on GPU, around 13 token/s decode, so quite usable if it not were for the horrendously slow prompt processing which was around 50 token/s. But it worked. Now it only caches 14 experts and I'm getting this error as soon as I send the prompt:
`[strata] the engine stopped unexpectedly (exit code 3221226505). The engine exited (code 3221226505). Its last log line: strata serve: WARNING: only 0 MiB of VRAM free at the start of this request (below 1024 MiB): when the card is this full the driver can page GPU memory to system RAM, decode falls to a few tok/s and a window may never finish (#2065). Add --vram-reserve-mib 1216 to the config's args, or lower --max-context / --batch - if that does not explain it, please report it at github.com/Niko1221/Strata/issues with the log. The next request starts the engine again. Its log: D:\KI\Strata\strata-iq3_xxs.log`
Reducing --vram-reserve-mib to 120 it doesn't even boot anymore (it did previously)
` strata serve: Windows commit: 23.1 GiB left of a 37.1 GiB limit (RAM + page file)
strata: the CPU prefill share is not armed by default: the experts it would read are 39.9 GiB of file pages and only 23.2 GiB of RAM is free to keep them. STRATA_PREFILL_CPU_SHARE=auto turns it on anyway
strata serve: the prompt path allocates its own buffers (too few cache slots to borrow)
strata serve: prefill: device buffers for a chunk of 1024 tokens do not fit (0 of 6143 MiB free at the failure; 470 MiB taken, the failing buffer wanted 10 MiB): trying a 512-token chunk
strata hc: CUDA0: the hyper-connection read runs as staged (the norm per token and stream, the down projection's activations staged ahead by cp.async); checked bit for bit against the plain read on this card (STRATA_HC_SPLIT=1 or 0 for the earlier ones)
strata serve: verify: the device arena (69 MiB) does not fit`
Setting the env variable STRATA_PREFILL_CPU_SHARE=auto doesn't help and only gives me this error:
` strata serve: Windows commit: 23.0 GiB left of a 37.1 GiB limit (RAM + page file)
strata: WARNING STRATA_PREFILL_CPU_SHARE=auto is set and kept, but the experts it would read are 39.9 GiB of file pages and only 23.1 GiB of RAM is free to keep them: the share can slow prompts or fail on a machine this short of memory (STRATA_PREFILL_CPU_SHARE=0 turns it off)
strata serve: the prompt path allocates its own buffers (too few cache slots to borrow)
strata serve: prefill: device buffers for a chunk of 1024 tokens do not fit (0 of 6143 MiB free at the failure; 470 MiB taken, the failing buffer wanted 10 MiB): trying a 512-token chunk
strata hc: CUDA0: the hyper-connection read runs as staged (the norm per token and stream, the down projection's activations staged ahead by cp.async); checked bit for bit against the plain read on this card (STRATA_HC_SPLIT=1 or 0 for the earlier ones)
strata serve: verify: the device arena (69 MiB) does not fit`
Setting STRATA_PREFILL_CPU_SHARE=0 doesn't help either.
[strata-iq3_xxs.log](https://github.com/user-attachments/files/33591076/strata-iq3_xxs.log)
本站相关内容
相关页面的快捷入口。