Issues / #2536

#2536 the engine's logits became non-finite before the first token (#879); it dropped its cached state: sending the request again once

open · @Arvintian · 2 comentarios · En GitHub

Server & APIMulti-GPUNVIDIA / CUDAModels & quants

Descripción

v0.1.44.1
dual 5060ti
iq3_s or iq3_xxs
NVIDIA-SMI 595.71.05              Driver Version: 596.49         CUDA Version: 13.2
Empty return, retry request, unable to recover unless Strata is restarted.

```
root           1  0.7  0.4 766640 220544 ?       Ssl  09:06   0:01 /opt/strata/.venv/bin/python /opt/strata/serve/server.py --engine strata --config /opt/strata/strata-iq3_xxs.json --port 8080 --open
root          47  0.6  1.3 140532444 680232 ?    Sl   09:07   0:01 /opt/strata/engine/strata-vision --mmproj /data/models/mmproj-Qwen3.8-Flash-Next-BF16.gguf --model /data/models/IQ3_XXS/Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS-00001-of-00002.gguf --gpu --max-tokens 1024
root          56 92.5 67.9 184304432 34920684 ?  SLl  09:07   4:03 /opt/strata/engine/strata --serve --pack /data/packs/iq3_xxs --native /data/models/IQ3_XXS/Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS-00001-of-00002.gguf --ple-gguf /data/models/IQ3_XXS/Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS-00002-of-00002.gguf --expert-profile /opt/strata/data/expert-profile.bin --expert-cache auto --prefill auto --spec 4 --spec-min-p 0.5 --mtp /data/mtp/rt --max-context 160000 --kv int8 --resident-experts --vision --vram-reserve-mib 200 --conversation-cache-mib 8192 --layer-split auto
```

En el sitio

Enlaces a install, modelos, releases.