Issues / #951

#951 crash after first inputs

open · @Maelstrom2014 · 2 comments · View on GitHub

BenchmarksServer & APIModels & quants

Description


stepan@BOMJ:~/Strata$ ./run-swift-iq3_xxs.sh
loading the vision encoder ...
loading the model (the first start takes a minute or two) ...
[strata] starting the engine: reading the model's weights ...
[strata] still starting (20 s) - please wait ...
[strata] still starting (40 s) - please wait ...
[strata] loading the experts into RAM (about 40 GB) and locking part of them for the GPU.
         YOUR PC CAN BE SLOW OR STOP RESPONDING FOR 1-3 MINUTES NOW - this is normal.
         Please wait and don't close this window; the browser opens when it is ready.
[strata] still starting (66 s) - please wait ...
[strata] still starting (86 s) - please wait ...
[strata] still starting (106 s) - please wait ...
[strata] still starting (126 s) - please wait ...
[strata] still starting (146 s) - please wait ...
[strata] still starting (166 s) - please wait ...
[strata] still starting (186 s) - please wait ...
[strata] still starting (206 s) - please wait ...
[strata] still starting (226 s) - please wait ...
[strata] still starting (246 s) - please wait ...
[strata] still starting (266 s) - please wait ...
[strata] still starting (286 s) - please wait ...
[strata] still starting (306 s) - please wait ...
[strata] still starting (326 s) - please wait ...
[strata] still starting (346 s) - please wait ...
[strata] still starting (366 s) - please wait ...
[strata] still starting (386 s) - please wait ...
[strata] still starting (406 s) - please wait ...
[strata] still starting (426 s) - please wait ...
[strata] still starting (446 s) - please wait ...
[strata] still starting (466 s) - please wait ...
[strata] still starting (486 s) - please wait ...
[strata] still starting (506 s) - please wait ...
[strata] still starting (526 s) - please wait ...
[strata] still starting (546 s) - please wait ...
[strata] still starting (566 s) - please wait ...
[strata] still starting (586 s) - please wait ...
[strata] still starting (606 s) - please wait ...
[strata] still starting (626 s) - please wait ...
[strata] still starting (646 s) - please wait ...
[strata] still starting (666 s) - please wait ...
[strata] still starting (686 s) - please wait ...
[strata] still starting (706 s) - please wait ...
[strata] still starting (726 s) - please wait ...
[strata] still starting (746 s) - please wait ...
[strata] experts loaded: 39.97 GiB at 0.06 GiB/s (761 s so far)
[strata] filling the GPU's expert cache (13204 experts, 21.47 GiB of VRAM) ...
[strata] almost ready ...
ready: http://127.0.0.1:8080/v1  (OpenAI: /v1/chat/completions, Anthropic: /v1/messages, context 131072 tokens, images on, API key required)
       open http://127.0.0.1:8080/ in a browser to chat; close this window to stop the model
       from other devices: http://192.168.0.25:8080/   (API: http://192.168.0.25:8080/v1)
[strata] reading the prompt: 27,841 tokens, 10 s so far
[strata] reading the prompt: 27,841 tokens, 20 s so far
[strata] reading the prompt: 27,841 tokens, 30 s so far
[strata] reading the prompt: 27,841 tokens, 40 s so far
[strata] reading the prompt: 27,841 tokens, 50 s so far
[strata] reading the prompt: 27,841 tokens, 60 s so far
[strata] the engine stopped unexpectedly (exit code -6). The engine stopped itself because it had stopped making progress - a hang it caught. Its log line: strata serve: no progress for 60 s during a request (reading the prompt (batched): layer 1 of the prompt chunk from token 0) - stopping the engine so the server starts it again (issue #29) - please report it at github.com/Niko1221/Strata/issues. The next request starts the engine again. Its log: /home/stepan/Strata/strata-swift-iq3_xxs.log
[strata] done: 0 tokens in 64 s (0.0 tok/s) (error, cancel=False)
[strata] the engine had stopped (exit code -6); starting it again (a minute or two) ...
[strata] starting the engine: reading the model's weights ...
[strata] still starting (20 s) - please wait ...
[strata] still starting (40 s) - please wait ...
[strata] loading the experts into RAM (about 40 GB) and locking part of them for the GPU.
         YOUR PC CAN BE SLOW OR STOP RESPONDING FOR 1-3 MINUTES NOW - this is normal.
         Please wait and don't close this window; the browser opens when it is ready.
[strata] still starting (66 s) - please wait ...
[strata] still starting (86 s) - please wait ...
[strata] still starting (106 s) - please wait ...
[strata] still starting (126 s) - please wait ...
[strata] still starting (146 s) - please wait ...
[strata] still starting (166 s) - please wait ...
[strata] still starting (186 s) - please wait ...
[strata] still starting (206 s) - please wait ...
[strata] still starting (226 s) - please wait ...
[strata] still starting (246 s) - please wait ...
[strata] still starting (266 s) - please wait ...
[strata] still starting (286 s) - please wait ...
[strata] still starting (306 s) - please wait ...
[strata] still starting (326 s) - please wait ...
[strata] still starting (346 s) - please wait ...
[strata] still starting (366 s) - please wait ...
[strata] still starting (386 s) - please wait ...
[strata] still starting (406 s) - please wait ...
[strata] still starting (426 s) - please wait ...
[strata] still starting (446 s) - please wait ...
[strata] still starting (466 s) - please wait ...
[strata] still starting (486 s) - please wait ...
[strata] still starting (506 s) - please wait ...
[strata] still starting (526 s) - please wait ...
[strata] still starting (546 s) - please wait ...
[strata] still starting (566 s) - please wait ...
[strata] still starting (586 s) - please wait ...
[strata] still starting (606 s) - please wait ...
[strata] still starting (626 s) - please wait ...
[strata] still starting (646 s) - please wait ...
[strata] still starting (666 s) - please wait ...
[strata] still starting (686 s) - please wait ...
[strata] still starting (706 s) - please wait ...
[strata] still starting (726 s) - please wait ...
[strata] experts loaded: 39.97 GiB at 0.06 GiB/s (743 s so far)
[strata] filling the GPU's expert cache (13204 experts, 21.47 GiB of VRAM) ...
[strata] almost ready ...
[strata] the engine is running again
[strata] done: 0 tokens in 64 s (0.0 tok/s) (cancel, cancel=True)
[strata] the engine had stopped (exit code -6); starting it again (a minute or two) ...
[strata] starting the engine: reading the model's weights ...
[strata] still starting (20 s) - please wait ...
[strata] loading the experts into RAM (about 40 GB) and locking part of them for the GPU.
         YOUR PC CAN BE SLOW OR STOP RESPONDING FOR 1-3 MINUTES NOW - this is normal.
         Please wait and don't close this window; the browser opens when it is ready.
[strata] still starting (51 s) - please wait ...
[strata] still starting (71 s) - please wait ...
[strata] still starting (91 s) - please wait ...
[strata] still starting (111 s) - please wait ...
[strata] still starting (131 s) - please wait ...
[strata] still starting (151 s) - please wait ...
[strata] still starting (171 s) - please wait ...
[strata] still starting (191 s) - please wait ...
[strata] still starting (211 s) - please wait ...
[strata] still starting (231 s) - please wait ...
[strata] still starting (251 s) - please wait ...
[strata] still starting (271 s) - please wait ...
[strata] still starting (291 s) - please wait ...
[strata] still starting (311 s) - please wait ...
[strata] still starting (331 s) - please wait ...
[strata] still starting (351 s) - please wait ...
[strata] still starting (371 s) - please wait ...
[strata] still starting (391 s) - please wait ...
[strata] still starting (411 s) - please wait ...
[strata] still starting (431 s) - please wait ...
[strata] still starting (451 s) - please wait ...
[strata] still starting (471 s) - please wait ...
[strata] still starting (491 s) - please wait ...
[strata] still starting (511 s) - please wait ...
[strata] still starting (531 s) - please wait ...
[strata] still starting (551 s) - please wait ...
[strata] still starting (571 s) - please wait ...
[strata] still starting (591 s) - please wait ...
[strata] still starting (611 s) - please wait ...
[strata] still starting (631 s) - please wait ...
[strata] still starting (651 s) - please wait ...
[strata] still starting (671 s) - please wait ...
[strata] still starting (691 s) - please wait ...
[strata] experts loaded: 39.97 GiB at 0.06 GiB/s (703 s so far)
[strata] filling the GPU's expert cache (13204 experts, 21.47 GiB of VRAM) ...
[strata] almost ready ...
[strata] the engine is running again
[strata] done: 0 tokens in 63 s (0.0 tok/s) (cancel, cancel=True)
[strata] the engine had stopped (exit code -6); starting it again (a minute or two) ...
[strata] starting the engine: reading the model's weights ...

Related on strata.com

Editorial links to help you install, pick models, or read release notes — not part of the upstream thread.