#6397 [Engine] Engine cannot use more than 8 videocards
open · @register83we-tech · 2 Kommentare · Auf GitHub
Setup & installServer & APIMulti-GPUNVIDIA / CUDAModels & quants
Beschreibung
Hi. 9 RTX 3060 12G, engine crashed with report ggml no cuda device. If I turn of one card - it works.
Traceback (most recent call last):
File "/root/.local/share/strata/terminal/0.1.62/serve/server.py", line 8720, in <module>
sys.exit(main())
^^^^^^
File "/root/.local/share/strata/terminal/0.1.62/serve/server.py", line 8073, in main
return _main(argv, lifecycle_event=lifecycle_event, interrupt_exit_code=interrupt_exit_code)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.local/share/strata/terminal/0.1.62/serve/server.py", line 8411, in _main
engine = start_engine_or_fallback(cfg, exe, effort_end, env, lazy, tok, offline=a.offline, batch_telemetry=a.batch_telemetry)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.local/share/strata/terminal/0.1.62/serve/server.py", line 7857, in start_engine_or_fallback
return StrataEngine(exe, engine_args(cfg, exe) + (effort_end or []), cwd=cfg.get("cwd"), log=cfg.get("log"), env=env, lazy=lazy,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.local/share/strata/terminal/0.1.62/serve/server.py", line 909, in __init__
raise EngineDied(why + (f" (see {log})" if log else "") + note +
serve.server_parts.engines.EngineDied: the engine exited before it was ready (see /root/.local/share/strata/install/strata-swift-iq3_s.log)
the engine process ended: exit code 2
strata generate: --layer-split K[,K2..]|auto needs --serve, rising K from 2, and one distinct GPU per K in --split-device (1..8; or 0 with one K: the same GPU). K is the FIRST LAYER of each later GPU, not a count of layers per card: "24,36,42" for 4 GPUs, not "24,12,6,6" (got "auto")Mehr auf der Site
Links zu Install, Modellen, Releases.