#2118 the engine stopped unexpectedly (exit code None)

Descripción

Hi

I can't get the unsloth-ud-q4_k_xl to run - ISTA IQ3_S works.
After calibration the benchmark eventually crashes and if i start up the engine without the benchmark, it crashes within 2 seconds of the first prompt without ever outputting a token.

Any idea where to look?

Thanks!

ZFS with 3 mirrored VDEVs (essentially RAID 10 across 6 Enterprise "Read Intensive" SATA SSDs)
128 GB RAM (DDR4-2133)
2x E5-2640v4 (only 8 cores exposed to VM)
4x 5060 Ti 16 GB

Strata - Qwen3.8-Flash-Next on a normal PC (a GPU + system RAM + CPU)

=== Step 1: checking your PC ===
  Your NVIDIA GPUs:
    GPU 0: NVIDIA GeForce RTX 5060 Ti, 16 GB VRAM - can be used
    GPU 1: NVIDIA GeForce RTX 5060 Ti, 16 GB VRAM - can be used
    GPU 2: NVIDIA GeForce RTX 5060 Ti, 16 GB VRAM - can be used
    GPU 3: NVIDIA GeForce RTX 5060 Ti, 16 GB VRAM - can be used

  Strata can run the model on one GPU, or share it across these: then each card holds the
  experts of its own layers, so together they hold about twice as many, and prompts are read about 20%
  faster (details: docs/MULTI_GPU.md). A much slower extra card can also make it slower.
  1) GPU 0 (NVIDIA GeForce RTX 5060 Ti, 16 GB) + GPU 1 (NVIDIA GeForce RTX 5060 Ti, 16 GB) together   (recommended)
  2) GPU 0 (NVIDIA GeForce RTX 5060 Ti, 16 GB) + GPU 1 (NVIDIA GeForce RTX 5060 Ti, 16 GB) + GPU 2 (NVIDIA GeForce RTX 5060 Ti, 16 GB) + GPU 3 (NVIDIA GeForce RTX 5060 Ti, 16 GB) together
  3) GPU 0 (NVIDIA GeForce RTX 5060 Ti, 16 GB) only
  4) GPU 1 (NVIDIA GeForce RTX 5060 Ti, 16 GB) only
  5) GPU 2 (NVIDIA GeForce RTX 5060 Ti, 16 GB) only
  6) GPU 3 (NVIDIA GeForce RTX 5060 Ti, 16 GB) only
Which GPUs? [1]: 2
  [ok] GPUs: GPU 0 (NVIDIA GeForce RTX 5060 Ti, 16 GB) + GPU 1 (NVIDIA GeForce RTX 5060 Ti, 16 GB) + GPU 2 (NVIDIA GeForce RTX 5060 Ti, 16 GB) + GPU 3 (NVIDIA GeForce RTX 5060 Ti, 16 GB) together (the model's layers are split across them)
  [ok] GPU: NVIDIA GeForce RTX 5060 Ti, 15.9 GB VRAM, compute capability 12.0, driver 610.57.04
  [ok] RAM: 121 GB
  [ok] CPU: Intel(R) Xeon(R) CPU E5-2640 v4 @ 2.40GHz (AVX2)

=== Step 2: your choices ===
  1) Qwen3.8-Flash-Next   Qwen; GSQ-RCO quants by ISTA-DASLab - the original model
  2) Swift 1.5            UkisAI's fine-tune of Qwen3.8-Flash-Next - thinks much shorter (-63% thinking tokens, 1.8x sooner answers by its authors' numbers)
  3) Qwen3.8-Flash-Next Coder ISTA-DASLab's coding version - half the experts (code, tools, images kept): needs ~32 GB of RAM, faster; weaker outside coding
  4) Qwen3.8-Flash-Next (Unsloth) Unsloth's ~4-bit quantizations - UD-IQ4_XS: a 94 GB download; with less than ~80 GB of RAM part of its experts are read from the SSD (UD-Q4_K_XL, 111 GB: experimental)
Which model? [1]: 4
  [ok] model: Qwen3.8-Flash-Next (Unsloth)

  1) UD-IQ4_XS ~4-bit i-quant (Unsloth Dynamic), between IQ3_S and UD-Q4_K_XL in quality; on a PC with less than ~80 GB of RAM part of its experts are read from the SSD; download 94 GB, keeps ~55 GB of its 60 GB of experts in RAM
  2) UD-Q4_K_XL 4-bit (Unsloth Dynamic), EXPERIMENTAL: the best quality, but most experts come from the SSD on a 64 GB PC (7-8.5 tokens/s measured); download 111 GB, keeps ~71 GB of its 77 GB of experts in RAM
Which size? [1]: 2
  [!]  UD-Q4_K_XL is EXPERIMENTAL (docs/UNSLOTH_Q4.md): most of its experts are read from the SSD while it answers, so it is several times slower than the 2-3-bit models; quality checked against llama.cpp

  UD-Q4_K_XL runs with a RAM budget of its experts, on one GPU or on GPU 0 (NVIDIA GeForce RTX 5060 Ti, 16 GB) + GPU 1 (NVIDIA GeForce RTX 5060 Ti, 16 GB) + GPU 2 (NVIDIA GeForce RTX 5060 Ti, 16 GB) + GPU 3 (NVIDIA GeForce RTX 5060 Ti, 16 GB) together:
  1) GPU 0 (NVIDIA GeForce RTX 5060 Ti, 16 GB) only   (recommended: the tested setup)
  2) GPU 0 (NVIDIA GeForce RTX 5060 Ti, 16 GB) + GPU 1 (NVIDIA GeForce RTX 5060 Ti, 16 GB) + GPU 2 (NVIDIA GeForce RTX 5060 Ti, 16 GB) + GPU 3 (NVIDIA GeForce RTX 5060 Ti, 16 GB) together: each card caches the experts of its own layers (#642)
UD-Q4_K_XL: which GPUs? [1]: 2
  [ok] UD-Q4_K_XL on GPU 0 (NVIDIA GeForce RTX 5060 Ti, 16 GB) + GPU 1 (NVIDIA GeForce RTX 5060 Ti, 16 GB) + GPU 2 (NVIDIA GeForce RTX 5060 Ti, 16 GB) + GPU 3 (NVIDIA GeForce RTX 5060 Ti, 16 GB), with its RAM budget (each card caches its own layers' experts, the RAM the hottest of the rest; #642)
  [ok] RAM budget: 71 GiB of UD-Q4_K_XL's experts in RAM, the rest read from the model files on the SSD
  [ok] size: UD-Q4_K_XL

  Context length = how much text the model can see at once (your chat, files, tool output).
  Longer needs more VRAM for it, so fewer experts fit on the GPU:
  1) 8K tokens
  2) 32K tokens   (recommended for your GPU)
  3) 64K tokens
  4) 128K tokens
  5) 200K tokens
  6) 256K tokens
  7) 384K tokens   (experimental: setup adds rope scaling)
  8) 512K tokens   (experimental: setup adds rope scaling)
Context? [2]: 2
  [ok] context: 32768 tokens

  KV cache precision (the model's memory of the conversation):
  1) 8-bit   (recommended: what every published number was measured with)
  2) 4-bit   half the memory (about 4% faster at 128K), but measurably less precise on long
             documents; long-context lookups (needle tests) still pass
KV cache? [1]: 1
  [ok] KV cache: 8-bit

  Images: the model can also read pictures (screenshots, photos, scanned pages). This adds a 0.9 GB
  download and keeps ~1.4 GB of VRAM free for the image encoder, so text is a few % slower.
Do you want images? [n]: n
  [ok] images: off

=== Step 3: Python packages ===
  [ok] numpy, jinja2, regex, pyyaml, tqdm, requests, cmake, ninja, pillow, psutil already installed

=== Step 4: the Strata engine ===
  [ok] llama.cpp 3cf0325 (gguf-py, ggml, mtmd)
  [ok] engine already built for this PC
  [ok] engine: /home/neumluc/Strata/engine/strata

=== Step 5: downloading Qwen3.8-Flash-Next (Unsloth) UD-Q4_K_XL ===
  [ok] Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf already downloaded
  [ok] Qwen3.8-Flash-Next-UD-Q4_K_XL-00002-of-00004.gguf already downloaded
  [ok] Qwen3.8-Flash-Next-UD-Q4_K_XL-00003-of-00004.gguf already downloaded
  [ok] Qwen3.8-Flash-Next-UD-Q4_K_XL-00004-of-00004.gguf already downloaded
  [ok] model files present

=== Step 6: preparing the model for Strata ===
  [ok] model prepared: /home/neumluc/Strata-data/packs/unsloth-ud-q4_k_xl
  [ok] MTP draft layer: /home/neumluc/Strata-data/mtp/rt

=== Step 7: writing the start script ===
  [ok] layer split across GPUs [0, 1, 2, 3] (auto)
  tip: with 121 GB of RAM, --prefill auto:32768 in the config's args read prompts 21-35% faster in community benchmarks (#433 #440 #834); not set, nothing changes
  tip: for several agents or clients at once, --conversation-cache-mib 8192 in the config's args keeps each one's conversation; without it they were measured re-reading ~90% of their prompts (#882 #440; docs/DETAILS.md, Multiple conversations)
  the earlier run config is kept as strata-unsloth-ud-q4_k_xl.json.bak; engine options it had that this one has not (setup chooses those): --kv-resident --vision --vram-reserve-mib --pcie-frac --pool-workers --adapt-every --adapt-swaps --adapt-decay
Tune Strata for this PC now? It measures a few engine settings (about 15-30 minutes, longer on a slow card; the PC is busy meanwhile; later: START-HERE --calibrate) [y]: 

  Tuning Strata for this PC: the output speed is measured with a few engine settings (the PCIe share, the
  draft depth, the CPU threads, the expert cache). It takes roughly 15-30 minutes (3 rounds of each value); the PC is busy meanwhile.
  Loading the model for the measurements ...
[strata] starting the engine: reading the model's weights ...
[strata] loading the most-used experts into RAM (up to 71 GiB; the rest are read from the model files as needed) and locking part of them for the GPU.
         YOUR PC CAN BE SLOW OR STOP RESPONDING FOR 1-3 MINUTES NOW - this is normal.
         Please wait and don't close this window; the browser opens when it is ready.
[strata] filling the GPU's expert cache (3386 experts, 10.12 GiB of VRAM) ...
[strata] still starting (38 s) - please wait ...
[strata] still starting (58 s) - please wait ...
[strata] still starting (78 s) - please wait ...
[strata] still starting (98 s) - please wait ...
[strata] still starting (118 s) - please wait ...
[strata] almost ready ...
[strata] still starting (141 s) - please wait ...
[strata] still starting (161 s) - please wait ...
[strata] still starting (181 s) - please wait ...
[strata] still starting (201 s) - please wait ...
    round 1/3 PCIe share: 0 56.0  0.19 60.2  0.2 58.5  0.35 59.1  0.55 49.6  0.75 45.0  0.9 48.8  1 42.7
    round 2/3 PCIe share: 0 56.7  0.19 60.0  0.2 58.8  0.35 55.7  0.55 49.7  0.75 52.9  0.9 47.6  1 42.9
    round 3/3 PCIe share: 0 55.9  0.19 60.8  0.2 57.4  0.35 51.9  0.55 51.6  0.75 48.9  0.9 48.2  1 42.3
    PCIe share, median of 3: 0: 56.0 (55.9-56.7), 0.19: 60.2 (60.0-60.8), 0.2: 58.5 (57.4-58.8), 0.35: 55.7 (51.9-59.1), 0.55: 49.7 (49.6-51.6), 0.75: 48.9 (45.0-52.9), 0.9: 48.2 (47.6-48.8), 1: 42.7 (42.3-42.9) -> best 0.19
    round 1/3 draft floor: 0.3 50.0  0.5 54.1  0.7 63.7
    round 2/3 draft floor: 0.3 49.5  0.5 54.2  0.7 51.3
    round 3/3 draft floor: 0.3 47.9  0.5 56.3  0.7 56.9
    draft floor, median of 3: 0.3: 49.5 (47.9-50.0), 0.5: 54.2 (54.1-56.3), 0.7: 56.9 (51.3-63.7) -> best 0.7
  Measuring with 7 CPU workers (restarts the engine) ...
[strata] starting the engine: reading the model's weights ...
[strata] loading the most-used experts into RAM (up to 71 GiB; the rest are read from the model files as needed) and locking part of them for the GPU.
         YOUR PC CAN BE SLOW OR STOP RESPONDING FOR 1-3 MINUTES NOW - this is normal.
         Please wait and don't close this window; the browser opens when it is ready.
[strata] filling the GPU's expert cache (3386 experts, 10.12 GiB of VRAM) ...
[strata] still starting (34 s) - please wait ...
[strata] still starting (54 s) - please wait ...
[strata] still starting (74 s) - please wait ...
[strata] almost ready ...
[strata] still starting (104 s) - please wait ...
[strata] still starting (124 s) - please wait ...
[strata] still starting (144 s) - please wait ...
[strata] still starting (164 s) - please wait ...
    7 workers: 59.7 tok/s
  Measuring with 5 CPU workers (restarts the engine) ...
[strata] starting the engine: reading the model's weights ...
[strata] loading the most-used experts into RAM (up to 71 GiB; the rest are read from the model files as needed) and locking part of them for the GPU.
         YOUR PC CAN BE SLOW OR STOP RESPONDING FOR 1-3 MINUTES NOW - this is normal.
         Please wait and don't close this window; the browser opens when it is ready.
[strata] filling the GPU's expert cache (3386 experts, 10.12 GiB of VRAM) ...
[strata] still starting (34 s) - please wait ...
[strata] still starting (54 s) - please wait ...
[strata] still starting (74 s) - please wait ...
[strata] still starting (94 s) - please wait ...
[strata] still starting (114 s) - please wait ...
[strata] almost ready ...
[strata] still starting (147 s) - please wait ...
[strata] still starting (167 s) - please wait ...
[strata] still starting (187 s) - please wait ...
[strata] still starting (207 s) - please wait ...
    5 workers: 63.1 tok/s
  Measuring with 4 CPU workers (restarts the engine) ...
[strata] starting the engine: reading the model's weights ...
[strata] loading the most-used experts into RAM (up to 71 GiB; the rest are read from the model files as needed) and locking part of them for the GPU.
         YOUR PC CAN BE SLOW OR STOP RESPONDING FOR 1-3 MINUTES NOW - this is normal.
         Please wait and don't close this window; the browser opens when it is ready.
[strata] filling the GPU's expert cache (3386 experts, 10.12 GiB of VRAM) ...
[strata] still starting (34 s) - please wait ...
[strata] still starting (54 s) - please wait ...
[strata] still starting (74 s) - please wait ...
[strata] still starting (94 s) - please wait ...
[stra