Issues / #2331
#2331 UD-Q4_K_XL: engine exits silently at startup (4 configs: budget page-lock, no-budget, split, single GPU) on Linux/AVX1/157 GB
open · @ifdog · 0 comentários · No GitHub
BenchmarksNVIDIA / CUDAModels & quantsDocumentationWindowsLinux
Descrição
## Summary UD-Q4_K_XL never reaches "session is up" on this box — the engine exits silently at the same place in four different configurations. The other packs (IQ3_XXS) run fine on the same machine with the same engine build, so this looks specific to the UD path. ## Machine - 2× RTX 2080 Ti 22 GB (sm_75), driver 615.71.09, Arch Linux, kernel 7.2.6 - Xeon E5-2650L v2 (Ivy Bridge, AVX1 → `STRATA_ISA_FLOOR=avx` build), **157 GB RAM** - Engine 0.1.43, source build, CUDA 12.8, gcc-13; also tested with the current master + the #2197 PR stack — same result - Model: `unsloth/Qwen3.8-Flash-Next-GGUF` UD-Q4_K_XL, 4 shards, 111,334,654,784 bytes, all four SHA-256-verified against the pinned revision (`38bb39e`) - The same box currently serves IQ3_XXS at 72-87 tok/s (after discovering that `--ple-io ram` is mandatory here, #2239), including vision, so the machine itself is in good shape - Storage is a mergerfs pool on spinning disks: ~70 MB/s streaming, ~222 rows/s random 4K (the reason PLE must be `--ple-io ram` here) ## What happens Every attempt gets through pack load, PLE load (`PLE table locked in RAM (--ple-io ram) in 454.5 s`), the draft layer, and the GPU line, then the engine exits with no message in its own log. `journalctl` shows only `strata-ud.service: Main process exited, code=exited, status=1/FAILURE` (once, with `--vision`, it was `RuntimeError: the vision encoder did not start`, and the vision crash dump showed frames inside `libgc.so.1` — though that may be a symbolization artifact of the crash handler). Four configurations, all dying at the same point: | # | config | dies at | memory peak | |---|---|---|---| | 1 | `--resident-budget-gib 71` + `--vision`, split 24, pipeline-windows 2 | `FileExpertSource: ... page-locking it in steps` | — | | 2 | no budget, `--vision`, split 24 | after `GPU 0: ... compute capability 7.5` | 105.8 GB | | 3 | no budget, no vision, split 24, pipeline-windows 2 | same | — | | 4 | no budget, no vision, **single GPU**, 32K context | same | 87.1 GB | Failure 1 has more detail: with the 71 GiB budget the engine says ``` FileExpertSource: RAM budget 71.00 GiB: 15517 of the 45.29 GiB of experts the GPU cache does not hold, by profile rank; the rest are read from the files FileExpertSource: allocating 67.77 GiB page-locked cache complement FileExpertSource: 1.36 GiB really free for a 67.77 GiB page-locked complement; the model files' cached pages were released: 31.05 GiB free FileExpertSource: the rest comes from reclaimed cache of other files: page-locking it in steps, as far as the RAM allows (#1250) ``` and then dies during the page-locking (no OOM in `dmesg`/`journalctl -k`, no core; `MemAvailable` was ~70 GB at the time). So the "as far as the RAM allows" path seems to end the process rather than continue with what it could lock. ## Questions 1. Is UD-Q4_K_XL expected to work on Linux at all? docs/UNSLOTH_Q4.md says it was validated on one Windows / RTX 5070 / AVX-512 / 64 GB PC; this is Linux + AVX1 + 157 GB + two cards, so it is off the tested path. 2. The silent exit with no message is the worst part — is there a flag to make the file-tier/budget path verbose (`STRATA_*` for FileExpertSource), or do you want a `catchsegv`/gdb run from me on a specific function? 3. For failure 1: should `page-locking it in steps` ever be fatal? If the answer is "it should fall back", that alone would let me run it with a smaller budget. Happy to run any instrumentation you want — engine rebuilds here take ~3 minutes and the model files are in place.
No site
Links install, modelos, releases.