#6281 [Engine] SSD conversation tier turns off when MTP comes from a GGUF (fingerprint looks for mtp/rt/dense.bin)
open · @mnott · 3 comments · View on GitHub
Setup & installModels & quantsWindows
Description
### Area [Engine] ### Desktop app, Terminal or source? Source (START-HERE.bat, setup.sh or a build) ### What happened On v0.2.16 the SSD tier for conversations evicted from the RAM conversation cache turns itself off at every start when the MTP drafter is loaded from a GGUF file (`--mtp <file>.gguf`) rather than from a runtime pack. The MTP runtime directory in this setup only holds `draft_vocab.bin`; `dense.bin`, `dense.txt` and `experts.bin` never exist when MTP comes from a GGUF, so the fingerprint can never be computed and the tier stays off. Why it matters: with several long agent conversations (60–135k tokens each) sharing one engine, the 4 RAM slots evict conversations constantly, and without the SSD tier every evicted conversation is read again from token 0 on its next turn (minutes per turn on Gyro-L). On a ~10 GB/s internal SSD a restore would take well under a second. Expected: the fingerprint covers the files the engine actually loaded (the MTP GGUF when MTP comes from a GGUF; the pack files when a pack is used), so the SSD tier stays on in both setups, while a spill file from a different model or config is still never restored. ### Smallest steps or command that reproduce it 1. Start a Gyro model with `--spec 2 --mtp <mtp>.gguf --conversation-cache-mib 12288 --conversation-cache-slots 4` (Metal build of v0.2.16). 2. Read the server log right after start: the SSD tier is announced and then disabled (see Logs). ### Reproduction status Reproduced locally ### Strata version v0.2.16 (engine built from source, Metal) ### Graphics card, memory and system Apple M5 Max, 128 GB unified memory, macOS ### Model, size and settings Qwen3.8-Flash-Next Gyro-L (TQ3_0 without the PLE table, PLE Q8_0 via --ple-gguf), --spec 2 with MTP from a Q2_0 GGUF, --expert-cache 64, --conversation-cache-mib 12288, --conversation-cache-slots 4 ### Logs ``` SSD tier in <dir>/conversation-spill/…, up to 32768 MiB no SSD tier from now on: the model cannot be fingerprinted (mtp/rt/dense.bin: No such file or directory) ``` A fix is in preparation and will be linked here.
Related on this site
Quick links to related pages.