#2410 [BUG] Strata never finishes reading a prompt through Hermes Agent.

Description

I have been trying to use Strata with Herme Agent, but the engine resets after making no progress at all on reading the prompt.   At the bottom of the post is my .dmp after I stopped Hermes and shut down Strata.

[strata-stall-36664.dmp](https://github.com/user-attachments/files/33431363/strata-stall-36664.dmp)

000000000000

load_hparams: Qwen-VL models require at minimum 1024 image tokens to function correctly on grounding tasks
load_hparams: if you encounter problems with accuracy, try adding --image-min-tokens 1024
load_hparams: more info: https://github.com/ggml-org/llama.cpp/issues/16842

strata-vision: model files loaded in 1.3 s
strata-vision: warmed up at 1024 image tokens in 1.0 s
[strata] 2026-10-14 02:45:46 engine started: C:\Users\Janus\Desktop\ARTIFICIAL\Strata\Strata-main\engine\strata.exe --serve --pack C:\Users\Janus\Desktop\ARTIFICIAL\Strata-data\packs\unsloth-ud-q4_k_xl --native C:\Users\Janus\Desktop\ARTIFICIAL\Strata-data\models\unsloth-UD-Q4_K_XL\Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf --expert-profile C:\Users\Janus\Desktop\ARTIFICIAL\Strata\Strata-main\data\expert-profile.bin --expert-cache auto --prefill auto --spec 4 --mtp C:\Users\Janus\Desktop\ARTIFICIAL\Strata-data\mtp\rt --max-context 262144 --kv int8 --kv-resident 32768 --vision --vram-reserve-mib 700 --spec-min-p 0.5 --adapt-every 1 --adapt-swaps 80 --adapt-decay 0.97 --layer-split auto
strata generate: memory guard: limit 127.5 GiB (source: meminfo), current 23.5 GiB, reclaimable cache credited 0.0 GiB; available 103.9 GiB (MemAvailable 103.9 GiB), floor 2560 MiB
strata generate: layer split across 3 GPUs: CUDA0, then CUDA1,2 (split auto)
strata generate: PCIe probe: 26.8 GB/s host->device (best of 26.8 26.8 26.8 26.8) -> pcie_frac 0.55 (default 0.55)
strata generate: native pack: C:\Users\Janus\Desktop\ARTIFICIAL\Strata-data\packs\unsloth-ud-q4_k_xl experts (largest blob 3.99 MB), token embedding Q8_0 in mapped host memory (644 MiB, 1.7 s)
strata generate: 1416 MiB of weights loaded from C:\Users\Janus\Desktop\ARTIFICIAL\Strata-data\packs\unsloth-ud-q4_k_xl in 0.3 s (303 canonical tensors skipped: served natively)
strata: STRATA_HC_Q8=1: 0.63 GiB of Q8_0 hyper-connection projections for the verify read
strata generate: 495 native projection matrices, 3619.14 MiB of weights, in 8.5 s
strata generate: PLE direct read probe: 72025 rows/s (fine)
strata generate: the SSD is kept awake while rows are read: one page of the table after 100 ms without a read, until 60 s after the last request (STRATA_SSD_KEEPALIVE=0 turns it off)
strata: STRATA_HC_Q8=1: 0.63 GiB of Q8_0 hyper-connection projections for the verify read
strata generate: layer split: CUDA1 PCIe probe 26.8 GB/s (best of 26.8 26.8 26.8 26.8) -> pcie_frac 0.55
strata generate: layer split: CUDA1 holds its weights; 5.77 GiB free (its session follows the split search)
strata: STRATA_HC_Q8=1: 0.63 GiB of Q8_0 hyper-connection projections for the verify read
strata generate: layer split: CUDA2 PCIe probe 26.8 GB/s (best of 26.8 26.8 26.8 26.8) -> pcie_frac 0.55
strata generate: layer split: CUDA2 holds its weights; 15.75 GiB free (its session follows the split search)
strata generate: profile C:\Users\Janus\Desktop\ARTIFICIAL\Strata\Strata-main\data\expert-profile.bin: 24576 ranked pairs, built for 24576 slots
strata generate: layer split auto: CUDA0 28 SMs at 1.81 GHz -> 1.43 ms per layer, 5.08 GiB free before its session carve
strata generate: layer split auto: CUDA1 28 SMs at 1.84 GHz -> 1.41 ms per layer, 4.99 GiB free before its session carve
strata generate: layer split auto: CUDA2 128 SMs at 2.60 GHz -> 0.22 ms per layer, 14.00 GiB free before its session carve
strata generate: layer split auto: CUDA0 link 26.8 GB/s -> miss cost x1.00
strata generate: layer split auto: CUDA1 link 26.8 GB/s -> miss cost x1.00
strata generate: layer split auto: CUDA2 link 26.8 GB/s -> miss cost x1.00
strata generate: layer split auto: K=2,3 would give a stage fewer than 3 layers (a tiny expert cache there clamps the prompt chunk of every card, #1616); chose K=3,6 instead (25.8 ms per decode window, not 23.2)
strata generate: layer split auto: K=3,6 - predicted 25.8 ms per decode window; the caches hold 7610 of 24576 profiled pairs (~95.8% of the routed mass), best of 1855 placements
strata generate: KV streaming: 32768 of 262144 cells per QSA layer in VRAM, the K/V in 0.26 GiB of pinned RAM
strata generate: PLE on, table 320001536 rows of C:\Users\Janus\Desktop\ARTIFICIAL\Strata-data\models\unsloth-UD-Q4_K_XL\Qwen3.8-Flash-Next-UD-Q4_K_XL-00002-of-00004.gguf
strata generate: layer split: CUDA1 holds its weights, session [3, 6); 5.63 GiB free
strata generate: layer split: CUDA2 holds its weights, session [6, 48) and the head; 14.25 GiB free
strata mtp: draft layer loaded, 835 MiB of VRAM (experts 675, dense 111), files read in 0.33 s (2382 MiB/s)
strata generate: GPU 0: NVIDIA GeForce RTX 3060, compute capability 8.6
strata generate: multi-GPU under WDDM: at most 8 GiB of the expert arena is pinned (STRATA_ARENA_PIN_GIB changes it)
strata generate: expert arena read unbuffered (1 of 48 probe reads from the file cache; 27.5 GiB available, 103.7 GiB of files)
strata generate: expert arena: locked 67003 MiB via working-set minimum + VirtualLock; cudaHostRegister limited to 8 GiB by the engine (multi-GPU under WDDM, or remote experts); 4 slices pinned (6 GiB); large pages refused for 77022101504 B (GetLargePageMinimum=2097152, VirtualAlloc error 1314); using 4 KB pages
strata generate: loaded 71.73 GiB at 0.52 GiB/s
strata generate: CPU pool tasks/phase: 72 (automatic), participating threads: 24
strata generate: 23 pool workers on logical processors 2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46, the host thread on 0 (draining too) (--host-core first)
strata aux cpus: not available on this system
strata generate: expert cache auto: 5.62 GiB free, 700 MiB reserved (+0 MiB for the draft head) -> 1327 slots
strata generate: only 0 MiB free once the slots are written (reserve 700 MiB); shrinking the expert cache
strata generate: expert cache 1226 slots, 3.83 GiB of VRAM; policy is
strata generate: the GPU computes the experts in the cache; it rounds differently from the CPU,
                 so a reply can differ slightly from a run without the cache (same quality:
                 bench/results/2026-09-27-cache-parity).
                 PROFILE, ranked by routing frequency, no eviction.
strata generate: pre-filled 1226 of 1226 slots from the profile in 0.2 s (25110 MB/s); slot 0 verified
strata generate: layer split, CUDA1: 5.63 GiB free of 12.00, room for experts 4.85 GiB
strata generate: layer split: CUDA1 runs layers 3-5, expert cache 1536 slots (4.64 GiB), 1536 of its 1536 profiled pairs; slot 0 verified
strata generate: layer split, CUDA2: 13.43 GiB free of 22.49, room for experts 12.65 GiB
strata generate: layer split: CUDA2 runs layers 6-47, expert cache 4374 slots (12.65 GiB), 4374 of its 21504 profiled pairs; slot 0 verified
strata generate: layer split: CUDA0 runs layers 0-2
strata generate: layer split auto: L2 probe: the split is startable on the live free figures
strata generate: R4 hit path ON - resident experts are computed on the GPU
strata generate: 23 expert-pool workers + the host thread
strata generate: session is up (engine 0.1.44.1)
strata generate: token graph hit path: 7136 resident experts, decided on the device
strata serve: Windows commit: 24.7 GiB left of a 181.5 GiB limit (RAM + page file)
strata serve: prompt chunk auto: 6656 tokens, a 96-slot ring
strata serve: the prompt path borrows 1015 CUDA0 cache slots (3.17 GiB)
strata serve:   CUDA1 prompt path borrows 1049 of its 1536 slots (3.17 GiB)
strata serve:   CUDA2 prompt path borrows 1096 of its 4374 slots (3.17 GiB)
strata hc: CUDA1: the hyper-connection read runs as staged (the norm per token and stream, the down projection's activations staged ahead by cp.async); checked bit for bit against the plain read on this card (STRATA_HC_SPLIT=1 or 0 for the earlier ones)
strata verify: window up to 6 tokens, 96.1 MiB of device buffers (100% VRAM resident: zero-doorbell graph)
strata hc: CUDA2: the hyper-connection read runs as staged (the norm per token and stream, the down projection's activations staged ahead by cp.async); checked bit for bit against the plain read on this card (STRATA_HC_SPLIT=1 or 0 for the earlier ones)
strata verify: window up to 6 tokens, 96.1 MiB of device buffers
strata serve: layer split: layers 0-2 (CUDA0), 3-5 (CUDA1), 6-47 (CUDA2), one hand-off per window
strata hc: CUDA0: the hyper-connection read runs as staged (the norm per token and stream, the down projection's activations staged ahead by cp.async); checked bit for bit against the plain read on this card (STRATA_HC_SPLIT=1 or 0 for the earlier ones)
strata verify: window up to 6 tokens, 96.1 MiB of device buffers
strata mtp: draft head over 106299 tokens (275.7 MiB)
strata serve: adaptive overlap: off: not on a layer split
strata serve: 677 MiB of VRAM free with everything loaded
strata serve: CUDA1: 751 MiB of VRAM free with everything loaded
strata serve: CUDA2: 0 MiB of VRAM free with everything loaded - LOW: its window graphs are captured on first use and need room too, and under WDDM a full card pages instead of failing, which can stall a request for good (#2192); add --vram-reserve-later-mib 1468 to the config's args (or lower --max-context)
strata serve: WARNING: only 677 MiB of VRAM free at the start of this request (below 1024 MiB): when the card is this full the driver can page GPU memory to system RAM, decode falls to a few tok/s and a window may never finish (#2065). Add --vram-reserve-mib 1088 to the config's args, or lower --max-context / --batch
strata verify: capturing the 6-token window (677 MiB of VRAM free)
strata verify: captured the 6-token window (upload no error, sync no error)
strata verify: capturing the 6-token window (751 MiB of VRAM free)
strata verify: captured the 6-token window (upload no error, sync no error)
strata verify: capturing the 6-token window (0 MiB of VRAM free)
strata verify: captured the 6-token window (upload no error, sync no error)
strata verify: capturing the 3-token window (677 MiB of VRAM free)
strata verify: captured the 3-token window (upload no error, sync no error)
strata verify: capturing the 3-token window (751 MiB of VRAM free)
strata verify: captured the 3-token window (upload no error, sync no error)
strata verify: capturing the 3-token window (0 MiB of VRAM free)
strata verify: captured the 3-token window (upload no error, sync no error)
strata verify: capturing the 1-token window (675 MiB of VRAM free)
strata verify: captured the 1-token window (upload no error, sync no error)
strata verify: capturing the 1-token window (751 MiB of VRAM free)
strata verify: captured the 1-token window (upload no error, sync no error)
strata verify: capturing the 1-token window (0 MiB of VRAM free)
strata verify: captured the 1-token window (upload no error, sync no error)
strata verify: capturing the 4-token window (675 MiB of VRAM free)
strata verify: captured the 4-token window (upload no error, sync no error)
strata verify: capturing the 4-token window (749 MiB of VRAM free)
strata verify: captured the 4-token window (upload no error, sync no error)
strata verify: capturing the 4-token window (0 MiB of VRAM free)
strata verify: captured the 4-token window (upload no error, sync no error)
strata serve: prompt 34 tokens = 0 reused + 34 read in 1126 ms (30.2 tok/s), 24 generated in 1416 ms (16.9 tok/s), drafts accepted 18 of 21, 1 checkpoints
strata serve: decode expert cache hit rate: 50.4% (6777 hits / 13458 lookups); 42 more read by the GPU over PCIe or from another GPU (0.3% of all 13500 routed)
strata serve: KV streaming: 100.00% of 0 block reads hit VRAM, 0.0 MiB read from RAM
strata serve: prompt 260 tokens = 0 reused + 260 read in 2283 ms (113.9 tok/s), 1 generated in 41 ms (24.2 tok/s), drafts accepted 0 of 0, 1 check