#2417 Inference stall & freeze caused by random I/O latency

本文

When the PLE shards (ple_gguf) are stored on a portable SSD, the model requires heavy random reads to fetch expert weights during long-context prefill. Limited by the USB protocol, cables, external enclosure controller and physical link, the portable SSD delivers far worse random I/O performance compared with an onboard NVMe SSD directly connected to the motherboard. During prefetching of long prompts, high I/O latency occurs, resulting in frozen inference, timeouts and broken sessions.
After I moved the model files to an internal SSD, everything works properly.