Issues / #2040

#2040 [Linux, SATA SSD] --ple-io boot auto-switch to mmap degrades PLE prefill ~7x: the 15K rows/s "direct is healthy" threshold is calibrated for NVMe, so it trips on ordinary SATA SSDs where direct is actually the faster mode

open · @lanceyang093-blip · 1 コメント · GitHub で見る

BenchmarksServer & APINVIDIA / CUDAModels & quantsWindowsLinux

本文

<html>
<body>
<!--StartFragment--><p style="box-sizing: border-box; scrollbar-width: thin; scrollbar-color: rgb(42, 47, 61) rgba(0, 0, 0, 0); margin: 0.8em 0px; line-height: 1.85; color: rgb(230, 232, 239); font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei&quot;, Roboto, Helvetica, Arial, sans-serif; font-size: 14px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgba(0, 0, 0, 0.22); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><strong style="box-sizing: border-box; scrollbar-width: thin; scrollbar-color: rgb(42, 47, 61) rgba(0, 0, 0, 0); color: rgb(243, 244, 246); font-weight: 720;">Environment.</strong> Ubuntu, <strong style="box-sizing: border-box; scrollbar-width: thin; scrollbar-color: rgb(42, 47, 61) rgba(0, 0, 0, 0); color: rgb(243, 244, 246); font-weight: 720;">SATA SSD</strong> (Lenovo SL750 240 GB, <code style="box-sizing: border-box; scrollbar-width: thin; scrollbar-color: rgb(42, 47, 61) rgba(0, 0, 0, 0); font-family: &quot;SF Mono&quot;, &quot;JetBrains Mono&quot;, ui-monospace, Menlo, Consolas, monospace; background: rgba(139, 92, 246, 0.1); border: 1px solid rgb(30, 34, 48); border-radius: 5px; padding: 0.16em 0.42em; font-size: 0.88em; word-break: break-word; color: rgb(221, 214, 254);">/sys/block/sda/queue/rotational</code> = 0), RTX 3070 Mobile16 GB, 61 GB RAM, engine 0.1.43 (git build), IQ3_S GSQ-RCO, PLE table 28.8 GB (320,001,536 rows × 90 B), <code style="box-sizing: border-box; scrollbar-width: thin; scrollbar-color: rgb(42, 47, 61) rgba(0, 0, 0, 0); font-family: &quot;SF Mono&quot;, &quot;JetBrains Mono&quot;, ui-monospace, Menlo, Consolas, monospace; background: rgba(139, 92, 246, 0.1); border: 1px solid rgb(30, 34, 48); border-radius: 5px; padding: 0.16em 0.42em; font-size: 0.88em; word-break: break-word; color: rgb(221, 214, 254);">--prefill auto:32768</code>, <code style="box-sizing: border-box; scrollbar-width: thin; scrollbar-color: rgb(42, 47, 61) rgba(0, 0, 0, 0); font-family: &quot;SF Mono&quot;, &quot;JetBrains Mono&quot;, ui-monospace, Menlo, Consolas, monospace; background: rgba(139, 92, 246, 0.1); border: 1px solid rgb(30, 34, 48); border-radius: 5px; padding: 0.16em 0.42em; font-size: 0.88em; word-break: break-word; color: rgb(221, 214, 254);">--ple-row-cache 2097152</code>, experts in a pinned RAM arena (~47 GiB), KV int8 resident.</p><p style="box-sizing: border-box; scrollbar-width: thin; scrollbar-color: rgb(42, 47, 61) rgba(0, 0, 0, 0); margin: 0.8em 0px; line-height: 1.85; color: rgb(230, 232, 239); font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei&quot;, Roboto, Helvetica, Arial, sans-serif; font-size: 14px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgba(0, 0, 0, 0.22); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><strong style="box-sizing: border-box; scrollbar-width: thin; scrollbar-color: rgb(42, 47, 61) rgba(0, 0, 0, 0); color: rgb(243, 244, 246); font-weight: 720;">Symptom.</strong> On 0.1.43, a fresh 77.8K-token prompt prefill runs at <strong style="box-sizing: border-box; scrollbar-width: thin; scrollbar-color: rgb(42, 47, 61) rgba(0, 0, 0, 0); color: rgb(243, 244, 246); font-weight: 720;">76–154 tok/s</strong> across 4 cold runs. Identical config on 0.1.41 (binary swapped, everything else the same): <strong style="box-sizing: border-box; scrollbar-width: thin; scrollbar-color: rgb(42, 47, 61) rgba(0, 0, 0, 0); color: rgb(243, 244, 246); font-weight: 720;">902.8 tok/s</strong>, verified after <code style="box-sizing: border-box; scrollbar-width: thin; scrollbar-color: rgb(42, 47, 61) rgba(0, 0, 0, 0); font-family: &quot;SF Mono&quot;, &quot;JetBrains Mono&quot;, ui-monospace, Menlo, Consolas, monospace; background: rgba(139, 92, 246, 0.1); border: 1px solid rgb(30, 34, 48); border-radius: 5px; padding: 0.16em 0.42em; font-size: 0.88em; word-break: break-word; color: rgb(221, 214, 254);">drop_caches</code> + fresh process (cold page cache, cold row cache). A 6–9× version-level regression that survives controlling every cache.</p><p style="box-sizing: border-box; scrollbar-width: thin; scrollbar-color: rgb(42, 47, 61) rgba(0, 0, 0, 0); margin: 0.8em 0px; line-height: 1.85; color: rgb(230, 232, 239); font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei&quot;, Roboto, Helvetica, Arial, sans-serif; font-size: 14px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgba(0, 0, 0, 0.22); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><strong style="box-sizing: border-box; scrollbar-width: thin; scrollbar-color: rgb(42, 47, 61) rgba(0, 0, 0, 0); color: rgb(243, 244, 246); font-weight: 720;">Boot log (0.1.43):</strong></p><div class="codeblock" style="box-sizing: border-box; scrollbar-width: thin; scrollbar-color: rgb(42, 47, 61) rgba(0, 0, 0, 0); margin: 1.1em 0px; position: relative; min-width: 0px; max-width: 100%; color: rgb(230, 232, 239); font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei&quot;, Roboto, Helvetica, Arial, sans-serif; font-size: 14px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgba(0, 0, 0, 0.22); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><button type="button" class="copy-btn" title="复制" style="box-sizing: border-box; scrollbar-width: thin; scrollbar-color: rgb(42, 47, 61) rgba(0, 0, 0, 0); z-index: 2; width: 28px; height: 28px; color: rgb(107, 114, 132); cursor: pointer; opacity: 0; background: 0px 0px; border: 0px; border-radius: 50%; flex: 0 0 auto; justify-content: center; align-items: center; margin: 0px; padding: 0px; font-size: 14px; line-height: 1; transition: opacity 0.15s, color 0.15s; display: inline-flex; position: absolute; top: 8px; right: 8px;"><svg stroke="currentColor" fill="none" stroke-width="2" viewBox="0 0 24 24" stroke-linecap="round" stroke-linejoin="round" height="1em" width="1em" xmlns="http://www.w3.org/2000/svg"><rect x="9" y="9" width="13" height="13" rx="2" ry="2"></rect><path d="M5 15H4a2 2 0 0 1-2-2V4a2 2 0 0 1 2-2h9a2 2 0 0 1 2 2v1"></path></svg></button><pre node="[object Object]" style="box-sizing: border-box; scrollbar-width: thin; scrollbar-color: rgb(42, 47, 61) rgba(0, 0, 0, 0); max-width: 100%; overflow-x: auto; border: 1px solid rgb(38, 42, 53); border-radius: 10px; margin: 0px; padding: 14px 0px; background: none 0% 0% / auto repeat scroll padding-box border-box color(srgb 0.101961 0.113725 0.14902 / 0.62) !important; min-width: 0px; font-size: 12.5px; line-height: 1.65;"><div class="code-line" style="box-sizing: border-box; scrollbar-width: thin; scrollbar-color: rgb(42, 47, 61) rgba(0, 0, 0, 0); align-items: flex-start; min-width: 0px; padding: 0px 16px; display: flex;"><code class="code-line-body hljs" style="box-sizing: border-box; scrollbar-width: thin; scrollbar-color: rgb(42, 47, 61) rgba(0, 0, 0, 0); color: rgb(221, 214, 254); background: 0px 0px; padding: 0px; font-family: &quot;SF Mono&quot;, &quot;JetBrains Mono&quot;, ui-monospace, Menlo, Consolas, monospace; border-color: currentcolor; border-style: none; border-width: medium; border-image: none; border-radius: 5px; font-size: 12.5px; word-break: break-word; display: block; overflow-x: auto; white-space: pre-wrap; flex: 1 1 0%; min-width: 0px; line-height: 1.55;">strata generate: <span class="hljs-comment" style="box-sizing: border-box; scrollbar-width: thin; scrollbar-color: rgb(42, 47, 61) rgba(0, 0, 0, 0); color: rgb(139, 148, 158);">--ple-io direct measures slow here (8588 rows/s, about 35 MB/s of random 4 KiB reads):</span></code></div><div class="code-line" style="box-sizing: border-box; scrollbar-width: thin; scrollbar-color: rgb(42, 47, 61) rgba(0, 0, 0, 0); align-items: flex-start; min-width: 0px; padding: 0px 16px; display: flex;"><code class="code-line-body hljs" style="box-sizing: border-box; scrollbar-width: thin; scrollbar-color: rgb(42, 47, 61) rgba(0, 0, 0, 0); color: rgb(221, 214, 254); background: 0px 0px; padding: 0px; font-family: &quot;SF Mono&quot;, &quot;JetBrains Mono&quot;, ui-monospace, Menlo, Consolas, monospace; border-color: currentcolor; border-style: none; border-width: medium; border-image: none; border-radius: 5px; font-size: 12.5px; word-break: break-word; display: block; overflow-x: auto; white-space: pre-wrap; flex: 1 1 0%; min-width: 0px; line-height: 1.55;">                 <span class="hljs-keyword" style="box-sizing: border-box; scrollbar-width: thin; scrollbar-color: rgb(42, 47, 61) rgba(0, 0, 0, 0); color: rgb(255, 123, 114);">using</span> <span class="hljs-comment" style="box-sizing: border-box; scrollbar-width: thin; scrollbar-color: rgb(42, 47, 61) rgba(0, 0, 0, 0); color: rgb(139, 148, 158);">--ple-io mmap instead (the OS file cache holds the pages; ...)</span></code></div></pre></div><p style="box-sizing: border-box; scrollbar-width: thin; scrollbar-color: rgb(42, 47, 61) rgba(0, 0, 0, 0); margin: 0.8em 0px; line-height: 1.85; color: rgb(230, 232, 239); font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei&quot;, Roboto, Helvetica, Arial, sans-serif; font-size: 14px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgba(0, 0, 0, 0.22); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><strong style="box-sizing: border-box; scrollbar-width: thin; scrollbar-color: rgb(42, 47, 61) rgba(0, 0, 0, 0); color: rgb(243, 244, 246); font-weight: 720;">Disk measurement (O_DIRECT 4 KiB reads on the PLE file, 16-thread pread = the engine's own DirectFile pattern):</strong></p>
mode | throughput | IOPS
-- | -- | --
rand 4 KiB, 1 thread | 13.6 MB/s | 3.3K
rand 4 KiB, 16 threads (direct pattern) | 41.3 MB/s | 10.1K
direct, 0.1.41 production prefill | ~59 MB/s | ~14.4K
direct, 0.1.43 boot probe (logged) | ~35 MB/s | ~8.6K
mmap, 0.1.43 production prefill | ~8.6 MB/s | ~2.1K

<p style="box-sizing: border-box; scrollbar-width: thin; scrollbar-color: rgb(42, 47, 61) rgba(0, 0, 0, 0); margin: 0.8em 0px; line-height: 1.85; color: rgb(230, 232, 239); font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei&quot;, Roboto, Helvetica, Arial, sans-serif; font-size: 14px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; fo

関連リンク

インストール・モデル・リリースへの站内リンク。