#4411 [Engine] session-file saves: ping-pong blocks and a writer thread so the source copy overlaps the previous block write

open · @sakuumomo · 0 comentarios · En GitHub

Setup & installAMD / HIPNVIDIA / CUDA

Descripción

### The problem

A busy-live session save crawls at ~110 MB/s while the storage beneath it measures 500+ MB/s (the owner's bcachefs pool on the desktop host). `FileSink` serializes: the producer copies one 16 MiB block out of the K/V sources, hands it to `write_all`, waits for the write, then copies the next block. The GPU-side copy and the disk write never overlap.

### What this does

`FileSink` now ping-pongs two 16 MiB buffers with a lazily started writer thread: while the writer drains block N to the OS, the producer already copies block N+1 out of the sources. When both buffers are busy the producer waits — at most one block behind.

Deliberately unchanged:

- the **byte stream**: blocks are written in the same fixed order, so hashes and file bytes are identical — the golden format tests pass untouched;
- progress semantics: reports still come from the calling thread, follow real movement (monotonic, may lag ≤1 block, never lead);
- fault latching and publish: a writer-side `write_all` failure merges into the existing `f.error()`/`kind()` channel after a clean join; the temp file is never published; `Publisher` untouched;
- `finish()` (pad → truncate → sync → close) runs only after the writer has joined.

New pieces:

- the save RAM preflight now reserves **2 × kBlock** (the two write buffers);
- if a writer thread cannot start (process/thread limits — `std::system_error`), `FileSink` falls back to synchronous draining: the exact old serialized path. The pipeline can never crash a save.

### Tests (all CPU, no new dependencies)

- **pipeline witness**: a tests-only `block_span` hook records each full block's OS-write span; the test asserts a write *begins during* a source read — causally impossible in the serialized loop (verified RED against it, GREEN against the pipeline). The hook sits beside the existing `fault` hook in `SessionWriteOptions` (same tests-only precedent).
- **starved writer**: a `"writer_thread"` fault seam forces the fallback; the save still completes, byte-identical, with no overlap witnessed.
- mid-pipeline write-fault test (existing fault injection) still passes unchanged.
- CPU gate `conversation_(file|split_failure|cache|memory)_test`, `session_save_reclaim_test`: 5/5. ThreadSanitizer run of `conversation_file_test`: 0 warnings.

Independent of #4310: the pipelined code is the `FileSink` core as it stands on `main`; the chain here is these four commits on `main` alone.

### Update (2026-10-30)

Rebased onto upstream `v0.2.8` (`870aedd9`); content unchanged, branch tip `2a8d10b8` (still these four commits). Gates re-run on the rebuilt lineage: the CPU session test family 7/7 (`conversation_append_test` 1969 checks, `conversation_file_test` 263, `session_watermark_test` 21), the ThreadSanitizer run 0 warnings (1969 + 260 checks), and the owner's HIP-only (gfx1030) and CUDA+HIP hybrid (gfx1150) flavor builds green with installCheck. Commits re-signed (GPG ed25519) on 2026-10-30; trees unchanged from the validated work.

### Update (2026-11-07)

Rebased onto upstream `v0.2.12` (`123bad4a`, engine 0.1.58); content unchanged, branch tip `8d944e7f` (still these four commits, zero conflicts). Gates re-run on the rebuilt lineage: CPU session family 7/7 (append 1969, file 263, watermark 31), ThreadSanitizer 1969 + 260 checks 0 warnings, and the owner's HIP-only (gfx1030) and CUDA+HIP hybrid (gfx1150) flavor builds green with installCheck on the redeployed train `nixos` @ `c3e4cdc4`. metal-link and title jobs are green on the new head; the PR is mergeable against current `main`. Commits re-signed (GPG ed25519) on 2026-11-07; trees unchanged from the validated work. The pre-rebase tip `2a8d10b8` stays fetchable as an ancestor of the train tag `pin-f6f00908`.

### Numbers (live battery on the owner's desktop host, post-merge)

A 166k-token, 4.94 GB session file: the old build saved it in 42–54 s uncontended (160 s worst, running through a heavy prefill); this build saves it in 54.5 s. Storage probes on the same pool at the same moment: 265 MB/s direct, 475 MB/s buffered+fsync — so the save ends at 91 MB/s because the **engine-side source chain** (K/V streamed out of the live state) is the binding rate on this host, not the disk. What overlap buys here is that the producer never stalls on disk mid-stream (the writer thread drains blocks while the source keeps copying; a save whose source runs at 91 MB/s cannot stream faster than its source, and the live rows confirm that); the byte stream is unchanged, and the pipelined write path is the foundation for append-delta saves — saving only the dirty tail (a few MB per turn at this size), which is where this host's save time actually collapses.




### Update (2026-11-20)

Rebased onto upstream `v0.2.17` (`12d491da4`); branch tip `6f0dcdf2` — 1:1 range-diff against the 0.2.12-era chain: the same four commits, hunks re-fitted to the surrounding 0.2.16/0.2.17 movement. Gates re-run on the rebuilt lineage `nixos` @ `4cfc06018` (88 signed commits): the CPU session family 7/7 (`conversation_append_test` 1969 checks, `conversation_file_test` 263, `session_watermark_test` 31, `session_save_reclaim_test` 167, plus the split-failure/cache/memory targets), the ThreadSanitizer build green with zero warnings, the HIP gfx1030 conversation battery 20/20, and the CUDA+HIP hybrid build linking `strata` plus all 20 battery targets in one process (rc=0); the HIP flavor build from a clean checkout of the new pin is green with installCheck. The owner's desktop host was switched to the new pin on 2026-11-20 and serves daily on it. Commits re-signed (GPG ed25519) on 2026-11-20; trees unchanged from the validated work. The pre-rebase tip `8d944e7f` stays fetchable as an ancestor of the tag `pin-pre0217-pr-engine-save-overlap`. No CI checks were reported on the new head; the PR is mergeable against current `main`.

### Update (2026-11-21)

Rebased onto upstream `v0.2.18` (`6e38f588a`); branch tip `6f0dcdf2` -> `698afd32c`. 1:1 range-diff against the 0.2.17-era chain: the same four commits, zero conflicts, zero adaptations (the 227-commit upstream release rode almost entirely through `engine/src/hy4flash/`). Gates re-run on the rebuilt lineage `nixos` @ `295872912` (88 signed commits): the CPU session family 7/7 (`conversation_append_test` 1969 checks, `conversation_file_test` 263 — the pipeline witness and fault-latching pins intact —, `session_watermark_test` 31, `session_save_reclaim_test` 167, plus the split-failure/cache/memory targets), the ThreadSanitizer build green with zero warnings (append 1969 + file 260 checks), the HIP gfx1030 conversation battery 20/20 under headless-first device enumeration, and the CUDA+HIP hybrid build linking `strata` plus all 20 battery targets in one process (CUDA arch 89, HIP arches gfx1030+gfx1150). The HIP flavor build from a clean checkout of the new pin is green with installCheck. The owner's desktop host was re-pinned to this lineage on 2026-11-21 and serves daily on it. Commits re-signed (GPG ed25519); trees unchanged from the validated work. The pre-rebase tip `6f0dcdf2a` stays fetchable as an ancestor of the tag `pin-4cfc06018`. No CI checks were reported on the new head; the PR is mergeable against current `main`.

En el sitio

Enlaces a install, modelos, releases.