Pull requests / #1810
#1810 feat: persist conversation checkpoints with RAM and disk restore
open · @CC-David-CC · 0 comments · View on GitHub
BenchmarksAMD / HIPNVIDIA / CUDAModels & quantsDocumentation
Description
Adds opt-in automatic disk-only conversation caching and disk fallback when RAM caching cannot admit a snapshot. Saved native state includes the appropriate main/draft layers; absent-MTP save/restore is handled without assuming a draft layer exists. Reproducible comparisons can keep caching enabled while fixing adaptive/CPU placement choices. Refreshed onto current main, preserving the active-layer guard, current session-reclaim/min-token/memory-cap options and diagnostics. Disk admission now uses the current host/cgroup memory sampler. #1489's native snapshot implementation is already preserved here; #1667 retains its live integration probe and API-history work. The two surviving PRs were validated together. Configure `--conversation-cache-spill-dir DIR` and `--conversation-cache-disk-mib N`, optionally `--conversation-cache-disk-only`; keep disposable validation state separate from deployed caches. Disk-only execution remains single-device. Defaults stay unchanged. [Usage and reproducible token checks](https://github.com/CC-David-CC/Strata-a5500/blob/d01538ec/docs/DETAILS.md). Fresh validation: six native cache/file/memory/spill/session-reclaim tests, combined serving suite (750 tests, 12 skipped), and all six exact-token configurations: RAM, disk-only and RAM-to-disk fallback, each greedy or seeded sampling (temperature 0.7, seed 12345), MTP width 2. Fresh A, switched-away A-return and repeated A output IDs matched, and returned conversations confirmed saved-prefix reuse. Combined Responses lifecycle checks passed 33/33 with MTP disabled; #1667 records the 32K checkpoint/retention probe. CUDA model measurements used RTX PRO 6000 Blackwell / ISTA IQ3_XXS. HIP gfx1012 builds/native cache tests and SYCL builds/host cache tests passed; those checks do not claim fresh model parity on those backends. [Fresh report, complete token IDs, scope and reproduction](https://github.com/CC-David-CC/Strata-a5500/blob/d01538ec/bench/results/2026-10-11-cache-refresh/README.md). Historical timings retain their original source/settings. The reproducible policy now also forces `STRATA_ROUTE_TAIL_SKIP=0`; normal routing defaults remain unchanged. Policy version 2 is included only in reproducible saved-state fingerprints. Native tests verify ordinary old snapshot compatibility, refusal of unversioned reproducible snapshots without terminating the engine, and a new-policy save/restore roundtrip. Seeded disk/fallback parity and HIP/SYCL checks passed again after versioning. Old reproducible snapshots must be regenerated.
Related on strata.com
Editorial links to help you install, pick models, or read release notes — not part of the upstream thread.