#2448 [Feature]: ~10x less SSD writing when switching long conversations — a small addition to #1480

Descrição

**How it was made.** I built it together with an AI coding assistant — it wrote the code, I set the goal and tested it on my PC. The code is here if you're curious: https://github.com/fqqdc/Strata-conversation-cache-delta/tree/conversation-cache-delta (it's #1480 plus my changes). I'm not planning to open a PR — I don't have any experience with that side of things — I just wanted to share it and hear what others think.

Hello! I work as a programmer, but this kind of low-level software really isn't my area — I don't write C/C++ and I know nothing about CUDA.

**My situation.** One Windows PC (64 GB RAM, RTX 5070 Ti 16 GB). My conversations get very long (up to ~256K), and switching back to one used to re-read the whole conversation every time — 40 to 85 seconds. PR #1480 (`--conversation-cache-disk-only`) fixed that for me: switching back now takes a couple of seconds. Thank you for it.

**The catch.** Every time you switch away, it saves the whole conversation to disk — over 2 GB for my 245K conversations. With daily use that adds up to tens of terabytes of writes a year, and I'd rather not wear out my SSD.

**What I did.** On top of #1480, I made it save only the part that actually changed. Two small housekeeping things came along: after a restart the cache folder cleans up its own leftovers, and when the folder is full it deletes the conversation you haven't used for the longest time (before, it deleted whichever file happened to sort first — sometimes one you'd just used).

Numbers from my own machine:
- switching away from a 245K conversation: over 2 GB → about 227 MB (roughly 10x less)
- switching away from a 121K conversation: about 1.2 GB → about 226 MB
- switching back still works, and nothing was lost in any of my tests (dozens of switches, plus engine restarts)

It's off by default — you add one option to turn it on. Without it, everything behaves exactly as before.

**Honest limits.** The saving is about 10x, not more: roughly 200 MB of each save is a chunk of the model's internal memory whose size doesn't depend on how long the conversation is, and I don't know how to shrink that. Also, I only tested this on my own computer (Windows, NVIDIA, single GPU) — no idea about Linux, AMD, or multi-GPU.

I hope to see this kind of feature in the main branch someday — conversations kept on disk, and only the changed part written each time.