Pull requests / #1667
#1667 feat: native caching and session save/restore extended for Codi
open · @CC-David-CC · 0 コメント · GitHub で見る
Setup & installAMD / HIPNVIDIA / CUDASecurityDocumentation
本文
## Summary
Extends native caching and session save/restore with **branching, checkpoint selection/eviction, and bookmarked-only restart retention**. Conversation history remains replayable when a checkpoint is gone.
Opt in with `--experimental-branch-checkpoints --responses-store-path ./responses`. Includes the Responses continuation groundwork from #1446. No native engine files change.

**Plain Strata, 32K:** first token **4.46 s fresh to 1.20 s with disk reuse**; complete response **5.99 s to 2.74 s**. Fresh/reuse are medians of three runs; restart/deletion are single samples. Same server and engine under different cache conditions. [Setup and raw results](https://github.com/CC-David-CC/Strata-a5500/blob/378f3f582e46c8eb734e1109ed584be60a9c9170/docs/measurements/branch-checkpoints-32k/README.md).
## Three bookmark examples
Use response IDs from two independent chats. Set `API=http://127.0.0.1:8095/v1`, `KEY` to your API key, and `A`/`B` to their response IDs.
**1. Bookmark the first chat.**
```sh
curl -fsS "$API/responses/$A/bookmark" -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' -d '{"protected":true}'
```
**2. Bookmark the second chat.**
```sh
curl -fsS "$API/responses/$B/bookmark" -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' -d '{"protected":true}'
```
**3. Remove the first bookmark.**
```sh
curl -fsS "$API/responses/$A/bookmark" -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' -d '{"protected":false}'
```
B stays protected; A's history is not deleted. Bookmarks select which available checkpoints survive restart; they do not reserve runtime cache capacity. Ordinary caches are cleaned at graceful shutdown or dirty boot.
## Validation
98 Python tests: 97 passed, one skipped. All nine plain-server 32K code-recall checks passed, including branching, protected restart and deleted-cache replay. The add-two/remove-one bookmark sequence is covered by an HTTP test. Experimental; CUDA exercised, HIP/SYCL inference not rerun.
関連リンク
インストール・モデル・リリースへの站内リンク。