#2843 Slot checkpoints for clients that rewrite the end of their history (3/3, depends on #2527 and #2528): STRATA_SLOT_CHECKPOINTS, STRATA_BRANCH_CHECKPOINT, BCHECKS

本文

## Summary

A client that rewrites the end of its history every time never finds a checkpoint to continue from. Claude Code's
safety classifier is one: each query is the agent's transcript with the new tool calls inserted before the closing
text (`... </transcript>` and the instruction), so the next query leaves this one a few dozen tokens before its end.
The turn checkpoint is past that point, the periodic ones are up to 16K tokens before it, and a slot that receives
the conversation (park=, an admission continuing in its slot) keeps only its deepest checkpoint. On our server
(3 x RTX A4000 layer split, --batch 6) every classifier query of 2026-10-16 read its whole 42-74K-token transcript
again: 15.6-27.4 s per call, several per agent step.

**Part 3, depends on #2527 and #2528**: the branch includes their commits; this PR's own are the last three
(`batch: STRATA_SLOT_CHECKPOINTS=N`, `generate: STRATA_BRANCH_CHECKPOINT=<margin>`, `batch, serve: BCHECKS`). The
placement side of the same problem - a slot holding an older version of the conversation is the one to replace
(`STRATA_SLOT_SIBLING_SHARE`) - is in #2528.

## Change (all opt-in)

- `STRATA_SLOT_CHECKPOINTS=N` (batch): a slot keeps the main session's N deepest checkpoints of its conversation (one
  per length) instead of one; a restore from a slot checkpoint brings the slot's shallower ones back with it. The
  log: `slot 2 keeps 2 checkpoints (at 32768 43372; 227.5 MiB)`. Pure helpers in
  `include/strata/program/slot_checkpoints.hpp`, unit test `slot_checkpoints_test` (20 checks).
- `STRATA_BRANCH_CHECKPOINT=<margin>` (serve, batch or not): before reading, the engine measures how far the prompt
  runs along each conversation it holds (the main session's, every idle slot's) before leaving it; when that point
  is `<margin>` tokens or more past where the read starts and before the turn boundary, the read stops there for one
  more checkpoint, like the root and turn cuts. The next query - which leaves this one later - resumes there.
- `BCHECKS <slot> <tokens> <lengths>` (with N > 1): after `PARKED`, `BADM 1` and `BYIELD` the engine says which
  checkpoints a slot keeps; the server keeps them per slot (forgotten before it sends anything that overwrites the
  slot), shows them in `/metrics` (`checkpoint_tokens`), and `place()` uses the deepest one that starts the prompt
  as the hint where no slot's history continues it (it was 0). Information only: the engine still picks what it
  reuses.

Unset, every path is the one before.

## Results

3 x RTX A4000, Qwen3.8-Flash-Next IQ3_S, 0.1.44.1 + #2527 / #2528, --batch 6 in 3 groups, 200K context. A 43-45K-token
classifier transcript growing by about 250 tokens per query, an agent conversation taking the main session between
the queries; without / with `STRATA_SLOT_CHECKPOINTS=2 STRATA_BRANCH_CHECKPOINT=1024`:

| Query | without: reused + read, ms | with: reused + read, ms |
| --- | ---: | ---: |
| 1 | 0 + 43,379, 14,146 | 0 + 43,379, 14,255 |
| 2 | 0 + 43,624, 14,112 | 32,768 + 10,856, 7,577 (checkpoint at 43,340) |
| 3 | 0 + 43,874, 14,277 | 43,340 + 534, 1,253 |
| 4 | 0 + 44,134, 14,399 | 43,340 + 794, 1,430 |
| 5 | 0 + 44,400, 14,535 | 43,340 + 1,060, 1,652 |
| 6 | 0 + 44,669, 14,596 | 43,340 + 1,329, 1,818 |

The same chain with a request kept decoding beside it (batch mode on throughout): 14.5-14.8 s per query without,
0.97-1.83 s with. On the solo path (no --batch, 262K): the third query 6,564 ms (32,768 + 11,110) without, 1,096 ms
(43,340 + 538) with.

Costs: a branch cut adds 0.8-0.9 s to the read it is in (the same 11,110 tokens from 32,768: 5.9 s straight, 6.8 s
with the cut; solo 6.5 / 7.3 s); for a transcript growing by g tokens per query a margin M costs about 0.9 s x g / M
in cuts and M / 2 more tokens per read (0.7 ms each), so g = 250-500 puts the best M at 800-1,100. Each checkpoint
is about 114 MiB here; with N = 2 a slot kept at most 227.6 MiB. Single user 63.7 / 63.9 tok/s, 6 concurrent x
2048 tokens 178.7 / 179.3 tok/s (`tools/serve_load.py`).

## How it was checked

Exactness, with docs/BATCHING.md's settings (STRATA_IQ_MT_MIN=1, --pcie-frac 0, --adapt-every 1000000,
--no-prefill-borrow), greedy: the 6 queries' short answers are the same tokens without and with; of 6 160-token
answers, 5 are the same and the third differs from its first token. That one depends only on whether the read had
a cut at 43,340: read at once, or resumed from 32,768 without the cut -> answer A; read through the cut in one
request (no restore), resumed from the branch checkpoint in the main session, or from it in a slot -> answer B. So
the saved, copied and restored state is the one the read had there; the cut changes the chunks of the read after it,
as `pin=` or a resume from an earlier turn checkpoint does.

Tests: `slot_checkpoints_test` (deepest-N selection, shared prefix, the branch boundary's conditions);
`serve/test_slot_alloc.py` 72 (the hint from reported checkpoints; BCHECKS lines kept per slot and forgotten before an
overwrite); test_parallel + test_slots + test_cache_trace + test_slot_alloc 140; the serve suite.

Built natively on CUDA (sm_86, Linux). The engine change is in `src/program/generate.cpp` only (shared by every
backend's serve loop); not built here: HIP, SYCL.

## Extra Notes

Both variables are in our production profiles on the machine above since 2026-10-17. The claim of the classifier's prompt layout comes
from our server's logs of Claude Code (a query's transcript grows inside `<transcript>...</transcript>`; each new
query inserts its tool calls before the closing tag).

🤖 Generated with [Claude Code](https://claude.com/claude-code)