Pull requests / #2094
#2094 Draft: immutable branching contexts with validated restore and serial benchmarks
open · draft · @CC-David-CC · 0 comments · View on GitHub
AMD / HIPNVIDIA / CUDAModels & quantsWindows
Description
Supersedes #2015. Foundation for #1938 and #1962; neither issue is closed by this draft. Adds immutable histories and controllers, movable branch references, shared checkpoints, rewind/retry, evaluation and recovery. Fixes omitted GPU expert contributions in the experimental branch/fresh drivers. Measured on RTX PRO 6000 Blackwell, Q8_0 weights / FP16 KV, one request at a time: | Eight serial calls | Request time sum | |---|---:| | Untouched Strata (7695859a) | 42.12s | | Modified ordinary serving | 42.18s | | Branching API | 50.35s | All paths produced identical tokens. All eight API frontiers matched an independent fresh oracle at zero logit error. Disk reload and complete checkpoint eviction/replay passed. Reused API calls took 2.4-3.4s; initial prefix creation took 12.6s and return to the prefix took 10.9s. The complete mixed hot/cold sequence was slower. These are single-run whole-path measurements, excluding startup and oracle checks. Prefill operators and cache budgets differ: no isolated cache-speedup claim. Separate 11-operation GPU smoke passed 14 fresh comparisons at zero error; G0 passed 51/51. Batched-prefill equivalence remains unresolved; help diagnosing it is welcome. Full G1-G5 certification, Windows and HIP/SYCL validation remain incomplete. [Measurements, fingerprints and limitations](https://github.com/CC-David-CC/Strata-a5500/blob/3445371f6b127575c44fd6e108c2b6683fea4ffa/experimental/branching_context/evidence/llm60-serial-speed.json). Reproduce with `experimental/branching_context/scripts/serial_speed.py --help` and a pinned frozen profile.
Related on strata.com
Editorial links to help you install, pick models, or read release notes — not part of the upstream thread.