Pull requests / #2094

#2094 Draft: immutable branching contexts with validated restore and serial benchmarks

open · draft · @CC-David-CC · 0 Kommentare · Auf GitHub

AMD / HIPNVIDIA / CUDAModels & quantsWindows

Beschreibung

Supersedes #2015. Foundation for #1938 and #1962; neither issue is closed by this draft.

Adds immutable histories and controllers, movable branch references, shared checkpoints, rewind/retry, evaluation and recovery. Fixes omitted GPU expert contributions in the experimental branch/fresh drivers.

Measured on RTX PRO 6000 Blackwell, Q8_0 weights / FP16 KV, one request at a time:

| Eight serial calls | Request time sum |
|---|---:|
| Untouched Strata (7695859a) | 42.12s |
| Modified ordinary serving | 42.18s |
| Branching API | 50.35s |

All paths produced identical tokens. All eight API frontiers matched an independent fresh oracle at zero logit error. Disk reload and complete checkpoint eviction/replay passed. Reused API calls took 2.4-3.4s; initial prefix creation took 12.6s and return to the prefix took 10.9s. The complete mixed hot/cold sequence was slower.

These are single-run whole-path measurements, excluding startup and oracle checks. Prefill operators and cache budgets differ: no isolated cache-speedup claim. Separate 11-operation GPU smoke passed 14 fresh comparisons at zero error; G0 passed 51/51.

Batched-prefill equivalence remains unresolved; help diagnosing it is welcome. Full G1-G5 certification, Windows and HIP/SYCL validation remain incomplete.

[Measurements, fingerprints and limitations](https://github.com/CC-David-CC/Strata-a5500/blob/3445371f6b127575c44fd6e108c2b6683fea4ffa/experimental/branching_context/evidence/llm60-serial-speed.json). Reproduce with `experimental/branching_context/scripts/serial_speed.py --help` and a pinned frozen profile.

Mehr auf der Site

Links zu Install, Modellen, Releases.