Pull requests / #1342

#1342 feat: progressive loading (streaming trunk) baseline

closed · @cybort · 0 Kommentare · Auf GitHub

Beschreibung

This PR introduces the baseline implementation for progressive loading (streaming trunk) in Strata.

The goal is to lower the memory floor required to run the engine by avoiding reading the entire `dense.bin` (trunk weights) into memory at once. Instead, `blk.*` layers are streamed from disk on demand using a ring buffer.

### Changes
- **TrunkStreamer Class**: Added `trunk_stream.hpp` and `trunk_stream.cpp` to handle asynchronous reading of dense weights via an unbuffered ring buffer slot strategy.
- **Generator Integration**: Updated `generate.cpp` to initialize the `TrunkStreamer` for dynamic layer mapping alongside the existing `WeightTable` integration.
- **Memory Management Fixes**: Modified `trunk_stream.cpp` to use standard `std::free` for deallocating slots.
- **Documentation**: Added documentation about the streaming trunk reader to `AGENTS.md`.
- **CMake**: Updated `CMakeLists.txt` to compile the new `trunk_stream.cpp` component.

This is a baseline implementation targeting proof-of-concept compilation and validation without breaking existing workflows.

Mehr auf der Site

Links zu Install, Modellen, Releases.