Pull requests / #1342

#1342 feat: progressive loading (streaming trunk) baseline

closed · @cybort · 0 comentários · No GitHub

Descrição

This PR introduces the baseline implementation for progressive loading (streaming trunk) in Strata.

The goal is to lower the memory floor required to run the engine by avoiding reading the entire `dense.bin` (trunk weights) into memory at once. Instead, `blk.*` layers are streamed from disk on demand using a ring buffer.

### Changes
- **TrunkStreamer Class**: Added `trunk_stream.hpp` and `trunk_stream.cpp` to handle asynchronous reading of dense weights via an unbuffered ring buffer slot strategy.
- **Generator Integration**: Updated `generate.cpp` to initialize the `TrunkStreamer` for dynamic layer mapping alongside the existing `WeightTable` integration.
- **Memory Management Fixes**: Modified `trunk_stream.cpp` to use standard `std::free` for deallocating slots.
- **Documentation**: Added documentation about the streaming trunk reader to `AGENTS.md`.
- **CMake**: Updated `CMakeLists.txt` to compile the new `trunk_stream.cpp` component.

This is a baseline implementation targeting proof-of-concept compilation and validation without breaking existing workflows.

No site

Links install, modelos, releases.