Pull requests / #1492
#1492 fix(serve): save and restore sessions without an MTP drafter
open · @Momoyeyu · 0 comentários · No GitHub
Server & APIAMD / HIPNVIDIA / CUDAWindowsLinux
Descrição
## Summary Related to #1444. This fixes a reproduced no-MTP path with the same error, not yet a confirmed reproduction of the reporter's Q6_K configuration. I asked for the missing launch arguments in the issue, so this PR intentionally does not close it. `--serve` supports running without `--mtp`, but SAVE always includes `mtp.kv_state()`. The unloaded drafter has `max_cells == 0`, so saving any nonempty conversation fails with: ``` conversation snapshot: invalid K/V extent ``` The in-memory snapshot APIs already accept an absent drafter. This makes the disk session path use the same distinction, including RESTORE's bounds and validation. ## What changed - Add nullable-draft overloads for streamed snapshot sources and session read limits; preserve the existing reference overloads. - Include the draft K/V source and its read bounds only when the drafter exists. - Pass the actual drafter presence through SAVE/RESTORE and its memory preflight. - Cover no-draft sources, byte bounds, mismatched draft presence, host-transfer restore, and a streamed disk round trip on the GPU. The file format and K/V extent checks are unchanged. Explicitly passing an uninitialized drafter still fails rather than bypassing validation. ## Test plan - [x] Reproduced the failure before the fix using the existing `CONVERSATION_TEST_TRANSFERS` fixture: `no-MTP disk save: conversation snapshot: invalid K/V extent`. - [x] After the fix, `conversation_transfer_test`, `conversation_validation_test`, and `conversation_file_test` all pass with `CUDA_VISIBLE_DEVICES=-1`. These are host-only checks, not GPU inference. - [x] Built the `strata` engine and `conversation_snapshot_test` with CUDA 13.0.88, GNU C++ 13.3.0, Release, architecture 89. - [x] Ran `conversation_snapshot_test` on an RTX 4090 D (24 GB), NVIDIA driver 595.84, Linux: **4055 checks passed**, including the added no-MTP disk round trip. - [x] `git diff --check`. ## Extra Notes The tests use synthetic session state; no full model was downloaded for this fix. The reporter's custom Q6_K pack, Windows, HIP and SYCL were not tested. The separate SYCL implementation is not changed by this PR.
No site
Links install, modelos, releases.