Pull requests / #3037

#3037 conversation snapshot: stage synchronous running-state copies through pinned memory (Windows/WDDM, #3036)

open · @denis-1337 · 0 comments · View on GitHub

BenchmarksServer & APIAMD / HIPNVIDIA / CUDAModels & quantsWindowsLinux

Description

Issue: Resolves #3036

## Summary
On Windows/WDDM the prompt-cache park path corrupts the model at 262144 context (all-logits NaN, the #879 symptom) and the checkpoint save itself is refused with `invalid argument`. One-file fix: stage every synchronous conversation snapshot copy through a real pinned buffer instead of letting HIP classify the heap destination. Full report with environment, reproduction and measurements: #3036.

## What changed
`src/core/conversation_state.cpp` only:
- `staged_copy()`: device <-> a real `cudaHostAlloc` 16 MiB buffer through the copy engine, pinned <-> destination as plain memcpy, one sync per chunk; used by default for all synchronous snapshot copies (`STRATA_CKPT_STAGE_FIRST=0` opts out for A/B).
- Save calls pass the copy direction (`to_host`) explicitly instead of relying on pointer classification, which WDDM gets wrong for pageable heap.
- A refused direct copy now consumes its sticky last-error (it previously surfaced as a bogus launch failure in the next `check_launch`) and retries with the direction explicit before falling back to staging.

## Extra Notes
Verified on RX 9070 XT (gfx1201), Windows 11 26300, driver 32.0.32015.2008, Strata 0.1.45, `--kv int8 --max-context 262144 --expert-cache auto`, prompt cache on:
- park-unique rounds: 35/41/17 non-finite events before -> 7/7 rounds `OK=11 ERR=0`, 0 non-finite after
- production stress: 3 runs, 22/22 OK, 0 non-finite since boot, ~55 tok/s decode
- the shipped `gfx1201-hipblaslt-100500.txt` tuning table (a second trigger of the same race) now runs clean with the patch
- Linux/Metal paths untouched; the patch applies cleanly to current `main`.

*Note: this issue and PR were prepared by an AI agent (Qwen3.8-Flash-Next IQ3_S, served by Strata itself) together with the account owner.*

Related on this site

Quick links to related pages.