改动 / #3037
#3037 conversation snapshot: stage synchronous running-state copies through pinned memory (Windows/WDDM, #3036)
open · @denis-1337 · 0 评论 · 去 GitHub 看
BenchmarksServer & APIAMD / HIPNVIDIA / CUDAModels & quantsWindowsLinux
说明
Issue: Resolves #3036 ## Summary On Windows/WDDM the prompt-cache park path corrupts the model at 262144 context (all-logits NaN, the #879 symptom) and the checkpoint save itself is refused with `invalid argument`. One-file fix: stage every synchronous conversation snapshot copy through a real pinned buffer instead of letting HIP classify the heap destination. Full report with environment, reproduction and measurements: #3036. ## What changed `src/core/conversation_state.cpp` only: - `staged_copy()`: device <-> a real `cudaHostAlloc` 16 MiB buffer through the copy engine, pinned <-> destination as plain memcpy, one sync per chunk; used by default for all synchronous snapshot copies (`STRATA_CKPT_STAGE_FIRST=0` opts out for A/B). - Save calls pass the copy direction (`to_host`) explicitly instead of relying on pointer classification, which WDDM gets wrong for pageable heap. - A refused direct copy now consumes its sticky last-error (it previously surfaced as a bogus launch failure in the next `check_launch`) and retries with the direction explicit before falling back to staging. ## Extra Notes Verified on RX 9070 XT (gfx1201), Windows 11 26300, driver 32.0.32015.2008, Strata 0.1.45, `--kv int8 --max-context 262144 --expert-cache auto`, prompt cache on: - park-unique rounds: 35/41/17 non-finite events before -> 7/7 rounds `OK=11 ERR=0`, 0 non-finite after - production stress: 3 runs, 22/22 OK, 0 non-finite since boot, ~55 tok/s decode - the shipped `gfx1201-hipblaslt-100500.txt` tuning table (a second trigger of the same race) now runs clean with the patch - Linux/Metal paths untouched; the patch applies cleanly to current `main`. *Note: this issue and PR were prepared by an AI agent (Qwen3.8-Flash-Next IQ3_S, served by Strata itself) together with the account owner.*
本站相关内容
相关页面的快捷入口。