Pull requests / #2209
#2209 CUDA: keep CPU expert servicing active during instrumented graph launches
open · @CC-David-CC · 0 Kommentare · Auf GitHub
Beschreibung
Compute Sanitizer can block `cudaGraphLaunch` while the verifier graph waits for CPU expert doorbells supplied by the calling thread. This is the launch/service cycle diagnosed in #1383 and #1341; #1383 added a warning. Add opt-in `STRATA_VERIFY_ASYNC_LAUNCH=1`: a separate host thread launches the CUDA graph while the caller serves its CPU requests. Join and propagate the real launch error before returning a frontier; release mapped GPU waits on an early service failure. Before/after diagnostic on RTX PRO 6000 Blackwell 96 GB, Qwen3.8 Flash Next Q8_0 / FP16 KV: - The original parent/two-sibling full-memcheck probe remained locked for **14+ hours**, with no completed forward operation. - The same probe completed in **43.6 seconds** after this scheduling fix: three exact fresh-oracle comparisons, unchanged parent bytes, four instrumented native processes and **zero sanitizer errors**. - The elapsed-time reduction comes from breaking the wait cycle. Current-main validation (`a0116b27`, RTX 4090, Linux, CUDA 13.3): - CUDA signal/cleanup/error-propagation regression passed; synchronous negative control failed after its eight-second watchdog; regression memcheck reported zero errors. - A two-token Q8_0 fixture produced byte-identical full 248,320-logit dumps on upstream main, patched default, threaded mode and memcheck. The CLI memcheck uses `--report-api-errors explicit`; default extended reporting also logged internal cuBLAS fatbinary-loading diagnostics. Minimal change: launch helper, `Verifier::run` integration, CUDA regression/CMake registration and usage documentation. The option defaults off. Scope is the CUDA single-request verifier; direct batch/pipelined entry points retain their scheduling. HIP active verifier source matches upstream; SYCL source is unchanged. G1 certification runs separately from this branch.
Mehr auf der Site
Links zu Install, Modellen, Releases.