Pull requests / #2209

#2209 CUDA: keep CPU expert servicing active during instrumented graph launches

open · @CC-David-CC · 0 comments · View on GitHub

AMD / HIPNVIDIA / CUDALinux

Description

Compute Sanitizer can block `cudaGraphLaunch` while the verifier graph waits for CPU expert doorbells supplied by the calling thread. This is the launch/service cycle diagnosed in #1383 and #1341; #1383 added a warning.

Add opt-in `STRATA_VERIFY_ASYNC_LAUNCH=1`: a separate host thread launches the CUDA graph while the caller serves its CPU requests. Join and propagate the real launch error before returning a frontier; release mapped GPU waits on an early service failure.

Before/after diagnostic on RTX PRO 6000 Blackwell 96 GB, Qwen3.8 Flash Next Q8_0 / FP16 KV:

- The original parent/two-sibling full-memcheck probe remained locked for **14+ hours**, with no completed forward operation.
- The same probe completed in **43.6 seconds** after this scheduling fix: three exact fresh-oracle comparisons, unchanged parent bytes, four instrumented native processes and **zero sanitizer errors**.
- The elapsed-time reduction comes from breaking the wait cycle.

Current-main validation (`a0116b27`, RTX 4090, Linux, CUDA 13.3):

- CUDA signal/cleanup/error-propagation regression passed; synchronous negative control failed after its eight-second watchdog; regression memcheck reported zero errors.
- A two-token Q8_0 fixture produced byte-identical full 248,320-logit dumps on upstream main, patched default, threaded mode and memcheck. The CLI memcheck uses `--report-api-errors explicit`; default extended reporting also logged internal cuBLAS fatbinary-loading diagnostics.

Minimal change: launch helper, `Verifier::run` integration, CUDA regression/CMake registration and usage documentation. The option defaults off. Scope is the CUDA single-request verifier; direct batch/pipelined entry points retain their scheduling. HIP active verifier source matches upstream; SYCL source is unchanged. G1 certification runs separately from this branch.

Related on strata.com

Editorial links to help you install, pick models, or read release notes — not part of the upstream thread.