#6543 [Engine] Guard negative page_table entries in qsa_prompt_attn and track monotonic kv_stream overflow_count

open · @YusenZhang0601 · 0 评论 · 去 GitHub 看

AMD / HIPNVIDIA / CUDAModels & quantsWindowsLinux

说明

## Problem and result

This small, focused patch addresses two defensive and diagnostic gaps in the QSA CUDA/HIP kernels:

1. **Unconfirmed static analysis — inconsistent `page >= 0` guards in `qsa_prompt_attn.cu`**:
   In `engine/src/kernels/cuda/qsa_prompt_attn.cu`, one `page_table` lookup (the HIP WMMA path at line 2034–2035) already checks `if (page >= 0)` before computing the physical pool row index, matching the decode attention kernels (`qsa_decode_attn.cu`). However, the other seven `page_table` lookups in `prompt_attn_kernel`, `prompt_attn_i8_kernel`, `decode_attn_i8_split_kernel`, `prompt_attn_i8v3_kernel`, and `prompt_attn_v70_kernel` (lines 174, 211, 603, 866, 1218, 1574, 1821) compute `(page * n_kv_heads + kvh) * page_size + (cell % page_size)` without checking `page >= 0`. If a streamed `page_table` ever contains `-1` (for example after a `kv_stream` slot overflow), those paths would compute a negative row offset rather than returning `-1` (masked). I did not reproduce a crash on upstream `main`; this part is **unconfirmed static analysis** for defensive consistency with line 2034 and `qsa_decode_attn.cu`.
2. **Observed sticky-boolean limitation in `kv_stream` overflow diagnostics**:
   In `engine/src/kernels/cuda/kv_stream.cu`, `resolve_kernel` records an overflow as a sticky boolean (`if (placed < need) m.ctl[3] = 1;`). When testing low-residency streamed KV (`--kv-resident 4096 --spec 4` on a 96K context), a multi-token decode step overflowed and latched `m.ctl[3] = 1` permanently until `kv_stream_reset`. Any subsequent caller reading `kv_stream_counters(m)` could not distinguish a historical decode overflow from a new overflow in the current call. Changing line 159 to `m.ctl[3] += 1;` and exposing `uint32_t overflow_count` in `KvStreamCounters` enables before/after delta checks while keeping `r.overflow = c[3] != 0;` fully backwards-compatible.

## Related discussion

none (small defensive/diagnostic fix with no default behavior or public CLI change).

## Approach

- **`engine/src/kernels/cuda/qsa_prompt_attn.cu`**: Add `page >= 0` checks to the remaining 7 `page_table` lookups so every lookup returns `-1` (already handled by every caller's row-masking path) when `page < 0`, matching line 2034–2035.
- **`engine/include/strata/kernels/kv_stream.hpp`** & **`engine/src/kernels/cuda/kv_stream.cu`**: Increment `m.ctl[3] += 1` instead of assigning `1` when `placed < need`, and populate `KvStreamCounters::overflow_count = (uint32_t) c[3]` alongside the unchanged `bool overflow = c[3] != 0;`.
- **`engine/src/kernels/kv_stream_parity.cpp`**: Assert `c.overflow_count == 0` alongside `c.overflow == false` in the existing parity test.
- **Compatibility**: Zero change to default behavior, memory layout (`kKvCtlInts` unchanged), or any existing caller reading `KvStreamCounters::overflow`.

## Validation

**Passed:**
- `origin/main` (`6e38f588a`) + `2e27acdd7` (`strata 0.1.64`, Windows x64, MSVC `14.37.32822`, CUDA `13.0.48`, `sm_86`, CMake + Ninja `-DSTRATA_BUILD_TESTS=ON`):
  - `ninja strata kv_stream_parity`: **PASS** (`32.5 s`, `strata --version` -> `strata 0.1.64`).
  - `kv_stream_parity` on RTX A6000 (`sm_86`, `rc=0`, `45.3 s`):
    ```text
    kv_stream_parity: streamed vs resident KV, bitwise
      int8: 306 batches, 702674 block lookups, 69720 misses (90.1% hit), overflow 0, 0 failures
      int8 ring restore: identical
      fp16: 306 batches, 678052 block lookups, 69552 misses (89.7% hit), overflow 0, 0 failures
      fp16 ring restore: identical
      q4_0: 306 batches, 684778 block lookups, 72149 misses (89.5% hit), overflow 0, 0 failures
      q4_0 ring restore: identical
      k8v4: 306 batches, 705100 block lookups, 73180 misses (89.6% hit), overflow 0, 0 failures
      k8v4 ring restore: identical
    PASS
    ```
- Historical fork verification (`0.1.56` fork `YusenZhang0601/[email protected]`, commit `a1c2c904`, RTX A6000 48 GB `sm_86`, Windows WDDM, Qwen3.8-Flash-Next IQ3_XXS 262K INT8 KV): under `--kv-resident 4096 --spec 4` after a 96K prime decode that latched `ctl[3]`, before/after `overflow_count` delta cleanly confirmed `ovf_delta == 0` for single-query prefill resolves while preserving decode's sticky `overflow` reporting.

**Failed:**
- none observed

**Not run:**
- HIP, SYCL, Metal, and Linux builds (I only have a Windows RTX A6000 CUDA 13 environment available for native engine compilation and GPU execution).
- Full 24,574-row `kvq` numerical sweep (not applicable; no numerical computation changes on valid `page >= 0` entries).

## Limits

- The `qsa_prompt_attn.cu` `page >= 0` guards are defensive consistency fixes identified by static inspection against line 2034; I have not constructed a dedicated fault-injection GPU test to trigger a negative `page_table` entry inside `qsa_prompt_attn.cu`.
- `m.ctl[3]` remains an `int32_t` counter reset by `kv_stream_reset`; it is not intended to survive more than `2^31 - 1` overflows within a single session without reset.

本站相关内容

相关页面的快捷入口。