Pull requests / #61

#61 text sampler: an empty penalty window must not touch the unsized shared bitmap

closed · @j-luwierski · 0 コメント · GitHub で見る

Server & API

本文

Follow-up from the #53 discussion (the penalties fix itself landed in v0.1.17 - this is the other thing I
mentioned there).

`use_bits` required only a non-null history, but the launch sizes the shared bitmap for `penalty_last_n > 0`
alone. A caller handing `sample_tokens` a history buffer with the penalties disabled (e.g. a stale buffer
from a previous penalised request) makes the kernel zero `bits_words` words of a **zero-byte** dynamic shared
allocation.

- unreachable from the serve path: `set_history` passes `nullptr` when the window is empty
- invisible to memcheck: the write lands in the unallocated SMEM window
- still undefined per the contract of the public API

The gate now also requires a non-empty window (`hlen > 0`, both kernels), and fixture 11 pins a stale
history with `last_n = 0` to the no-history result in both kernels.

## Verification

- `sampler_parity --selftest`: 0 failures (fixtures 1-11)
- randomised fuzz, kernels vs a serial single-pass reference (updated to the new top_p-before-min_p order):
  2000 cases, 0 mismatches
- `compute-sanitizer`: racecheck 0 hazards, memcheck 0 errors

関連リンク

インストール・モデル・リリースへの站内リンク。