Pull requests / #61
#61 text sampler: an empty penalty window must not touch the unsized shared bitmap
closed · @j-luwierski · 0 commentaires · Sur GitHub
Description
Follow-up from the #53 discussion (the penalties fix itself landed in v0.1.17 - this is the other thing I mentioned there). `use_bits` required only a non-null history, but the launch sizes the shared bitmap for `penalty_last_n > 0` alone. A caller handing `sample_tokens` a history buffer with the penalties disabled (e.g. a stale buffer from a previous penalised request) makes the kernel zero `bits_words` words of a **zero-byte** dynamic shared allocation. - unreachable from the serve path: `set_history` passes `nullptr` when the window is empty - invisible to memcheck: the write lands in the unallocated SMEM window - still undefined per the contract of the public API The gate now also requires a non-empty window (`hlen > 0`, both kernels), and fixture 11 pins a stale history with `last_n = 0` to the no-history result in both kernels. ## Verification - `sampler_parity --selftest`: 0 failures (fixtures 1-11) - randomised fuzz, kernels vs a serial single-pass reference (updated to the new top_p-before-min_p order): 2000 cases, 0 mismatches - `compute-sanitizer`: racecheck 0 hazards, memcheck 0 errors
Sur le site
Liens install, modèles, releases.