#2743 iq_embed_rows: missing the >65535-row guard - fixed --prefill 65536 fails deterministically at a 64K chunk (invalid argument)

本文

Environment: Windows 11, RTX 4090 48GB (non-D), driver 617.42, engine/project 0.1.44.1 (source tree), NVFP4 pack (Qwen3.8-Flash-Next-NVFP4-GPTQ-Strata).

Symptom: with a FIXED `--prefill 65536`, every launch fails when the 131072-token tier is requested: the engine's last log line is `iq_embed_rows: invalid argument`, the request returns HTTP 503, and the engine stops serving (the Python server stays up). Reproduced 3/3 launches (deterministic). The same config completes every tier up to 65536 (actual 65527 tokens).

Log excerpt (run 1 of 3, tune/logs/r4-1.log; identical shape in runs 2 and 3 at lines 86 and 85):

    strata serve: prompt 65527 tokens = 0 reused + 65527 read in 12706 ms (5157.2 tok/s), 223 generated in 1651 ms (135.1 tok/s), drafts accepted 131 of 206, 1 checkpoints
    strata serve: decode expert cache hit rate: 99.3% (143966 hits / 144957 lookups); 3 more read by the GPU over PCIe or from another GPU (0.0% of all 144960 routed)
    strata serve: KV streaming: 95.89% of 1885524 block reads hit VRAM, 312.4 MiB read from RAM
    strata serve: suffix drafts: 5 windows, 19 of 21 drafts accepted
    iq_embed_rows: invalid argument

The failing request (the next tier, 131072 tokens) produced no `prompt ... read` line at all - the failure is at prompt ingestion, before any chunk progressed.

Analysis: `iq_embed_rows` launches one row per prompt token. Three sibling kernels guard the same CUDA grid-y limit of 65535 (src/kernels/cuda/qsa_select.cu:1168 `grid.y > 65535`; native_rope.cu:191,223 rows > 65535; sampler.cu:1196 tokens > 65535), but the iq_embed_rows path has no guard or row tiling - a 65536-row launch exceeds grid.y (max 65535). Consistent with this: a FIXED chunk of 61440 (below the limit) completes a full 9-tier x 3-run ladder on the same machine/config with zero occurrences of the error (prompt-path borrow 23.70 GiB vs 9988 slots at 65536, proportional).

Suggestion: add the guard + row tiling (or clamp) to the embedding upload path, like the sibling kernels.

Secondary: `--prefill auto:NN` silently clamps to 32768 for NN >= 32768 (src/program/generate.cpp:2041-2049); I first tried `auto:65536` expecting a larger chunk and got 32768 with no notice. Documenting (or reporting) the ceiling would help.

Happy to paste the full three run logs if useful.