#2745 [Bug]: NVFP4 2-GPU layer split + STRATA_PREFILL_HELP=1: deterministic non-finite logits (#879) on 1024..3276-token single-chunk prompts (PeerPrefill omits tail_off scales)

Descrição

### What happened

When running an NVFP4 pack (`Maximilian228--Qwen3.8-Flash-Next-NVFP4-GPTQ-Strata`) with a 2-GPU layer split (`--layer-split 23`), `--prefill auto` (chunk 8192), and `STRATA_PREFILL_HELP=1`, every single-chunk prompt segment whose uncached length `T` falls in `1024 <= T <= 3276` deterministically fails with `non-finite logits (#879) in a prompt window`.

When `#879` fires:
1. The engine drops the live session and all KV checkpoints (`after non-finite logits (#879): dropped the live session, 6 checkpoints ... reading this prompt from token 0`).
2. `server.py` automatically retries the request from token 0.
3. On the retry from token 0, the full prompt (`13K` to `86K` tokens) spans multiple 8192-token chunks (`single_chunk == false`), so `bind_stage_helper` does not activate and the retry succeeds after re-reading the entire 13K–86K prompt from scratch.

In a single multi-turn coding/agent session (growing from 11K to 86K tokens), this triggered **9 times** (at prompt deltas of `1189`, `1292`, `1292`, `1297`, `1738`, `1288`, `1289`, `1289`, `1296` tokens — every single one in `[1024, 3276]`), wiping the KV cache each time. Setting `STRATA_PREFILL_HELP=0` completely resolves the issue.

#### Root cause in `src/prefill/prefill.cpp`

1. **Activation condition (`src/prefill/prefill.cpp:1571-1579, 1624-1633, 2125-2126`)**:
   - `Prefill::bind_stage_helper(int64_t T)` activates only when `single_chunk` (`n <= m.T`), `T >= stream_all_min()` (`1024`), and `split_help_frac(T) >= 0.3` (`T <= 3276`).
   - `bind_stage_helper` checks `mmq_plan()` and `fused::native_supported()`, but does **not** check `lay.fmt[l].tail_off != 0`.
2. **Missing NVFP4 `tail_off` global scales in `PeerPrefill` (`src/prefill/prefill.cpp:3626-3657` and `3707-3732`)**:
   - On NVFP4 packs, each expert blob carries a 16-byte tail `{s_gate, s_up, s_down, 0}` (`f.tail_off != 0`, each scale `~1e-4`).
   - The primary card's MMQ path (`src/prefill/prefill.cpp:3888-3930`) gathers `f.tail_off` into `m.grp_tail`, runs `mmq::swiglu_scaled` (applying `s_gate` and `s_up`), and writes `s_down` into `m.row_sd` via `mmq::down_row_scales` before `moe_combine` (`line 4055-4056`) multiplies all `T * K` rows by `m.row_sd`.
   - However, `PeerPrefill` (`P.compact` at lines `3626-3657` for `STRATA_PREFILL_HELP=1`, and `!P.compact` at lines `3707-3732` for `--peer-device`) calls `mmq::gather_native` **without** `f.tail_off` / `grp_tail`, runs unscaled `mmq::swiglu` instead of `mmq::swiglu_scaled` (leaving hidden activations `~1e8x` too large), and **never writes `m.row_sd[rows_local .. rows_local + rows_peer)`** (leaving those entries uninitialized/stale when `moe_combine` reads `m.row_sd` across all `T * K` rows).

### Strata version, GPU, OS

0.1.44.1 · RTX 4070 Ti 12GB (CUDA0) + RTX 2080 Ti 22GB (CUDA1) · Debian 13 Linux 6.12.111 (Driver 615.71.09, CUDA 13.1)

### Engine log

```text
[strata] 2026-10-16 16:08:17 engine started: .../strata --serve --pack .../pack --native .../qwen-nvfp4-gptq-dense.gguf --native-dense-gguf .../qwen-nvfp4-gptq-dense.gguf --ple-gguf .../ple-fp8.gguf --embd-gguf .../token-embd-bf16.gguf --expert-profile .../expert-profile.bin --expert-cache auto --prefill auto --spec 4 --mtp .../mtp/rt --max-context 524288 --rope-scaling yarn --rope-scale 2 --kv int8 --resident-budget-gib 48 --vision --vram-reserve-mib 760 --spec-min-p 0.5 --pool-workers 4 --adapt-every 1 --adapt-swaps 80 --adapt-decay 0.97 --pipeline-windows 2 --vram-reserve-later-mib 850 --kv-resident 65536 --layer-split 23
strata generate: layer split across 2 GPUs: CUDA0, then CUDA1 (split 23)
strata generate: expert cache 1975 slots, 5.09 GiB of VRAM
strata generate: layer split: CUDA1 runs layers 23-47, expert cache 5132 slots (13.22 GiB), 5132 of its 12800 profiled pairs; slot 0 verified
strata generate: resident RAM mode: 48.00 GiB of experts in RAM (page-locked), 1975 in the GPU cache; adaptive swaps exchange them with the GPU cache (no file reads)
strata serve: prompt chunk auto: 8192 tokens, a 96-slot ring
strata serve: layer split: layers 0-22 (CUDA0), 23-47 (CUDA1), one hand-off per window
...
# Multi-chunk (>8192) and short (<1024) prompt deltas succeed:
strata serve: prompt 11202 tokens = 0 reused + 11202 read in 40444 ms (277.0 tok/s), 192 generated in 4605 ms (41.7 tok/s), drafts accepted 121 of 165, 2 checkpoints
strata serve: prompt 12253 tokens = 11393 reused + 860 read in 7358 ms (116.9 tok/s), 223 generated in 4683 ms (47.6 tok/s), drafts accepted 139 of 211, 3 checkpoints

# Every single-chunk prompt delta in 1024..3276 triggers STRATA_PREFILL_HELP -> #879 non-finite logits:
strata serve: non-finite logits (#879) in a prompt window at position 13660 (row 0): its tokens are not emitted, the request ends (DONE nonfinite) and the next request reads its prompt from token 0
strata serve: prompt 13665 tokens = 12476 reused + 1184 of 1189 read in 10245 ms (115.6 tok/s), 0 generated in 0 ms (0.0 tok/s), drafts accepted 0 of 0, 4 checkpoints (cancelled)
strata serve: after non-finite logits (#879): dropped the live session, 4 checkpoints, 0 parked conversations and 0 idle slots (0 running slots not kept after their reply); reading this prompt from token 0
strata serve: prompt 13665 tokens = 0 reused + 13665 read in 33361 ms (409.6 tok/s), 18633 generated in 326870 ms (57.0 tok/s), drafts accepted 11628 of 16271, 2 checkpoints
...
strata serve: non-finite logits (#879) in a prompt window at position 34089 (row 0): its tokens are not emitted, the request ends (DONE nonfinite) and the next request reads its prompt from token 0
strata serve: prompt 34094 tokens = 32802 reused + 1287 of 1292 read in 10782 ms (119.4 tok/s), 0 generated in 0 ms (0.0 tok/s), drafts accepted 0 of 0, 5 checkpoints (cancelled)
strata serve: after non-finite logits (#879): dropped the live session, 5 checkpoints, 0 parked conversations and 0 idle slots (0 running slots not kept after their reply); reading this prompt from token 0
strata serve: prompt 34094 tokens = 0 reused + 34094 read in 57325 ms (594.8 tok/s), 1786 generated in 35842 ms (49.8 tok/s), drafts accepted 988 of 1420, 3 checkpoints
...
strata serve: non-finite logits (#879) in a prompt window at position 42413 (row 0): its tokens are not emitted, the request ends (DONE nonfinite) and the next request reads its prompt from token 0
strata serve: prompt 42418 tokens = 41126 reused + 1287 of 1292 read in 10535 ms (122.2 tok/s), 0 generated in 0 ms (0.0 tok/s), drafts accepted 0 of 0, 6 checkpoints (cancelled)
strata serve: after non-finite logits (#879): dropped the live session, 6 checkpoints, 0 parked conversations and 0 idle slots (0 running slots not kept after their reply); reading this prompt from token 0
strata serve: prompt 42418 tokens = 0 reused + 42418 read in 65016 ms (652.4 tok/s), 828 generated in 15942 ms (51.9 tok/s), drafts accepted 475 of 661, 4 checkpoints
...
strata serve: non-finite logits (#879) in a prompt window at position 64415 (row 0): its tokens are not emitted, the request ends (DONE nonfinite) and the next request reads its prompt from token 0
strata serve: prompt 64420 tokens = 62682 reused + 1733 of 1738 read in 10901 ms (159.0 tok/s), 0 generated in 0 ms (0.0 tok/s), drafts accepted 0 of 0, 6 checkpoints (cancelled)
strata serve: after non-finite logits (#879): dropped the live session, 6 checkpoints, 0 parked conversations and 0 idle slots (0 running slots not kept after their reply); reading this prompt from token 0
strata serve: prompt 64420 tokens = 0 reused + 64420 read in 93249 ms (690.8 tok/s), 39 generated in 916 ms (42.6 tok/s), drafts accepted 27 of 37, 5 checkpoints
...
strata serve: non-finite logits (#879) in a prompt window at position 86399 (row 0): its tokens are not emitted, the request ends (DONE nonfinite) and the next request reads its prompt from token 0
strata serve: prompt 86404 tokens = 85108 reused + 1291 of 1296 read in 10902 ms (118.4 tok/s), 0 generated in 0 ms (0.0 tok/s), drafts accepted 0 of 0, 6 checkpoints (cancelled)
strata serve: after non-finite logits (#879): dropped the live session, 6 checkpoints, 0 parked conversations and 0 idle slots (0 running slots not kept after their reply); reading this prompt from token 0
```