#3287 [Engine] Metal: the #879 guard reads the argmax sentinel where the device row check cannot run

本文

## Problem and result

On the Metal engine a non-finite logits row (#879) reaches the user as a crash instead of a recovery. The #879 guard reads per-window flags that `logits_nonfinite_rows` raises on the device beside the greedy pick, but the Metal port has no kernel of that name — it is a trap stub (`metal/src/engine/stubs.cpp`) — so the flags stay zero and `nonfinite_hit` reports a clean window. What a row that is *all* NaN leaves behind is Metal's sampler sentinel: `metal/kernels/sample.metal` seeds the argmax at `bi = 0x7fffffff` and skips NaN elements, so the row keeps the sentinel, `pl_finish` copies it out, and the engine prints `T 2147483647`. The serve loop then dies twice: the detokenizer raises a raw `IndexError: token id 2147483647 is outside the vocabulary (248320)` and kills the request thread, and the engine feeds the sentinel into the drafter and the embedding and segfaults (exit -11) shortly after.

Measured on an M1 Max (macOS, engine built from `main` at `e88ee39d`, Qwen3.8 vocabulary 248320), reproduced when the machine is under enough memory pressure for a row to go NaN.

After this change the sentinel takes the existing #879 recovery, on both sides:
- the engine marks that row non-finite at the pick read, emits none of the window's tokens, and answers `DONE … nonfinite` — the cached state is dropped before the sentinel can reach the drafter, so the follow-on segfault goes away;
- the server raises `EngineNonFinite` if a sentinel id appears in the token stream anyway (an engine binary without this fix), so the request gets the normal path: retried once when nothing went out yet, ends as `"length"` mid-reply, a clean error after a second pass — never a traceback.

## Approach

`Verifier::note_nonfinite` takes the picks that were just read beside it and marks the row of any pick holding `INT32_MAX`. That reading is true on every platform: the id is past every vocabulary, so a pick that holds it is exactly the fact #879 records. All five pick-read sites pass their buffer (`run`, `run_slot_rows`, `run_slots`, `batch_poll`, `pl_finish`). Where the device check already runs, the scan can only agree with it; on Metal it is the whole guard.

The server's arm is one test in `_run` before the detokenizer, raising the same `EngineNonFinite` the `DONE … nonfinite` line raises, so it needs no new recovery code.

## Validation

Passed:
- `python3 -m unittest serve.test_server.NonFiniteLogits serve.test_server.SentinelToken` — 7/7 on this branch off `main` (`d986ecbc`). Two new mock tests: a sentinel before the first token is retried once and answers normally; a sentinel mid-reply ends the reply as `"length"` with the text up to it. The five existing #879 tests still pass.
- Metal engine builds clean; `ctest` 2/2 (`runtime_hello`, `window_cells_test`) on the M1 Max.

Not run:
- the sentinel path with the real model on Metal: it needs an all-NaN row, which only appeared under memory pressure and cannot be provoked on demand (and `STRATA_NAN_INJECT` reports the row through the flags, beside this path). The engine-side arm is therefore compile-and-static evidence; the server-side arm is proven by the tests and by this machine's own crash tracebacks.

Failed: none.

## Limits

- A row with *some* NaN logits still samples its best finite pick on Metal (the sampler skips NaN per element); only an all-non-finite row leaves the sentinel, and that is the case this guard catches.
- The default CUDA/HIP path keeps its byte-identical behaviour unless a pick holds `INT32_MAX`, which on those platforms the device check already reports.