#3602 [Engine] Deterministic all-logits NaN (#879) (IQ3_S, CUDA, Windows)

Descripción

# [Engine] 0.1.47: #879 with direct PLE I/O; mmap passes the failing prompts

Follow-up to [#3465](https://github.com/Niko1221/Strata/issues/3465) and [#879](https://github.com/Niko1221/Strata/issues/879).

Two captured prompts still fail on engine **0.1.47** with default `--ple-io direct`: non-finite logits in the **first decode window**, after the entire prompt was read batched. The automatic retry also fails, even with **`--short-read 0`**. With **`--ple-io mmap`**, both prompts and a repeat of the 33K prompt pass the same first-output checks.

A separate diagnostic build finds non-finite values already in the **gathered/uploaded PLE embedding**, while the residual before PLE is finite. This narrows the investigation to PLE input acquisition/decoding/upload; it does not yet identify the exact faulty operation.

## Environment and configuration

- Official v0.2.2 Windows CUDA 13 engine **0.1.47**, `f-022-rel`, commit `76f90e58436d`; unmodified release executable.
- Windows 10 Pro 19045 in a Proxmox/QEMU VM, GPU passthrough.
- RTX 4070 SUPER 12 GB, sm_89, driver 610.88.
- Xeon E5-2697 v4, 16 vCPUs, 96 GiB RAM. Guest CPU flags include AVX2/FMA/F16C/BMI2, without AVX-512/VNNI.
- Existing Python server: `f57c0ad` plus a request-capture patch, **not** the v0.2.2 Python server. The engine itself reports #879 before the API returns 503.
- Model: `ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF`, **IQ3_S**, two GGUF shards and the corresponding native pack. No external refusal-direction/control vector. No STRATA diagnostic or CPU/MMQ override environment during the release tests.

The same first-decode NaN was separately reproduced on **SC117/Qwen3.8-Flash-Next-GSQ-RCO-abliterated-GGUF, IQ3_S**. Its direct-I/O failure logs and new mmap/cold-restart tests are included below. A shared exact root cause is not proven.

Relevant arguments; paths below are placeholders:

```text
--pack <iq3_s-pack> --native <shard1.gguf> --ple-gguf <shard2.gguf>
--expert-profile <expert-profile.bin> --expert-cache auto --prefill auto
--spec 4 --spec-min-p 0.5 --mtp <rt>
--max-context 262144 --kv int8 --kv-resident 32768
--vram-reserve-mib 1600 --pool-workers 8 --short-read 0 --vision
```

Vision encoding is on CPU. The candidate run adds only the explicit engine argument `--ple-io mmap`. Build and model identifiers, hashes and CPU flags are in `environment.txt`; download-receipt hashes are distinguished from hashes checked in this session.

## Release 0.1.47: what was replayed

Requests were sent sequentially. The direct sequence used one cold-start engine process; mmap used a separate cold-start process. Each request below had **zero prompt-prefix reuse**, although expert-cache/process history persists between requests in a process.

| PLE I/O | Request | HTTP / output | Total time |
|---|---|---|---:|
| direct | Short, 20 tokens | 200 / 5 tokens, `Strata works.` | 2.19 s |
| direct | Public min-wiki content from #3465, 1,717 tokens | 200 / 8 tokens | 9.09 s |
| direct | Captured user message, 33,133 tokens | 503 / 0 tokens; both attempts fail | 102.04 s |
| direct | Captured agent request, 61,244 tokens | 503 / 0 tokens; both attempts fail | 195.98 s |
| direct | Public-only Wikimedia repetition, 33,336 tokens | 200 / 8 tokens | 53.95 s |
| direct | Short after the long requests | 200 / 5 tokens, `Strata works.` | 1.31 s |
| mmap | Captured 33,133-token prompt | 200 / 8 tokens | 54.47 s |
| mmap | Captured 61,244-token prompt | 200 / 8 tokens | 98.76 s |
| mmap | Repeat of the 33,133-token prompt | 200 / 8 tokens | 52.34 s |

The 33K case is one user message, no tools, thinking disabled, prefixed with `Diagnostic 1234567890.\n`. The 61K case includes tools; streaming was disabled and its output capped. Long-request output limits were eight tokens: **these are checks for finite first output, not validation of a complete answer**. The 61K success contains reasoning tokens rather than a completed final answer.

Direct failures occur at positions **33132 / 61243**, after all **33133 / 61244** prompt tokens were read. The retry drops cached state and rereads from token zero but fails again. Error:

```text
the model's output became non-finite (NaN) twice in a row (#879);
the engine has dropped its cached state - send the request again
```

Private request bodies are excluded. The public controls pass, so they must not be presented as failing reproducers. Length alone does not explain the failure, and a failed long request does not invariably poison later short requests.

## Additional SC117 mmap validation across a restart

SC117/Qwen3.8-Flash-Next-GSQ-RCO-abliterated-GGUF IQ3_S was then tested on the **unmodified release 0.1.47**, with `--ple-io mmap`, `--short-read 0` and the other engine settings retained. No diagnostic/CPU/MMQ override environment was enabled. After the first replay set, the server was stopped and started by its tray controller; the replacement engine had a different process ID. The post-restart requests waited for READY.

| SC117 / release 0.1.47 / mmap | Prompt tokens | HTTP / output tokens | Total time |
|---|---:|---|---:|
| Short after startup | 20 | 200 / 5 | 1.55 s |
| Failing 33K prompt replay | 33133 | 200 / 8 | 51.68 s |
| Failing 61K prompt replay | 61244 | 200 / 8 | 92.14 s |
| Short after cold restart | 20 | 200 / 5 | 1.68 s |
| 33K replay after cold restart | 33133 | 200 / 8 | 51.31 s |

All five requests had zero prompt-prefix reuse. Short requests returned `Strata works.`; long cases were capped at eight output tokens, so they check finite first generation rather than full-answer quality. The same captured 33K/61K prompts were used. This extends the measured mmap workaround to SC117 and checks a further cold-start process; it still does not identify a particular I/O code defect.

## Where non-finite values first appear in the diagnostic trace

Separate local CUDA build: source **`f57c0ad` plus the attached diagnostic patch**, sm_89, Release, `STRATA_Q6K_EXPERTS=ON`, `STRATA_MMQ_KQUANTS=ON`. This is **not the release 0.1.47 binary**. `STRATA_DBG_NAN_WIN=1` and `STRATA_DBG_NAN_END=1` enabled the probes. The 33K request was its first inference after startup; it failed on both attempts, HTTP 503 after 219.51 s.

First observed bad PLE input in this run:

```text
pre-PLE residual: 0 non-finite; max finite |x| 0.522
PLE embedding: 96 non-finite; first token 4162, column 1632; max finite |x| 7.92e+06
post-PLE residual: 81920 non-finite; first token 4162
```

The embedding probe runs **after its upload, before PLE projection GEMMs**. Projections then become non-finite, followed by the layer-1 GDN input, recurrence state and MoE. Thus the downstream MoE/GDN reports do not establish those operations as the source. The trace has not yet separated disk/direct reads, row cache, decoding, host-buffer lifetime and upload.

The patch's label `PLE normalized query` scans a reused buffer after postops: at that point it holds **normalized gated values**, not the pristine query. Instrumentation adds copies/synchronizations and can change timing and failure boundaries.

## Other checks already performed

| Check | Observation and limit |
|---|---|
| Token-to-table-row mapping | Reconstructed 33,133 IDs and compared the entire stream with `Service.encode_prompt`, not just its length. |
| PLE data on disk | Reread **3,120 FP16 scales** at 39 positions around the first three bad embedding positions: **0 non-finite**, max absolute scale about **0.00033236**. This is not yet a byte-for-byte comparison of gathered CPU/GPU data with disk. |
| Dense weights | Scanned **38 stored F32/BF16/F16 tensors** in layers 0/1: **0 non-finite**. Native expert tensors and the entire PLE table were not exhaustively scanned. |
| Earlier #3465 narrowing | `--short-read 0` initially passed prompts of 1,717 / 1,759 / 11,096 / 29,425 tokens. The later 33K/61K failures show that this was a workaround for the tested tail-window cases, not a general elimination of NaN. |
| CPU parity, earlier local build | `iq_avx2_parity` and `iq_avx2_parity_iq3s_mt1`: **2/2 passed**, zero reported failures. This does not validate every inference/I/O path. |
| Earlier 0.1.46 CPU switches | `STRATA_IQ256_GATHER=0` and `STRATA_FORCE_AVX2=1` each passed a 33K replay, but a fresh-process control **without either switch also passed**. These runs do not establish a fix. |
| Earlier 0.1.46 debug scan | `STRATA_DBG_NAN=1` reproduced the failure and detected bad MoE intermediates during batched reading. Its finite GDN slot-0 scan did not rule out other slots or an earlier bad input. |
| Earlier context/vision checks | With vision and KV residency removed and context falling back to 4096, short slices passed; the 33K prompt was rejected as too large. Restoring context 262144 without KV residency could not start with that measured VRAM budget. This does **not** isolate vision or KV streaming as the cause. |
| Earlier SC117 checks | `STRATA_PREFILL_MMQ=0` did not eliminate #879. The FP16-KV attempt was interrupted/timed out and gives **no conclusive result**. These are not controlled 0.1.47 tests of the ISTA-DASLab model. |

Earlier checks used 0.1.46 or earlier local builds; they were not all rerun as a controlled 0.1.47 matrix. Small reduced triggers also changed across processes/instrumentation, so no reliable public minimal failing prompt is claimed.

## Interpretation and remaining limits

- `--ple-io mmap` is a **tested workaround for this replay set**, currently left enabled. It is not a demonstrated code fix or a guarantee for all future requests.
- Auto prefill selected **3840 tokens with direct**, **4352 with mmap**, and **3584 in the diagnostic build**. Separate processes, memory layout, history and chunk sizes are confounders; the I/O path has not been isolated with fixed chunks and repeated cold A/B runs.
- Static inspection of #3465's patch: its fallback handles NaN in a **prompt-tail window** by disabling short-read on retry. This failure is **first decode with short-read already zero**, so that fallback does not address this observed path.

## Suggested next investigation

1. **Find the first data mismatch:** compare the selected GGUF row bytes with Direct/PleReader output, decoded host floats, and uploaded GPU floats. Record the first row/block/column, expected/actual raw bits and whether values are finite. Start near token 4162 in this trace; the boundary may change between runs.
2. **Create a CPU-only row fixture:** replay the implicated row IDs through `read_row` and `gather_batch` with direct and mmap, comparing against buffered file reads. Test row-cache hits/misses, page-crossing rows, prefetch matching, completion tags and buffer reuse. These are investigation targets, not defects already proven.
3. **Control the A/B:** use the same fixed prefill chunk (for example 3840), model and arguments, multiple cold starts, and report every pass/failure. First verify that the fixed-chunk direct control still reproduces. Compare Windows storage/VM paths if a reader-level mismatch is found.
4. **Improve early diagnostics:** an opt-in finiteness check on gathered PLE input could identify the first bad row before it contaminates recurrent state and surfaces later as #879. Keep the NaN guard enabled; do not hide the error or globally change defaults based on this one machine.

Not yet done: CPU/GPU byte-for-byte row comparison, CPU-only failing fixture, fixed-chunk multi-restart A/B, an isolated MTP-off test. The attached patch is instrumentation, not a production fix; its CUDA build was tested, while HIP/SYCL builds were not tested. No exact faulty operation or patch fixing it is claimed.

Attachments: `logs.txt` (relevant excerpts), `measurements.json` (sanitized results), `environment.txt` (configuration/identity), and `diagnostic-probes.patch` (instrumentation). No private prompts, local paths, account names, IP addresses or credentials are included.


The sanitized supporting files are inlined below.

<details>
<summary>logs.txt</su