#3364 [Engine] expert-cache auto does not reserve memory for --batch slots; MTP allocation fails and the engine exits with code 1

Beschreibung

# Strata issue draft — `expert-cache auto` ignores `--batch` slot sessions
## Title

`expert-cache auto` does not reserve memory for `--batch` slot sessions; MTP allocation then fails and the engine exits with code 1

## Summary

With a long context (`--max-context 1048576`) and `--batch 4`, Strata's
`expert-cache auto` fills the expert cache first. The four batch slot sessions
need ~1.27 GiB each per card, but by the time they are allocated each card has
0.00 GiB free. The engine then falls back to 1/0 slot sessions, and the
following MTP allocation fails:

```text
mtp: dense weights allocation failed (unspecified launch failure), requested 110 MiB, CUDA0 free unknown
```

The serve layer reports:

```text
RuntimeError: the engine exited before it was ready
```

and the process exits with code **1**. The engine should either reserve the
batch-slot memory before sizing the expert cache, or degrade safely instead of
exiting.

## Environment

- Strata: `0.1.45.3` Windows HIP build
  (`strata-engine-0.1.45.3-windows-x86_64-hip-rx9070.tar.gz`,
  release tag `v0.2.0-beta`)
- OS: Windows 11
- GPUs: 2× AMD Radeon AI PRO R9700 (gfx1201), 32 GB each
- RAM: 64 GB class (63.4 GiB visible)
- Model: Qwen3.8-Flash-Next GSQ-RCO-Abliterated IQ3_S, native context 262144
- KV: `--kv int8`, `--kv-resident 32768`
- Layer split: `--layer-split auto`
- Experts: `--expert-cache auto --mmap-experts`
- MTP: enabled (`--mtp <strata-mtp/rt> --mtp-hnorm stream`)
- Vision: `--vision`
- Serve config: `"parallel": 4` → engine gets `--batch 4`

The model path is omitted here; full local logs can be attached.

## Exact engine arguments

```text
strata.exe --serve
  --pack <model>/strata-pack/IQ3_S
  --native <model>/...-00001-of-00002.gguf
  --ple-gguf <model>/...-00002-of-00002.gguf
  --expert-profile <model>/strata-pack/IQ3_S/expert-profile.bin
  --expert-cache auto
  --prefill auto
  --spec 4 --spec-min-p 0.5
  --kv-resident 32768
  --mtp <model>/strata-mtp/rt
  --mtp-hnorm stream
  --max-context 1048576
  --rope-scaling yarn --rope-scale 4 --yarn-orig-ctx 262144
  --kv int8
  --ple-io mmap
  --vram-reserve-mib 500
  --vram-reserve-later-mib 500
  --layer-split auto
  --mmap-experts
  --vision
  --batch 4
```

`--batch 4` comes from the serve config's `"parallel": 4`.

## Steps to reproduce

1. Start the above configuration with the long-context model and `"parallel": 4`.
2. Wait for model loading / layer-split selection to finish.
3. Observe the batch-slot allocations and the final MTP failure.

## Expected behavior

`expert-cache auto` should account for the memory needed by the requested
`--batch N` slot sessions **before** filling the expert cache, or it should
reduce the cache until the slots fit. If the requested batch size still cannot
fit, it should:

- fall back to fewer slots (or serial) with a clear warning, and
- keep MTP loading if only the batch-size request is impossible.

It should not terminate the engine with exit code 1.

## Actual behavior

Relevant engine log, in order:

```text
strata generate: layer split: CUDA1 holds its weights, session [26, 48) and the head; 24.58 GiB free
strata generate: --batch-groups auto: 2 groups of 2 slots
strata generate: WARNING: --batch 4: slot 1's session does not fit on CUDA0 (1.27 GiB each, 0.00 GiB free); 1 slot
strata generate: --batch: 1 slot sessions on CUDA0 (1.27 GiB each); 0.00 GiB free
strata generate: WARNING: --batch 4: slot 0's session does not fit on CUDA1 (1.26 GiB each, 0.00 GiB free); 0 slots
strata generate: --batch: 0 slot sessions on CUDA1 (1.26 GiB each); 0.00 GiB free
strata generate: WARNING: --batch is off: not two slot sessions fit (a smaller --max-context or KV streaming, --kv-resident, makes them smaller)
strata generate: mtp: dense weights allocation failed (unspecified launch failure), requested 110 MiB, CUDA0 free unknown
```

The serve layer then reports:

```text
RuntimeError: the engine exited before it was ready
```

Process result: exit code `1`.

## Why this looks like an engine-side accounting/ordering bug

The expert cache is sized before the `--batch` slots are accounted for:

1. `expert-cache auto` computes and fills the cache from the currently free VRAM.
2. Only afterwards does the engine try to allocate the `--batch 4` slot sessions.
3. Each slot needs ~1.27 GiB per card at this context; zero memory is left.
4. The batch silently degrades to 1/0 slots.
5. MTP then fails its much smaller 110 MiB allocation, terminating the process.

The slot sessions are real VRAM consumers and are also visible in the engine's
own log; they should be part of the auto-cache calculation.

## Contrast: same engine at a shorter context

At `--max-context 65536` with `--batch 2`, the same model starts successfully:

```text
strata generate: KV streaming: 32768 of 65536 cells per QSA layer in VRAM, the K/V in 0.39 GiB of pinned RAM
strata generate: --batch-groups auto: 2 groups of 1 slot
strata generate: --batch: 2 slot sessions on CUDA0 (0.32 GiB each); 25.38 GiB free
strata generate: --batch: 2 slot sessions on CUDA1 (0.32 GiB each); 24.90 GiB free
```

So the mechanism works when slot sessions fit; the problem is that auto cache
does not leave room for them at long context.

## Possible fixes

- Subtract `batch_slots × per_slot_session` before computing the expert cache.
- Cap `--batch` to the number of slot sessions that fit after reserving MTP/vision,
  and emit a warning instead of failing MTP.
- If the requested `--batch` cannot fit, fall back to serial or a smaller batch
  rather than exiting.
- Alternatively, document that `--batch N` must be lowered manually for long
  contexts; currently the failure mode is an engine exit, not a usable warning.

## Attachments to include

- `Base/logs/server-20261020-120736.log` (full serve traceback)
- relevant excerpt of `Base/logs/strata-serve.log` around the lines quoted above
- the exact `strata-config.console.json` with `"parallel": 4`

---