#3281 [Engine] add request-scoped prefill VRAM loans

Beschreibung

This the rebased version of https://github.com/Niko1221/Strata/pull/1581 for the VRAM loaning part of the PR.
Reopened PR of https://github.com/Niko1221/Strata/pull/3105 as I couldn't use /reopen.

## Problem and result

Current main can borrow expert-cache storage for prompt buffers, and
`--prefill-head-alias` can reuse the native output-head allocation. On a tight
8 GiB serve configuration, however, verifier, native-head, and MTP decode
resources still prevent the automatic planner from selecting a large prompt
geometry.

This PR adds a manual request-scoped transaction that releases those
decode-only resources during prefill, grows the elastic expert cache, and
restores all resources and runtime state before generation. It also supports
repeating the transaction for later long messages.

On an RTX 2080 Super Max-Q, enabling the loan in the same binary changed the
automatic plan from a 2,304-token chunk to 5,376 and increased median prefill
from 513.6 to 774.8 tokens/s (+50.9%). Median generation was 30.9 tokens/s in
both arms. The default path is unchanged when the master switch is absent.

## Approach

Example full configuration:

```json
{
  "STRATA_PREFILL_ELASTIC_LOAN": "1",
  "STRATA_PREFILL_MTP_LOAN": "1",
  "STRATA_PREFILL_HEAD_LOAN": "1",
  "STRATA_PREFILL_RETAIN_STARTUP_CHUNK": "1",
  "STRATA_PREFILL_REPEAT_LOAN": "1",
  "STRATA_PREFILL_REPEAT_MIN_TOKENS": "50000"
}
```

All six variables are introduced here, but they are not six independent
requirements:

| Variable | Purpose |
| --- | --- |
| `STRATA_PREFILL_ELASTIC_LOAN` | Master opt-in; required |
| `STRATA_PREFILL_MTP_LOAN` | Also defer MTP routed experts and draft head |
| `STRATA_PREFILL_HEAD_LOAN` | Also defer the native output head and verifier |
| `STRATA_PREFILL_RETAIN_STARTUP_CHUNK` | Keep the startup planner geometry for the first request |
| `STRATA_PREFILL_REPEAT_LOAN` | Repeat the transaction for later substantial prompts |
| `STRATA_PREFILL_REPEAT_MIN_TOKENS` | Unread-token threshold for repeats; default 1,024, minimum 256 |
| `--vram-elastic`, `--serve`, `--mtp`, `--spec` | Existing upstream options used by the transaction; not introduced here |

The existing `--vram-elastic` option supplies the segmented expert-cache
backend. Startup loans cannot use a prompt-length threshold because the first
request is not known at initialization; the repeat threshold applies only to
later prompts.

Current main already loans expert-cache slots to prompt workspace. This PR
extends the lifetime transaction to resources that are needed only for decode:
the verifier arena/graphs, native output head, MTP routed experts, and MTP draft
head. Reload restores PCIe balancing, adaptive-overlap gating, residency
pre-launch synchronization, logit bias, and MTP draft sampling before decode.

## Safety and supported configurations

The feature is admitted only on the serialized, one-GPU NVIDIA CUDA serve path.
It requires `--serve`, `--vram-elastic`, and CUDA virtual-memory segments of at
least 64 MiB. Startup rejects batch slots, pipeline windows, layer split,
helper/remote/peer caches, HIP, and `--prefill-head-alias`. MTP/head loans
require `--mtp` with `--spec >= 2`; the head loan also requires a native output
head. Invalid combinations fail before resources are deferred.

Multiple conversations are supported when requests are processed sequentially
by one server. Concurrent batch sessions are deliberately rejected. Separate
Strata processes sharing a GPU are not coordinated and must not take loans
concurrently.

The transaction does not change the routing algorithm. A larger cache and a
different chunk geometry can change which experts execute on GPU and the order
of floating-point operations, so generated tokens are not promised to be
bit-identical to a loan-off run.

## Validation

Tested revision: `cf14700f`, based on upstream `e88ee39d` (0.2.0-beta). The
combined build used this branch plus the separate SM75 native-IQ branch at
`db02a533`.

### Passed

- Windows Release CUDA production build with CUDA 13.4, MSVC 19.51, and SM75
  SASS.
- Three fresh-engine end-to-end runs per configuration, serialized on the same
  machine. The production server was stopped during each run and restored and
  health-checked afterward.
- A default-off A/B against the exact upstream executable: the combined
  rebased binary with every new flag absent measured 513.6 PP median versus
  514.6 PP for current main (-0.2%, within the run ranges), and selected the
  same 2,304/226 plan.
- A later-message lifecycle run: after a short warm-up request, a 31,258-token
  prompt released verifier, native-head, and MTP resources; retried smaller
  geometries when necessary; restored all resources; and generated all 256
  requested tokens without a CUDA/runtime failure.

### Failed

- The repository-wide test build did not complete on Windows because current
  upstream `src/kernels/gdn_chunk_parity.cpp` calls POSIX `setenv`. The
  production target compiles and the end-to-end loan paths run; that unrelated
  current-main test compile failure remains.

### Not run

- HIP and SYCL builds.
- Multi-GPU, batch, pipeline, helper, remote-cache, and peer-cache execution;
  the feature rejects these configurations at startup.
- Other GPU generations after this current-main rebase.
- Full visible-answer parity: the throughput requests reached their 256-token
  reasoning cap, so these runs validate lifecycle and rates, not complete
  answer equivalence.

## Benchmark configuration

- GPU: NVIDIA GeForce RTX 2080 Super Max-Q, 8 GiB, 105 W, compute capability
  7.5.
- CPU/RAM: Intel i7-10875H, 64 GiB RAM.
- PCIe host-to-device probe: approximately 12.2--12.3 GB/s.
- Windows, CUDA 13.4, MSVC Release build with SM75 SASS.
- Model: Qwen3.8-Flash-Next IQ3_XXS native GGUF plus IQ3_XXS expert pack.
- Runtime: 65,536 context, Q4_0 KV, 24,576 resident KV cells, MTP `--spec 4`,
  `--short-read 0`, experimental speed projection, `STRATA_BF16_TC=1`, and
  `STRATA_PROMPT_ATTN_Q4_TC=1` held constant.
- Request: 31,258 fresh prompt tokens, 256 generated tokens, fresh process for
  every repetition. Values below are medians of three engine timing lines.

## Results

| Configuration | Prompt chunk / ring | PP median (range) | TG median (range) |
| --- | ---: | ---: | ---: |
| Exact current-main executable | 2,304 / 226 | 514.6 (513.6--516.1) | 29.9 (29.7--30.0) |
| Combined rebased executable, every new flag absent | 2,304 / 226 | 513.6 (512.8--514.5) | 30.9 (30.7--31.0) |
| Loan flags enabled; SM75 native-IQ extension off | 5,376 / 227 | **774.8** (772.7--779.9) | **30.9** (30.2--31.9) |
| Loans plus [separate SM75 native-IQ extension](https://github.com/Niko1221/Strata/pull/3104) | 4,864 / 512 | 859.4 (859.1--860.5) | 30.9 (30.7--32.6) |

The same-binary isolated loan effect in this workload is +50.9% PP. The
combined row is included to show composition, not attributed to this PR alone;
the [SM75 native-IQ PR](https://github.com/Niko1221/Strata/pull/3104) adds a
separate +10.9% over loans, and the combined gain over current main is +67.0%.

For comparison, current main with `--prefill-head-alias` selected a
1,536-token chunk and measured 370.3 PP median on this exact workload. That is
not a general claim against head alias; it explains why its existing narrower
loan did not replace this transaction on this memory-constrained setup.

The later-message lifecycle run selected 4,352 tokens after bounded retries,
completed prefill at 841.0 PP, restored the deferred resources, and generated
at 28.0 TG. Compared with the 30.9 TG fresh-request median, that is a 9.4%
reduction and remains inside the 10% decode-regression limit used for this
experiment.

## Additional qualification evidence

### Fixed-geometry mechanism check

A matched 2,304/227 experiment held prompt geometry constant to distinguish
the transaction cost from the planner benefit:

| Configuration | Prompt plan | PP | TG |
| --- | ---: | ---: | ---: |
| Loan flags off | 2,304 / 227 | 424.7 | 29.8 |
| Loan flags on, same plan | 2,304 / 227 | 425.1 | 29.3 |

The +0.1% PP difference is noise. The loan is not a faster kernel; its value is
allowing a larger cache/chunk geometry to fit. Benefit therefore depends on
prompt length and on whether releasing the decode resources crosses a useful
planner threshold.

### Cross-GPU loan qualification

The transaction is not SM75-specific. An earlier qualification used an RTX
4070 Ti (12 GiB, SM89), INT8 KV, a 196,608-token context, and a 512-slot staging
ring. These measurements were made on an older v0.1.41 source revision, so they
support the cross-GPU mechanism but are not current-main performance numbers.
Loan-off and loan-on runs were interleaved, serialized, and used fresh engine
processes. The 36k result is a three-run median and the 72k result is a five-run
median; completions were capped at 512 tokens.

| Fresh prompt | Loan | Prompt chunk | PP | TG | GPU prefill |
| ---: | --- | ---: | ---: | ---: | ---: |
| 36,061 tokens | Off | 8,448--8,960 | **2,562.9** | **54.8** | 13.811 s |
| 36,061 tokens | On | 15,104 | 2,539.4 | 52.9 | **12.987 s** |
| 72,061 tokens | Off | 8,960 | 2,617.0 | 54.1 | 27.271 s |
| 72,061 tokens | On | 15,104 | **2,680.9** | **54.9** | **25.772 s** |

At 36k, loans reduced GPU prompt work by 6.0%, but the fixed resource-reload
cost left engine PP 0.9% lower. At 72k, they reduced GPU prompt work by 5.5%
and improved engine PP by 2.4%. The five-run 72k PP ranges were
2,604.3--2,633.7 off and 2,501.1--2,696.3 on; four of the five loan-on runs
were at least 2,673.5 PP, with one retained slow outlier. This crossover is why
the feature remains a manual opt-in rather than a default.