#3267 [Engine] LoRA adapter support with per-request on/off switch

说明

## Problem and result

Strata cannot apply a LoRA adapter, so a fine-tuned variant of the model needs a merged GGUF. This adds optional LoRA support in the CUDA engine (llama.cpp's adapter GGUF format) with a per-request on/off switch.

- `--lora FILE` / `--lora-scaled FILE:SCALE[,...]` load a llama.cpp LoRA GGUF (`general.type = adapter`, `adapter.type = lora`; f32/f16/bf16). It is applied to the mixer output projections: `ssm_out` on GDN layers and `attn_output` on QSA layers. For each adapted layer, `y += s·B(A·x)` with `s = scale·alpha/rank`; several files stack their ranks.
- **Per request:** the engine key is `lora=0|1`, and the server takes `"lora": true|false` in the request body (OpenAI and Anthropic APIs) or `"sampling": {"lora": ...}` in the config. The switch is a device flag the kernels read, so captured CUDA graphs serve both states with no re-capture. Turned off, the output is bit-identical to the engine without an adapter (checked, see Validation).
- **State:** the conversation cache, batch slots and session files are keyed by the switch, next to `cvec`. Session files stay backward compatible (a new bit in the existing flag word; the fingerprint changes only when an adapter is loaded). While batch slots are decoding, a request for the other state is refused.
- **Not adapted:** the MTP draft layer (it only drafts; the verified tokens are the adapted model's).

Without `--lora` nothing changes: the feature is opt-in.

## Related discussion

Continues #1523, closed in the 0.2.0-beta tracker reset. `/reopen` passed its check but GitHub would not reopen it (the branch was rebased and force-pushed after it was closed), so this is the same change, not a competing attempt. No separate scope issue.

## Approach

Six commits: kernels (+ the `lora_parity` test); core hooks in decode, verify and the prompt path; conversation cache/session keying; `generate` loader, CLI flags and request key; `serve` request field + unit test; the decode/verify up kernel as a programmatic dependent launch (rank > 64 and HIP use the plain launch).

Rebased on `main` `f57c0adc` (0.2.0-beta). Conflicts were in `layer.cpp`, `conversation_state.cpp`, `conversation_cache.hpp` and `generate.cpp`; `main`'s changes were kept and the LoRA lines re-added, including the LoRA switch in the #879 slot checkpoint move and the donor take-over.

## Validation

Revision tested: this branch's code (`799c0c17`) as built in a Docker image from `0e320510`, which has the same source files (it differs only in docs/bench files kept out of this PR). Engine on `main` `f57c0adc` base. One NVIDIA RTX 5090, CUDA 13, driver 615.71. Model: Qwen3.8-Flash-Next Coder IQ1_M (`--native`, `--spec 4`, `--expert-cache auto`), adapter `ara-trial9-lora-F32.gguf`: 48 layers adapted, 79.8 MiB.

**Passed:**
- `lora_parity` (synthetic, GPU): `lora_parity: all ok`. Error against double precision 3.0e-7 to 7.9e-7 for 1, 3, 8 and 300 tokens, fp32 and fp16 activations. Switched off: bitwise unchanged for 1, 8 and 300 tokens and from a graph captured while on. Switching back on re-applies through the same graph. Kernel time, rank 50, 6144 -> 2560: 3.7 µs (1 token), 7.6 µs (4), 10.5 µs (8), 31.7 µs (300); in a captured 48-layer graph 3.71 µs per layer at 1 token and 10.49 µs at 8 tokens.
- End to end (`bench/e2e.py`-style, greedy, 48 tokens, 2 prompts, with and without MTP): `lora=0` gives the same tokens as an engine without an adapter; `lora=1` repeated gives the same tokens. On these two short prompts `lora=1` and `lora=0` also gave the same tokens, so this run does not show the adapter changing the output; over the API, 120-token answers did differ between `lora: true` and `lora: false`.
- Over the Anthropic API on the running container (`/v1/messages`, `temperature 0`): a request without the field behaves as `lora: true`; `lora: false` differs from it; after a `lora: true` request, a `lora: false` request reads 0 cached tokens (the cache is keyed by the switch) and a repeat in the same state reads the cached prefix.
- Multi-GPU layer split, same revision and engine as above, with an RTX 4090 as the second card over Thunderbolt: `--layer-split 24 --split-device 1`, `--spec 4`, with and without MTP, same two prompts, greedy, 48 tokens. The log shows `layers 0-23 (CUDA0), 24-47 (CUDA1), one hand-off per window` and `LoRA: 1 file, 48 layers adapted, 79.8 MiB per device`. Results as on one card: `lora=1` repeated gives the same tokens, and `lora=0` gives the same tokens as an engine without an adapter. As on one card, `lora=1` and `lora=0` gave the same tokens on these short prompts, so this run shows the adapter loads on both cards and the switch is bit-neutral, not that the second card's layers change the output. Speed on the split was not measured.
- Speed (3 runs per setting after one warm-up, prompts of about 2,500-2,900 tokens, 256 generated tokens):

| | decode, MTP | decode, no MTP |
|---|---|---|
| no adapter | 271.8 tok/s | 164.0 tok/s |
| loaded, `lora=0` | 269.9 (-0.7%) | 161.9 (-1.3%) |
| loaded, `lora=1` | 272.4 (+0.2%) | 158.6 (-3.3%) |

The MTP differences are within the run-to-run noise of a single benchmark process per setting. Prompt read is not compared: each setting reads its own texts (2,527 to 2,880 tokens), so its times are not comparable.
- Earlier measurements on a B200 (sm_100, Qwen4Exp IQ2_XS, `main` v0.1.40.4, rank 50) are in the first revision of this PR: decode -3.3% (MTP) and -5.5% (no MTP) with `lora=1`, `lora=0` -1.3% and -2.8%; they were taken before the rebase and are not repeated here.

## Limits

No SYCL/HIP port, no web UI toggle, no VRAM accounting in the layer-split auto planner (80 MiB at rank 50). The MTP draft layer is not adapted. Docs (`docs/DETAILS.md`) and benchmark scripts/results are kept out of this PR.