#445 Large arena prefill regression on Linux: 42-43 GiB pin threshold stalls GPU

Description

Greetings! 

first, thanks for the great work!

Found a performance regression testing on multiple P100s (sm_60 / Pascal). Unclear if Pascal-specific or affects newer architectures.

Commit `ed6c124` ("#253: the 8 GiB arena pin cap only under WDDM") causes large expert arenas to lose most prompt speed on Linux. The environment variable from that commit recovers performance, so this likely needs a different default or release note.

## Setup

| | |
|---|---|
| Model | Qwen3.8-Flash-Next **IQ3_S** (`arena_gb` 50.3) |
| GPUs | 3 × Tesla P100-PCIE-16GB (sm_60) |
| Topology | GPU0 in PCIe x4, GPU1/2 in x16 — auto layer split |
| RAM | 125 GB |
| CPU | Xeon E5-2699 v3 (AVX2 only) |
| CUDA / driver | 12.0 / 580.178.04 |
| Engine | `-DSTRATA_EXPERIMENTAL_SM60=ON -DCMAKE_CUDA_ARCHITECTURES=60` |

### Key Triggers
- **Arena Size:** Performance cliff at ~84% of arena size. Smaller CI arenas won't reproduce this.
- **Layer Split:** Required condition per `ed6c124`.
- **RAM Headroom:** Available RAM stays 67-68 GB — this is a threshold issue, not exhaustion.
- **Architecture:** sm_60 without AVX-512. GPU arch might matter; CPU likely doesn't.

## Results

Median of ≥2 rounds, fresh ~7.3K-token prompts (cached prompts ~0.45s, not measured).

| build | `STRATA_ARENA_PIN_GIB` | prefill tok/s |
|---|---|---|
| `30ec18e` (v0.1.30) | — | **179.7** |
| `ed6c124` (first bad) | — | 134.6 |
| `c499bd1` (v0.1.32) | — | 133.5 |
| `aeb35be` (v0.1.33) | — | 127.0 |
| **`aeb35be`, unmodified** | **40** | **192.2** |

Same pattern at 69,144 tokens: 125.0 → 204.3 tok/s. Gap widens at larger prompts.

## Why this matters

#253 was resolved but didn't fix this for my setup. With layer split, default pins entire arena (past 42-43 GiB cliff): 127 tok/s default vs 192 tok/s with undocumented variable. Severely hinders large-context performance. Unclear if this affects single-GPU or only multi-GPU, but #253's fix clearly didn't cover all cases.

## Discrete Performance Threshold

| `STRATA_ARENA_PIN_GIB` | 4 | 8 | 16 | 32 | 40 | 41 | 42 | **43** | 44 | unset |
|---|---|---|---|---|---|---|---|---|---|---|
| prefill | 177.8 | 179.6 | 183.3 | 184.2 | **192.2** | fast | fast | **slow** | 134.0 | 127.0 |
| prompt loan | 1.49 | 1.49 | 1.49 | 1.49 | **1.49** | 1.49 | 1.49 | **0.68** | 0.68 | 0.68 |

Degradation is discrete (loan halves between 42-43 GiB), consistent across 7.3K and 69K tokens. RAM stable at 67-68 GB confirms threshold logic issue.

`STRATA_PREFILL_TIMING=1` shows 96.5% loss in `embed+steps` ("gap where GPU waits" per `prefill.cpp` comments) — GPU stalls on host expert feed, not slower kernels. These env vars didn't help: `STRATA_OLD_GROUPED=1`, `STRATA_REFILL_SERIAL=1`, `STRATA_STAGER_THREADS=4`, `STRATA_SPLIT_OWN=0`, `STRATA_PREFILL_AUTO_MAX=8192`, `STRATA_PREFILL_LEND_PCT` (60-90).

## Where to Look

Loan halves at 83-85% arena pinned. `generate.cpp` has `pinned_share() >= 0.9 ? 90 : 85` threshold. Values are close but `STRATA_PREFILL_LEND_PCT` didn't affect loan, so likely different thresholds. Coupling between arena pin % and prompt loan logic is probably there.