#3127 [Feature]: an AMD card as an expert helper beside an NVIDIA card (CUDA engine + HIP helper process, Linux, opt-in)

Descripción

### What do you want to do?

Use an NVIDIA card and an AMD card in the same PC together, without Vulkan. #1214 asked for this (closed as not planned, since a Vulkan backend would be a separate engine), and the later comments there ask for exactly NVIDIA + AMD. I have a working prototype that keeps the existing backends and asks whether you want it, and in what shape, before I open any pull request.

**How it works.** The CUDA engine stays the primary. With `--external-experts <path to strata-expert-helper>` it starts a second process, built from the HIP backend, that loads the hottest experts the NVIDIA card does not hold into the AMD card's VRAM. Per layer, the engine writes the routed tokens for those experts into a shared memory block(a memfd that both `cudaHostRegister` and `hipHostRegister` accept, so neither side copies). The helper computes them with the existing HIP expert kernels and writes the rows back. The CPU pool computes whatever neither GPU holds, as it does today. With `--external-experts-ram exclusive`, the RAM copy leaves out the helper's experts, so on 32 GB the remaining experts fit in RAM.

**Measured** on one box: Ryzen 9 3900X (AVX-2 only), 32 GB DDR4, RTX 5060 Ti 16 GB on CPU lanes (engine), RX 9060 XT 16 GB on the chipset's x4 Gen3 link (helper), Ubuntu, engine 0.1.41. Five fixed prompts (4K and 32K code, two short chats, one tool call), greedy, one fresh server and one pass each, 64K context, `--spec 4 --mtp`, int8 K/V.

| Model | RTX only, decode tok/s | RTX + helper, decode tok/s |
|---|---|---|
| Flash-Next IQ3_XXS, `--resident-experts` | 14-31 (RAM copy does not fit, the rest read from the SSD) | 63-74 (exclusive; 0 expert reads from the SSD in decode) |
| Coder IQ1_M, `--resident-experts` | 47-54 | 71-98 |

- Prompt speed: the inclusive mode leaves it as it was (Coder 4K: 1545 vs 1482 tok/s). The exclusive mode reads the helper's experts from the files during a prompt: Coder 4K drops to 855 tok/s, and IQ3_XXS's 70-75-token prompts get slower (7.5-12.4 vs 14.7-24.3 tok/s), while its 4K and 32K prompts get faster (227 / 332 vs 184 / 311).
- The helper sets the pace in 64-94% of layers. Round trip about 200 us median, up to 740 us p99. The x4 Gen3 link is most of that. 
- 100 requests in a row: all answered, 0 helper timeouts. If the helper process is killed mid-request, the CPU pool computes that layer's rows from the model files and the engine stays up.
- Default path: without `--external-experts`, a 4K prompt gave identical tokens in two runs of the build before the change and two runs of the branch. The short prompts differ run to run on the base build too, so they cannot show it either way.

**IQ3_S at full context (256K): the setup I run every day.** Flash-Next IQ3_S has 46.84 GiB of experts. At 256K the int8 K/V cache leaves room for only 5.40 GiB of them in the RTX's cache. With the helper, 15.36 GiB sit on the RX 9060 XT and 22.38 GiB in RAM, so about 3.7 GiB come from the SSD. On the RTX alone, the same RAM would hold about 23 of the remaining 41.44 GiB, so about 18 GiB would come from the SSD. That last figure is arithmetic from the startup lines, not a measured run: my RTX-only IQ3_S attempt ran the PC so low on memory that it was stopped in its first request, so there is no RTX-only IQ3_S speed here.

Same prompts and settings as above, helper exclusive, `--resident-experts`:

| Context | Engine | Decode tok/s (p1-p5) | Prompt tok/s, 4K / 32K | Experts read from the SSD |
|---|---|---|---|---|
| 256K | 0.1.45 (merged with `main` at 477ac13c) | 29-44 | 149 / 249 | about 3.7 GiB |
| 256K | 0.1.41 | 27-41 | 138 / 238 | about 2.5 GiB |
| 128K | 0.1.41 | 31-55 | 129 / 246 | about 0.5 GiB |
| 64K | 0.1.41 | 47-60 | 147 / 244 | none: everything fits |

- Each step up in context takes VRAM from the RTX's experts for the K/V cache (0.95 GiB more at 128K, 2.87 GiB more at 256K), and on 32 GB those experts can only go to the SSD. At 64K they all fit on the two cards and in RAM.
- Short prompts (70-407 tokens) run at only 4-16 tok/s at every size: in exclusive mode the prompt path reads the helper's 15.4 GiB of experts from the files.

**Size.** About 1,780 lines of engine code: mostly new files (`include/strata/xhelper/protocol.hpp`,
`src/core/external_experts.cpp`, `src/xhelper/helper_main.cpp`, `src/xhelper/helper_tier.cpp`), and about 230 lines in existing files (`expert_source.cpp/.hpp`, `generate.cpp`, `CMakeLists.txt`). Plus tests: a protocol test that runs in both builds without a GPU, a helper parity test against the CPU kernels, and an engine-side test with a real helper.

**Limits.**
- Linux only (memfd).
- Only tested on the one box above, with one NVIDIA and one AMD card.
- Refused, with a one-line message, alongside `--layer-split`, `--expert-cache-device1..3`, `--remote-expert-opt`, `--peer-device`, `--batch`, `--pipeline-windows 2`, `--vram-elastic` and `--kv-grow`.
- Not tested: Windows, other AMD or NVIDIA cards, a helper on x16 lanes.

**How it was made.** Both the code and the plan were written with Claude Code Opus 5.5. I ran every
measurement above on my own hardware. I have not yet reviewed the code line by line myself, and unlikely to do so since I am not proficient in CPP. I just compiled and have been using it as my daily driver.

**Questions.**
1. Do you want this in Strata at all, as an opt-in?
2. Your helper caches (`--expert-cache-device1`, `--remote-expert-opt`) do the same job for a second card of the same vendor. Would you rather this reuse that path, with the helper process as one more kind of remote tier, than add a separate `--external-experts` flag?
3. If yes, would you take it as a series of small PRs: (1) the shared-memory protocol and its test, (2) the HIP helper binary and its parity test, (3) the engine side behind the flag, (4) docs and the numbers?

The branch is at https://github.com/alizand1992/Strata (`main`). The plan, the per-phase results and the raw numbers are in `docs/plans/OPTION_A_EXCLUSIVE_RAM.md` and `docs/plans/results/`.