#6160 [Engine] STRATA_PIPED_MTP is turned off on Linux CUDA by the engine's own parallel defaults (STRATA_BATCH_SELF_COMMIT=1)

open · @fioruccione · 7 评论 · 去 GitHub 看

Multi-GPUAMD / HIPNVIDIA / CUDAModels & quantsWindowsLinux

说明

**Reproduction status:** Measured locally (the code path is from reading the source, confirmed by the engine's log line)

I was trying `"parallel"` on my box and `STRATA_PIPED_MTP=1` never engaged. The log said:

```
strata generate: WARNING: STRATA_PIPED_MTP is off: it does not combine with STRATA_BATCH_SELF_COMMIT (the window would keep its rows itself)
```

I never set `STRATA_BATCH_SELF_COMMIT`. On a Linux CUDA build with `--batch` 2 or more, the engine sets it itself: `generate.cpp:4200` calls `apply_parallel_defaults_cuda(true, o.batch)`, and the table in `arch_defaults.cpp:235` has `STRATA_BATCH_SELF_COMMIT=1`. Nothing changes it again before the check in `generate.cpp:7583`. So with the defaults, PIPED_MTP can't run on Linux CUDA. Setting `STRATA_BATCH_SELF_COMMIT=0` by hand fixes it, and the log then says `STRATA_PIPED_MTP: 2 pipelined groups of 1 slot draft up to 1 MTP proposal per slot each window`.

If I read the history right, it's just an ordering thing. PIPED_MTP came in on 10-27 (17878ac57, #4121), and it was measured then. The parallel stack became the launcher's default on 10-30 (dd471d314), and moved into the engine on 11-07 (c9abd38a7, Linux CUDA only). Windows gfx1151 gets the same table since 8becec7d5, but I don't think PIPED_MTP matters there.

**What it gives once it's on.** P620, Threadripper PRO 3975WX, 2x RTX 4060 Ti 16 GB (PCIe 4.0 x8), layer split auto 24/24, Strata 0.2.15 (engine 0.1.60), Qwen3.8-Flash-Next IQ3_S, MTP `--spec 4`, int8 KV. Parallel arms: `--batch-mtp`, `STRATA_PIPED_MTP=1`, `STRATA_PIPED_CONCURRENT=1`, `STRATA_BATCH_SELF_COMMIT=0`, no `--remote-expert-opt`. A fresh engine per arm, order serial, 2, 4, 4, 2, serial, 3 rounds each, T=0, thinking off. Requests in a burst start together; aggregate = all tokens / time until the last one ends.

| Burst | serial | `"parallel": 2` | `"parallel": 4` |
|---|---|---|---|
| 2 short | 91 | 122 | 121 |
| 3 short | 88 | 109 | 139 |
| 4 short | 91 | 121 | 177 |
| 20K prompt + 3 short | 56 | 59 | 81 |
| ... slowest first token in that burst | 22.5 s | 20 s | 12.6 s |
| one request alone | 92-105 | 90-102 | 92-99 |
| expert slots (both cards) | 11,496 | 10,235 | 9,120 |
| VRAM free, lowest seen | 401 MiB | 606 MiB | 442 MiB |

So on this box it's +34 % with two requests and almost 2x with four, and a request on its own doesn't lose anything. Four slots take 5.5 minutes to start instead of 2-3 (all the window captures), and with `--vram-reserve-mib 800` I got one "below 512 MiB" warning per start. I'd give it a bit more reserve.

One thing I did not measure is `STRATA_BATCH_SELF_COMMIT=0` on its own, without PIPED_MTP, so I can't say what it costs the plain pipeline.

**Smaller things from the same runs** (`"parallel": 2`, before PIPED_MTP):
- Without drafts the default pipeline lost to serial on short pairs (83-85 vs 87-99), and `--batch-groups 1` was worse (71-74). So the 0.2.7 choice of the pipeline over one group looks right on two 4060 Ti too.
- `STRATA_PIPED_CONCURRENT=1` does what BATCHING.md says. With a 20K prompt read beside a short request, the short one's longest stop went from 9.7 s to 0.65 s, and the aggregate went up 3 %. `STRATA_PIPED_READ_PIECE=16384` was worse on everything (−17 %).
- `--remote-expert-opt` makes no difference here (serial with and without: same numbers), which matches the code comment that it's inert on a plain layer split.

**Notes that are out of date** (from reading the source, not measured): `verify.cpp:322` still calls `STRATA_BATCH_SELF_COMMIT` opt-in. `BATCHING.md:496` says PIPED_MTP is off with SELF_COMMIT on, but not that the engine turns SELF_COMMIT on for you. And `data/settings-register.json` points to `generate.cpp:7579` for the read, which is at 7583.

I don't know which fix you'd prefer: leave SELF_COMMIT off when `STRATA_PIPED_MTP` is set, or just document the `=0`. I'm happy to run whatever you want on this box.

本站相关内容

相关页面的快捷入口。