Pull requests / #1790

#1790 sycl mtp: slot drafters copy the shared draft head's type (#1121 on the SYCL port, as #1110 did for CUDA)

closed · @leadgtr7 · 0 comments · View on GitHub

BenchmarksAMD / HIPNVIDIA / CUDAModels & quantsWindows

Description

## Title
Issue: Resolves #1121 for the SYCL port (the CUDA source was fixed in #1110)

## Summary
`--batch-mtp` still ends the Intel (SYCL) engine on the first batch admission with `strata batch: MTP admission for slot 0 failed: mtp: unsupported native MMVQ GGML type`. The cause is the one #1110 fixed in `src/core/mtp.cpp`: a slot drafter that shares the main drafter's draft-vocab head copies `dhead_`, `dvocab_` and `n_dvocab_` but not `dhead_type_`, which stays `-1`. `sycl/src/core/mtp.cpp` never got that line. This adds it. Nothing else changes; without `--batch-mtp` no drafter shares a head, so that path is untouched.

## What changed
- `sycl/src/core/mtp.cpp`, `MtpDrafter::bind`: `dhead_type_ = shared->dhead_type_;` next to the other three copies (same line and comment as the CUDA source).

## How it was checked
Arc Pro B65 32 GB (xe, PCIe Gen4 x16), engine 0.1.41 built with `sycl/tools/build.sh` (AOT `bmg-g31`), Swift 1.5 IQ3_S GSQ-RCO (all 512 experts; ~12.5k in VRAM, the rest in the pinned host mirror), `--mtp rt --spec 4`, `rt/draft_vocab.bin` present, `"parallel"` 2 and 4. Server with #1721's `_pump` fix (needed for any `"parallel"` on SYCL).

| | result |
|---|---|
| without this line, `--batch-mtp`, parallel 2 | engine exits at the first admission (the error above), both clients get nothing |
| without this line + `STRATA_MTP_FULL_HEAD=1` (the workaround in #1121) | works; single-slot decode 34.4 tok/s vs 38.4 without the variable (the full head costs ~10%) |
| **with this line**, `--batch-mtp`, parallel 2, no env | works: 199 batch windows, avg 2.60 rows, both replies coherent |
| **with this line**, parallel 4 | works: 175 windows, avg 6.83 rows |

Throughput on this card is a separate story and not this PR's claim (4 concurrent 300-token requests: 29.5 tok/s queued on one slot, 31.9 with 2 slots + `--batch-mtp`, 32.4 with 4; the batch windows wait on mirrored experts over PCIe). The time to first token drops from up to 29 s to under 6 s with 4 slots.

Not tested: CUDA/HIP (untouched), greedy token equality of batched vs solo (on SYCL that already differs without `--batch-mtp`; noted on #1721).

## Extra Notes
Only the SYCL port is touched. Assisted-by: Claude Code.

Related on strata.com

Editorial links to help you install, pick models, or read release notes — not part of the upstream thread.