#5557 [Engine] SYCL: weight arena is ONE USM device allocation — returns NULL silently on devices with a max-alloc cap (WSL2 GPU-PV, 1 GiB measured); crash 'cudaMemcpy failed for output_hc_down.weight'

closed · @villangames252-a11y · 2 commentaires · Sur GitHub

BenchmarksSetup & installServer & APINVIDIA / CUDAModels & quantsDocumentationWindowsLinux

Description

# [Engine] SYCL: the weight arena is one USM device allocation — on devices whose max single alloc is below the pool size it returns NULL silently and crashes as "cudaMemcpy failed for output_hc_down.weight"

## Environment (all measured on one machine)

- Windows 11 26300 (build 10.0.26300.9711), Core Ultra 285H-class, **Intel Arc 140T iGPU (Panther Lake Xe2, PCI 8086:7d51)**, 96 GB RAM, graphics driver 32.0.101.8993.
- WSL2 Ubuntu 26.04 (WSL 2.6.x, `/dev/dxg` present, networkingMode=mirrored).
- Intel GPU compute runtime in the distro: `clinfo` → "Intel(R) Graphics [0x7d51]", OpenCL 3.0 NEO, **Global memory: 50.74 GiB, Max memory allocation: 1024 MiB** (the Level-Zero/oneAPI path enumerates the same device, `sycl-ls` equivalents confirm).
- Strata control 0.1.58, engine `strata-engine-0.1.58-linux-x86_64-sycl` (signature-verified via `strata update --backend sycl`).

## What works on this route

- Engine installs and `--version` test-start passes; feed selects `linux-x86_64-sycl` for the PC.
- `strata launch --backend intel --model-path ...` packs Qwen3.8-Flash-Next IQ3_XXS in place from `/mnt/c` GGUFs (76 GB, ~25 MB/s over 9p), writes `native_experts.txt`, runs the AVX2 expert path on a no-AVX-512 CPU, downloads + packs the DFlash2 drafter, probes PCIe host→device at **9.5 GB/s** with a mapped token embedding (260 MiB).

## The crash (deterministic, 32k and 16k context)

```
strata generate: PCIe probe: 9.5 GB/s host->device (best of 8.7 9.1 8.7 9.5) -> pcie_frac 0.26
strata generate: this CPU has no AVX-512: the expert kernels run on AVX-2
strata generate: native pack: .../iq3_xxs experts (largest blob 2.33 MB), token embedding IQ3_S in mapped host memory (260 MiB, 3.4 s)
NULL pointer argument in memory copy operation.
strata generate: cudaMemcpy failed for output_hc_down.weight
```

## Root cause (read from the source, 0.1.58)

`sycl/include/strata/sycl_queue.hpp:267` — `malloc_device_guarded()` calls `sycl::malloc_device(bytes, q)`, and when it returns `nullptr` it **registers and returns the null pointer without an error** (its null path exists only for the alias-check guard). The caller writes the arena as one `malloc_device_guarded(pool_bytes)` (generate.cpp:3030); the DPCT error-check wraps an error-code, so a null result passes silently (the informative "cudaMalloc(...) for the weight arena failed" message at generate.cpp:~3040 never prints). The loader then does `dpct queue memcpy(dst_base + r.dst_off + out_at, host_src, ...)` for the first arena row — `output_hc_down.weight`, 6,553,600 bytes at index `dst_off 0` (pack index v3: `pool 1538035200`, 1079 tensors) — and SYCL raises "NULL pointer argument in memory copy operation" → the typed `EngineMissing`-ish crash "cudaMemcpy failed for <row>".

The pool here is 1.43 GiB, i.e. **over the 1 GiB per-allocation cap the WSL2 GPU-PV Level-Zero device reports** (`Max memory allocation: 1024 MiB`; global window 50.74 GiB is fine). Any device with a max-alloc below the pool size hits the same silent-null chain. On native Linux with i915/xe the cap is far higher, which is why the port's other users have not met it.

## Suggestions

1. Make the silent-null impossible: after the arena allocation, refuse with a typed error that names the pool size, the device's `maxMemAllocAlloc` (ZE / `CL_DEVICE_MAX_MEM_ALLOC_SIZE`) and the fix, instead of letting the null reach the loader.
2. If the allocation fails or the pool exceeds the device's max alloc: fall back to a **mapped-host arena** (the code already maps the token embedding in host memory) or split the pool into a series of ≤ cap chunks with a segment-aware `WeightTable`.
3. With (1)/(2), the engine could run on WSL2 GPU-PV Arc and other alloc-limited devices — today Windows Arc users get no engine at all (docs/INTEL_ARC.md: "Windows has no path"), and this WSL2 route is the only GPU route they have; it gets to 99% of a working start and then dies on this one allocation.

## Data point (same machine, same model, llama.cpp SYCL on native Windows)

`llama-b12198-bin-win-sycl-x64` (Arc via native Level Zero, `-ngl 99`, 262k q4_0 KV): loads fine, answers fine, **~5.5 tok/s decode**, vs the same machine's CPU llama-server (16 cores, mmap) at **6.8 tok/s** for this hybrid model — so on driver 32.0.101.8993 the iGPU currently adds nothing for Qwen3-Next-class hybrids; native Windows SYCL support for the port (#5236, #3607) would be the real unlock. Related: #4718 (Hyper-V/WSL-class allocation quirks), #3629 (WSL2 host memory probe).

Happy to re-run anything on this box; it is a standing Arc 140T + 96 GB test bench.

Sur le site

Liens install, modèles, releases.