#6541 [Engine] strata-hy4 0.1.64: EXCEPTION_ILLEGAL_INSTRUCTION (AVX-512 EVEX) on Arrow Lake-HX — current workaround disables the 1.5x AVX-VNNI cold-expert kernels

open · @akiry09 · 4 评论 · 去 GitHub 看

BenchmarksServer & APINVIDIA / CUDAModels & quantsWindows

说明

## Environment
- Strata 0.2.18 / strata-hy4 engine 0.1.64 (windows-x86_64-cuda-cuda13)
- CPU: Intel Core Ultra 9 275HX (Arrow Lake-HX, Family 6 Model 198 / 0xC6) — no AVX-512 support
- GPU: RTX 5090 Laptop 24 GB
- Model: Hy4-Flash SQ-96 pack (native + Engram-Q8 + Vision, 131072 context, DFly2 draft)

## Problem
The engine loads successfully and serves `/v1/chat/completions`, but crashes with
exit code 3221225501 = 0xC000001D = EXCEPTION_ILLEGAL_INSTRUCTION immediately
when the first prompt arrives (prefill path). It never crashes during startup
or weight loading.

## Root cause evidence (from WER crash dump)
I parsed `strata-hy4.exe.<pid>.dmp` with the Python `minidump` library:
- Exception address: RVA 0x115ba0 inside strata-hy4.exe
- Machine code at fault: 62 f2 5d 28 50 04 01 ...
- 0x62 is the AVX-512 EVEX prefix (e.g. vmovdqu64 zmm). Arrow Lake-HX has no
  AVX-512, so this raises an illegal-instruction fault.

The engine log shows `kernel=fast(avx2-vnni)` at the time of the crash, i.e.
the CPU-expert kernel itself already selected the VEX/VNNI path. The offending
EVEX instruction is therefore from a different hot path (compiler-generated
vectorized code in prefill), not from the expert GEMM kernel. Selecting
`avx2-vnni` in the kernel picker does not avoid it.

## Current workaround and its cost
Setting `STRATA_FORCE_ISA=avx2` avoids the crash. However, per
`hy4_cpu_experts.cpp:1462`, this clears BOTH avx512 and avx_vnni capability
bits, so the engine falls back to the plain AVX2 kernel (`kernel=fast(avx2)`).

This disables exactly the optimization that your v0.2.18 release notes
highlight: "AVX-VNNI (k2_vnni) 1.5x faster CPU cold-expert kernels on Arrow
Lake / hybrid CPUs". The k2_vnni kernel source even names Core Ultra 9 275HX
as its target CPU, so this is a regression specifically on the hardware the
optimization was written for.

Measured on this machine: decode ~8 tok/s (plain 212 ms/step), vs the 90-111
tok/s you report for a desktop RTX 5090. Part of that gap is the 24 GB laptop
GPU, but the missing VNNI path is a measurable avoidable loss.

## Request
Could you please locate and fix the AVX-512 EVEX instruction in the prefill
path (RVA 0x115ba0) so it falls back to a VEX/AVX2 sequence when CPUID reports
no AVX-512? That would let Arrow Lake / Meteor Lake users keep AVX-VNNI enabled
for the cold-expert kernels.

If a code fix is not immediately available, a finer-grained override would be
a useful stopgap — e.g. an option that suppresses only AVX-512 while keeping
AVX-VNNI for the expert kernels and all other hot paths.

I can provide the full crash dump, the exact byte sequence, or run any build
you produce to verify the fix.

本站相关内容

相关页面的快捷入口。