改动 / #3622
#3622 [Engine] Build and run the engine on ARM64 (DGX Spark / GB10)
open · @ferstar · 0 评论 · 去 GitHub 看
BenchmarksSetup & installNVIDIA / CUDAModels & quants
说明
## Problem and result The engine only builds and runs on x86 today: the CPU expert path assumes the x86 AVX kernels and CPUID probes. On an ARM64 machine (NVIDIA DGX Spark / GB10, CUDA 13.0) the build fails at the x86 kernel sources and intrinsics. After this change the engine builds on aarch64 and loads and answers requests there. On x86 the x86 sources and flags are still selected as before. ## Related discussion none. This is a platform port, not a feature or default change, but it touches CMake and the CPU kernel selection, so maintainer review of scope is requested. ## Approach - CMake: `STRATA_CPU_X86` is decided from `CMAKE_SYSTEM_PROCESSOR`. x86-only kernel sources and per-file flags are added only on x86. On aarch64, `src/kernels/cpu/portable.cpp` provides the kernel entry points, and native experts use the ggml-cpu NEON dot products. `-ffp-contract=off` on aarch64 so the host quantizers do not diverge from x86 at exact ties. - `include/strata/platform/cpu_relax.hpp` maps the x86 spin-wait intrinsics (`_mm_pause`, `_mm_sfence`) per architecture. - The x86 CPU probes (CPUID, XCR0) compile only on x86 and report no AVX-512 elsewhere. - `setup.py`: when `nvidia-smi` reports `[N/A]` for memory (GB10 unified memory), the pool size comes from `/proc/meminfo`. The CPU feature check no longer expects the x86 `flags:` line. - `--cpu-oracle-q8-0` is rejected with a message on non-x86 builds. Compatibility: the x86 build keeps every x86 source and flag it had before, but the header and intrinsic indirection touches shared files (`verify.cpp`, `pool.cpp`, `session.cpp`, `mtp.cpp`, `graph.cpp`), so x86 review is needed. ## Validation **Passed:** - GB10 (aarch64, CUDA 13.0, driver 580, branch `gb10-022` at `d891f006`): `strata` engine target builds; model load and chat completions work with Qwen3.8-Flash IQ3_S (three completions of 512 tokens at temperature 0 observed at 55–59 tok/s; no comparison run, so no speedup is claimed). - x86 (RTX 4090 D, sm_89, CUDA 13.2, same commit): full engine build via `setup.py` succeeds. **Failed:** - none observed on the builds above. An earlier pushed revision (`2a148442`) failed to compile on both platforms; it was replaced by `d891f006`. **Not run:** - `ctest` on either machine. - Inference on the x86 build of this exact commit. - Any x86 regression benchmark. - CUDA builds other than 13.0 (GB10) and 13.2 (x86). ## Limits - The aarch64 CPU path uses the portable kernels; it has not been benchmarked against x86. - Only validated on one GB10 and one RTX 4090 D.
本站相关内容
相关页面的快捷入口。