Pull requests / #2419
#2419 arm64: build + run Strata on NVIDIA DGX Spark (GB10, sm_121) - rebased onto v0.1.41, IQ3_S validated
open · @ccatzin-zumwalt · 0 comments · View on GitHub
BenchmarksSetup & installServer & APIAMD / HIPNVIDIA / CUDAModels & quantsSecurityDocumentation
Description
## What this adds
Builds and runs Strata on the NVIDIA DGX Spark: an aarch64 host (10 Cortex-X925 +
10 Cortex-A725) with a Blackwell GPU (GB10, sm_121) sharing 128 GB of unified
memory. Up to now the Spark could only use the community aarch64 port; this branch
rebases that port onto v0.1.41 (after the upstream history cleanup), fixes the
drift the new base introduced, and validates it end-to-end with IQ3_S.
## Attribution
This is a rebase of the community aarch64 port — PRs #409 (eelgaev), #901
(githubhjs) and #1077 (johnlockejrr) — onto v0.1.41 (`fb58e0d`), plus the drift
fixes the new base required. All original authorship is preserved in the
history; our own changes are the rebase fixes, marked "DGX Spark GB10 (aarch64)"
in the affected blocks.
## Changes
- `include/strata/platform/cpu_relax.hpp`: portable `_mm_pause` / `_mm_sfence`
(yield / `dmb oshst` on aarch64).
- `src/kernels/cpu/portable.cpp`: scalar + NEON activation quantizer and the BF16
router dot. The AVX-only kernels are refused on ARM; native packs (IQ3_S
included) fall through to ggml-cpu's NEON dot products.
- `CMakeLists.txt`: `STRATA_CPU_X86` guard drops the AVX kernel files on aarch64
and sets `-ffp-contract=off`; passes `GGML_CPU_ARM_ARCH` to ggml-cpu.
- `setup.py` / `Dockerfile`: aarch64 detection, ggml CPU features read from
`/proc/cpuinfo` (gcc's `-mcpu=native` finds none on GB10's big.LITTLE part),
sm_121 auto-selection, and GB10 unified-memory handling (the GPU reports no
memory of its own; setup counts system RAM).
- x86-intrinsic guards in graph/session/verify/mtp/pool/expert_layout/generate,
including the `expert_layout.cpp` / `native_expert.cpp` blocks this base had
moved.
- `docs/DGX_SPARK.md`: documents the rebase, the install path, and the IQ3_S
validation numbers.
x86 builds are untouched: every change is behind an aarch64 guard or in
aarch64-only files.
## Validation (one DGX Spark, 2026-10-08)
IQ3_S (GSQ-RCO), 256K context, `--parallel 4`, `--kv int8`, `--mmap-experts`,
all 24,576 experts in the cache:
- decode 58.8–66.0 tok/s; prompt 103–111 tok/s on short requests;
- 127K-token long prompt prefilled at 1512 tok/s, with a needle-in-haystack
marker retrieved correctly from that depth;
- MTP accepting 13/14 and 60/79 proposals;
- above the community reference on Spark (47 tok/s, UD-IQ4_XS).
## Testing notes / known gaps
- Validated on aarch64 only; we could not re-run the x86 build matrix on an
x86_64 host, though the diff is guard-only for x86.
- Needle-in-haystack was verified at 127K depth, not the full 256K.
- A 1h+ soak (docs/TEST_REQUESTS.md) has not been run on this rebase yet.
Happy to run any of these if you want them before merge.Related on this site
Quick links to related pages.