Pull requests / #2419

#2419 arm64: build + run Strata on NVIDIA DGX Spark (GB10, sm_121) - rebased onto v0.1.41, IQ3_S validated

open · @ccatzin-zumwalt · 0 コメント · GitHub で見る

BenchmarksSetup & installServer & APIAMD / HIPNVIDIA / CUDAModels & quantsSecurityDocumentation

本文

## What this adds

    Builds and runs Strata on the NVIDIA DGX Spark: an aarch64 host (10 Cortex-X925 +
    10 Cortex-A725) with a Blackwell GPU (GB10, sm_121) sharing 128 GB of unified
    memory. Up to now the Spark could only use the community aarch64 port; this branch
    rebases that port onto v0.1.41 (after the upstream history cleanup), fixes the
    drift the new base introduced, and validates it end-to-end with IQ3_S.

    ## Attribution

    This is a rebase of the community aarch64 port — PRs #409 (eelgaev), #901
    (githubhjs) and #1077 (johnlockejrr) — onto v0.1.41 (`fb58e0d`), plus the drift
    fixes the new base required. All original authorship is preserved in the
    history; our own changes are the rebase fixes, marked "DGX Spark GB10 (aarch64)"
    in the affected blocks.

    ## Changes

    - `include/strata/platform/cpu_relax.hpp`: portable `_mm_pause` / `_mm_sfence`
      (yield / `dmb oshst` on aarch64).
    - `src/kernels/cpu/portable.cpp`: scalar + NEON activation quantizer and the BF16
      router dot. The AVX-only kernels are refused on ARM; native packs (IQ3_S
      included) fall through to ggml-cpu's NEON dot products.
    - `CMakeLists.txt`: `STRATA_CPU_X86` guard drops the AVX kernel files on aarch64
      and sets `-ffp-contract=off`; passes `GGML_CPU_ARM_ARCH` to ggml-cpu.
    - `setup.py` / `Dockerfile`: aarch64 detection, ggml CPU features read from
      `/proc/cpuinfo` (gcc's `-mcpu=native` finds none on GB10's big.LITTLE part),
      sm_121 auto-selection, and GB10 unified-memory handling (the GPU reports no
      memory of its own; setup counts system RAM).
    - x86-intrinsic guards in graph/session/verify/mtp/pool/expert_layout/generate,
      including the `expert_layout.cpp` / `native_expert.cpp` blocks this base had
      moved.
    - `docs/DGX_SPARK.md`: documents the rebase, the install path, and the IQ3_S
      validation numbers.

    x86 builds are untouched: every change is behind an aarch64 guard or in
    aarch64-only files.

    ## Validation (one DGX Spark, 2026-10-08)

    IQ3_S (GSQ-RCO), 256K context, `--parallel 4`, `--kv int8`, `--mmap-experts`,
    all 24,576 experts in the cache:

    - decode 58.8–66.0 tok/s; prompt 103–111 tok/s on short requests;
    - 127K-token long prompt prefilled at 1512 tok/s, with a needle-in-haystack
      marker retrieved correctly from that depth;
    - MTP accepting 13/14 and 60/79 proposals;
    - above the community reference on Spark (47 tok/s, UD-IQ4_XS).

    ## Testing notes / known gaps

    - Validated on aarch64 only; we could not re-run the x86 build matrix on an
      x86_64 host, though the diff is guard-only for x86.
    - Needle-in-haystack was verified at 127K depth, not the full 256K.
    - A 1h+ soak (docs/TEST_REQUESTS.md) has not been run on this rebase yet.

    Happy to run any of these if you want them before merge.

関連リンク

インストール・モデル・リリースへの站内リンク。