#6561 [Engine] Inkling-2-Mini AMD ROCm/HIP support on top of #6520
closed · @jingx8885 · 0 コメント · GitHub で見る
BenchmarksSetup & installServer & APIAMD / HIPNVIDIA / CUDAModels & quantsSecurityWindowsLinux
本文
## Problem and result PR #6520 (@baptisterajaut, with @merbanan's patches) adds `strata-ink2`, `strata-ink2ref`, and `install_inkling2` for Thinking Machines Lab's official `Inkling-2-Mini` NVFP4/MXFP8 checkpoint, targeting NVIDIA CUDA only. This PR builds directly on #6520 (`dd7fa18d`, keeping all upstream commits and authorship intact) and adds three commits (`a1757f27`, `dec61c6c`, `ac13f14a`) porting both binaries and the Linux setup build path to AMD ROCm/HIP wave32 RDNA GPUs (`gfx10` / `gfx11` / `gfx12`) without changing CUDA code paths. On an AMD Ryzen AI MAX+ 395 (Radeon 8060S, `gfx1151`, ROCm 7.14), `strata-ink2` matches `strata-ink2ref` at mean KL `4.105e-12` (top-1 `100.00%`), runs 128K FP16 KV + DSpark `--spec 1` all-resident in `23.10 GiB` VRAM with `128/128` exact greedy token match against `--spec 0`, and passes end-to-end serving for chat, vision, tool calls, `/v1/responses`, `/v1/messages`, and conversation cache reuse. ## Related discussion Built on #6520 and #6442. All base Inkling-2-Mini work is by @baptisterajaut and @merbanan; our contribution is the three AMD ROCm/HIP commits on top (`a1757f27`, `dec61c6c`, `ac13f14a`). ## Approach - **HIP compatibility (`engine/include/strata/hip_compat/`):** add `cudaStreamPerThread`, `cudaStreamCaptureModeRelaxed`, `cudaErrorUnknown`, `cudaDeviceGetPCIBusId`, `cudaMemcpyToSymbolAsync`, `__HIP_STREAM_PER_THREAD` symbol address handling (`&symbol` for `_spt` APIs), `CUBLAS_PEDANTIC_MATH` -> `HIPBLAS_DEFAULT_MATH`, `cublasSgemm` in the hipBLAS 0.54 branch, and system-scope `__ldcv`. - **WMMA adapter (`engine/src/inkling2/ink2_wmma_hip.cuh`):** drop-in `16x16x16` FP16->FP32 `wmma` layer using `__builtin_amdgcn_wmma_f32_16x16x16_f16_w32` on `gfx11`, `..._gfx12` on `gfx12`, and a wave32 `__shfl` fallback on non-WMMA wave32 cards. `kb_moe_gemm_tc`, `kb_moe_gemm_tc_gg`, and `kb_attn_tc` stay untouched. - **Engine and CMake (`engine/src/inkling2/ink2_engine.cu`, `engine/cmake/strata_gpu_engine.cmake`):** enable `strata-ink2` and `strata-ink2ref` under `STRATA_ENABLE_CUDA OR STRATA_ENABLE_HIP` with `-fgpu-default-stream=per-thread` on HIP, implement `Vmm`/`VRange` directly via HIP VMM (`hipMem*`, minimum `2 MiB` granularity rounding), precheck `FA_SMEM` against `cudaDevAttrMaxSharedMemoryPerBlockOptin` on HIP, and default `c_q8w` to `0` on HIP. Static preprocessing projection of `#if !defined(STRATA_USE_HIP)` on `ink2_engine.cu` is byte-identical to #6520. - **Setup (`terminal/setup.py`, `terminal/strata_control/ink2_family.py`, `terminal/tests/terminal/test_inkling2_setup.py`, `terminal/tests/terminal/test_gh_0217_hipflags.py`):** offer Inkling-2-Mini on Linux AMD wave32 RDNA cards (`INK2_HIP_ARCHS`), build `strata-ink2` with ROCm/HIP CMake flags into `build-ink2-hip`, write `backend: "hip"` to the run config, cover HIP build, stamp reuse, and arch refusal in `test_inkling2_setup.py`, and update `test_gh_0217_hipflags.py` for the third `CMAKE_HIP_FLAGS` call site in `setup.py`. ## Validation Environment: Linux x86_64, AMD Ryzen AI MAX+ 395 / Radeon 8060S (`gfx1151`), ROCm 7.14 (AMD Clang 23.0.0), official `thinkingmachines/Inkling-2-Mini` checkpoint at `ed40e7da625f594fc9c4990f5f66be24873c22d7` (SHA-256 verified), plus GitHub Actions backend compile checks on the same three HIP commits. **Passed:** - GitHub Actions `Backend compile checks`: `CUDA 12.9`, `CUDA 13.0`, `HIP ROCm 7.2`, `SYCL oneAPI 2026.1`, and `Metal strata` all passed; `Checks` CPU CMake (Linux, Windows MSVC), host builds (Linux, macOS, Windows), and `Config snapshot` (ubuntu-24.04, windows-2022) all passed. - Local `gfx1151` build: `cmake -S engine -B build-ink2-hip -G Ninja -DSTRATA_ENABLE_HIP=ON -DSTRATA_ENABLE_CUDA=OFF -DCMAKE_HIP_ARCHITECTURES=gfx1151` and `cmake --build build-ink2-hip --target strata-ink2ref strata-ink2` (`strata-ink2` 1.7 MB, `strata-ink2ref` 136 KB). - Real `gfx1151` hardware probes: `ink2_wmma_hip.cuh` (`col_major` max err `2.68e-7`, `row_major` + `kb_attn_tc` probe max err `4.29e-6`), per-thread stream symbol copy + graph capture (exact), HIP VMM (`raw_gran=4096`, rounded to `2 MiB`, pass). - `strata-ink2ref` vs `strata-ink2` (`--kv f32 --chunk 0 --graph 1`): 28 prompt positions mean KL `4.105e-12`, max KL `4.574e-11`, top-1 `100.00%`, 8/8 greedy tokens exact. - Offload parity (`--vram-reserve-mib 4096 --kv-grow 1 --tc 1 --chunk 256`): `--graph 0` vs `--graph 1` bit-identical across `5,601,624` logits (`max_abs_diff=0`); `--cpu-threads 8` (AVX-512 + HIP doorbell + `__ldcv`) `max_abs=0`, top-1 `100.00%`. - All-resident 128K (`--max-context 131072 --kv f16 --tc 1 --chunk 1024 --spec 1`): loads in `9.7 s` (`23.10 GiB` VRAM), `--spec 1` matches `--spec 0` on `128/128` greedy tokens (`32.2` -> `37.4 tok/s` on high-acceptance sample, up to `50.6 tok/s` under `serve.server`). - Live `serve.server --engine strata`: verified `/v1/models`, streaming `/v1/chat/completions` (`reasoning_content` + `content`), tool calls, PNG vision (`2.02 s` including lazy ViT load), `/v1/responses` (`1.35 s`), `/v1/messages` (`2.12 s`), and 10.5K-token prompt conversation-cache reuse (`19.36 s` cold at `~604 tok/s` prefill -> `2.07 s` warm). - Unit tests on `ac13f14a`: `test_gh_0217_hipflags.py` (2/2), `test_inkling2_setup.py` (19/19), `test_setup_amd.py` (53/53), `test_inkling_images.py` (7/7), `test_inkling_frontend.py` (11/11). **Failed:** - Pre-existing failures on `v0.2.18` / #6520 (unrelated to the HIP commits): `serve` (6 tests in `test_mcp_image_default.py` and `test_stop_closed_pipe.py`), `terminal/tools` (3 `hy4-flash` tests plus 2 `test_setup_model_pins.py` assertions on `ink2_family.py`). **Not run:** - Real GPU execution on NVIDIA CUDA (compiled cleanly in CI under CUDA 12.9 and 13.0; `ink2_engine.cu` CUDA projection verified byte-identical statically). - Real hardware runs on `gfx1100`, `gfx1201`, or `gfx1030` (`ink2_wmma_hip.cuh` `gfx11`, `gfx12`, and wave32 fallback paths passed a local C++20 wave unit test under `clang++ -Wall -Wextra -Werror`; real GPU tests ran on `gfx1151` only). - Windows runtime execution (remains excluded by `ink2_offered`). ## Limits - `setup.py` skips `nvidia-smi` desktop VRAM probing (`display = False`, `used = 0.0`) on HIP; on small-VRAM Linux AMD desktop cards this can overestimate free VRAM for offload pools unless `--vram-reserve-mib` is passed. - Wave64 AMD GPUs (`gfx906`) are excluded.
関連リンク
インストール・モデル・リリースへのサイト内リンク。