#6038 [Engine] gfx1030 (2x Radeon PRO W6800 64GB, Linux HIP 0.1.59): DeepSeek-V4.1-Flash @ 256k context benchmarks, >15k multi-chunk prefill deadlock (#4211), and DS4_COPY_PIN kfd_ioctl stall
closed · @Net-Sentinel · 2 commentaires · Sur GitHub
BenchmarksMulti-GPUAMD / HIPModels & quantsWindowsLinux
Description
We run `Strata v0.2.13` (`strata-dsv41 0.1.59-linux-x86_64-hip-rx6000`) 24/7 in production on a Linux `gfx1030` (RDNA2) workstation serving both an autonomous OpenClaw agent (`25k–196k` token sessions) and Home Assistant Voice. Since there is very little Linux `gfx1030` / dual-W6800 data in the tracker (only single-card Windows in #5144, and #4211 noted that Linux HIP was untested for the layer-split chunked prompt fix), we wanted to share our production benchmarks, two reproducible Linux HIP / `gfx1030` failure modes, and the workarounds that made the rig 100% rock-solid at **262,144 (`256k`) context**. --- ## 1. Hardware & Software Rig - **GPUs:** **2x AMD Radeon PRO W6800 32GB** (`64 GB` GDDR6 total, `gfx1030`, `HIP_VISIBLE_DEVICES=1,2`) + 1x RX 6800 16GB (`GPU[0]`, isolated for Whisper/Plex) - **CPU & RAM:** AMD Ryzen 9 5950X (16C/32T) + **128 GB DDR4-3600** (4x32GB matched) - **OS & Kernel:** Ubuntu 24.04 LTS, kernel `6.8.0-149-generic` (`lru_gen.enabled=0 amd_pstate=active`), `THP=always` (`shmem_enabled=within_size`), `vm.swappiness=1` - **Engine:** `0.1.59-linux-x86_64-hip-rx6000/strata-dsv41` - **Pack:** `deepseek-v4.1-flash-ist-das-2.00bpw` (`552B` MoE + `110.6 GB` Engram on PCIe 4.0 NVMe) --- ## 2. Finding #1: Multi-Chunk Layer-Split Prefill Deadlock on Linux HIP (`>15k` tokens) — Extends #4211 With `--tp 2` on `deepseek-v4.1-flash-ist-das-2.00bpw`, `strata-dsv41` falls back to a 2-GPU layer split (`layers 0..20` on GPU 0, `layers 21..39 + head` on GPU 1) because `n_ff = 2304` is not evenly divisible by `2 * 256`. In #4211, `host_wait_at_least` was added before the HIP stream wait to fix a chunked-prefill deadlock, noting at the time that the HIP build was not tested. On `0.1.59-linux-x86_64-hip-rx6000`: - **Prompts `<= 10,000` tokens (single chunk):** Complete chunked prefill cleanly. - **Prompts `>= 15,000–17,000` tokens (crossing the multi-chunk handoff boundary):** Deadlock reproducibly inside **`run_dev_chunks` → `host_wait_at_least`**. - **Symptom:** `/proc/<pid>/task/*/wchan` shows all worker threads parked in `futex_wait_queue` except one thread spinning at `99.9%` CPU (`R`, user-space `host_wait_at_least` polling loop). Both W6800 GPUs drop to **`0%` utilization (`10–13W`)** and `rocm-smi` shows no active compute until `engine_silence_s` kills the engine. - **Workaround (100% reliable):** Passing **`--prefill-chunk 0`** forces the 16-token non-blocking windowed prefill path (`prefill_chunk = 0`). With `--prefill-chunk 0`, 20k–196k token agent prompts complete with zero deadlocks, and once cached in `ds4-slots`, warm turns prefill in **`366 ms`**. --- ## 3. Finding #2: `DS4_COPY_PIN=1` Causes a 303-Second `kfd_ioctl` Stall on Linux `amdgpu` Without Pre-Allocated `hugetlb` - On Linux `amdgpu` (`gfx1030`), forcing `DS4_COPY_PIN=1` without pre-allocated `2MB hugetlb` pages causes `hipHostRegister` / `kfd_ioctl` to stall for **~303 seconds** in kernel space while attempting to pin the `75.9 GB` CPU expert arena, triggering startup timeouts. - **Solution:** Leave `DS4_COPY_PIN` unset (respecting Strata's default `off` behavior on RDNA2 when `hugetlb` is absent) and instead set Linux Transparent Huge Pages to `always` (`shmem_enabled=within_size`) with `vm.swappiness=1` and `MemoryMax=infinity` in the systemd unit. - With that configuration, all **`8,994 / 8,994` CPU-lane experts load in `13.4s`** (`100%` RAM resident, zero SSD expert spill). --- ## 4. Production Benchmarks: Dual W6800 (`64 GB`) + `128 GB` DDR4 at `262,144` (`256k`) Context ### A. `deepseek-v4.1-flash-ist-das-2.00bpw` (`strata-dsv41 0.1.59`) Flags: `--tp 2 --max-context 262144 --reserve-mib 900 --cpu-threads 12 --prefill-chunk 0 --layer-split auto` Env: `STRATA_DS4_SLOT_KEEP=1` (`4` RAM slots + disk `ds4-slots`), `STRATA_DS4_PREFIX_KEEP=2` | Metric | Measured Result (`2x W6800 32GB + 5950X`) | |---|---| | **Max Context Window** | **`262,144` tokens (`256k`)** | | **Global KV Cache Footprint in VRAM** | **`0.34 GB` total** across the 4 global KV layers (`2, 8, 14, 20`) + 48 sliding-window (`512`) layers | | **CPU-Lane Expert Residency** | **`8,994 / 8,994` (`100%` in DDR4 RAM, `0` on SSD)** | | **Decode Throughput** | **`24.8 – 26.9 tok/s`** | | **Warm Prefix-Cache Reuse (`ds4-slots`)** | **`366 ms`** (`68` reused + `28` read) | | **Slot Swap Latency (Agent `35k` ctx ↔ Home Assistant Voice `3k` ctx)** | **`0.0 – 0.1 s`** | | **True Idle Power (`strata-dsv41` sleeping in `S` state)** | `0–4%` CPU, **`8W + 9W`** across the two W6800s | ### B. `qwen3.8-flash-next-iq3_s` (`125B`, `strata 0.1.59 rx6000`) - On `0.1.59`, `VRAM expert cache` holds **`9,728` experts (`17.52 GiB` VRAM)** (`80.7%–81.0%` expert-cache hit rate). - Combining `STRATA_HIP_PROMPT_F16=1`, `STRATA_SPEC_PROB=1`, and `--mtp .../rt --spec 2 --spec-min-p 0.95 --suffix-draft 0` completely eliminated earlier RDNA2 FP16 prefill NaN (`!`) collapses and MTP low-confidence token splices, sustaining **`45.0 tok/s` decode** and `1.0s` warm prefill on `7.5k`-token tool-calling turns. Happy to run any diagnostic builds or test a patch for the Linux HIP multi-chunk `run_dev_chunks` handoff on our dual-W6800 rig if useful to the team!
Sur le site
Liens install, modèles, releases.