#6409 [Engine] Qwen3.8-Next-Flash on Strix Halo AI Max 395 (128GB UMA): tuning for 2 concurrent streams at >100 tok/s each @ 200k context
open · @bobwei192-star · 2 comentários · No GitHub
BenchmarksSetup & installAMD / HIPModels & quantsSecurityLinux
Descrição
### Area Engine ### Your question I run Strata on a Strix Halo (Ryzen AI Max+ 395) machine with 128GB unified memory and no discrete VRAM. Use case: CI/CD daily work — Jenkins pipeline maintenance, test-case authoring, debug and log analysis. My prompts repeat a lot of template content, and I need two streams to run at the same time. Target configuration: - Model: Qwen3.8-Next-Flash - Concurrency: 2 simultaneous requests - Context: 200k tokens per request - Goal: sustain >100 tok/s decode per stream while both requests run at the same time Two things I am asking about: 1. Which optimization directions are worth tuning for this hardware and workload — prefix cache reuse for the repeated template prompts, KV cache layout under unified memory, ROCm/HIP kernel settings, and scheduling strategy for concurrent streams? 2. Which metrics and test points should a benchmark on this machine cover to validate the goal above (e.g. prefill vs decode split, per-stream tok/s under concurrency, TTFT, KV/prefix cache hit rate, memory headroom at 200k)? ### Strata version 0.2.14.1 (build 0.1.60.1), from `strata --version`. ### System AMD Ryzen AI Max+ 395 (Strix Halo), 128GB unified memory, Linux. ### What you tried Strata 0.2.14.1 is installed here, but I have not downloaded the model or run any measurement yet. This issue is the scoping question before I set up a benchmark, so I am not reporting any numbers or claiming any speed.
No site
Links install, modelos, releases.