Pull requests / #2166
#2166 bench: RTX 5090 + RTX 3090 two-host stage-RPC UD-Q4_K_XL results
open · @petria · 0 comments · View on GitHub
BenchmarksServer & APINVIDIA / CUDADocumentationLinux
Description
## Summary Community speed and repository-only correctness results for original Unsloth Qwen3.8-Flash-Next UD-Q4_K_XL, running on an RTX 5090 coordinator and RTX 3090 prefix worker in **two separate Linux hosts**. The engine is Strata 0.1.43 plus local, uncommitted stage-RPC changes, not stock Strata network RPC support. ## Results The original speed measurements remain unchanged: three repetitions per test, 36 measured requests plus 4 warm-up/calibration requests, no failures or skips. Short-chat decode median **71.3 tokens/s**; fresh 4K/16K/32K prefill **1226.4 / 1420.4 / 1446.1 tokens/s**. Agent follow-up decode median **88.7 tokens/s**, conversation-prefix reuse **68.781%**. The separate quality follow-up used only tests/fixtures already in this repository: - Unchanged `tools/needle_bench.py`: **6/6 recall cases passed**, at 32k/128k and depths 10/50/90; actual inputs 32,804–32,805 and 124,961–124,963 tokens. - The three existing RX 9060 XT community TypeScript fixtures, through fresh Codex CLI sessions with medium reasoning: **9/9 attempts passed** (three per fixture). Independently reran the original acceptance tests in read-only, network-disabled Bubblewrap copies: **243/243 checks**, representing **81 distinct acceptance cases repeated three times**, not 243 distinct problems. - **64 completed tool calls**, no failed agent turns or timeouts. Two nonzero shell exits were `git status` in a non-Git workspace and `grep -c` finding zero failed tests; neither was a backend or acceptance-test failure. - **77 native requests**, 3,364,168 cumulative prompt tokens, 2,189,767 reused, and 26,866 generated (including reasoning). Weighted throughput across the quality collection: **1427.8 fresh-prompt tokens/s**, **81.2 decode tokens/s**; MTP acceptance **72.02%**. These are separate from the original speed suite. - Both-node follow-up telemetry: **515 samples**; observed Strata worker-process VRAM **17.63 GiB**, distinguished from whole-card desktop allocations. ## What changed Only the results folder `bench/results/2026-10-12-community-rtx5090-rtx3090-stage-rpc-udq4/` changes: the original README/reproduction notes link to a new `quality/` directory. It contains method/scoring limitations, raw recall results, per-attempt coding results, unchanged generated source, independent acceptance logs, sanitized tool events, native per-request measurements, both-node telemetry, provenance and portable reproduction scripts. No gistfile1/fable task or custom coding challenge. ## Validation - All 77 native request records reconcile exactly with start/end counters. - Fixture tests/instructions unchanged, allowed source-file scope checked, generated solution hashes preserved, required refactoring helpers reviewed. - Both engines' configurations, binary hashes, base revision, tracked-source diff, service PID and invocation identities stayed unchanged. Existing stack was not restarted, rebuilt or reconfigured and remains running. - Original speed data validated unchanged; JSON/JSONL, local links, Python/shell syntax, privacy and results-only diff checked. Generated sources preserve original bytes/CRLF line endings; Git whitespace checks use `cr-at-eol`. - Published collector has an offline-tested fallback for disabled optional request-body monitoring; HTTP 404 still yields native metrics, while genuine HTTP errors remain visible. This collection-only correction does not change any measured inference request; measured/published controller hashes differ explicitly in provenance. ## Limitations and collection incident This PR **does not add RPC code to upstream**. Custom engine source changes and binaries are not distributed, so the upstream base commit alone cannot recreate the experimental network deployment. That limitation remains explicit. The quality follow-up's first collector required optional `/api/requests`, which was disabled (HTTP 404). A metrics-only collector recovered sampling without engine changes or replaying tests. Hardware sampling begins **138.5 seconds** after collection started: early memory peaks are unmeasured. All completed request timings were recovered from retained native history. Native request-body monitor TTFT/separate reasoning-token counts are unavailable; Codex usage is kept separately where exposed. Warm expert cache and an active desktop; no stock/single-GPU or same-day A/B comparison, cold-cache, vision/concurrency, full 192K coding, or broad coding leaderboard assessment. Coding acceptance tests were visible to the agent. No universal quality/speedup claim. No models, generated packs, executable binaries, credentials, private addresses, raw request bodies, personal session history or TLS keys are included.
Related on strata.com
Editorial links to help you install, pick models, or read release notes — not part of the upstream thread.