Pull requests / #2166
#2166 bench: RTX 5090 + RTX 3090 two-host stage-RPC UD-Q4_K_XL results
open · @petria · 0 comentários · No GitHub
BenchmarksServer & APINVIDIA / CUDADocumentationLinux
Descrição
## Summary Community speed and repository-only correctness results for original Unsloth Qwen3.8-Flash-Next UD-Q4_K_XL, running on an RTX 5090 coordinator and RTX 3090 prefix worker in **two separate Linux hosts**. The engine is Strata 0.1.43 plus local, uncommitted stage-RPC changes, not stock Strata network RPC support. ## Results The original speed measurements remain unchanged: three repetitions per test, 36 measured requests plus 4 warm-up/calibration requests, no failures or skips. Short-chat decode median **71.3 tokens/s**; fresh 4K/16K/32K prefill **1226.4 / 1420.4 / 1446.1 tokens/s**. Agent follow-up decode median **88.7 tokens/s**, conversation-prefix reuse **68.781%**. The separate quality follow-up used only tests/fixtures already in this repository: - Unchanged `tools/needle_bench.py`: **6/6 recall cases passed**, at 32k/128k and depths 10/50/90; actual inputs 32,804–32,805 and 124,961–124,963 tokens. - The three existing RX 9060 XT community TypeScript fixtures, through fresh Codex CLI sessions with medium reasoning: **9/9 attempts passed** (three per fixture). Independently reran the original acceptance tests in read-only, network-disabled Bubblewrap copies: **243/243 checks**, representing **81 distinct acceptance cases repeated three times**, not 243 distinct problems. - **64 completed tool calls**, no failed agent turns or timeouts. Two nonzero shell exits were `git status` in a non-Git workspace and `grep -c` finding zero failed tests; neither was a backend or acceptance-test failure. - **77 native requests**, 3,364,168 cumulative prompt tokens, 2,189,767 reused, and 26,866 generated (including reasoning). Weighted throughput across the quality collection: **1427.8 fresh-prompt tokens/s**, **81.2 decode tokens/s**; MTP acceptance **72.02%**. These are separate from the original speed suite. - Both-node follow-up telemetry: **515 samples**; observed Strata worker-process VRAM **17.63 GiB**, distinguished from whole-card desktop allocations. ## What changed Only the results folder `bench/results/2026-10-12-community-rtx5090-rtx3090-stage-rpc-udq4/` changes: the original README/reproduction notes link to a new `quality/` directory. It contains method/scoring limitations, raw recall results, per-attempt coding results, unchanged generated source, independent acceptance logs, sanitized tool events, native per-request measurements, both-node telemetry, provenance and portable reproduction scripts. No gistfile1/fable task or custom coding challenge. ## Validation - All 77 native request records reconcile exactly with start/end counters. - Fixture tests/instructions unchanged, allowed source-file scope checked, generated solution hashes preserved, required refactoring helpers reviewed. - Both engines' configurations, binary hashes, base revision, tracked-source diff, service PID and invocation identities stayed unchanged. Existing stack was not restarted, rebuilt or reconfigured and remains running. - Original speed data validated unchanged; JSON/JSONL, local links, Python/shell syntax, privacy and results-only diff checked. Generated sources preserve original bytes/CRLF line endings; Git whitespace checks use `cr-at-eol`. - Published collector has an offline-tested fallback for disabled optional request-body monitoring; HTTP 404 still yields native metrics, while genuine HTTP errors remain visible. This collection-only correction does not change any measured inference request; measured/published controller hashes differ explicitly in provenance. ## Limitations and collection incident This PR **does not add RPC code to upstream**. Custom engine source changes and binaries are not distributed, so the upstream base commit alone cannot recreate the experimental network deployment. That limitation remains explicit. The quality follow-up's first collector required optional `/api/requests`, which was disabled (HTTP 404). A metrics-only collector recovered sampling without engine changes or replaying tests. Hardware sampling begins **138.5 seconds** after collection started: early memory peaks are unmeasured. All completed request timings were recovered from retained native history. Native request-body monitor TTFT/separate reasoning-token counts are unavailable; Codex usage is kept separately where exposed. Warm expert cache and an active desktop; no stock/single-GPU or same-day A/B comparison, cold-cache, vision/concurrency, full 192K coding, or broad coding leaderboard assessment. Coding acceptance tests were visible to the agent. No universal quality/speedup claim. No models, generated packs, executable binaries, credentials, private addresses, raw request bodies, personal session history or TLS keys are included.
No site
Links install, modelos, releases.