Pull requests / #2166

#2166 bench: RTX 5090 + RTX 3090 two-host stage-RPC UD-Q4_K_XL results

open · @petria · 0 comments · View on GitHub

BenchmarksServer & APINVIDIA / CUDADocumentationLinux

Description

## Summary

Community speed and repository-only correctness results for original Unsloth
Qwen3.8-Flash-Next UD-Q4_K_XL, running on an RTX 5090 coordinator and RTX 3090
prefix worker in **two separate Linux hosts**. The engine is Strata 0.1.43 plus
local, uncommitted stage-RPC changes, not stock Strata network RPC support.

## Results

The original speed measurements remain unchanged: three repetitions per test,
36 measured requests plus 4 warm-up/calibration requests, no failures or skips.
Short-chat decode median **71.3 tokens/s**; fresh 4K/16K/32K prefill
**1226.4 / 1420.4 / 1446.1 tokens/s**. Agent follow-up decode median
**88.7 tokens/s**, conversation-prefix reuse **68.781%**.

The separate quality follow-up used only tests/fixtures already in this repository:

- Unchanged `tools/needle_bench.py`: **6/6 recall cases passed**, at 32k/128k and
  depths 10/50/90; actual inputs 32,804–32,805 and 124,961–124,963 tokens.
- The three existing RX 9060 XT community TypeScript fixtures, through fresh
  Codex CLI sessions with medium reasoning: **9/9 attempts passed** (three per
  fixture). Independently reran the original acceptance tests in read-only,
  network-disabled Bubblewrap copies: **243/243 checks**, representing **81
  distinct acceptance cases repeated three times**, not 243 distinct problems.
- **64 completed tool calls**, no failed agent turns or timeouts. Two nonzero
  shell exits were `git status` in a non-Git workspace and `grep -c` finding zero
  failed tests; neither was a backend or acceptance-test failure.
- **77 native requests**, 3,364,168 cumulative prompt tokens, 2,189,767 reused,
  and 26,866 generated (including reasoning). Weighted throughput across the
  quality collection: **1427.8 fresh-prompt tokens/s**, **81.2 decode tokens/s**;
  MTP acceptance **72.02%**. These are separate from the original speed suite.
- Both-node follow-up telemetry: **515 samples**; observed Strata worker-process
  VRAM **17.63 GiB**, distinguished from whole-card desktop allocations.

## What changed

Only the results folder
`bench/results/2026-10-12-community-rtx5090-rtx3090-stage-rpc-udq4/` changes:
the original README/reproduction notes link to a new `quality/` directory.
It contains method/scoring limitations, raw recall results, per-attempt coding
results, unchanged generated source, independent acceptance logs, sanitized tool
events, native per-request measurements, both-node telemetry, provenance and
portable reproduction scripts. No gistfile1/fable task or custom coding challenge.

## Validation

- All 77 native request records reconcile exactly with start/end counters.
- Fixture tests/instructions unchanged, allowed source-file scope checked,
  generated solution hashes preserved, required refactoring helpers reviewed.
- Both engines' configurations, binary hashes, base revision, tracked-source
  diff, service PID and invocation identities stayed unchanged. Existing stack
  was not restarted, rebuilt or reconfigured and remains running.
- Original speed data validated unchanged; JSON/JSONL, local links, Python/shell
  syntax, privacy and results-only diff checked. Generated sources preserve
  original bytes/CRLF line endings; Git whitespace checks use `cr-at-eol`.
- Published collector has an offline-tested fallback for disabled optional
  request-body monitoring; HTTP 404 still yields native metrics, while genuine
  HTTP errors remain visible. This collection-only correction does not change
  any measured inference request; measured/published controller hashes differ
  explicitly in provenance.

## Limitations and collection incident

This PR **does not add RPC code to upstream**. Custom engine source changes and
binaries are not distributed, so the upstream base commit alone cannot recreate
the experimental network deployment. That limitation remains explicit.

The quality follow-up's first collector required optional `/api/requests`, which
was disabled (HTTP 404). A metrics-only collector recovered sampling without
engine changes or replaying tests. Hardware sampling begins **138.5 seconds**
after collection started: early memory peaks are unmeasured. All completed
request timings were recovered from retained native history. Native request-body
monitor TTFT/separate reasoning-token counts are unavailable; Codex usage is
kept separately where exposed.

Warm expert cache and an active desktop; no stock/single-GPU or same-day A/B
comparison, cold-cache, vision/concurrency, full 192K coding, or broad coding
leaderboard assessment. Coding acceptance tests were visible to the agent.
No universal quality/speedup claim. No models, generated packs, executable
binaries, credentials, private addresses, raw request bodies, personal session
history or TLS keys are included.

Related on strata.com

Editorial links to help you install, pick models, or read release notes — not part of the upstream thread.