贡献 / #2166

#2166 bench: RTX 5090 + RTX 3090 two-host stage-RPC UD-Q4_K_XL results

open · @petria · 0 评论 · 去 GitHub 看

BenchmarksServer & APINVIDIA / CUDADocumentationLinux

说明

## Summary

Community speed and repository-only correctness results for original Unsloth
Qwen3.8-Flash-Next UD-Q4_K_XL, running on an RTX 5090 coordinator and RTX 3090
prefix worker in **two separate Linux hosts**. The engine is Strata 0.1.43 plus
local, uncommitted stage-RPC changes, not stock Strata network RPC support.

## Results

The original speed measurements remain unchanged: three repetitions per test,
36 measured requests plus 4 warm-up/calibration requests, no failures or skips.
Short-chat decode median **71.3 tokens/s**; fresh 4K/16K/32K prefill
**1226.4 / 1420.4 / 1446.1 tokens/s**. Agent follow-up decode median
**88.7 tokens/s**, conversation-prefix reuse **68.781%**.

The separate quality follow-up used only tests/fixtures already in this repository:

- Unchanged `tools/needle_bench.py`: **6/6 recall cases passed**, at 32k/128k and
  depths 10/50/90; actual inputs 32,804–32,805 and 124,961–124,963 tokens.
- The three existing RX 9060 XT community TypeScript fixtures, through fresh
  Codex CLI sessions with medium reasoning: **9/9 attempts passed** (three per
  fixture). Independently reran the original acceptance tests in read-only,
  network-disabled Bubblewrap copies: **243/243 checks**, representing **81
  distinct acceptance cases repeated three times**, not 243 distinct problems.
- **64 completed tool calls**, no failed agent turns or timeouts. Two nonzero
  shell exits were `git status` in a non-Git workspace and `grep -c` finding zero
  failed tests; neither was a backend or acceptance-test failure.
- **77 native requests**, 3,364,168 cumulative prompt tokens, 2,189,767 reused,
  and 26,866 generated (including reasoning). Weighted throughput across the
  quality collection: **1427.8 fresh-prompt tokens/s**, **81.2 decode tokens/s**;
  MTP acceptance **72.02%**. These are separate from the original speed suite.
- Both-node follow-up telemetry: **515 samples**; observed Strata worker-process
  VRAM **17.63 GiB**, distinguished from whole-card desktop allocations.

## What changed

Only the results folder
`bench/results/2026-10-12-community-rtx5090-rtx3090-stage-rpc-udq4/` changes:
the original README/reproduction notes link to a new `quality/` directory.
It contains method/scoring limitations, raw recall results, per-attempt coding
results, unchanged generated source, independent acceptance logs, sanitized tool
events, native per-request measurements, both-node telemetry, provenance and
portable reproduction scripts. No gistfile1/fable task or custom coding challenge.

## Validation

- All 77 native request records reconcile exactly with start/end counters.
- Fixture tests/instructions unchanged, allowed source-file scope checked,
  generated solution hashes preserved, required refactoring helpers reviewed.
- Both engines' configurations, binary hashes, base revision, tracked-source
  diff, service PID and invocation identities stayed unchanged. Existing stack
  was not restarted, rebuilt or reconfigured and remains running.
- Original speed data validated unchanged; JSON/JSONL, local links, Python/shell
  syntax, privacy and results-only diff checked. Generated sources preserve
  original bytes/CRLF line endings; Git whitespace checks use `cr-at-eol`.
- Published collector has an offline-tested fallback for disabled optional
  request-body monitoring; HTTP 404 still yields native metrics, while genuine
  HTTP errors remain visible. This collection-only correction does not change
  any measured inference request; measured/published controller hashes differ
  explicitly in provenance.

## Limitations and collection incident

This PR **does not add RPC code to upstream**. Custom engine source changes and
binaries are not distributed, so the upstream base commit alone cannot recreate
the experimental network deployment. That limitation remains explicit.

The quality follow-up's first collector required optional `/api/requests`, which
was disabled (HTTP 404). A metrics-only collector recovered sampling without
engine changes or replaying tests. Hardware sampling begins **138.5 seconds**
after collection started: early memory peaks are unmeasured. All completed
request timings were recovered from retained native history. Native request-body
monitor TTFT/separate reasoning-token counts are unavailable; Codex usage is
kept separately where exposed.

Warm expert cache and an active desktop; no stock/single-GPU or same-day A/B
comparison, cold-cache, vision/concurrency, full 192K coding, or broad coding
leaderboard assessment. Coding acceptance tests were visible to the agent.
No universal quality/speedup claim. No models, generated packs, executable
binaries, credentials, private addresses, raw request bodies, personal session
history or TLS keys are included.

本站相关内容

相关页面的快捷入口。