Pull requests / #2320

#2320 bench: Unsloth UD-Q4_K_XL two-host RTX 5090/3090 rerun and quality checks

open · @petria · 0 comentários · No GitHub

BenchmarksServer & APINVIDIA / CUDAModels & quantsWindowsLinux

Descrição

## Summary

New 2026-10-13 results for Unsloth Qwen3.8-Flash-Next **UD-Q4_K_XL** on an RTX 5090 coordinator and RTX 3090 prefix worker in **two separate Linux hosts**. This is a rerun of the model reported in #2166, not a different model or an update to that PR.

The deployment uses Strata 0.1.43 plus our retained custom stage-RPC changes; this PR **does not add RPC support to upstream**.

## Configuration and method

- 196,608 context, INT8 GPU-resident KV, 24/24 layers, fixed 2048 prefill, resident experts, MTP4/minimum probability .5 and retained suffix lookup.
- Worker/coordinator VRAM reserves 4096/1000 MiB; CPU pools 11/31; one active sequence. Vision enabled but not exercised.
- Frozen, unchanged upstream tools and public corpus at `7695859aa13afb248bdb4d36f9b2e95707db079d`; 517 consumed source inputs and all 14 coding-fixture files verified against that commit.
- Both-node telemetry begins before the first request; optional disabled request monitoring no longer causes a sampling gap.

## Results

- Built-in speed suite: **36/36 measured requests**, three repeats, four excluded warm-up/calibration requests. Chat decode median **66.4 tok/s**; fresh 4K/16K/32K prefill **1177.2 / 1362.8 / 1399.1 tok/s**; agent-cache follow-up decode **81.1 tok/s**.
- Unchanged repository recall test: **6/6**, 32K/128K targets at depths 10/50/90, fixed seed 7, reasoning off.
- Three existing RX 9060 XT community TypeScript fixtures through nine fresh ephemeral Codex sessions with medium reasoning: **9/9 attempts passed**. Independent read-only, network-disabled acceptance checks: **243/243**, representing **81 distinct cases repeated three times**.
- **65 completed tool calls**; no failed agent turns, timeouts or scope violations. One nonzero shell exit was `git status` in a non-Git fixture, not a backend or acceptance failure.
- Quality collection: **76 native requests**, **3,315,714 prompt tokens**, **2,138,255 reused**, **22,678 generated** including reasoning/tool serialization; weighted fresh prefill **1313.8 tok/s**, decode **76.4 tok/s**, draft acceptance **73.03%**. These are separate from the reasoning-off speed summaries.
- **100 speed + 601 quality telemetry samples**, no collector errors. Worker inference-process sampled VRAM peak **15.91 GiB**, below the 20-GiB guard; desktop allocations are reported separately.

## Comparison and limitations

The worker automatic expert cache is now **3522 slots / 10526 MiB**, versus **4110 / 12292 MiB** in #2166; desktop load and generated answers also differ. The report includes the original Unsloth speed comparison and retained GSQ IQ3_S shared-workload raw data, with the 128K/16+32 versus 192K/24+24 differences explicit. This is not an isolated model-quality, hardware A/B or RPC-overhead claim.

Only repository-provided speed/recall/coding fixtures were run: no gistfile1/fable or custom challenge. Visible acceptance tests, three small coding fixtures, adaptive/warm cache, no vision/concurrency/cold-cache/full-192K quality assessment. NIC counters include other interface traffic. Peaks are sampled lower bounds.

The custom engine source/binaries are not distributed, so upstream base alone cannot reproduce network RPC. Replay scripts assume a compatible existing deployment. An initial missing remote workload-directory preflight issued zero inference requests; corrected/verified staging preceded the published run and its failed preflight archive remains private.

## Validation and contents

Only `bench/results/2026-10-13-community-rtx5090-rtx3090-stage-rpc-unsloth-udq4-rerun/` is added: report, per-request results, both-node telemetry, configs/provenance, generated source, acceptance/tool logs and replay scripts.

- All native request/token/timing/draft records reconcile with boundary counters.
- Engine configurations, binaries, source/diff identities and service invocations stayed unchanged throughout the tests.
- Instructions/tests preserved; source-only scope and helper usage reviewed; generated bytes/CRLF and hashes preserved.
- Offline collector/redaction/accounting regressions pass; public JSON/JSONL, relative links, Python/shell syntax, privacy and results-only diff checked. Git whitespace checks use `cr-at-eol` for preserved Windows-origin CRLF.
- No models, packs, executables, TLS keys, credentials, private addresses, request bodies or personal sessions included. Existing #2166 is unchanged.

No site

Links install, modelos, releases.