#3061 Community benchmark: R9700 vs RTX 5060 Ti — Qwen3.8-Flash-Next Q8_0 at 256K on HP Z4 G4; 3945WX server results

Description

Community benchmark report for the same HP Z4 G4 with two GPUs. At approximately 260K input, the R9700 measured 2.75× prefill and 1.50× decode versus the historical RTX 5060 Ti result. The later tuning sweep did not establish a further overall full-window speed improvement.

Benchmark orchestration, data processing, report preparation and sampled static code review were assisted by Codex. The evidence includes per-request measurements, exact paired prompts, generated answers, settings, checksums and review findings.

Measured 17–18 October 2026. I tested both cards in the same HP Z4 G4: i9-10940X (14 cores / 28 threads), 192 GB DDR4-2666 across four memory channels. The R9700 ran at PCIe 3.0 x16; the 5060 Ti ran at PCIe 3.0 x8. This is a host-specific Strata comparison, including CPU/RAM offload.

The deployed model is Qwen3.8-Flash-Next Q8_0, with a configured 262,144-token total context, int8 KV streaming and 32,768 resident KV cells. Strata 0.1.45 was pinned to commit 477ac13cbe8e582c20484871ce459d20d82df09d; AMD used ROCm 7.2.4. The R9700 results below use a 270 W ceiling. The service retained a 24-logical-core CPU quota, 148 GiB memory cap and no swap. Model weights were unchanged; no expert pruning was enabled.

## Matched comparison — medians of three fresh requests per size

| Input | 5060 Ti prefill t/s | R9700 prefill t/s | 5060 Ti decode t/s | R9700 decode t/s | First token, NVIDIA / AMD (s) |
| --- | ---: | ---: | ---: | ---: | ---: |
| Short | 70.0 | 50.9 | 22.4 | 37.0 | 2.90 / 3.92 |
| ~4K | 217.9 | 472.4 | 21.9 | 34.9 | 18.81 / 8.70 |
| ~16K | 407.0 | 1477.9 | 21.4 | 34.7 | 40.41 / 11.22 |
| ~32K | 536.8 | 1932.4 | 21.8 | 35.2 | 61.22 / 17.06 |
| ~260K | 635.2 | 1744.0 | 20.6 | 31.0 | 409.67 / 149.39 |

The same saved request payloads were used on both GPUs: thinking off, greedy generation, maximum 768 output tokens, natural stopping and zero cached input tokens. Output lengths vary. NVIDIA results are historical measurements on this HP, rather than interleaved reruns.

At about 260K input, the R9700 achieved 2.75x prefill and 1.50x decode; first-token latency fell from 6m 50s to 2m 29s. At about 32K, the ratios were 3.60x and 1.61x. These are per-request phase averages, not dashboard peaks.

## Actual full-window tests with 10,000 output tokens

Each measured request used 252,136 input + 10,000 output tokens, leaving eight tokens within the 262,144 capacity. Input and output share the context window. Three paired requests were completed for each configuration, with zero prompt-cache reuse.

| Configuration | Pooled prefill t/s | Pooled decode t/s | Median first token | Mean total time |
| --- | ---: | ---: | ---: | ---: |
| Existing 13-worker configuration | 1793.3 | 36.20 | 146.62 s | 417.06 s |
| 21-worker tuning candidate | 1718.1 | 36.74 | 146.94 s | 419.21 s |

The existing configuration completed the three full requests in 402.09–424.94 seconds (mean 6m 57s). Its individual decode averages were 36.7, 35.9 and 36.0 t/s. Pooled throughput means total tokens divided by total measured phase time; it is not an average of instantaneous samples.

The tuning candidate used 21 workers, 192 tasks and adapt-every 8, plus short-chunk CPU sharing. It had slightly higher pooled decode but about 0.52% worse mean total completion time. It was rejected. Forty eligible three-request 65K screening trials were completed, but their apparent winner did not establish an overall full-window improvement. Two trials overlapping a large file copy were excluded and repeated after the copy finished.

These 10K-output requests use synthetic retrieval records followed by long Python reference generation, with temperature 0.7, top_p 0.8, top_k 20, presence_penalty 1.5 and seed 42. Thinking was disabled and EOS was allowed. They are a different workload from the short-output matched GPU table, so 36.20 t/s must not be divided by the NVIDIA table's 20.6 t/s to claim another matched speedup.

## A useful short-prompt improvement

On the original 13-worker configuration, STRATA_PREFILL_CPU_SHARE=auto with STRATA_PREFILL_CPU_SHARE_MAX=1024 reduced median first-token latency from 3.93 s to 1.58 s. All three paired short requests improved, and all six passed the automatic checks. Mean latency reduction was 59.1%. The inputs were 195–196 tokens and answers were 219–251 tokens. Median total response time fell from 10.19 s to 7.80 s, although output lengths differed. Median decode was 37.0 versus 36.3 t/s: the gain was prompt latency.

Three 252,136-input/1,500-output follow-up screens also completed with that flag. Final confirmation with another three 10K-output requests was cancelled when I stopped testing to return the card. This remains a measured short-prompt improvement, not a fully validated combined replacement configuration.

## Correctness, images and operating limits

All six full-window 10K-output requests passed transport, token count, retrieval/arithmetic, repetition and complete-code syntax checks. Sampled manual review covered 18 initial code examples plus middle/end excerpts and found semantic errors: incorrect edge-case handling, an incorrect interval-intersection assertion, and inaccurate complexity claims. Generated code was not executed. The final incomplete entry at the 10K limit is expected truncation.

Earlier image validation passed four fresh-image tests, one cached image follow-up and two image-plus-long-context requests. A separate earlier document test passed 5/6 requests and 26/27 checked fields; one arithmetic answer was wrong. This shows usable context and image support, with ordinary model accuracy limitations.

At 300 W, one earlier long-context workload reached a 101 C hotspot and triggered the configured 100 C guard; subsequent reported comparison runs used 270 W. The earlier 270 W validation recorded peaks of 75 C edge / 96 C hotspot / 86 C memory and 76 C CPU. Those figures describe that validation interval, not an eight-hour thermal soak. No clocks, voltage or firmware were overclocked.

CPU usage could be raised above the observed 54% by adding workers, but that did not guarantee better completion time. A sampled 25-second full-window generation interval with 21 workers used about 23 logical cores on average without quota throttling. More CPU activity alone was not a reliable optimization target. AMD's missing live PCIe-traffic counter also did not imply a slow link: the saved startup transfer probe was about 13.7–13.8 GB/s on Gen3 x16; it is not an in-request bandwidth measurement.

## Additional server result: RTX 5060 Ti on Threadripper PRO 3945WX

A separate worker completed testing on the owner's AMD Threadripper PRO 3945WX server with 256 GB installed RAM and an RTX 5060 Ti 16 GB. AMD here identifies the CPU; the GPU is NVIDIA. Inference ran in a dedicated VM with GPU passthrough and 160 GiB allocated RAM. The service retained a 16-logical-core-equivalent CPU quota, 152 GiB memory limit and 148 GiB memory-high threshold. The benchmark exports do not independently establish the PCIe link mode, so no Gen4 attribution is made here.

The model was again Qwen3.8-Flash-Next Q8_0 on Strata 0.1.45, with 262,144 total context, int8 KV and 32,768 resident KV positions. Each full trial processed **251,681 input + 10,000 output tokens**, with zero prompt-cache reuse. There was **one full request per configuration**; model loading was excluded and a short warm-up followed each restart.

| Server configuration | Prefill t/s | Decode t/s | Total elapsed seconds |
| --- | ---: | ---: | ---: |
| long-baseline | 1040.2 | 25.0 | 642.14 |
| long-candidate | 758.9 | 25.3 | 727.26 |
| long-refined | 1071.2 | 26.5 | 612.95 |

The selected recorded run took **10m 13s**, versus 10m 42s for its baseline: **4.55% less elapsed time** in these single trials. Its launch settings included five CPU workers, prefill 14,848, spec-min-p 0.85, spec 4 and a 1,024 MiB VRAM reserve. The 16,384-prefill candidate was reduced by the engine to 8,192 because its buffers did not fit, and its full-request time regressed.

Earlier short-context confirmation used one 512-output-token request each for coding, explanation and fiction. The geometric mean decode rate rose from **26.51 to 27.66 t/s** (+4.3%) with five workers and spec-min-p 0.7. That earlier profile was subsequently superseded by the long-context 0.85 profile.

**Comparability:** these server trials used different synthetic prompts, temperature 0, a fixed seed and benchmark-only EOS suppression. The HP 10K-output suite used temperature 0.7, allowed natural EOS and had three runs per configuration. The server results are therefore a separate observation, not a controlled HP-vs-server or R9700-vs-5060 Ti speedup. Changes in CPU, VM allocation, PCIe, runtime settings and prompt content are not isolated. The 1,863-entry synthetic input recovered all three check values in each full trial, but the generated handbook was not comprehensively fact-checked. Image support was disabled on this server.

The attached ZIP covers the HP tests. The additional server timings below were transcribed from the other worker's saved Strata-256K-Benchmarks.json and Strata-Throughput-Benchmarks.json. The full request payload is not included for the server suite; its recorded SHA-256 is provided.

<details>
<summary>Additional server per-request evidence (JSON)</summary>


```json
{
  "measurement_date": "2026-10-18",
  "model": "qwen3.8-flash-next-q8_0",
  "strata_version": "0.1.45",
  "context_capacity": 262144,
  "request_sha256": "4d5e2aabe5748f06b906ad50243f9ff31c530398ad81d4012001d674b85c670c",
  "full_runs_per_configuration": 1,
  "benchmark_only_eos_suppression": true,
  "full_runs": [
    {
      "label": "long-baseline",
      "input_tokens": 251681,
      "output_tokens": 10000,
      "cached_tokens": 0,
      "prompt_ms": 241948.2,
      "decode_ms": 399977.3,
      "prefill_tps": 1040.2,
      "decode_tps": 25,
      "total_wall_s": 642.1366689650004,
      "draft_tokens": 7114,
      "accepted_draft_tokens": 5404,
      "all_three_retrieval_markers_present": true,
      "finish_reason": "length",
      "output_sha256": "92d8807ca7e23e877b44e486cbab42698650cf5cdcfc17ad2509ff98ecf0c552"
    },
    {
      "label": "long-candidate",
      "input_tokens": 251681,
      "output_tokens": 10000,
      "cached_tokens": 0,
      "prompt_ms": 331631.6,
      "decode_ms": 395424.1,
      "prefill_tps": 758.9,
      "decode_tps": 25.3,
      "total_wall_s": 727.2642409880009,
      "draft_tokens": 5660,
      "accepted_draft_tokens": 4599,
      "all_three_retrieval_markers_present": true,
      "finish_reason": "length",
      "requested_prefill": 16384,
      "effective_prefill": 8192,
      "spec_min_p": 0.85,
      "output_sha256": "a3c006ba906aa601f1fbaabaf4d4d1d35380efa08fc2f7c307ea85f29c1c4521"
    },
    {
      "label": "long-refined",
      "input_tokens": 251681,
      "output_tokens": 10000,
      "cached_tokens": 0,
      "prompt_ms": 234954.5,
      "decode_ms": 377767.4,
      "prefill_tps": 1071.2,
      "decode_tps": 26.5,
      "total_wall_s": 612.9474047219992,
      "draft_tokens": 6019,
      "accepted_draft_tokens": 5129,
      "all_three_retrieval_markers_present": true,
      "finish_reason": "length",
      "effective_prefill": 14848,
      "pool_workers": 5,
      "spec_min_p": 0.85,
      "output_sha256": "2bf0385ab300e11eed61e405f688927d49f5ba8601e89785ed7380df02d686cd"
    }
  ],
  "short_confirmation": {
    "method": "One 512-output-token request per task/configuration, geometric mean across the three task rates; earlier profile with spec-min-p 0.7",
    "original_decode_tps": {
      "coding": 27.9,
      "explanation": 26.4,
      "story": 25.3,
      "geometric_mean": 26.5120239726954
    },
    "candidate_decode_tps": {
      "coding": 28.7,
      "explanation": 26.9,
      "story": 27.4,
      "geometric_mean": 27.656353214633892
    }
  }
}
```

</details>

## Conclusion from the measurements

The R9700 materially im