#3562 [Benchmark] Four RTX 4090: v0.2.1.4 TTFT and cache-hit comparison

本文

## Problem and result

On four RTX 4090 cards with 16 serving slots and four groups, the same-day upgrade from engine 0.1.45.3 to v0.2.1.4/source engine 0.1.46.3 increased 128K cache-hit TTFT from 0.933 to 1.619 seconds, despite faster decode. Sixteen-client total throughput fell from 256.07 to 208.17 token/s. Add the report and sanitized request evidence so the observed regressions can be reviewed.

## Related discussion

Refs #3560. This report provides evidence and does not close the issue.

## Approach

One results-only directory contains the human report, per-request timings/usage/hashes, source/build and configuration facts, conservative comparison exclusions, and the 52 public short-request payload references. Twelve long-context payload references remain private because they contain network/path material; the report explicitly limits byte-for-byte reproduction. Native inference, server code, model assets and defaults are unchanged by this PR.

## Validation

**Passed:**
- Real-model old/new arms: 64 requests and 19 groups each; all output length/retrieval checks passed. All old groups and 18/19 new groups had clean accounting.
- Three valid pairs for the reported 128K and 16-client comparisons matched input/output hashes, output tokens and native cache/prompt counts.
- Public short payload hashes were checked against the actual HTTP serialization and both arms' recorded hashes; original long payloads and private values are excluded.
- New engine/vision builds and offline checks passed; functional checks were separate (26 requests before a fresh performance restart, three text/image requests afterwards). Configuration, authentication and running source/binary hashes were verified.
- Report arithmetic, JSON/gzip parsing, artifact checksums, private-marker/credential scans and `git diff --check` were checked before publication.

**Failed / excluded:**
- The observed regressions are retained, not replaced by reruns.
- New 256K first-read repeat 1 was contaminated and had mismatched cache/prompt counts. Its following repeat is conservatively excluded too. Both 256K categories are INCONCLUSIVE for a three-pair claim.
- Initial service-test collection lacked two support-test modules; those import failures were preserved, then the 11 missing tests passed against unchanged Python production code. The combined evidence is 563 passes/eight optional-dependency skips.

**Not run:**
- A root-cause fix or profiling of the responsible function; standard-v1; full model quality; HIP/SYCL/Metal hardware; a new full model-weight hash collection; interleaved A/B/A or sustained-load qualification.

## Limits

This is a custom serving diagnostic on one host and three repetitions, with sequential arms. Long fixture privacy limits independent exact replay. Native decode timing and end-to-end throughput are separate measurements. Source build engine 0.1.46.3 is distinguished from the release's prebuilt engine package version. CI/maintainer approval and upstream merge are separate from the local validation above.