#2260 bench: 4xV100 32GB Q4_K_XL-DN4, 8-way concurrent 204 tok/s

Beschreibung

## bench: 4xV100 32GB Q4_K_XL-DN4, 8-way concurrent 204 tok/s

Date: 2026-10-13 (Asia/Shanghai)
Hardware: G292-Z20, 4 x Tesla V100 32GB (PCIe), EPYC 7K62, 128GB RAM
Engine: Strata 0.1.43.1
Model: Qwen3.8-Flash-Next Q4_K_XL-DN4
Config: 4-GPU auto layer split, 32K context, KV int8, temperature=0, thinking=false

### Method
- Same model and 4-GPU config as previous 4-way test; temporary parallel changed from 4 to 8. Production config untouched.
- 8 simultaneously-started requests, 512 tokens max each, 2 rounds. Distinct request IDs.
- Total throughput includes prompt processing and first-token wait. Model load time excluded.
- Short synthetic load, not representative of long-context or sustained full-load.

### Results
- Round 1: 200.2 tok/s total, per-request 26.4-40.2 tok/s
- Round 2: 207.8 tok/s total, per-request 27.4-40.4 tok/s
- Average: 204.0 tok/s
- Previous 4-way: 154.7 tok/s, +31.9 percent with 8-way
- 16/16 requests succeeded; 8192 tokens total; all finish_reason=length.

### Thermals (no throttling observed)
- GPU0: 56C core / 58C HBM / 26463 MiB
- GPU1: 52C core / 55C HBM / 26255 MiB
- GPU2: 50C core / 54C HBM / 26255 MiB
- GPU3: 57C core / 59C HBM / 28029 MiB
- Fans at 15600-16200 RPM during test, restored to quiet after.

### Verdict
8-way increases total throughput but reduces per-request speed and increases first-token wait. Better for batch generation; for interactive chat, 4-way may be preferable.