#2314 [Community Research] Strata Hardware Requirements — 4 CPUs × 4 GPUs × 3 Quantizations

Description

# Strata Hardware Requirements — Multi-Generation CPU/GPU Benchmark Research

## Overview

This issue tracks a community-driven hardware research and benchmarking project focused on understanding the **real-world hardware requirements and performance characteristics of Strata**.

The goal is to systematically benchmark Strata across multiple generations of consumer CPUs and NVIDIA GPUs, using hardware available for testing.

Rather than focusing exclusively on maximum tokens per second, this research aims to determine:

- What hardware is actually required to run Strata effectively?
- How much does CPU performance affect inference?
- How important are system RAM capacity and memory bandwidth?
- How much GPU VRAM is practically necessary?
- How significantly does PCIe bandwidth affect performance?
- What are the performance differences between GPU generations?
- How does quantization affect hardware requirements and throughput?
- Can older, inexpensive hardware provide a practical local LLM experience?
- How much performance can be retained at reduced GPU power limits?
- Which hardware configurations provide the best practical performance per watt?

The project will use a consistent benchmark methodology and publish raw results through separate pull requests.

The intention is to produce a reproducible dataset that can help other users make informed hardware decisions when deploying Strata.

---

## 1. Research Objectives

### 1.1 GPU Requirements

Evaluate Strata across four generations of NVIDIA GPUs:

| GPU | Architecture | VRAM |
|---|---|---:|
| RTX 2070 SUPER | Turing | 8 GB |
| RTX 3080 | Ampere | 10 GB |
| RTX 4080 SUPER | Ada Lovelace | 16 GB |
| RTX 5080 | Blackwell | 16 GB |

Research questions:

- How does GPU architecture influence inference performance?
- How important is VRAM capacity compared with GPU compute performance?
- What happens when model weights cannot fit entirely in VRAM?
- How much does partial CPU/RAM offloading affect performance?
- Can an 8 GB GPU provide a usable long-context inference experience?
- How much practical improvement do newer GPU generations provide?
- Is a faster GPU always better when VRAM capacity is limited?

### 1.2 CPU Requirements

Evaluate four generations of consumer CPU platforms:

| CPU | Architecture | Memory Platform | PCIe Generation |
|---|---|---|---|
| Intel Core i7-6700K | Skylake | DDR4 | PCIe 3.0 |
| AMD Ryzen 9 3900X | Zen 2 | DDR4 | PCIe 4.0 |
| AMD Ryzen 9 5950X | Zen 3 | DDR4 | PCIe 4.0 |
| AMD Ryzen 9 7950X3D | Zen 4 | DDR5 | PCIe 5.0 |

Research questions:

- How much does CPU performance influence GPU inference?
- Does CPU architecture significantly affect prefill throughput?
- Does higher CPU core count improve inference performance?
- How important is CPU single-thread performance?
- Can older CPUs effectively support modern GPUs?
- How much performance is lost when using PCIe 3.0 instead of PCIe 4.0?
- Does DDR5 provide meaningful improvements over DDR4?
- When does CPU-side inference become the primary bottleneck?

**Important:** CPU generation, memory bandwidth, PCIe capabilities, and platform characteristics are not independent variables.

Cross-platform comparisons will therefore be treated as whole-system comparisons unless individual factors can be isolated through controlled experiments.

### 1.3 System RAM Requirements

Investigate the relationship between:

- Installed system RAM
- Available system RAM
- Model quantization
- Context length
- KV cache configuration
- GPU VRAM capacity
- CPU/GPU offloading
- Inference throughput

Research questions:

- What is the minimum practical RAM requirement?
- How much RAM is consumed by different quantizations?
- How does RAM consumption scale with context length?
- How much additional RAM is required when using GPUs with limited VRAM?
- Does additional RAM capacity improve performance, or only allow larger workloads?
- How important is system memory bandwidth?

Where possible, RAM capacity and memory configuration will be tested independently.

If different systems use different RAM capacities, those differences will be documented.

### 1.4 PCIe Bandwidth

Investigate the impact of PCIe bandwidth on inference.

Particular attention will be paid to:

- PCIe 3.0 vs PCIe 4.0
- Negotiated PCIe link width
- CPU/GPU data transfers
- Partial model offloading
- Long-context prefill performance
- Time to first token

Where supported, controlled PCIe link-speed experiments may be performed on the same CPU/GPU configuration.

This will help distinguish PCIe limitations from differences caused by CPU architecture.

### 1.5 Quantization

The following Strata quantizations will be evaluated:

| Quantization | Included |
|---|---|
| IQ3_S | Yes |
| IQ3_XXS | Yes |
| IQ2_XS | Yes |
| Q2_0 | No |

The objective is to determine how quantization influences:

- Prefill throughput
- Decode throughput
- VRAM utilization
- System RAM utilization
- CPU/GPU offloading
- Long-context performance
- Energy consumption
- Minimum practical hardware requirements

This benchmark series primarily evaluates **performance and hardware requirements**.

Model quality, coding accuracy, reasoning reliability, and agent task completion require separate evaluations.

No conclusions about model quality will be drawn from throughput measurements alone.

---

## 2. Hardware Test Matrix

The objective is to test **every listed GPU with every listed CPU**, using all three quantizations.

Each completed CPU/GPU combination will include all supported power levels and context lengths.

### Intel Core i7-6700K

| GPU | IQ3_S | IQ3_XXS | IQ2_XS |
|---|---|---|---|
| RTX 2070 SUPER | [ ] | [ ] | [ ] |
| RTX 3080 | [ ] | [ ] | [ ] |
| RTX 4080 SUPER | [ ] | [ ] | [ ] |
| RTX 5080 | [ ] | [ ] | [ ] |

### AMD Ryzen 9 3900X

| GPU | IQ3_S | IQ3_XXS | IQ2_XS |
|---|---|---|---|
| RTX 2070 SUPER | [ ] | [ ] | [ ] |
| RTX 3080 | [ ] | [ ] | [ ] |
| RTX 4080 SUPER | [ ] | [ ] | [ ] |
| RTX 5080 | [x] | [ ] | [ ] |

### AMD Ryzen 9 5950X

| GPU | IQ3_S | IQ3_XXS | IQ2_XS |
|---|---|---|---|
| RTX 2070 SUPER | [ ] | [ ] | [ ] |
| RTX 3080 | [ ] | [ ] | [ ] |
| RTX 4080 SUPER | [ ] | [ ] | [ ] |
| RTX 5080 | [ ] | [ ] | [ ] |

### AMD Ryzen 9 7950X3D

| GPU | IQ3_S | IQ3_XXS | IQ2_XS |
|---|---|---|---|
| RTX 2070 SUPER | [ ] | [ ] | [ ] |
| RTX 3080 | [ ] | [ ] | [ ] |
| RTX 4080 SUPER | [ ] | [ ] | [ ] |
| RTX 5080 | [ ] | [ ] | [ ] |

### Optional: AMD Threadripper 2950X

If time permits and the platform is operational, additional benchmarks will be performed using an AMD Threadripper 2950X.

| GPU | IQ3_S | IQ3_XXS | IQ2_XS |
|---|---|---|---|
| RTX 2070 SUPER | [ ] | [ ] | [ ] |
| RTX 3080 | [ ] | [ ] | [ ] |
| RTX 4080 SUPER | [ ] | [ ] | [ ] |
| RTX 5080 | [ ] | [ ] | [ ] |

Threadripper platform characteristics:

- Zen+ architecture
- 16 cores / 32 threads
- Quad-channel DDR4
- PCIe 3.0
- Support for large memory configurations

Research questions:

- Can quad-channel DDR4 compensate for older CPU architecture?
- How much does memory bandwidth affect CPU-side inference?
- How does the platform compare with Ryzen 3900X and 5950X?
- How significantly does PCIe 3.0 affect modern GPUs?
- Can an older workstation platform provide a cost-effective Strata deployment?

**Threadripper testing is optional and will not block completion of the primary research matrix.**

---

## 3. Benchmark Methodology

### 3.1 Context Lengths

Target context lengths:

| Context | Included |
|---|---|
| 1K | Yes |
| 4K | Yes |
| 8K | Yes |
| 16K | Yes |
| 32K | Yes |
| 64K | Yes |
| ~128K | Yes |

Each configuration will be tested using the same workload wherever technically possible.

Unsupported context lengths, allocation failures, and out-of-memory conditions will be recorded explicitly.

They will not be silently omitted from the results.

### 3.2 Performance Measurements

Collect:

- Prefill throughput (tokens/s)
- Decode throughput (tokens/s)
- Time to first token (TTFT)
- Total request latency
- Total benchmark execution time
- Successful and failed requests
- Long-context retrieval accuracy

Where supported, tests will be repeated three times to reduce measurement noise.

### 3.3 Long-Context Retrieval

Needle-in-a-haystack tests will be performed at:

- 32K context
- 64K context
- 96K context
- ~128K context

Target needle positions:

- 10%
- 50%
- 90%

The purpose is to verify that long-context configurations remain functional rather than merely accepting large input sizes.

These tests do not establish general reasoning or coding quality.

### 3.4 GPU Power Limits

Target power-limit sweep:

| Power Limit | Included |
|---|---|
| 100% | Yes |
| 95% | Yes |
| 90% | Yes |
| 85% | Yes |
| 80% | Yes |
| 75% | Yes |
| 70% | Yes |

Power limits will be applied relative to each GPU's configured default power limit.

Unsupported limits will be documented.

The benchmark will record the actual applied limit in watts.

### 3.5 Hardware Telemetry

Collect:

**GPU**
- Power consumption (W)
- Total GPU energy consumption (Wh)
- GPU utilization (%)
- GPU memory utilization
- GPU clock frequency
- GPU temperature
- Applied power limit
- PCIe link generation and width

**CPU**
- CPU utilization
- CPU model
- Core/thread count
- CPU frequency, where available
- CPU temperature, where available

**Memory**
- Installed system RAM
- Peak system RAM consumption
- RAM configuration and speed
- Peak VRAM consumption

**System**
- Operating system
- Kernel version
- NVIDIA driver version
- CUDA/runtime versions
- Strata version and commit
- Benchmark script version and commit

GPU energy consumption will be calculated from sampled GPU power telemetry.

**GPU energy is not equivalent to total system energy consumption.**

Whole-system power measurements may be added separately if appropriate measurement equipment becomes available.

---

## 4. Reproducibility Requirements

Every benchmark result should include sufficient metadata to reproduce the test.

### Required Configuration Metadata

```yaml
hardware:
  cpu: "AMD Ryzen 9 3900X"
  gpu: "NVIDIA RTX 5080"
  vram_gb: 16
  ram_gb: 64
  ram_type: "DDR4"
  pcie_generation: 4
  pcie_width: 16

software:
  os: "Ubuntu"
  kernel: "<version>"
  nvidia_driver: "<version>"
  strata_commit: "<commit>"
  benchmark_commit: "<commit>"

model:
  name: "Qwen3.8-Flash-Next"
  quantization: "IQ3_S"
  context_length: 131072
  kv_cache: "int8"
  speculation: 4

benchmark:
  power_limit_percent: 100
  power_limit_watts: 360
  repetitions: 3
```

Values shown above are examples and must be replaced with the actual configuration used.

### Configuration Consistency

The following settings should remain consistent wherever possible:

- Model family and version
- Quantization-specific model weights
- Context length
- KV cache configuration
- Speculative decoding configuration
- CPU/GPU offloading settings
- Benchmark workload
- Benchmark repetitions
- Telemetry collection method

Any differences must be explicitly documented.

When possible, all configurations should use the same Strata commit.

If software versions change during the research, affected results will be clearly identified.

---

## 5. Initial Results — Ryzen 9 3900X + RTX 5080

The first completed benchmark covers:

- AMD Ryzen 9 3900X
- NVIDIA RTX 5080 16 GB
- 64 GB DDR4
- PCIe 4.0
- IQ3_S quantization
- Seven GPU power limits

### Initial Observations

| Metric | 100% Power | 70% Power |
|---|---:|---:|
| Configured power limit | 360 W | 252 W |
| Benchmark duration | 671.97 s | 704.80 s |
| GPU energy consumption | 53.29 Wh | 46.11 Wh |
| Average GPU power | 285.74 W | 235.73 W |
| Prefill at ~128K | 3614.9 tok/s | 3412.0 tok/s |

Preliminary findings:

- Approximately 13.5% lower GPU energy consumption at 70% power.
- Approximately 4.9% longer total benchmark execution time.
- Approximately 5.6% lower prefill throughput at ~128K context.

These results suggest that reduced GPU power limits may provide meaningful energy savings with relatively