#6463 [Engine] Claude Code's small side requests evict the agent's parked conversation (default conversation cache)

open · @ThatsR4d · 0 评论 · 去 GitHub 看

BenchmarksServer & APINVIDIA / CUDAModels & quantsLinux

说明

### Area

[Engine]  (speed, kernels, models, the server and API)

### Desktop app, Terminal or source?

Terminal (the `strata` command)

### What happened

Claude Code sends small side requests (80-190 tokens: titles, summaries, quick classifications) between the agent's own turns. With the default conversation cache (4 parked conversations, no minimum size) these tiny conversations take the parked slots and evict the agent's main conversation, so the next agent turn re-reads 18K-34K tokens from scratch. In our Claude Code sessions about 29% of task wall time was prompt reading because of it.

Setting `--conversation-cache-min-tokens 2048 --conversation-cache-slots 8 --conversation-cache-mib 16384` stopped it: the agent's follow-up turns now log e.g. `prompt 12395 tokens = 12280 reused + 115 read`.

Related, for the Claude Code notes: Claude Code 2.1.328 sends `output_config.effort: "medium"` (not "high") when the model name is not an Anthropic one, so the Qwen template adds no effort instruction. LLAMA_SWAP.md says Claude Code sends "high". On our SWE-bench Verified and Terminal-Bench runs, medium solved as much as xhigh and was faster.

Suggestion: a minimum parked size by default (e.g. 1,024-2,048 tokens), or recommending it in the Claude Code connect notes.

### Smallest steps or command that reproduce it

1. Default run config, Claude Code against `/v1/messages`.
2. An agent task of 15+ turns; watch the engine log for `prompt N tokens = 0 reused + N read` on a turn that continues the same conversation, right after a small side request.

### Reproduction status

Reproduced locally

### Strata version

Strata 0.2.16 / engine 0.1.62

### Graphics card, memory and system

RTX 4090 24 GB, i9-14900K, 128 GB DDR5, Ubuntu 26.04 (Linux 7.0)

### Model, size and settings

Qwen3.8-Flash-Next IQ3_S, 262K context, `--kv int8 --kv-resident 32768`, DFlash2; Claude Code 2.1.328

本站相关内容

相关页面的快捷入口。