#6469 [GUI] The desktop agent's compaction request re-reads the conversation from the system prompt on Gyro (hybrid); --prompt-cache-spread cuts it from 14.9 to 2.7 s here

open · @ric20527 · 0 comentários · No GitHub

BenchmarksSetup & installModels & quantsWindowsLinux

Descrição

### Area

[GUI]  (the desktop app: the agent and the engine arguments the app writes)

### What do you want to do?

When the desktop agent's automatic compaction (`dsh-compaction-basic`) sends its summary request, the engine re-reads
most of the conversation. With `--prompt-cache-spread`, the same request resumes near the cut. I would like the app to
start the engine with `--prompt-cache-spread` when the agent is used, or to make it a visible setting.

**Why it happens** (from the engine log, `STRATA_TRACE=1`, and the token ids of each request). Gyro is a hybrid model,
so the engine can only resume from a saved checkpoint, not from any token (`--prompt-cache 6`). The checkpoints are the
root, at the end of the system prompt, plus the most recent turn boundaries. The summary request keeps the first ~70 %
of the history and appends the summary instruction. No checkpoint is left at that point, so the engine falls back to
the root. `--prompt-cache-spread` (DETAILS.md: "the kept points thin out geometrically back from the end") keeps one
near the cut.

**Measured on this PC.** A synthetic agent session built like the app's agent requests: a system prompt, four tools, tool
results up to ~60K tokens, then a summary request that cuts the history at 70 %. Each arm is a fresh server process,
with the conversation hook off. The common prefix with the previous request was checked on the exact token ids.

| engine | arm | summary request, first token | tokens reused |
| --- | --- | ---: | ---: |
| 0.1.62 (DFlash2) | default | 14.91 s | 6,311 |
| 0.1.62 (DFlash2) | `--prompt-cache-spread` | **2.74 s** | 35,936 |
| 0.1.62 (without DFlash2) | default → spread | 14.7 → 2.6 s | |
| 0.1.63 (DFlash2), 2 sessions | default | 15.35 / 15.22 s | 6,311 |
| 0.1.63 (DFlash2), 2 sessions | `--prompt-cache-spread` | **2.85 / 2.88 s** | 35,936 |

On 0.1.63 (order D S S D, each a fresh process): returning to the full history after the summary went from 15.05 /
15.00 s to 11.79 / 11.81 s (reused 27,494 → 35,936). The pruning request was the same in both arms (19.5-19.7 s,
reused 6,311), as expected.

Common prefix with the previous request: 41,572 tokens. Same chats without compaction (a 30K document, a follow-up, a
second 20K chat, returning to the first): no difference, every arm within ±0.7 s, all intervals include 0.

**In real use** (885 agent requests over two days on this PC, from the app's session files): normal steps reuse 98.2 %
of the prompt. The large re-reads were:

| cause | cases | tokens read again | helped by `--prompt-cache-spread` |
| --- | ---: | ---: | --- |
| compaction summary request | 8 | 457K | yes, in 6 of 8 (estimated, from the ids) |
| first request after the summary | 9 | 249K | no: the history really changed |
| `tool-result-pruner` (8,192 characters) | 5 | 346K | no: it rewrites a result in the first ~8K tokens |
| new chat or another chat | 7 | 211K | no |

At ~2,000 tok/s here, that is about 20 s saved per compaction, in 6 of 8 cases (an estimate, not a measurement in the
app). The pruning row is an app-side cost. It prunes when the chat reaches the pressure threshold, so a result near the
start changes and everything after it is read again. Pruning when the result arrives would avoid that. I did not test
this.

**Related, not duplicates:**
* #6379 / #6380: the conversation cache reuses 0 tokens on a single GPU. That is a different bug; here reuse works and
  only the summary request misses.
* #4744: when compaction runs and how much it keeps. This issue is about what compaction costs in prefill once it
  runs.

**What I did not test:** other models (Gyro-M/L, the non-hybrid families), Linux, contexts beyond ~84K, and the real
desktop agent with the flag on (I added it to my config on 2026-11-21; I can report after some days of use).

**Setup:** desktop app 0.2.17, engine 0.1.63 (and 0.1.62 for the first rows; the 0.1.63 rows at stock GPU clocks), RX 9070 XT 16 GB, Ryzen 5 7600X3D,
32 GB, Windows 11 Pro, Gyro-S TQ1_0, 131,072 context, `--kv int8 --kv-resident 32768`, DFlash2.

### What do you do today instead?

`--prompt-cache-spread` in the model's user args.

No site

Links install, modelos, releases.