Pull requests / #3311

#3311 [Docs] --prompt-cache-every does not set the prompt piece size

open · @gopinath87607 · 0 Kommentare · Auf GitHub

DocumentationLinux

Beschreibung

> **Why this PR was recreated.** The original (#2834) was opened from the `gopinath87607/Strata` fork. Deleting that fork deleted its branches, and GitHub closes every pull request whose branch lived in a deleted fork - a closed PR whose source branch is gone cannot be reopened. This is the same work from a new fork, based on `477ac13c`, and GitHub reports that it merges cleanly with current `main` (`e88ee39d`). It replaces the closed #2834, which was closed by the fork's deletion, not by a maintainer.

## Problem and result

Readers can take `--prompt-cache-every` for a piece-size dial, as it is in the Maya checkout, where its marks bound the segments the prompt path reads and every segment pays a fill and a drain of the part chain. In this engine a periodic mark is saved at a chunk boundary and the run carries on, so no piece ends at it. After this docs change, `docs/DETAILS.md` says that, gives the Maya checkout's numbers with what they were measured on, and names the cuts that actually split a fresh long read here.

## Related discussion

The original pull request and its discussion: #2834 (closed when its fork was deleted; this PR replaces it). The numbers quoted are the Maya checkout's, local commit `3095719` (2026-10-15).

## Approach

A docs bullet for `--prompt-cache-every` in `docs/DETAILS.md`: the knob is **not** a piece-size dial in this engine. What the bullet records:

- The Maya checkout's numbers for its `STRATA_GLM_CKPT_EVERY` (same 16384 default), local commit `3095719`, on a 100,535-token fresh read (Maya-L split, 8192-token chunks, one 580-token warm prompt before each arm, engine restarted between arms): `16384` cut the read into 5 pieces read in 148.8 s, `32768` into 4 in 134.9 s (-9.3%), `0` into one of 13 chunks in 93.2 s (-37.4%) - that checkout's numbers on its rig, a piece fitting ~14.6 s + 6.5 s a chunk there.
- Why they do not carry over here: a periodic mark is saved at a chunk boundary and the run carries on, so no piece ends at it. What cuts this engine's read into runs is the cut list - a re-read (`STRATA_CKPT_REREAD`), the system-prompt root, a message boundary (`STRATA_CACHE_MESSAGE_BOUNDARY`), the start of the last turn and a pinned prefix - and each of those is one fill and drain.
- So `--prompt-cache-every 0` here skips the periodic marks and the near-tail one (a `--prompt-cache-tail` mark needs `--prompt-cache-every` above 0), while the root, the last turn, a message boundary and a pin are still saved - the same trade the Maya checkout documents for its `0`: a client that rewrote a turn *before* the last one mounts a farther mark. The lever for a fresh long read here is those cuts, not this cadence.

Documentation only: not one line of code or default changes.

## Validation

Tested revision: `4a5fa03f`, one commit on `477ac13c`; not rebased, and GitHub reports it merges cleanly with current `main` (`e88ee39d`). Environment: Linux; docs-only, so there is no build or GPU run.

**Passed:**
- `git grep -n -e 'prompt-cache-every' -e 'prompt-cache-tail' -e 'STRATA_CKPT_REREAD' -e 'STRATA_CACHE_MESSAGE_BOUNDARY' -- src docs` - every name and flag the bullet cites exists in the tree it documents (`docs/DETAILS.md` lines 1050 and 1064-1067; `src/program/generate.cpp` lines 900-902 and 2518-2520).
- The default the bullet cites is the one the engine prints: `src/program/generate.cpp:900`, `--prompt-cache-every N ... (default 16384, 0 = off)`.
- The bullet's Maya numbers are quoted with what they were measured on (that checkout's rig, Maya-L split, the warm prompt, the restarts between arms). No number is claimed for this engine.

**Failed:**
- none observed.

**Not run:**
- No build: the change is Markdown only.
- No separate rendered-Markdown check; GitHub renders the page for this PR and the bullet uses the file's existing list style.

## Limits

- The numbers are the Maya checkout's, not this engine's. They are there to show why the cadence was a dial there and is not one here.
- The bullet documents the engine as it is; it does not change how any flag behaves.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Mehr auf der Site

Links zu Install, Modellen, Releases.