Pull requests / #2175
#2175 serve: "env_to_user" (opt-in) moves opencode's <env> block into the first user message (new session's first request 11.9 -> 1.1 s)
open · @zsariboga · 0 コメント · GitHub で見る
Server & APINVIDIA / CUDAModels & quantsDocumentation
本文
## What `"env_to_user": true` in `strata-<model>.json` (opt-in): opencode's `<env>` block (with its "Here is some useful information about the environment you are running in:" line) is taken out of the system message and put at the front of the first user message, for OpenAI and Anthropic requests (string content, or a leading text part before content parts). Without the key nothing changes. ## Why opencode writes the block (working directory, today's date) near the start of its system message, about 3,000 characters in, before ~41,000 characters of instructions and tool definitions that are the same in every session. The prefix cache (and `--prefix-cache-dir` on disk) only reuses a prompt's start, so every new folder and every new day read the whole system turn again: ~14K tokens, 8-12 s on a 12 GB laptop, before the first token of every new session. I captured two first requests from two folders with a fake server: the block was the only difference in the system message. With the block in the user turn the system turn (and the tools the template writes into it) is the same for every session, and the cache restores it. The model still reads the same lines, one turn later. ## Measured RTX 5070 Ti Laptop 12 GB, Swift 1.5 IQ3_XXS, `--prefix-cache-dir` on, opencode, a new session in a second folder: | | first request | from cache | read | time | | --- | --- | --- | --- | --- | | `env_to_user` off | 14,753 tokens | 0 | 14,753 | 11.9 s | | `env_to_user` on | 14,073 tokens | 13,693 | 380 | **1.1 s** | Quality check: a task that asks for the working directory and the date answered both correctly 3/3 with the switch on. On my machine's history, 34 such cold reads averaged 10.3 s each. ## Tests `serve/test_env_to_user.py` (6): the block moves, two folders give the same system turn, content parts get a leading text part, Anthropic requests too, and off / no block changes nothing. `serve/test_frontend.py` passes. `docs/DETAILS.md` has a paragraph next to `tool_call_recovery`.
関連リンク
インストール・モデル・リリースへの站内リンク。