#2732 2x2080Ti 44G 64G RAM UD-IQ4 512K 配置 KV 复用

Beschreibung

# Strata 配置留档（2026-10-16，主机 192.168.1.6:/home/sunny/Strata）

> 升级 Strata 后需要恢复的三样东西：
> 1. 配置 JSON 里的两个缓存参数（下文已标注）
> 2. 启动脚本的调优环境变量（脚本一般不会被升级覆盖，确认即可）
> 3. **serve/frontend.py 的键序规范化补丁（升级会覆盖此文件，必须重打！）**

---

## 一、引擎配置 strata-unsloth-ud-iq4_xs.json

```json
{
 "exe": "/home/sunny/Strata/engine/strata",
 "args": [
  "--pack", "/home/sunny/Strata-data/packs/unsloth-ud-iq4_xs",
  "--native", "/models/unsloth-UD-IQ4_XS/Qwen3.8-Flash-Next-UD-IQ4_XS-00001-of-00003.gguf",
  "--expert-profile", "/home/sunny/Strata/data/expert-profile.bin",
  "--expert-cache", "auto",
  "--prefill", "auto",
  "--spec", "4",
  "--spec-min-p", "0.70",
  "--mtp", "/home/sunny/Strata-data/mtp/rt",
  "--max-context", "524288",
  "--rope-scaling", "yarn",
  "--rope-scale", "2",
  "--kv", "int8",
  "--kv-resident", "32768",
  "--pipeline-windows", "2",
  "--vram-reserve-later-mib", "1134",
  "--vram-reserve-mib", "896",
  "--pcie-frac", "0.35",
  "--ple-row-cache", "8388608",
  "--ple-inflight", "1024",
  "--conversation-cache-mib", "20480",
  "--conversation-cache-slots", "8",
  "--conversation-cache-min-free-mib", "2048",
  "--conversation-cache-min-tokens", "8192",
  "--aux-cpus", "auto",
  "--resident-experts"
 ],
 "cwd": "/home/sunny/Strata",
 "tokenizer": "/home/sunny/Strata-data/packs/unsloth-ud-iq4_xs/tokenizer",
 "model_name": "qwen3.8-flash-next-unsloth-ud-iq4_xs",
 "log": "/home/sunny/Strata/strata-unsloth-ud-iq4_xs.log",
 "lib_dirs": ["/usr/local/cuda-13.0/bin", "/usr/local/cuda-13.0/lib64"],
 "port": 8000,
 "env": { "STRATA_HC_Q8": "1" },
 "gpu": [0, 1],
 "gpus_asked": true,
 "layer_split": "26",
 "host": "0.0.0.0",
 "api_key": "local",
 "draft_vocab": "cjk",
 "fit_max_tokens": true,
 "effort_position": "end"
}
```

### KV 复用相关的关键参数（升级后重点核对）

| 参数 | 值 | 说明 |
|---|---|---|
| `--conversation-cache-min-free-mib` | **2048**（默认 4096 太保守） | parking 内存准入底线，保证 ~3.7GB 的对话快照能 park 进 RAM |
| `--conversation-cache-min-tokens` | **8192**（旧版无此参数，默认 0） | 小于 8192 token 的请求不占缓存槽，防止小请求驱逐大对话 |
| `--conversation-cache-mib` | 20480 | 缓存总预算 |
| `--conversation-cache-slots` | 8 | 最多同时 park 8 个对话 |
| `--kv-resident` | 32768 | 常驻 KV 窗口 |

---

## 二、启动脚本 run-unsloth-ud-iq4_xs.sh

```sh
#!/bin/sh
cd "/home/sunny/Strata"
# Turing/NVLink 调优开关（#1349/#2239 实测）
export STRATA_SPLIT_OWN=0
export STRATA_STAGE_TRIM=1
export STRATA_PREFILL_HELP=1
export STRATA_BF16_TC=1
export STRATA_SELECT_SIMT=1
export STRATA_SPLIT_MTP_BATCH=1
export STRATA_SPLIT_RING=384
exec "/home/sunny/Strata/.venv/bin/python" "/home/sunny/Strata/serve/server.py" "--engine" "strata" "--config" "/home/sunny/Strata/strata-unsloth-ud-iq4_xs.json" "--port" "8000"
```

启动方式：`cd /home/sunny/Strata && setsid nohup ./run-unsloth-ud-iq4_xs.sh > /tmp/strata-run.log 2>&1 < /dev/null &`

---

## 三、serve/frontend.py 补丁（升级必重打！）

升级会覆盖 `serve/frontend.py`，KV 复用的根因修复在里面。重打方法：

在 `/home/sunny/Strata` 下执行：

```sh
python3 - <<'PYEOF'
p = '/home/sunny/Strata/serve/frontend.py'
src = open(p, encoding='utf-8').read()

helper = '''def _canonical_tools(obj):
    """Rebuild dicts with sorted keys, recursively.  Clients (Trae, Codex) reshuffle tool JSON property
    order between requests of one conversation while the content stays the same; the rendered prompt must
    not follow it, or the engine's conversation cache loses its whole prefix.  Lists keep their order:
    only object key order is unstable."""
    if isinstance(obj, dict):
        return {k: _canonical_tools(v) for k, v in sorted(obj.items())}
    if isinstance(obj, list):
        return [_canonical_tools(v) for v in obj]
    return obj


'''
if 'def _canonical_tools' in src:
    print('already patched')
else:
    assert src.count('class ChatTemplate:') == 1
    src = src.replace('class ChatTemplate:', helper + 'class ChatTemplate:', 1)
    sig = '               **kwargs) -> str:\n'
    assert src.count(sig) == 1, 'render signature not found'
    src = src.replace(
        sig,
        sig + '        if tools is not None:                            # render client key-order shuffles stably\n'
              '            tools = _canonical_tools(tools)\n',
        1,
    )
    open(p, 'w', encoding='utf-8').write(src)
    print('patched OK')
PYEOF
python3 -m py_compile serve/frontend.py && echo COMPILE_OK
```

补丁位置参考（2026-10-16 版本 frontend.py）：
- 第 40-50 行：`_canonical_tools()` 模块级函数（在 `class ChatTemplate` 之前）
- 第 71-73 行：`ChatTemplate.render()` 开头，`**kwargs) -> str:` 之后加两行，对 `tools` 做规范化

### 验证补丁生效

```sh
# 重启后跑一轮对话，看日志：
grep -E 'prompt [0-9]+ tokens = ' strata-unsloth-ud-iq4_xs.log | tail -1
# 期望: prompt N tokens = (N - 几百) reused ...  即 reused 接近 prompt 大小
```

### 当前版本备份位置（服务器上）

- `serve/frontend.py.bak-20261016`（补丁前原始版）
- `strata-unsloth-ud-iq4_xs.json.bak-20261016-opt`（改配置前原始版）