Pull requests / #2195

#2195 setup --update: an MTP folder shared by several configs keeps the draft subset of the most recently used one (the oldest config's default subset replaced cyrillic: Russian decode 68 -> 54 tok/s)

open · @bulhenvaster · 0 Kommentare · Auf GitHub

BenchmarksSetup & installAMD / HIPNVIDIA / CUDAModels & quantsWindowsLinux

Beschreibung

## Title
setup --update: an MTP folder shared by several configs keeps the draft subset of the most recently used one

Issue: none open that I could find (searched issues and pull requests for "draft vocab update cyrillic" and "draft subset shared mtp").

## Summary
`setup.py --update` (`update.sh` / `UPDATE.bat`) refreshes the draft subset once per installed config, most recently
used first. When two configs name the same MTP folder, the folder ends with the subset of the config used **longest
ago**. A config without `"draft_vocab"` counts as `cjk`, so it puts the default subset over the `cyrillic` (or `fr`,
`en`) one of the config that is actually in use.

Seen on one install after the update to 0.1.43: `strata-iq3_xxs.json` has `"draft_vocab": "cyrillic"`, an older
`strata-iq3_xxs-helper.json` (same model, another GPU mode, same `--mtp` folder) has no key. After `update.sh` the
folder's `draft_vocab.bin` was byte-identical to `data/draft_vocab.bin`, and the engine started with
`draft head over 106299 tokens` instead of `58963`.

Measured on that install (RTX 5060 Ti 16 GB + RTX 3060 12 GB with `--peer-device`, Xeon E5-2683 v4, IQ3_XXS, engine
0.1.43, Linux), a greedy 600-token Russian story, two alternating runs per subset:

| Subset in the MTP folder | Decode tok/s | Tokens per window |
| --- | --- | --- |
| default (what the update left) | 54.0 / 54.0 | 1.39 |
| cyrillic (what the config asks for) | 68.7 / 68.6 | 1.90 |

English prose and code were within the run-to-run spread between the two.

## What changed
- `setup.py`: `update_install` calls a new `refresh_shared_draft_vocab(cfg, done)`, which refreshes an MTP folder once,
  for the first (most recently used) config that names it, and skips the folder for the later ones.
  `refresh_draft_vocab` itself is unchanged.
- `tools/test_setup_draft_vocab.py`: three tests - the most recently used config decides for a shared folder, separate
  folders each keep their own subset, a config without `--mtp` is skipped.

What it leaves alone: an install where every config has its own MTP folder, or where all configs choose the same
subset, gets exactly the files it got before. Starting a model still installs that config's own subset (so starting
the older config switches a shared folder, as before); only the update path is changed. No engine code is touched.

How I checked: `python -m unittest tools.test_setup_draft_vocab` (10 pass, 3 new),
`tools.test_setup_config` and `tools.test_setup_configs` (18 pass). No GPU or download is involved in the change.

## Extra Notes
Not tested on Windows or with the AMD engine; the changed code is pure file work and has no platform branch.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Mehr auf der Site

Links zu Install, Modellen, Releases.