问题 / #3448

#3448 [Engine] data/expert-profile-coder.bin does not match the 2026-10-19 Coder release (the experts were re-indexed)

open · @rodi4eg · 0 评论 · 去 GitHub 看

BenchmarksAMD / HIPNVIDIA / CUDAModels & quants

说明

## The shipped Coder expert profile no longer matches the model

`data/expert-profile-coder.bin` is the shipped ranking of the Coder's kept experts, mapped onto them through the release's `rco-allocation.txt`. ISTA replaced the Coder's first shard on 2026-10-19 and **re-indexed the kept experts** ("Update expert allocation to match the replaced first shard"), so the mapping the profile was built on is gone.

**What I measured:** the shipped profile has 12,288 ranked `(layer, expert)` pairs. Against the new release, **2 of 12,288 still point at the same expert** (0.0%). The new allocation moved 18,378 of the 74,952 allocation entries at the same position, every one of them a different expert index, across 48 of the 49 layers (210-516 entries per layer).

The profile is a *priority order*: the VRAM expert cache holds ~1,872 of the model's 12,288 experts, and with the per-layer layout each layer keeps its 39 highest-priority ones. A stale profile therefore makes the cache prefetch the wrong 39 per layer at the start.

**Why it does not bite much today:** `rank_learned_profile` (`src/core/expert_cache.cpp:160-164`) sorts the cache by (1) already resident, (2) the learned **heat** (the decayed routing counts, which is also what drives the adaptive tier's swaps), and only then (3) the shipped profile as a tie-breaker. The heat term re-learns the hot experts within the first minutes, so the initial placement self-corrects: on this box the adaptive tier exchanged 2,166 experts with the VRAM tier in the first requests, and a profile rebuilt from the new model's own routing traces measured the same as the stale one (43.6 vs 44.8 tok/s decode, inside the +/-5% run-to-run spread of this box).

**What I would suggest**

- Re-map the shipped profile onto the new `rco-allocation.txt` (the same way it was built the first time), or
- document that the profile is only a tie-breaker, so a release that re-indexes the experts does not need one.

I can post a profile rebuilt from the new model's routing traces (11,182 of the 12,288 pairs from the traces, 1,106 filled in) if that is useful - it is at `data/expert-profile-coder-new.bin` in my tree and I can attach it to a PR.

**Environment:** RTX 4070 12 GB, Ryzen 7 5700X, 31 GiB RAM, Strata source build at `f57c0adc` (engine 0.1.45.3), Coder IQ1_M, the 2026-10-19 revision `bba88856f8c3c0c9e40e45f82d05e228b814ac43`.

本站相关内容

相关页面的快捷入口。