Pull requests / #1284

#1284 Adaptive tier: --adapt-decay outside (0, 1) is refused (inf counts, NaN swap gains)

open · @sergqwer · 0 评论 · 在 GitHub 查看

Multi-GPUNVIDIA / CUDA

描述

Replaces #1047, which GitHub closed when `main`'s history was rewritten. The closing note says this was not a rejection. It is the same commit, cherry-picked onto the new `main` (v0.1.40.1). The range check sits beside 0.1.40's `--adapt-async`. Its default output equals v0.1.40 on IQ2_XS: the logits and the 32 greedy tokens after a 32K prompt are identical bytes. The discussion before that is in #591.

---

The range check from #407, as offered there. `--adapt-decay` still accepts any value.

**What goes wrong at 1 or more.** `usage` is multiplied by the decay after every adaptation, so with 1 or more the counts never shrink. At 2, an expert routed early in the conversation passes `FLT_MAX` after 128 adaptations, which is about 128 verify rounds with `--adapt-every 1` or ~500 with the default 4. Then, in `adapt`:
- `cand[i].first < vict[i].first + 1.5f` is false for inf against inf, so the pair is pushed with `gain = inf - inf = NaN`.
- `std::sort(swaps, gain > gain)` then compares NaNs. That breaks the strict weak ordering `std::sort` requires, which is undefined behaviour.

**Measured on v0.1.38** (RTX 5090, IQ2_XS, fixed `--expert-cache 14900`, a 1,000-token chat, `--adapt-every 1`, two runs each):

| `--adapt-decay` | rounds | ms a round | experts swapped | hits |
| --- | --- | --- | --- | --- |
| 0.7 | 178 | 13.67 / 13.70 | 1,095 | 0.981 |
| 2 | 188 | 13.47 / 13.54 | 3,523 | 0.990 |

At 2 the tier swapped 3.2x as many experts. Nothing visible broke in a run this short (same output both times, no crash), so the case for refusing is the undefined sort, not a measured failure.

**The change:** the flag parse refuses values outside (0, 1) with "--adapt-decay must be between 0 and 1 (exclusive)" and exit code 2 (checked with 1.5). Values inside the range behave as before.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01VZy1yKaDDiA8a7svdwaHio

站内延伸阅读

链到安装、模型与版本说明,便于 SEO/GEO,非官方 issue 正文。