Pull requests / #591

#591 Adaptive tier: --adapt-decay outside (0, 1) is refused (inf counts, NaN swap gains)

closed · @sergqwer · 0 commentaires · Sur GitHub

Multi-GPUNVIDIA / CUDA

Description

The range check from #407, alone on 0.1.38, as offered there. 0.1.38's `--adapt-decay` accepts any value.

**What goes wrong at 1 or more.** `usage` is multiplied by the decay after every adaptation, so with 1 or more the counts never shrink. At 2, an expert routed early in the conversation passes `FLT_MAX` after 128 adaptations, which is about 128 verify rounds with `--adapt-every 1` or ~500 with the default 4. Then, in `adapt`:
- `cand[i].first < vict[i].first + 1.5f` is false for inf against inf, so the pair is pushed with `gain = inf - inf = NaN`.
- `std::sort(swaps, gain > gain)` then compares NaNs. That breaks the strict weak ordering `std::sort` requires, which is undefined behaviour.

**Measured on v0.1.38** (RTX 5090, IQ2_XS, fixed `--expert-cache 14900`, a 1,000-token chat, `--adapt-every 1`, two runs each):

| `--adapt-decay` | rounds | ms a round | experts swapped | hits |
| --- | --- | --- | --- | --- |
| 0.7 | 178 | 13.67 / 13.70 | 1,095 | 0.981 |
| 2 | 188 | 13.47 / 13.54 | 3,523 | 0.990 |

At 2 the tier swapped 3.2x as many experts. Nothing visible broke in a run this short (same output both times, no crash), so the case for refusing is the undefined sort, not a measured failure.

**The change:** the flag parse refuses values outside (0, 1) with "--adapt-decay must be between 0 and 1 (exclusive)" and exit code 2 (checked with 1.5). Values inside the range behave as before.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01VZy1yKaDDiA8a7svdwaHio

Sur le site

Liens install, modèles, releases.