Pull requests / #2957

#2957 engine: a WARNING when --max-context is past the model's trained context and no rope scaling is active

open · @sergqwer · 0 comentários · No GitHub

Setup & installServer & APINVIDIA / CUDAModels & quantsWindows

Descrição

A config written by hand with `--max-context 524288` and no `--rope-scaling` starts without a word. Positions past 262144 then rotate by angles the model never saw. Setup adds yarn above the trained context, but a config written by hand does not go through setup.

**The change.** Where the engine resolves the rope config to `none`, it now prints one WARNING to stderr when `--max-context` is past the trained context (the model's, or `--yarn-orig-ctx`). The WARNING names the fix. Nothing else changes, and a context within the trained one prints nothing new.

Measured on the builds of 477ac13c (upstream/main, v0.1.45) and this branch:

| start | 477ac13c | this PR |
| --- | --- | --- |
| `--max-context 524288`, no `--rope-scaling` | starts, no line | starts, one line (below) |
| `--max-context 524288 --rope-scaling yarn --rope-scale 2` | - | no WARNING (the existing "rope scaling yarn, factor 2 ..." line) |
| `--max-context 262144` | no line | no line |

```
strata generate: WARNING: --max-context 524288 is past the model's trained context of 262144 and no rope scaling is active: positions past it rotate by angles the model never saw, and it can lose track there. To extend the context: --rope-scaling yarn --rope-scale 2 (what setup writes above 262144)
```

**Identity at the default 262144.**
- The first-token logits and the greedy tokens (32 max; both prompts stop at the end of turn) are byte-identical to 477ac13c's for a 2K and a 32K prompt.
- 2K: `1e9c612c2e6c7a32...`, 4 tokens. 32K: `f5b2872c7ac30b17...`, 20 tokens.

**Setup.**
- Machine: RTX 5090, Ryzen 9 9950X3D, 128 GB, Windows 11.
- Builds: MSVC 2022, CUDA 13.3, portable, `75-real;86-real;89-real;120-real`.
- Model: the NVFP4 GPTQ pack.

**How to reproduce.**
- Install with upstream's `python setup.py --family nvfp4 --model NVFP4` (Maximilian228/Qwen3.8-Flash-Next-NVFP4-GPTQ-Strata).
- For the WARNING: start `strata --serve` with the model arguments setup writes, `--max-context 524288` and no `--rope-scaling`. The line comes before the experts load.
- For the identity:
  - flags: `--expert-cache 3000 --prefill auto --spec 6 --spec-min-p 0.7 --max-context 262144 --kv int8 --pcie-frac 0 --adapt-every 1000000`;
  - env: `STRATA_PREFILL_CPU_SHARE=0 STRATA_PCIE_BALANCE=0 STRATA_IQ_MT_MIN=1 STRATA_DUMP_FIRST_LOGITS=<file>`;
  - send a 2K and a 32K prompt, greedy, 32 tokens.
- All of it was measured on that pack with upstream/main builds.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01VZy1yKaDDiA8a7svdwaHio

No site

Links install, modelos, releases.