问题 / #2767

#2767 [Bug]: --rope-scaling and the other rope flags are resolved (for linear/yarn logged as active) on the Mac (Metal) engine but change no rotation

open · @oinani0721 · 0 评论 · 去 GitHub 看

Setup & installModels & quantsDocumentationWindows

说明

### What happened

On the Metal engine, the rope flags that docs/DETAILS.md:1427-1430 lists (`--rope-scaling`, `--rope-scale`, `--rope-freq-base`, `--rope-freq-scale`, `--yarn-*`) and a model file's `rope.scaling.*` keys are resolved, and for linear and yarn the engine logs the scaling as active (`strata generate: rope scaling yarn, factor 4 ...`, src/program/generate.cpp:3002-3008). But no rotation on a Mac reads them:

- `rope_table_set` does nothing on Metal (metal/src/engine/setup_host.cpp:151, "the window keeps its own table");
- every RoPE rotation on Metal is in `qsa_prep`, which reads only the decode window's own cos/sin table (metal/kernels/qsa.metal:74);
- `Window::init` builds that table without scaling, from the model file's `rope.freq_base` (metal/src/engine/window.cpp:468-475).

So, by the code, `--rope-scaling yarn --rope-scale 4` gives the unscaled rotation, and a context past the trained 262,144 tokens runs with the stock angles.

Expected: the flags change the rotation on a Mac as they do on the PC.

On an Apple M5 Max (128 GB), main 296b59d's Metal build gave the same tokens, the same timing-free DONE fields and byte-identical first-window logits for 86 requests (a question, 64 tokens of a fixed text forced one per request, 20 prompts of 256 to 1,000 tokens and 128 tokens of a story), and byte-identical log-probability rows (the target token's and the top 64, `STRATA_METAL_LOGPOS`) at every position that the next one-token window continued (the 64 forced tokens and the story), with and without `--rope-scaling yarn --rope-scale 4 --yarn-orig-ctx 262144` (the other rope flags were not run), with one-token decode windows (no MTP draft layer, no suffix drafts). On setup's default config its 6 requests gave identical tokens, DONE fields and first-window logits with and without `--rope-scaling yarn --rope-scale 4 --yarn-orig-ctx 262144` too.

The model was Qwen3.8-Flash-Next-GSQ-RCO-abliterated-IQ3_S (two shards, revision eec4e10a of its Hugging Face repository), greedy, thinking off. I have a change that makes the window build its table with `build_rope_table` under the resolved scaling, with a CPU-only test, and will open it as a pull request that references this issue.

### Strata version, GPU, OS

0.1.44.1-metal (main 296b59d) · Apple M5 Max, 128 GB · macOS 27.0.1

### Engine log

```
strata generate: rope scaling yarn, factor 4 (freq_scale 0.25, base 1e+07, mscale 1.138629), --max-context 65536 against a trained context of 262144
```

本站相关内容

相关页面的快捷入口。