Pull requests / #1633

#1633 setup: offer the experimental speed projection for Swift 1.5 too

open · @lineape · 0 comentários · No GitHub

BenchmarksSetup & installMulti-GPUAMD / HIPModels & quantsSecurity

Descrição

Setup offers the speed projection only for the original model and the Coder, and leaves it off for Swift 1.5
("made for the original Qwen3.8-Flash-Next"). We measured it on Swift 1.5, and it behaves as on the original, so
this PR offers it there too. It stays off by default, experimental, and switchable per request.

**Why the vector transfers.** The projection acts on the residual stream (layers 4-44). Swift 1.5's post-training
barely touched the weights that write into it. Comparing the BF16 checkpoints (published by SC117 on the abliterated
Swift model card):
- the routed experts' `ffn_down_exps` are byte-identical to the original's on all 48 layers;
- `ssm_out`, `attn_output` and `ffn_down_shexp` differ by 0.21-2.42%.

**Measured.**
- Swift 1.5 IQ3_XXS, the exact files of setup's pinned revision `b22d729` (sha256-verified), on Strata 0.1.41;
  GTX 1080 + 1070 layer split.
- Shipped vector, `--control-vector-layer-range 4 44 --cvec-mode project`.
- 25 AdvBench harmful behaviours and 25 Alpaca instructions (random sample, seed 20261008); thinking off, greedy,
  160 tokens; keyword refusal check.

| | harmful refused | harmless refused | decode |
|---|---|---|---|
| Swift 1.5, projection off | 25/25 | 1/25 | 31.1 tok/s |
| Swift 1.5, projection on | **0/25** | 1/25 | 32.9 tok/s |
| (for reference) SC117's abliterated Swift 1.5 | 0/25 | 1/25 | 32.1 tok/s |

- The one harmless "refusal" is the same prompt in every arm, a false positive of the keyword check.
- With the vector loaded but the request field `false`, all 25 harmless answers were byte-identical to stock.
- A code prompt gave the same function in every arm.
- Not measured: teacher-forced KL on Swift, thinking-on prompts, IQ2_XS/Q2_0, and long agent sessions.

**Possibly of interest.** SC117's abliterated Swift is orcarouter's single-direction abliteration, `W ← W − r rᵀW`
with one `r` for all layers, transplanted. `r` can be recovered from the quantized files: the top singular vector
of `ΔW` agrees across 12 layers at |cos| ≥ 0.998, and matches the BF16 edit at 0.9998. Used as a project-mode
control vector (the same `r` on layers 1-47), it also gave 0/25 and 1/25. Its harmless answers stayed closer to
stock than the shipped vector's (difflib similarity 0.56 vs 0.42). It agrees with the shipped per-layer directions
at |cos| 0.29-0.77 (peak at layer 24). We are not proposing to ship it: it derives from orcarouter's gated weights.
We mention it as evidence that the projection approach works on Swift.

No site

Links install, modelos, releases.