#2965 setup: a separate opt-in layers 8-33 projection vector for the original Qwen3.8-Flash-Next (any quantization)

Description

## What this adds

A **separate, opt-in** second projection vector for the **original Qwen3.8-Flash-Next in any quantization**: layers 8-33, strength 1.0, `project`, `per-layer`, the same engine flags and mechanism as the existing experimental speed projection. It sits beside the existing one; **the existing vector, its option (`--experimental-speed-projection`), its families (`qwen`, `coder`) and its layers 4-44 are untouched.**

- `data/experimental-speed-projection/Qwen3.8-Flash-Next-experimental-speed-projection-l8-33.gguf` (268 KB, sha256 `a2ec3e999724022a55d48d30822bd10b1b25916ccc221013a119a674d06a46b5`), its `.json`, and a README section (origin, method, numbers, license).
- `--experimental-projection-l8-33 on|off` and its own setup question, "Turn on the experimental layers 8-33 projection?" (default no), for the `qwen`, `unsloth` and `nvfp4` families (in `nvfp4`, the GPTQ pack `NVFP4` only). Not the Coder (not measured there), not Swift 1.5, and not NVIDIA's own checkpoint `NVFP4-NVIDIA` (see Safety).
- It is meant to be one projection that works across the quantizations of the original model, with smaller side effects than the current vector in the measurements below.

## The two are never on at once

- Both flags on the command line: setup stops with an error naming both flags.
- Interactive: in `qwen` the existing question comes first; the new one is asked only if that was answered no. In `unsloth` and `nvfp4` (`NVFP4`) only the new one exists; for `NVFP4-NVIDIA` neither is asked. Naming the new flag on the command line skips the existing question.
- Re-running setup over a config: the loaded vector is recognised by its file name (`choices_from_config` now returns `esp` and `esp_l8`), and turning one on on the command line replaces the other.
- Server: `serve/server.py` refuses a config whose `args` load both shipped vectors, with a message naming them.
- Per request: `"experimental_speed_projection": false` (and the web switch) turns off whichever single vector is loaded; one key, no new API.

## Measurements (stated as measured; the fork's code, not 0.1.45)

Made by sergqwer (strata-nvfp4) from the activations of a GPTQ all-NVFP4 quantization of the original model (per layer: mean harmful minus mean harmless residual at the 7 assistant-header positions of 320 + 320 prompts, summed over the 4 hyper-connection streams, unit-normalized; about 30 window / strength variants scored on held-out prompts). Measured 2026-10-10 on this fork's 0.1.41/0.1.42-era code, RTX 5090, against the stock model of the same quantization: KL is teacher-forced on 5,153 rows (exact mode); "</think> shift" is the mean change of log p(`</think>`) over the model's own thinking (above 0 = ends thinking sooner); refusals are counted by a phrase classifier on the first 300 characters (the thinking-off held-out answers were also read by hand), greedy.

GPTQ NVFP4 pack, this vector vs the existing one (layers 4-44):

| | layers 8-33 (this) | existing 4-44 |
|---|---|---|
| refusals, validation 96, thinking off | 4/96 | 4/96 |
| refusals, held-out 104 EN, thinking off | 3/104 (1 real, the rest disclaimers followed by an answer) | 3/104 |
| refusals, held-out 104 EN, thinking on | 0/104 | 2/104 |
| KL, all rows | 0.0259 | 0.0433 |
| KL, the model's thinking text | 0.0144 | 0.0298 |
| </think> shift, thinking text (nats) | +0.019 | +0.092 |
| thinking tokens on a long agentic request, 5 greedy runs, mean (stock 1,699) | 1,947, 0 loops | 1,825, 0 loops |

(Stock refuses 96/96 and 102/104 of the validation / held-out prompts with thinking off.)

Other quantizations (both vectors against that quantization's own stock; KL and shift on the base texts and 4,818 thinking rows; 104 held-out EN prompts, thinking off):

| quantization | refusals this / existing | KL this / existing | </think> shift this / existing | thinking length, 3 greedy runs |
|---|---|---|---|---|
| ISTA Q2_0 | 1/104 / 1/104 | 0.016 / 0.027 | +0.027 / +0.087 | stock 965, LOOP, 1,766; this 1,306, 1,350, 906 |
| ISTA IQ2_XS | 1/104 / 1/104 | 0.017 / 0.031 | -0.003 / +0.123 | stock 3/3 loop, this 2/3 loop (the quantization loops by itself) |
| Unsloth UD-Q4_K_XL | 4/104 / 4/104 | 0.028 / 0.042 | +0.050 / +0.163 | stock 888, 1,183, 1,731; this 1,204, 1,255, 1,343 |

**Not measured:** the other ISTA and Unsloth sizes, the Coder, and anything on 0.1.45. The sets are small (104 prompts): differences of 1-2 refusals are noise, and the refusal classifier counts capability disclaimers. The existing vector was measured on the fork's engine for this comparison, not with its authors' harness.

## Try it with Strata's own setup

```
python setup.py --family nvfp4 --model NVFP4 --experimental-projection-l8-33 on
python setup.py --family qwen --model IQ3_S --experimental-projection-l8-33 on
python setup.py --family unsloth --model UD-Q4_K_XL --experimental-projection-l8-33 on
```

## Checked here

- New `tools/test_setup_projection.py` (18 tests, on the golden-test PC mocks): each family gets the right vector and layer range; `NVFP4-NVIDIA` never gets it, is not asked, and `--experimental-projection-l8-33 on` there is left off with the reason; the Coder and the existing option are unchanged; off by default; both flags on stop setup; the interactive questions cannot end with both; configs read back for each vector; the server refuses a config with both.
- Live, RTX 5090, official 0.1.45 engine, the NVFP4 GPTQ pack, config as setup writes it for `nvfp4` plus the new flags (engine log: `control vector mode = project, dir = per-layer, layers 8..33 (26 steered)`), thinking off, temperature 0, 200 tokens: 3 prompts from the harmful validation set that the stock model refuses were answered with the vector on and refused with `"experimental_speed_projection": false` in the request (the bomb answer begins with a safety warning, then complies); an ordinary prompt got the same answer either way.
- `tools` suite: 853 tests; the only failures are three golden `unsloth` cases whose vision `threads` depend on the host's CPU (16 here, 6 in the golden file; identical on unmodified `upstream/main`) and `test_iq_pack`, which needs `third_party/llama.cpp` (passes with it linked). `serve` suite: 848 OK.

## Safety and license

Off by default. It removes the model's refusal behaviour, a safety behaviour: you are responsible for what the model writes with it on. The vector comes from Qwen3.8-Flash-Next activations, so the model's Qwen Community License 1.0 applies, as for the existing vector. Not offered for NVIDIA's checkpoint (`NVFP4-NVIDIA`): the NVIDIA Open Model License does not allow reducing that model's safeguards, so setup does not ask there, and `--experimental-projection-l8-33 on` with it is left off with a warning that says why (`ESP_L8_NOT_FOR` in `setup.py`).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01VZy1yKaDDiA8a7svdwaHio