Pull requests / #15

#15 generate: residual control-vector projection from GGUFs

closed · @j-luwierski · 0 コメント · GitHub で見る

Multi-GPUModels & quantsSecurity

本文

## What

Adds residual control-vector projection to the engine: per-layer control vectors loaded from GGUF files are projected onto the residual stream during generation, so model behaviour can be steered at inference time without modifying the weights.

## New options

- `--control-vector-scaled` - enable the projection with a given strength
- `--control-vector-layer-range` - apply the vector only to a range of layers
- `--cvec-mode project` - projection mode
- `--cvec-dir` - directory with the per-layer GGUFs

## Ready-to-use control vector

A refusal-projection control vector for Qwen3.8-Flash-Next can be downloaded from:

https://huggingface.co/Cudecnik/Qwen3.8-Flash-Next-refusal-projection

関連リンク

インストール・モデル・リリースへの站内リンク。