Pull requests / #15

#15 generate: residual control-vector projection from GGUFs

closed · @j-luwierski · 0 comentarios · En GitHub

Multi-GPUModels & quantsSecurity

Descripción

## What

Adds residual control-vector projection to the engine: per-layer control vectors loaded from GGUF files are projected onto the residual stream during generation, so model behaviour can be steered at inference time without modifying the weights.

## New options

- `--control-vector-scaled` - enable the projection with a given strength
- `--control-vector-layer-range` - apply the vector only to a range of layers
- `--cvec-mode project` - projection mode
- `--cvec-dir` - directory with the per-layer GGUFs

## Ready-to-use control vector

A refusal-projection control vector for Qwen3.8-Flash-Next can be downloaded from:

https://huggingface.co/Cudecnik/Qwen3.8-Flash-Next-refusal-projection

En el sitio

Enlaces a install, modelos, releases.