Issues / #3149
#3149 support for q5 hadamard rotation for K or V or both
open · @Schumi08 · 0 comentarios · En GitHub
Descripción
support for q5 hadamard rotation for K or V or both, will make them nearly lossless even at 256 ctx... with a small cost of compute, it will allow massive increase in ctx reducing KV size
En el sitio
Enlaces a install, modelos, releases.