问题 / #3149

#3149 support for q5 hadamard rotation for K or V or both

open · @Schumi08 · 0 评论 · 去 GitHub 看

说明

support for q5 hadamard rotation for K or V or both, will make them nearly lossless even at 256 ctx... with a small cost of compute, it will allow massive increase in ctx reducing KV size

本站相关内容

相关页面的快捷入口。