Issues / #739

#739 [enhancement] Variance-normalized KV-cache quantization (KVarN) & KV cache precision tail

open · @gregfromo · 1 commentaires · Sur GitHub

Models & quants

Description

I'm comming from beellama.cpp and use the KVarN kv-cache types & tail. 
Maybe you have time to take a look at this, if it's possible to implement in Strata, if you consider it as valueadding?

Great work sofar, many thanks!

Sur le site

Liens install, modèles, releases.