Issues / #739

#739 [enhancement] Variance-normalized KV-cache quantization (KVarN) & KV cache precision tail

open · @gregfromo · 1 comentários · No GitHub

Models & quants

Descrição

I'm comming from beellama.cpp and use the KVarN kv-cache types & tail. 
Maybe you have time to take a look at this, if it's possible to implement in Strata, if you consider it as valueadding?

Great work sofar, many thanks!

No site

Links install, modelos, releases.