#739 [enhancement] Variance-normalized KV-cache quantization (KVarN) & KV cache precision tail

open · @gregfromo · 1 comments · View on GitHub

Description

I'm comming from beellama.cpp and use the KVarN kv-cache types & tail. 
Maybe you have time to take a look at this, if it's possible to implement in Strata, if you consider it as valueadding?

Great work sofar, many thanks!

Related on strata.com

Install Strata
Model sizes & RAM
Release notes

Editorial links to help you install, pick models, or read release notes — not part of the upstream thread.