Pull requests / #1340

#1340 core: retain fixed RAM originals during native async exchanges

open · @CC-David-CC · 0 评论 · 在 GitHub 查看

AMD / HIPNVIDIA / CUDADocumentation

描述

Experimental default-off resident-RAM option. Q8/FP16 KV on RTX PRO 6000 Blackwell, 32K/128K plain and MTP: retains approximately 8 GiB and avoids 13.1-14.0% of counterfactual eviction-download payload. TPS nearly unchanged; enabled outputs and work differ. No isolated speedup or quality-equivalence claim. Default-off exact tokens, ownership/ASan/UBSan and STOP checks passed.

![Results](https://raw.githubusercontent.com/CC-David-CC/Strata-a5500/b47225cee8f26ce47bd5fd837f1fe9b62e0829ec/docs/experiments/retained-originals-main/results.png)

站内延伸阅读

链到安装、模型与版本说明,便于 SEO/GEO,非官方 issue 正文。