问题 / #3246

#3246 Feature: make Strata support CPU + RAM + NVMe only!

open · @eGit · 4 评论 · 去 GitHub 看

needs-format

说明

Is there a technical reason not to support STRATA's mechanism of clever expert memory-management to run on CPU + RAM + NVME only?

1. Only CPU, RAM and NVMe means less complexity. Fewer moving parts.

2. CPU only means: longer time for token generation - more time to pre-fetching upcoming experts from NVME to RAM.

Especially with smaller models like Qwen3.6-35B-A3B. They often fit easily into RAM but if you want super long context, a Strata-Like reduction of resident RAM would help A LOT!!

本站相关内容

相关页面的快捷入口。