Pull requests / #595

#595 No-AVX2 CPUs on a native build; run-script context override

closed · @4EverBuilder · 0 评论 · 在 GitHub 查看

Setup & installNVIDIA / CUDAModels & quants

描述

- engine: a native (non-portable) build runs a native pack's CPU experts on ggml-cpu's own kernels when the CPU has no AVX2 (q2_native_kernels, AVX2 kernel gates in native_expert/pool/expert_source); the portable build (STRATA_PORTABLE_BUILD) still refuses. iq_avx2's sign table is built on first use rather than in a static constructor, so startup never executes AVX2 on such a CPU. native_expert_parity skips the AVX2 checks there.
- setup: a CPU without AVX2 warns and compiles the engine locally instead of failing.
- run-<model>.sh takes an optional context argument (131072, 512K, 1M), using the new tools/context_config.py to derive a config next to the setup-written one.

Developed on a Xeon X5690 (SSE4.2, no AVX) + RTX 4060 + Tesla P40.

站内延伸阅读

链到安装、模型与版本说明,便于 SEO/GEO,非官方 issue 正文。