Pull requests / #595

#595 No-AVX2 CPUs on a native build; run-script context override

closed · @4EverBuilder · 0 comentários · No GitHub

Setup & installNVIDIA / CUDAModels & quants

Descrição

- engine: a native (non-portable) build runs a native pack's CPU experts on ggml-cpu's own kernels when the CPU has no AVX2 (q2_native_kernels, AVX2 kernel gates in native_expert/pool/expert_source); the portable build (STRATA_PORTABLE_BUILD) still refuses. iq_avx2's sign table is built on first use rather than in a static constructor, so startup never executes AVX2 on such a CPU. native_expert_parity skips the AVX2 checks there.
- setup: a CPU without AVX2 warns and compiles the engine locally instead of failing.
- run-<model>.sh takes an optional context argument (131072, 512K, 1M), using the new tools/context_config.py to derive a config next to the setup-written one.

Developed on a Xeon X5690 (SSE4.2, no AVX) + RTX 4060 + Tesla P40.

No site

Links install, modelos, releases.