#678 Feature request: Optional Intel NPU support for the vision encoder

Descrição

Hi Niko,

First of all, thank you for Strata — it’s an impressive piece of engineering and has made running the 125B MoE model on consumer hardware actually practical.

I’d like to propose a possible future improvement: adding optional support for Intel’s NPU (Neural Processing Unit) for the vision encoder.

Current situation

Right now strata-vision can run on:

GPU (fast, but reserves ~1.4 GB of VRAM)CPU (much slower)

On systems with Intel Core Ultra processors (Meteor Lake, Arrow Lake, Lunar Lake, etc.) there is a dedicated **NPU** that is specifically good at vision workloads (ViT-style models) and is power-efficient.

Why this could be useful
The vision encoder (mmproj / ViT + projector) is relatively small and fits the type of workload NPUs handle well.
Running the encoder on the NPU would:
Free the main GPU completely from vision work (no VRAM reservation)
Avoid slowing down the main text generation pathBe significantly faster and more efficient than the current CPU fallback

Many users with recent Intel CPUs already have the NPU available and idle most of the time.
Suggested approach (low priority / optional)

It would be great if support could be added as an optional backend, for example:

--vision npu or a config option "backend": "npu"Using OpenVINO (which already has mature NPU support) or the Level Zero / NPU driver pathKeeping the current GPU and CPU paths unchanged

I understand this is not a high priority given the main focus on the MoE text engine, and that implementing OpenVINO integration would require extra work. I’m mainly sharing the idea in case it aligns with any future plans or if other users have also requested something similar.

Thanks again for the excellent work on Strata.

Best regards,
Edgar