Pull requests / #1880

#1880 setup: the images question now asks where the encoder runs (GPU / CPU)

open · @xtremexp · 0 comentarios · En GitHub

Setup & installAMD / HIPModels & quantsDocumentation

Descripción

## Description
--vision cpu (the image encoder on the CPU, no VRAM) has existed since 0.1.32 (#304), and setup uses it itself when a ready-made encoder has no code for the card (#331). Only the interactive prompt never offered it: Do you want images? [y/n], and yes always meant the GPU. So the people who would want it most — a card full of experts — cannot find it without knowing the flag.

The GPU encoder keeps ~1.4 GB of VRAM free, which leaves ~1,000 fewer experts cached and costs 2-8% on prompt and output speed (docs/DETAILS.md, "Speed with images on", 4K context, Q2_0 / IQ2_XS / IQ3_XXS). The CPU encoder costs nothing on the GPU and reads a picture in about 3 s at 300 image tokens on 8 cores.

## What changed
Saying yes to "Do you want images?" now asks a second question: Image encoder? [1]: 1) GPU (recommended) 2) CPU (no VRAM). The intro line names both costs. --help for --vision mentions cpu. Docs: the four questions in DETAILS.md, the Images section, the question list in INSTALL.md.

## What it leaves alone
GPU stays the default, so Enter and --yes behave exactly as before (--yes leaves images off, as documented). The --vision flags, their parsing, the build, the config format and the server are untouched — the CPU answer writes the same config --vision cpu writes. The AMD path is untouched: it has no GPU encoder, so it never reaches this prompt (hip_vision).

En el sitio

Enlaces a install, modelos, releases.