Issues / #1080

#1080 RTX 5000 PRO slow unsloth ud iq4_xs fix

open · @ukrolelo · 4 comentarios · En GitHub

NVIDIA / CUDAModels & quants

Descripción

I didin't read what is this settings but i removed this
  "--resident-budget-gib",
  "55",

from the json and now it's flying, otherwise it was dead slow. 262K context, parallel 2, vision cpu
usage 47.1/47.8 VRAM

En el sitio

Enlaces a install, modelos, releases.