Issues / #806
#806 Strange behavior for x2 RTX 5060 Ti (16Gb each) and UD-Q4_K_XL
closed · @art-den · 1 Kommentare · Auf GitHub
Setup & installNVIDIA / CUDAModels & quantsWindows
Beschreibung
SETUP.bat told for my hardware: `[!] UD-Q4_K_XL runs on one GPU here: on several it has no RAM budget and needs ~135 GB of RAM (its GGUF files and 24 GB more), this PC has 128 - using GPU 0 (NVIDIA GeForce RTX 5060 Ti, 16 GB) only` even though I selected two graphics cards when answering his question. So it configured only one GPU (`"gpu": 0`) But if then to change in model configuration file to `"gpu": [0,1]` and remove `--resident-budget-gib` strata works fine with two GPUs and consumes only 84 GB of memory (not 135 as SETUP.bat told)!
Mehr auf der Site
Links zu Install, Modellen, Releases.