Issues / #806
#806 Strange behavior for x2 RTX 5060 Ti (16Gb each) and UD-Q4_K_XL
closed · @art-den · 1 comments · View on GitHub
Setup & installNVIDIA / CUDAModels & quantsWindows
Description
SETUP.bat told for my hardware: `[!] UD-Q4_K_XL runs on one GPU here: on several it has no RAM budget and needs ~135 GB of RAM (its GGUF files and 24 GB more), this PC has 128 - using GPU 0 (NVIDIA GeForce RTX 5060 Ti, 16 GB) only` even though I selected two graphics cards when answering his question. So it configured only one GPU (`"gpu": 0`) But if then to change in model configuration file to `"gpu": [0,1]` and remove `--resident-budget-gib` strata works fine with two GPUs and consumes only 84 GB of memory (not 135 as SETUP.bat told)!
Related on strata.com
Editorial links to help you install, pick models, or read release notes — not part of the upstream thread.