Issues / #806

#806 Strange behavior for x2 RTX 5060 Ti (16Gb each) and UD-Q4_K_XL

closed · @art-den · 1 comments · View on GitHub

Setup & installNVIDIA / CUDAModels & quantsWindows

Description

SETUP.bat told for my hardware:

`[!]  UD-Q4_K_XL runs on one GPU here: on several it has no RAM budget and needs ~135 GB of RAM (its GGUF files and 24 GB more), this PC has 128 - using GPU 0 (NVIDIA GeForce RTX 5060 Ti, 16 GB) only` even though I selected two graphics cards when answering his question.

So it configured only one GPU (`"gpu": 0`)

But if then to change in model configuration file to `"gpu": [0,1]` and remove `--resident-budget-gib` strata works fine with two GPUs and consumes only 84 GB of memory (not 135 as SETUP.bat told)!

Related on strata.com

Editorial links to help you install, pick models, or read release notes — not part of the upstream thread.