问题 / #3610
#3610 [Engine] It is not possible to set cuda_device in vision
open · @FranBarInstance · 2 评论 · 去 GitHub 看
说明
I have a second GPU with 4GB of VRAM, which I use for my monitors and applications; I also utilize it for vision-related tasks, thereby freeing up all the memory on the primary GPU for inference.
I am trying to migrate my configuration from version 1.45 to version 1.46, but I cannot find a way to use `cuda_device`
```
"vision": {
"exe": "...",
"mmproj": "...",
"model": "...",
"gpu": true,
"cuda_device": 1,
"max_tokens": 1024
}
```
I haven't been able to get it to work, and as a result, performance is significantly lower in the new version.
Strata version. 1.46.x本站相关内容
相关页面的快捷入口。