Issues / #1738

#1738 [Feature] hot-switching model with API

open · @yookoala · 0 comentários · No GitHub

Server & API

Descrição

Strata, although do not support a lot of model, do support a few. It would be very useful if there is a way to actually unload a model to load another one without restarting Strata with a different configuration.

The items to implement includes at least:
- A way to discover and list all models in the data folder.
- The `/v1/models` endpoint to actually list all discovered models.
- The `/v1/unload` endpoint to actually unload the model from VRAM / system memory.
- The `/v1/load` endpoint to actually load the specified model from the file system.

No site

Links install, modelos, releases.