Issues / #3651
#3651 [TUI] `launch --model-path` rejects working OrcaRouter GGUF models
open · @CyberHackademy · 2 comentarios · En GitHub
Setup & installNVIDIA / CUDAModels & quantsWindows
Descripción
## Bug Description After updating to Strata v0.2.2 on Windows 11, I am unable to launch my existing OrcaRouter GGUF models using `strata.py launch --model-path`. The same models load successfully through `setup.py` using their existing configuration profiles. However, the `launch` command rejects them with the following error: ``` strata: These files cannot be used as they are (this is not a model size Strata knows). Choose the matching model in the list, or check that every shard is in the folder. ``` ## Environment - OS: Windows 11 - Strata version: v0.2.2 - GPUs: NVIDIA RTX 3090 24 GB + RTX 5060 Ti 16 GB - RAM: 128 GB DDR5 - Models: Qwen3.8-Flash-Next-Uncensored (OrcaRouter) - Quantizations: Q4_K_M, Q5_K_M, Q6_K - Format: Multi-shard GGUF ## Steps to Reproduce From a Strata installation directory: ``` cd C:\Strata .\.venv\Scripts\python.exe .\strata.py launch --model-path "C:\Strata-data\models\Orca_Q4_K_M" ``` Alternatively, specify the first GGUF shard: ``` .\.venv\Scripts\python.exe .\strata.py launch --model-path "C:\Strata-data\models\Orca_Q4_K_M\Qwen3.8-Flash-Next-Uncensored-Q4_K_M-00001-of-00003.gguf" ``` Both approaches produce the same error. ## What Works Using the existing installer: ``` .\.venv\Scripts\python.exe .\setup.py --gguf-dir "C:\Strata-data\models\Orca_Q4_K_M" ``` Selecting `Orca_Q4_K_M_DUAL_GPU` successfully initializes the engine. Relevant output: ``` [strata] layer split across GPUs [0, 2] (30) [strata] experts loaded: 75.00 GiB at 5.76 GiB/s [strata] filling the GPU's expert cache (5479 experts, 16.61 GiB of VRAM) ... ready: http://127.0.0.1:8080/v1 ``` This confirms that the model files are usable by the installed Strata engine. ## Possible Root Cause Looking at the v0.2.2 source code, `strata_control/models.py` uses a `SUPPORTED_SELECTIONS` list and GGUF fingerprint matching to identify local models. OrcaRouter quantizations such as Q4_K_M, Q5_K_M, and Q6_K do not appear in this list. The model import validation may therefore be rejecting otherwise functional custom GGUF models before the engine can initialize. ## Expected Behavior Since the v0.2.2 release notes mention improved support for importing local GGUF models, I would expect `strata launch --model-path` to recognize and load models that already work with an existing Strata configuration. Ideally, Strata should also support importing previously configured custom GGUF models without requiring them to match a predefined model catalog entry. ## Actual Behavior `launch --model-path` rejects the model during identification, even though the same files load successfully through `setup.py`. ## Additional Information The same recognition issue also occurs with my existing GSQ-RCO abliterated IQ3_S model. Would it be possible to extend the local GGUF recognition mechanism to support existing custom model profiles and additional compatible quantizations, including OrcaRouter Q4_K_M, Q5_K_M, and Q6_K? Thank you for your work on Strata!
En el sitio
Enlaces a install, modelos, releases.