Issues / #3651

#3651 [TUI] `launch --model-path` rejects working OrcaRouter GGUF models

open · @CyberHackademy · 2 comentarios · En GitHub

Setup & installNVIDIA / CUDAModels & quantsWindows

Descripción

## Bug Description

After updating to Strata v0.2.2 on Windows 11, I am unable to launch my existing OrcaRouter GGUF models using `strata.py launch --model-path`.

The same models load successfully through `setup.py` using their existing configuration profiles.

However, the `launch` command rejects them with the following error:

```
strata: These files cannot be used as they are (this is not a model size Strata knows). Choose the matching model in the list, or check that every shard is in the folder.
```

## Environment

- OS: Windows 11
- Strata version: v0.2.2
- GPUs: NVIDIA RTX 3090 24 GB + RTX 5060 Ti 16 GB
- RAM: 128 GB DDR5
- Models: Qwen3.8-Flash-Next-Uncensored (OrcaRouter)
- Quantizations: Q4_K_M, Q5_K_M, Q6_K
- Format: Multi-shard GGUF

## Steps to Reproduce

From a Strata installation directory:

```
cd C:\Strata

.\.venv\Scripts\python.exe .\strata.py launch --model-path "C:\Strata-data\models\Orca_Q4_K_M"
```

Alternatively, specify the first GGUF shard:

```
.\.venv\Scripts\python.exe .\strata.py launch --model-path "C:\Strata-data\models\Orca_Q4_K_M\Qwen3.8-Flash-Next-Uncensored-Q4_K_M-00001-of-00003.gguf"
```

Both approaches produce the same error.

## What Works

Using the existing installer:

```
.\.venv\Scripts\python.exe .\setup.py --gguf-dir "C:\Strata-data\models\Orca_Q4_K_M"
```

Selecting `Orca_Q4_K_M_DUAL_GPU` successfully initializes the engine.

Relevant output:

```
[strata] layer split across GPUs [0, 2] (30)
[strata] experts loaded: 75.00 GiB at 5.76 GiB/s
[strata] filling the GPU's expert cache (5479 experts, 16.61 GiB of VRAM) ...
ready: http://127.0.0.1:8080/v1
```

This confirms that the model files are usable by the installed Strata engine.

## Possible Root Cause

Looking at the v0.2.2 source code, `strata_control/models.py` uses a `SUPPORTED_SELECTIONS` list and GGUF fingerprint matching to identify local models.

OrcaRouter quantizations such as Q4_K_M, Q5_K_M, and Q6_K do not appear in this list.

The model import validation may therefore be rejecting otherwise functional custom GGUF models before the engine can initialize.

## Expected Behavior

Since the v0.2.2 release notes mention improved support for importing local GGUF models, I would expect `strata launch --model-path` to recognize and load models that already work with an existing Strata configuration.

Ideally, Strata should also support importing previously configured custom GGUF models without requiring them to match a predefined model catalog entry.

## Actual Behavior

`launch --model-path` rejects the model during identification, even though the same files load successfully through `setup.py`.

## Additional Information

The same recognition issue also occurs with my existing GSQ-RCO abliterated IQ3_S model.

Would it be possible to extend the local GGUF recognition mechanism to support existing custom model profiles and additional compatible quantizations, including OrcaRouter Q4_K_M, Q5_K_M, and Q6_K?

Thank you for your work on Strata!

En el sitio

Enlaces a install, modelos, releases.