#6621 [Engine] Inkling SQ-10 slow on my Windows 11 / RX 9060 XT 16GB / 64GB RAM setup; mismatched API model IDs silently use Qwen
open · @998chde · 1 commentaires · Sur GitHub
BenchmarksSetup & installServer & APIAMD / HIPModels & quantsWindowsLinux
Description
## Area
[Engine] (speed, kernels, models, the server and API)
## Desktop app, Terminal or source?
Desktop app on Windows 11; an external agent connects through the OpenAI-compatible API.
## Summary
On **my Windows 11 / Radeon RX 9060 XT 16 GB / 64 GB RAM configuration**, Inkling-2-Mini SQ-10 shows approximately **127–130 tokens/s prefill and 10 tokens/s decode**. A roughly 40K-token agent request exceeded the client's timeout before prefill completed.
Separately, after switching the server's loaded model to Qwen IQ3_XXS, a client still requesting `inkling-2-mini-sq-10` continued receiving successful completions from Qwen. Even an invented model name was accepted. This made the performance comparison misleading until the raw API responses were checked. These are two observations, not a claim that they have the same root cause.
## Hardware and versions
- Windows 11
- AMD Radeon RX 9060 XT, 16 GB VRAM (gfx1200)
- AMD Ryzen 7 3700X, 8 cores / 16 threads
- 64 GB system RAM
- Desktop app: Strata 0.2.19 (confirmed in Settings > About)
- Inkling shape engine: 0.1.65, Windows x86_64 HIP rx9070 package; Settings > Engine recognizes inkling2 0.1.65 as current
- Model: Inkling-2-Mini SQ-10 (`inkling-2-mini-sq-10`), configured context 256K
- GPU driver version, actual KV mode, DSpark activation, expert placement, and hipBLASLt loading status have not yet been collected. I am not assuming they are enabled or disabled.
## 1. Inkling performance observed in the Desktop monitor
The user-facing test asks for only a short reply (the Chinese word “好”), but the agent attaches system instructions and tool definitions, resulting in approximately 40K input tokens. **This is not a one-token-input benchmark.**
Screenshot observations:
- Earlier monitor: reading approximately 14K / 40K tokens; current prefill approximately 130 tok/s; GPU utilization approximately 100%; VRAM approximately 12 / 16 GB.
- Later monitor: reading approximately 8.7K / 42K tokens; current prefill 127 tok/s; last-request decode 10.0 tok/s; GPU utilization 100%; VRAM 15.0 / 16 GB; GPU temperature 53 C; GPU power approximately 121 W.
- CPU approximately 28%; system RAM approximately 11.6 / 64 GB; disk read 0.0 MB/s at that instant.
- Recent-request table: one stopped request with 40,424 prompt tokens, 0 output tokens, 4m 5s; another completed request with 40,468 prompt tokens, 41 output tokens, reported decode 10.0 tok/s and duration 1m 15s.
- The agent timed out once; subsequent activity eventually completed prefill. Cache reuse between attempts is unknown, so the completed request's wall time should not be treated as a cold-prefill benchmark.
### Steps
1. Load Inkling-2-Mini SQ-10 in Strata 0.2.19 on the hardware above, with the recognized Windows HIP rx9070 shape engine and 256K context.
2. Connect an agent via the OpenAI-compatible endpoint.
3. Send a simple short-answer prompt while the client includes its tool schemas/system instructions (approximately 40K total prompt tokens).
4. Observe long prefill/client timeout and approximately 10 tok/s decode in the Desktop monitor.
### Expected / questions
I would like to know whether this throughput is expected for this specific hardware and build, or whether a known fallback / Windows HIP setting / missing runtime dependency could explain it. I am not claiming the release-note results on other GPUs should apply to this card.
Are there recommended settings for this configuration, and which startup-log lines would help confirm hipBLASLt, prefill chunking, KV allocation, DSpark, and VRAM/host expert residency?
Related reports: #6594 (Windows RX 9060 XT SQ-15 NaNs) and #6601 (Linux RX 9070 prefill NaNs). My observed issue is slow ongoing prefill, **not a confirmed NaN/crash**, and my pack is SQ-10, so I am not asserting those reports explain it.
## 2. Unknown / mismatched API model names silently use the loaded Qwen model
After switching the loaded server model to `iq3_xxs`, a client conversation was accidentally left set to `inkling-2-mini-sq-10`. It became faster, initially appearing to be an Inkling speed change. A direct API control test showed that it was actually Qwen responding.
`GET /v1/models` returned only:
```json
{
"id": "iq3_xxs",
"status": {"value": "loaded"},
"context_length": 262144,
"meta": {"label": "Qwen3.8-Flash-Next IQ3_XXS", "aliases": ["iq3_xxs"]}
}
```
This is an excerpt; other response fields are omitted.
Three sequential non-streaming requests used identical messages (`计算 17*23,只回复结果。`), `temperature: 0`, and `max_tokens: 256`:
| Requested model | HTTP | Response model | Content | Reported decode tok/s |
|---|---:|---|---|---:|
| `inkling-2-mini-sq-10` | 200 | `iq3_xxs` | `391` | 36.4 |
| `iq3_xxs` | 200 | `iq3_xxs` | `391` | 47.1 |
| `diagnostic-nonexistent-model-47291` | 200 | `iq3_xxs` | `391` | 52.7 |
The last two requests reused 59/64 prompt tokens. Their speed is **not an Inkling measurement** and is not comparable to the long-agent-prompt measurements above.
### Minimal reproduction
With Qwen `iq3_xxs` loaded, replace the base URL below with the running server:
```bash
curl "$BASE_URL/models"
curl "$BASE_URL/chat/completions" \
-H 'Content-Type: application/json' \
-d '{"model":"diagnostic-nonexistent-model-47291","messages":[{"role":"user","content":"计算 17*23,只回复结果。"}],"temperature":0,"max_tokens":256,"stream":false}'
```
Observed: HTTP 200 and response `model: "iq3_xxs"`, rather than an unknown-model error.
### Expected / question
Is permissive model-name handling intentional in this serving mode? If so, could it be documented clearly or controlled by an explicit setting? Preferably an unrecognized model ID would return an error, while documented aliases remain accepted. The response correctly identifies Qwen, but the requesting client's selected model label can remain Inkling, making accidental cross-model comparisons very easy.
## Reproduction status and evidence limits
Both observations were seen locally. Performance evidence is from the Desktop monitor; model-name acceptance was additionally reproduced with direct HTTP requests. No engine startup log or profiler trace is attached yet; root cause is unknown.
Three screenshots were reviewed for this report: performance monitor, Settings > Engine, and Settings > About. The measurements and version details above are transcribed from them. Image attachments are not yet uploaded through this CLI submission.
Sur le site
Liens install, modèles, releases.