#6130 [Engine] RX 7900 XTX (gfx1100, Linux), 0.2.15: picture text misread ("STRATA 412" for "STRATA 42") after earlier requests on the default HIP build; the -DSTRATA_MMQ=ON build reads it right
open · @roverdrubber · 2 comments · View on GitHub
BenchmarksSetup & installServer & APIAMD / HIPModels & quantsWindowsLinux
Description
### Area
[Engine] (speed, kernels, models, the server and API)
### Desktop app, Terminal or source?
Source (START-HERE.bat, setup.sh or a build)
### What happened
The same picture (`test-image.png`, 640x480, below as base64: the text "STRATA 42" below a red circle and a blue square) and the same request (temperature 0, `max_tokens` 40, `reasoning_effort` none, the question "What text does the picture show, and which two shapes? One short sentence.") on a freshly started server, after the same 34 earlier requests (10 tool-call requests, then 24 short questions; scripts attached):
| Build (same config, same model) | Picture read right | Answer |
|---|---|---|
| 0.2.15 as `setup.build_engine_hip` builds it (`-DSTRATA_PREFILL_MMQ=ON -DSTRATA_MMQ=OFF`) | 0 of 4 | "The text in this image is STRATA 412, which is located at the bottom of a red circle and blue square." every time |
| The same source with `-DSTRATA_MMQ=ON`, the same `strata-vision` binary | 3 of 3 | "STRATA 42" |
| 0.2.15 default build with `STRATA_VISION_TOWER=mtmd` | 2 of 3 | "STRATA 42" twice, "STRATA 412" once |
| 0.2.13.1 (engine 0.1.59.1, built here the same way) | 3 of 3 | "STRATA 42", the same sentence every time |
Expected: "STRATA 42", as 0.2.13.1 and the `-DSTRATA_MMQ=ON` build answer.
- As the **first** request on a fresh server the default 0.2.15 build reads it right ("The text in the image is STRATA 42."), so the earlier requests matter. The picture prompt is 330 tokens with 0 reused in both cases (`prompt 330 tokens = 0 reused + 330 read`).
- The default build prints `prompt kernels: chunks of 1024 tokens or more run 42 of 48 layers on the fused kernels and 6 on the int8 kernels; shorter chunks run 42 on the staged fused entry and 6 on the int8 kernels.` The `-DSTRATA_MMQ=ON` build does not print this line.
- The other checks matched: tool calls 10 of 10 and short questions 23 of 24 (the same miss) in every run above; on the default build also payload fidelity 40 of 40 and needles 3 of 3 at 32K and 1 of 1 at 128K. On a fresh server, this picture and five others with printed text and numbers (made with DejaVu Sans), each asked three ways, were read right by both picture readers on the default build (18 of 18 each).
- For context, on this card the default 0.2.15 build reads prompts faster than 0.2.13.1 (x1.075 at 16K, 6 of 6 pairs; x1.115 at 64K, 3 of 3), and the `-DSTRATA_MMQ=ON` build is level with it. (Unlike #6067 on 0.2.14, the start line here is the fused one above, not the FP16 path.)
- Every run: no gfx timeouts or resets in the kernel log; the card's memory clock is capped at 1225 MHz by LACT on this PC (for stability, on every build alike).
### Smallest steps or command that reproduce it
1. Start the server freshly with the config below.
2. `sh repro.sh http://127.0.0.1:8090` (the five files are in the folded sections below; save them in one folder. Standard library only, about 45 s; `repro.sh` decodes the picture from `test-image.png.b64`).
3. The `image check:` line has the answer.
Config (local paths left out): `--pack <IQ3_M pack> --native <IQ3_M GGUF> --ple-gguf <the same GGUF> --expert-profile data/expert-profile.bin --expert-cache auto --prefill auto:32768 --spec 4 --spec-min-p 0.5 --mtp <MTP files> --max-context 204800 --kv int8 --vram-reserve-mib 3072 --kv-resident 32768 --vision`; env `STRATA_HIP_WMMA=1 STRATA_ROUTE_TAIL_SKIP=7 STRATA_HIPBLASLT_TUNING=tools/hip/gfx1100-hipblaslt-100202.txt`; picture reader on the CPU, 8 threads, the model's F16 mmproj. Expert cache 6,882 slots (15.11 GiB) on every build.
Builds: `setup.build_engine_hip({"arch": "gfx1100", "archs": ["gfx1100"], "driver": "amdgpu"}, setup.llama_override(), vision="cpu")` in a v0.2.15 checkout with ROCm 7.2.4 (vendored ggml); the MMQ build is the same cmake line with `-DSTRATA_MMQ=ON` instead of `OFF`.
Not tried: the stock Qwen3.8-Flash-Next; without the MTP drafter, `STRATA_ROUTE_TAIL_SKIP` or WMMA; a shorter list of earlier requests.
<details><summary><code>repro.sh</code></summary>
```sh
#!/bin/sh
# Run against a freshly started Strata server: 10 tool-call requests, 24 short questions, then the picture (about 45 s).
# sh repro.sh [BASE] BASE defaults to http://127.0.0.1:8090
BASE=${1:-http://127.0.0.1:8090}
cd "$(dirname "$0")" || exit 1
[ -f test-image.png ] || base64 -d test-image.png.b64 > test-image.png # the picture, 640x480 PNG
python3 toolcall.py --base "$BASE" --out toolcall-out.json
python3 quality.py --base "$BASE" --label q --out quality-out.json
python3 picture_check.py --base "$BASE" --image test-image.png
```
</details>
<details><summary><code>picture_check.py</code></summary>
```python
#!/usr/bin/env python3
# One picture question against one OpenAI-compatible server (Strata), greedy, reasoning off, then three short texts.
# picture_check.py --base http://127.0.0.1:8090 --image test-image.png
import argparse, base64, json, time, urllib.request
ap = argparse.ArgumentParser()
ap.add_argument("--base", required=True)
ap.add_argument("--image", required=True)
a = ap.parse_args()
img = base64.b64encode(open(a.image, "rb").read()).decode()
def ask(content, n):
body = {"model": "strata", "max_tokens": n, "temperature": 0, "reasoning_effort": "none",
"messages": [{"role": "user", "content": content}]}
t = time.time()
r = json.load(urllib.request.urlopen(urllib.request.Request(a.base + "/v1/chat/completions", data=json.dumps(body).encode(),
headers={"Content-Type": "application/json"}), timeout=300))
return time.time() - t, r["choices"][0]["message"]["content"]
print("image check: %.1f s -> %r" % ask([{"type": "image_url", "image_url": {"url": "data:image/png;base64," + img}},
{"type": "text", "text": "What text does the picture show, and which two shapes? One short sentence."}], 40))
for i in range(3):
print("text after a picture %d: %.1f s -> %r" % ((i,) + ask("Name three primary colours, in one short line. (%d)" % i, 24)))
```
</details>
<details><summary><code>quality.py</code></summary>
```python
#!/usr/bin/env python3
# Fixed questions with one checkable answer each, greedy, thinking off, against one OpenAI-compatible server.
# quality.py --base http://127.0.0.1:8090 --label strata --out quality-strata.json
import argparse
import json
import re
import urllib.request
from pathlib import Path
ITEMS = [
("mul", "What is 17 * 23? Reply with the number only.", "391"),
("add", "What is 1234 + 5678? Reply with the number only.", "6912"),
("pow", "What is 2 to the power of 16? Reply with the number only.", "65536"),
("sub", "What is 987 - 349? Reply with the number only.", "638"),
("div", "What is 144 / 12? Reply with the number only.", "12"),
("mod", "What is the remainder when 1000 is divided by 7? Reply with the number only.", "6"),
("reverse", "Reverse the string \"strata\". Reply with the reversed string only.", "atarts"),
("count", "How many times does the letter r occur in the word \"strawberry\"? Reply with the number only.", "3"),
("sort", "Sort these numbers in ascending order: 42, 7, 19, 3, 88. Reply with the numbers separated by commas and no spaces.", "3,7,19,42,88"),
("py-sum", "What does this Python program print? Reply with the output only.\n\nprint(sum(i*i for i in range(5)))", "30"),
("py-slice", "What does this Python program print? Reply with the output only.\n\nprint('abc'[::-1].upper())", "CBA"),
("py-set", "What does this Python program print? Reply with the output only.\n\nprint(len(set('mississippi')))", "4"),
("py-alias", "What does this Python program print? Reply with the output only.\n\nx = [1, 2, 3]\ny = x\ny.append(4)\nprint(len(x))", "4"),
("capital", "What is the capital of Australia? Reply with one word.", "Canberra"),
("symbol", "What is the chemical symbol for gold? Reply with the symbol only.", "Au"),
("planet", "Which planet is closest to the Sun? Reply with one word.", "Mercury"),
("leap", "How many days are in a leap year? Reply with the number only.", "366"),
("hex", "Convert the decimal number 255 to hexadecimal. Reply with the lowercase hex digits only, no prefix.", "ff"),
("bin", "Convert the binary number 101101 to decimal. Reply with the number only.", "45"),
("json", "From this JSON, reply with the value of \"port\" only.\n\n{\"host\": \"10.0.0.5\", \"port\": 8443, \"tls\": true}", "8443"),
("order", "Alice is taller than Bob, and Bob is taller than Carol. Who is the shortest? Reply with one word.", "Carol"),
("series", "What is the next number in this series: 2, 6, 12, 20, 30, ? Reply with the number only.", "42"),
("fib", "With F(1) = 1 and F(2) = 1, what is the 10th Fibonacci number? Reply with the number only.", "55"),
("primes", "How many prime numbers are there below 20? Reply with the number only.", "8"),
]
ap = argparse.ArgumentParser()
ap.add_argument("--base", required=True)
ap.add_argument("--label", required=True)
ap.add_argument("--out", required=True)
a = ap.parse_args()
rows, right = [], 0
for key, question, want in ITEMS:
body = {"model": "strata", "max_tokens": 32, "temperature": 0, "stream": False,
"chat_template_kwargs": {"enable_thinking": False}, "reasoning_effort": "none",
"messages": [{"role": "user", "content": question}]}
req = urllib.request.Request(a.base + "/v1/chat/completions", data=json.dumps(body).encode(),
headers={"Content-Type": "application/json"})
try:
r = json.load(urllib.request.urlopen(req, timeout=600))
answer = (r["choices"][0]["message"].get("content") or "").strip()
except Exception as e: # keep the failure in the record
answer = f"ERROR {e}"[:200]
norm = re.sub(r"[`'\"*\s]", "", answer).rstrip(".").lower()
ok = norm == want.lower()
right += ok
rows.append({"engine": a.label, "item": key, "want": want, "answer": answer[:120], "ok": ok})
print(f"{a.label:7s} {key:9s} {'ok ' if ok else 'WRONG'} want {want!r:16} got {answer[:60]!r}", flush=True)
Path(a.out).write_text(json.dumps(rows, indent=1))
print(f"{a.label}: {right} of {len(ITEMS)} correct", flush=True)
```
</details>
<details><summary><code>toolcall.py</code></summary>
```python
#!/usr/bin/env python3
# Tool-calling checks against one OpenAI-compatible server (Strata or llama-server), as an agent would use it.
# toolcall.py --base http://127.0.0.1:8090 --out toolcall-strata.json
import argparse
import json
import urllib.request
from pathlib import Path
ap = argparse.ArgumentParser()
ap.add_argument("--base", required=True)
ap.add_argument("--out", required=True)
a = ap.parse_args()
WEATHER = {"type": "function", "function": {
"name": "get_weather", "description": "Get the current weather for a city.",
"parameters": {"type": "object", "properties": {
"city": {"type": "string", "description": "City name"},
"unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}}, "required": ["city"]}}}
READ = {"type": "function", "function": {
"name": "read_file", "description": "Read a text file and return its content.",
"parameters": {"type": "object", "properties": {
"path": {"type": "string"}, "offset": {"type": "integer"}, "limit": {"type": "integer"}}, "required": ["path"]}}}
WRITE = {"type": "function", "function": {
"name": "write_file", "description": "Write content to a file, replacing it.",
"parameters": {"type": "object", "properties": {
"path": {"type": "string"}, "content": {"type": "string"}}, "required": ["path", "content"]}}}
BASH = {"type": "function", "function": {
"name": "bash", "description": "Run a shell command and return its output.",
"parameters": {"type": "object", "properties": {
"command": {"type": "string"}, "timeout": {"type": "integer"}}, "required": ["command"]}}}
def post(paRelated on this site
Quick links to related pages.