Issues / #2210

#2210 [Bug]: Windows HIP gfx1031: prompt reading ~30 % slower on the 0.1.43 engine than 0.1.40 (same model and server)

open · @huy-sec · 0 Kommentare · Auf GitHub

BenchmarksSetup & installServer & APIAMD / HIPModels & quantsWindowsLinux

Beschreibung

### What happened

After updating from 0.1.40 to 0.1.43, prompt reading (prefill) is about 30 % slower on this card. Decode is unchanged. Running the **0.1.40 engine under the 0.1.43 server**, same model, same prompts, brings the speed back, so the change is in the engine.

**Method.** One server start per setting, a short warm-up, then three fresh prompts (~16k, ~19k, ~16k tokens; non-overlapping slices of Markdown text, a unique first line so nothing is reused, `max_tokens: 1`). Speed is taken from the engine's `prompt N tokens = 0 reused + N read in T ms` line. The same slices in the same order for every setting. E–H ran back to back in alternating order, so file-cache warmth or background load can't favour one engine.

| Run | Model | Engine | Setting | Read tok/s |
|---|---|---|---|---|
| A | Swift 1.5 IQ2_XS | 0.1.43 | as set up | 393, 303, 381 |
| B | Swift 1.5 IQ2_XS | 0.1.43 | `--ple-io direct`, `STRATA_WATCHDOG_S=900` | 396, 386, 388 |
| C1 | IQ3_XXS | 0.1.43 | as set up | 384, 380, 394 |
| C2 | IQ3_XXS | **0.1.40** | as set up | **560, 541, 592** |
| D | Swift 1.5 IQ2_XS | **0.1.40** | as set up | **530, 525, 554** |
| E | Swift 1.5 IQ2_XS | 0.1.43 | rematch | 397, 386, 396 |
| F | Swift 1.5 IQ2_XS | **0.1.40** | rematch | **525, 524, 553** |
| G | Swift 1.5 IQ2_XS | 0.1.43 | rematch | 398, 383, 395 |
| H | Swift 1.5 IQ2_XS | **0.1.40** | rematch | **524, 492, 533** |
| I | Swift 1.5 IQ2_XS | 0.1.43 | `--prefill 8192` | 421, 393, 444 |

Request history on this PC agrees: fresh prompts of 4k+ tokens read at a median 540–578 tok/s on 0.1.40 in every size bucket (160 requests), and 303–395 on 0.1.43 (42 requests).

**Ruled out**
- PLE reads: 0.1.43 usually measures the drive at ~18–34 MB/s random 4 KiB and switches to `--ple-io mmap`. Forcing `direct` (run B, watchdog raised so a slow first read can't trip it) changes nothing, and in the rematch (E, G) its probe kept `direct` on its own: still ~395.
- Mostly not the Windows HIP prompt-chunk cap (#1630): 0.1.40 reads in 8192-token chunks, 0.1.43 in 6144. Giving 0.1.43 8192 chunks (run I) recovers a little (~420 against ~395), far short of 0.1.40's ~525; `--prefill 7168` earlier measured about the same as 6144.
- The Stager's sleeping waits (bc089d5d): Linux-only since 2a8cdedd.
- ROCm: both engines built against the same ROCm, identical DLLs.

Same model on both engines: decode 39.2 vs 39.3 tok/s, and the same answers on a small set of tool-use, coding and log tasks.

Settings (from setup): `--expert-cache auto --prefill auto --spec 4 --spec-min-p 0.5 --mtp ... --max-context 131072 --kv int8 --kv-resident 32768`, env `STRATA_HIP_PROMPT_F16=1`.

I can bisect v0.1.40..v0.1.43 on this card, or run a build with diagnostics, if that helps.

### Strata version, GPU, OS

Server 0.1.43 (7695859a); engines 0.1.43 and 0.1.40, both self-built with `tools\hip\build_windows.bat`, `STRATA_HIP_ARCHS=gfx1031`, ROCm 10.2.0a20260930 · RX 6700 XT 12 GB (gfx1031) · Ryzen 5 5500, 64 GB RAM · Windows 11 Pro (26200)

### Engine log

0.1.43 (run E), start and prompt lines (generation part trimmed):
```
strata generate: the SSD is kept awake while rows are read: one page of the table after 100 ms without a read, until 60 s after the last request (STRATA_SSD_KEEPALIVE=0 turns it off)
strata generate: CPU pool tasks/phase: 18 (automatic), participating threads: 6
strata generate: session is up (engine 0.1.43)
strata serve: prompt chunk auto: 6144 tokens, a 384-slot ring
strata serve: prompt 16408 tokens = 0 reused + 16408 read in 41324 ms (397.1 tok/s)
strata serve: prompt 18690 tokens = 0 reused + 18690 read in 48407 ms (386.1 tok/s)
```
0.1.40 (run F), the same prompts:
```
strata generate: the SSD is kept awake while rows are read: one page of the table after 100 ms without a read, until 60 s after the last request (STRATA_SSD_KEEPALIVE=0 turns it off)
strata generate: session is up (engine 0.1.40)
strata serve: prompt chunk auto: 8192 tokens, a 384-slot ring
strata serve: prompt 16408 tokens = 0 reused + 16408 read in 31251 ms (525.0 tok/s)
strata serve: prompt 18690 tokens = 0 reused + 18690 read in 35677 ms (523.9 tok/s)
strata serve: prompt 16234 tokens = 0 reused + 16234 read in 29373 ms (552.7 tok/s)
```

Mehr auf der Site

Links zu Install, Modellen, Releases.