Issues / #2210
#2210 [Bug]: Windows HIP gfx1031: prompt reading ~30 % slower on the 0.1.43 engine than 0.1.40 (same model and server)
open · @huy-sec · 0 Kommentare · Auf GitHub
BenchmarksSetup & installServer & APIAMD / HIPModels & quantsWindowsLinux
Beschreibung
### What happened After updating from 0.1.40 to 0.1.43, prompt reading (prefill) is about 30 % slower on this card. Decode is unchanged. Running the **0.1.40 engine under the 0.1.43 server**, same model, same prompts, brings the speed back, so the change is in the engine. **Method.** One server start per setting, a short warm-up, then three fresh prompts (~16k, ~19k, ~16k tokens; non-overlapping slices of Markdown text, a unique first line so nothing is reused, `max_tokens: 1`). Speed is taken from the engine's `prompt N tokens = 0 reused + N read in T ms` line. The same slices in the same order for every setting. E–H ran back to back in alternating order, so file-cache warmth or background load can't favour one engine. | Run | Model | Engine | Setting | Read tok/s | |---|---|---|---|---| | A | Swift 1.5 IQ2_XS | 0.1.43 | as set up | 393, 303, 381 | | B | Swift 1.5 IQ2_XS | 0.1.43 | `--ple-io direct`, `STRATA_WATCHDOG_S=900` | 396, 386, 388 | | C1 | IQ3_XXS | 0.1.43 | as set up | 384, 380, 394 | | C2 | IQ3_XXS | **0.1.40** | as set up | **560, 541, 592** | | D | Swift 1.5 IQ2_XS | **0.1.40** | as set up | **530, 525, 554** | | E | Swift 1.5 IQ2_XS | 0.1.43 | rematch | 397, 386, 396 | | F | Swift 1.5 IQ2_XS | **0.1.40** | rematch | **525, 524, 553** | | G | Swift 1.5 IQ2_XS | 0.1.43 | rematch | 398, 383, 395 | | H | Swift 1.5 IQ2_XS | **0.1.40** | rematch | **524, 492, 533** | | I | Swift 1.5 IQ2_XS | 0.1.43 | `--prefill 8192` | 421, 393, 444 | Request history on this PC agrees: fresh prompts of 4k+ tokens read at a median 540–578 tok/s on 0.1.40 in every size bucket (160 requests), and 303–395 on 0.1.43 (42 requests). **Ruled out** - PLE reads: 0.1.43 usually measures the drive at ~18–34 MB/s random 4 KiB and switches to `--ple-io mmap`. Forcing `direct` (run B, watchdog raised so a slow first read can't trip it) changes nothing, and in the rematch (E, G) its probe kept `direct` on its own: still ~395. - Mostly not the Windows HIP prompt-chunk cap (#1630): 0.1.40 reads in 8192-token chunks, 0.1.43 in 6144. Giving 0.1.43 8192 chunks (run I) recovers a little (~420 against ~395), far short of 0.1.40's ~525; `--prefill 7168` earlier measured about the same as 6144. - The Stager's sleeping waits (bc089d5d): Linux-only since 2a8cdedd. - ROCm: both engines built against the same ROCm, identical DLLs. Same model on both engines: decode 39.2 vs 39.3 tok/s, and the same answers on a small set of tool-use, coding and log tasks. Settings (from setup): `--expert-cache auto --prefill auto --spec 4 --spec-min-p 0.5 --mtp ... --max-context 131072 --kv int8 --kv-resident 32768`, env `STRATA_HIP_PROMPT_F16=1`. I can bisect v0.1.40..v0.1.43 on this card, or run a build with diagnostics, if that helps. ### Strata version, GPU, OS Server 0.1.43 (7695859a); engines 0.1.43 and 0.1.40, both self-built with `tools\hip\build_windows.bat`, `STRATA_HIP_ARCHS=gfx1031`, ROCm 10.2.0a20260930 · RX 6700 XT 12 GB (gfx1031) · Ryzen 5 5500, 64 GB RAM · Windows 11 Pro (26200) ### Engine log 0.1.43 (run E), start and prompt lines (generation part trimmed): ``` strata generate: the SSD is kept awake while rows are read: one page of the table after 100 ms without a read, until 60 s after the last request (STRATA_SSD_KEEPALIVE=0 turns it off) strata generate: CPU pool tasks/phase: 18 (automatic), participating threads: 6 strata generate: session is up (engine 0.1.43) strata serve: prompt chunk auto: 6144 tokens, a 384-slot ring strata serve: prompt 16408 tokens = 0 reused + 16408 read in 41324 ms (397.1 tok/s) strata serve: prompt 18690 tokens = 0 reused + 18690 read in 48407 ms (386.1 tok/s) ``` 0.1.40 (run F), the same prompts: ``` strata generate: the SSD is kept awake while rows are read: one page of the table after 100 ms without a read, until 60 s after the last request (STRATA_SSD_KEEPALIVE=0 turns it off) strata generate: session is up (engine 0.1.40) strata serve: prompt chunk auto: 8192 tokens, a 384-slot ring strata serve: prompt 16408 tokens = 0 reused + 16408 read in 31251 ms (525.0 tok/s) strata serve: prompt 18690 tokens = 0 reused + 18690 read in 35677 ms (523.9 tok/s) strata serve: prompt 16234 tokens = 0 reused + 16234 read in 29373 ms (552.7 tok/s) ```
Mehr auf der Site
Links zu Install, Modellen, Releases.