#1356 Stream corruption with `STRATA_QFUSE=1` on GFX1151 when using `parallel: 2`

open · @ChrisDeadman · 2 comments · View on GitHub

Description

A parallel request leads to the model responding with garbage.
I have nailed this down to `STRATA_QFUSE=1`, which is set by default (`STRATA_GFX1151_DEFAULTS=1`).
I suggest to remove this from the defaults, because it is not safe for parallel requests.

For parallel to work on Halo, I also have to run with `--no-prefill-borrow` by the way, otherwise the second stream hangs at _"reading the prompt: 8192 of ... tokens"_.

Related on strata.com

Install Strata
Model sizes & RAM
Release notes

Editorial links to help you install, pick models, or read release notes — not part of the upstream thread.