Issues / #1356
#1356 Stream corruption with `STRATA_QFUSE=1` on GFX1151 when using `parallel: 2`
open · @ChrisDeadman · 2 Kommentare · Auf GitHub
Beschreibung
A parallel request leads to the model responding with garbage. I have nailed this down to `STRATA_QFUSE=1`, which is set by default (`STRATA_GFX1151_DEFAULTS=1`). I suggest to remove this from the defaults, because it is not safe for parallel requests. For parallel to work on Halo, I also have to run with `--no-prefill-borrow` by the way, otherwise the second stream hangs at _"reading the prompt: 8192 of ... tokens"_.
Mehr auf der Site
Links zu Install, Modellen, Releases.