Issues / #1356

#1356 Stream corruption with `STRATA_QFUSE=1` on GFX1151 when using `parallel: 2`

open · @ChrisDeadman · 2 commentaires · Sur GitHub

AMD / HIP

Description

A parallel request leads to the model responding with garbage.
I have nailed this down to `STRATA_QFUSE=1`, which is set by default (`STRATA_GFX1151_DEFAULTS=1`).
I suggest to remove this from the defaults, because it is not safe for parallel requests.

For parallel to work on Halo, I also have to run with `--no-prefill-borrow` by the way, otherwise the second stream hangs at _"reading the prompt: 8192 of ... tokens"_.

Sur le site

Liens install, modèles, releases.