Issues / #1356

#1356 Stream corruption with `STRATA_QFUSE=1` on GFX1151 when using `parallel: 2`

open · @ChrisDeadman · 2 コメント · GitHub で見る

AMD / HIP

本文

A parallel request leads to the model responding with garbage.
I have nailed this down to `STRATA_QFUSE=1`, which is set by default (`STRATA_GFX1151_DEFAULTS=1`).
I suggest to remove this from the defaults, because it is not safe for parallel requests.

For parallel to work on Halo, I also have to run with `--no-prefill-borrow` by the way, otherwise the second stream hangs at _"reading the prompt: 8192 of ... tokens"_.

関連リンク

インストール・モデル・リリースへの站内リンク。