#1356 Stream corruption with `STRATA_QFUSE=1` on GFX1151 when using `parallel: 2`

open · @ChrisDeadman · 2 评论 · 在 GitHub 查看

描述

A parallel request leads to the model responding with garbage.
I have nailed this down to `STRATA_QFUSE=1`, which is set by default (`STRATA_GFX1151_DEFAULTS=1`).
I suggest to remove this from the defaults, because it is not safe for parallel requests.

For parallel to work on Halo, I also have to run with `--no-prefill-borrow` by the way, otherwise the second stream hangs at _"reading the prompt: 8192 of ... tokens"_.

站内延伸阅读

Install Strata
Model sizes & RAM
Release notes

链到安装、模型与版本说明，便于 SEO/GEO，非官方 issue 正文。