Issues / #1944
#1944 Long story creation leads to single token responses when context grows
open · @RobinMeyer · 2 comments · View on GitHub
Description
I used a story generator with strata 0.1.42, which sends a request for every section with the content of all earlier request as context and there it happened, that after about 12000 tokens, the responses are cut in the middle and all following responses return only one token anymore. I'm using RTX5070ti 16GB, 7950X3d and 64GB RAM. The configuration is: "--expert-cache", "auto", "--prefill", "auto", "--spec", "4", "--mtp", "D:\\AI\\Strata-data\\mtp\\rt", "--max-context", "131072", "--kv", "int8", "--kv-resident", "32768", "--spec-min-p", "0.5", "--pool-workers", "8", "--adapt-every", "1", "--adapt-swaps", "160", "--adapt-decay", "0.97" With strata 0.1.39, the error seems not to happen. There, I just wrote a 90000 character story without any broken chapters with the same config.
Related on strata.com
Editorial links to help you install, pick models, or read release notes — not part of the upstream thread.