#6530 [Engine] Hy4-Flash SQ-64 (strata-hy4 0.1.64): long reasoning falls into an endless repetition loop (2 of 2 attempts)
open · @m4rantza · 4 Kommentare · Auf GitHub
BenchmarksServer & APIMulti-GPUNVIDIA / CUDAModels & quantsWindows
Beschreibung
### Area Engine (strata-hy4 0.1.64, Tencent Hy4-Flash SQ-64) ### Desktop app, Terminal or source? Terminal (strata launch), prompts sent from the built-in web chat at http://127.0.0.1:8080 ### What happened On a longer coding task, Hy4-Flash SQ-64 starts with a sensible plan in its thinking, but after a few thousand tokens it gets stuck repeating the same lines forever, for example: "Need maybe include test for `max_size` eviction with explicit ttl None and default_ttl None. Good." It never leaves the thinking phase and never writes an answer. It keeps going until stopped. - Attempt 1: thought for 1,038 s, then the chat showed "Error in input stream". After typing "resume", it went into the loop. - Attempt 2: fresh model start, new chat, same prompt: the same loop again. While looping, decode jumps to about 80 tok/s and the engine log shows very high draft acceptance (about 4.6 accepted drafts per step, against about 1.4 in the standard benchmark), so DFly2 keeps confirming the repeated text. I can't tell whether the drafting causes the loop or only speeds it up. Short questions work well: a 10-question reasoning, troubleshooting and coding quiz scored 95%, and the standard benchmark passes 12/12. The problem only shows up in long reasoning. ### Smallest steps or command that reproduce it 1. strata launch --family hy4-flash --model SQ-64 --data-dir "G:\Strata\Strata-data" --context 128k --no-tui 2. Open http://127.0.0.1:8080 and start a new chat 3. Send: Write a thread-safe TTL cache in Python with get/set/delete, per-key expiry, cleanup of expired entries, and a max size with LRU eviction, including unit tests. Put the cache in a file ttl_cache.py and the tests (unittest) in test_ttl_cache.py. 4. Wait: after a few thousand tokens of thinking, the same lines repeat endlessly. Chat settings (defaults): Thinking "thorough", temperature 1.00, top-p 0.95, top-k 20, max tokens empty. ### Reproduction status Reproduced 2 of 2 times (the second time after a fresh model start and in a new chat). ### Strata version 0.2.18: engine and control 0.1.64, strata-hy4 0.1.64 (cuda13) ### Graphics card, memory and system RTX 4090 24 GB (PCIe 4.0 x8), Ryzen 9 7950X3D, 64 GB DDR5-6200, Windows 11 Pro. Models on 2x Optane P5800X RAID. While Hy4 runs, system RAM is almost full (62.8 of 63 GB in the monitor). ### Model, size and settings Tencent Hy4-Flash SQ-64, 128K context, defaults (DFly2 on, vision on). Engram file on the Optane RAID. Standard benchmark: 28 / 28 / 28 tok/s decode (short / 4k / 16k), prompt 632 / 1061 tok/s, 12/12. The same prompt on Qwen3.8-Flash-Next UD-IQ4_XS-v31 (same PC) finishes in one go without looping. ### Logs Engine log: \AppData\Local\Strata\launch\sq-64-128k.engine.log (I can attach it). Last speculative-decoding line during the loop: [spec] verify steps 20111: accepted drafts per step 4.549 (verified slots 5.40/step, tree_steps 966, branch_hits 404, prob_iters 49, chain_evals 136597, early_exits 2384), draft 2.7 ms/step, verify 83.5 ms/step Thanks for Hy4 support. It's impressive to run a 220B model on a 64 GB PC, and short answers are really good.
Mehr auf der Site
Links zu Install, Modellen, Releases.