#6530 [Engine] Hy4-Flash SQ-64 (strata-hy4 0.1.64): long reasoning falls into an endless repetition loop (2 of 2 attempts)

open · @m4rantza · 2 commentaires · Sur GitHub

BenchmarksServer & APIMulti-GPUNVIDIA / CUDAModels & quantsWindows

Description

### Area
Engine (strata-hy4 0.1.64, Tencent Hy4-Flash SQ-64)

### Desktop app, Terminal or source?
Terminal (strata launch), prompts sent from the built-in web chat at http://127.0.0.1:8080

### What happened
On a longer coding task, Hy4-Flash SQ-64 starts with a sensible plan in its thinking, but after a few thousand tokens it gets stuck repeating the same lines forever, for example:

"Need maybe include test for `max_size` eviction with explicit ttl None and default_ttl None. Good."

It never leaves the thinking phase and never writes an answer. It keeps going until stopped.

- Attempt 1: thought for 1,038 s, then the chat showed "Error in input stream". After typing "resume", it went into the loop.
- Attempt 2: fresh model start, new chat, same prompt: the same loop again.

While looping, decode jumps to about 80 tok/s and the engine log shows very high draft acceptance (about 4.6 accepted drafts per step, against about 1.4 in the standard benchmark), so DFly2 keeps confirming the repeated text. I can't tell whether the drafting causes the loop or only speeds it up.

Short questions work well: a 10-question reasoning, troubleshooting and coding quiz scored 95%, and the standard benchmark passes 12/12. The problem only shows up in long reasoning.

### Smallest steps or command that reproduce it
1. strata launch --family hy4-flash --model SQ-64 --data-dir "G:\Strata\Strata-data" --context 128k --no-tui
2. Open http://127.0.0.1:8080 and start a new chat
3. Send:
   Write a thread-safe TTL cache in Python with get/set/delete, per-key expiry, cleanup of expired entries, and a max size with LRU eviction, including unit tests. Put the cache in a file ttl_cache.py and the tests (unittest) in test_ttl_cache.py.
4. Wait: after a few thousand tokens of thinking, the same lines repeat endlessly.

Chat settings (defaults): Thinking "thorough", temperature 1.00, top-p 0.95, top-k 20, max tokens empty.

### Reproduction status
Reproduced 2 of 2 times (the second time after a fresh model start and in a new chat).

### Strata version
0.2.18: engine and control 0.1.64, strata-hy4 0.1.64 (cuda13)

### Graphics card, memory and system
RTX 4090 24 GB (PCIe 4.0 x8), Ryzen 9 7950X3D, 64 GB DDR5-6200, Windows 11 Pro. Models on 2x Optane P5800X RAID.
While Hy4 runs, system RAM is almost full (62.8 of 63 GB in the monitor).

### Model, size and settings
Tencent Hy4-Flash SQ-64, 128K context, defaults (DFly2 on, vision on). Engram file on the Optane RAID.
Standard benchmark: 28 / 28 / 28 tok/s decode (short / 4k / 16k), prompt 632 / 1061 tok/s, 12/12.

The same prompt on Qwen3.8-Flash-Next UD-IQ4_XS-v31 (same PC) finishes in one go without looping.

### Logs
Engine log: \AppData\Local\Strata\launch\sq-64-128k.engine.log (I can attach it). Last speculative-decoding line during the loop:
[spec] verify steps 20111: accepted drafts per step 4.549 (verified slots 5.40/step, tree_steps 966, branch_hits 404, prob_iters 49, chain_evals 136597, early_exits 2384), draft 2.7 ms/step, verify 83.5 ms/step

Thanks for Hy4 support. It's impressive to run a 220B model on a 64 GB PC, and short answers are really good.

Sur le site

Liens install, modèles, releases.