Pull requests / #3229
#3229 [Engine] loop guard: early sentence-run detection, medium-effort splice and sampling-raise-only recovery
open · @Yunado · 0 Kommentare · Auf GitHub
Server & APINVIDIA / CUDAWindows
Beschreibung
**The problem and the result.** The opt-in ``"reasoning_loop_recovery"`` guard (#728) misses, or catches too late, or catches and does nothing, in three production shapes: 1. **Too late.** A coding-agent loop repeated one ~20-word sentence over and over; the coverage look (what fraction of the tail belongs to a thrice-repeated 12-word passage) stayed near 15%, under its 25% line, while the run burned well over a thousand tokens before anything fired. 2. **Medium left out.** The recovery splices only the *xhigh* effort sentence to low; a prompt carrying the *medium* sentence returned None and the caller silently kept generating. 3. **Nothing to splice.** On a low-effort run, or a client that sends no effort sentence at all, ``focused_recovery_prompt`` returns None and the caller (``if recover_prompt is not None: break``) silently keeps generating: a production loop at low burned 29,476 output tokens with ``reasoning_recoveries=0`` while the detector itself reported coverage 1.0. After this change the guard fires on a young sentence run (the same sentence 5 times in a row; sentences under 8 words and 40 characters never count, so repeated code and short lines cannot trigger it), the recovery splices the medium sentence exactly as the xhigh one, and when a **template prompt** (one whose first turn is a system message) carries no effort sentence at all the resume goes on from the same ids with the sampling raise the resume block already performs (temperature to at least 1.0, presence penalty to at least 1.5), the server window saying "on the sampling raise alone". A prompt without a system message was never a native-effort run and stays without recovery, exactly as ``test_missing_native_high_hint_skips_recovery`` contracts: ``focused_recovery_prompt`` conflates the two None shapes, so the caller separates them with ``prompt_has_system_turn`` (the same head test the splice itself makes). **The approach.** The hunks all live inside the guarded block: the sentence-run counter (``reasoning_sentence_run``) feeds the same trigger as coverage and the periodic tail; ``focused_recovery_prompt`` matches either effort sentence and splices the first occurrence (the xhigh splice stays byte-identical to the old single-sentence algorithm); a ``spliced`` flag records which recovery ran and only changes the window line; ``SYSTEM_HEAD`` is spelled once, built from ``IM_END``. ``"stop"``, the default path (both keys off) and requests with images are byte-identical. **Validation.** - passed: ``py_compile`` on serve/server.py; ``tools/loop_guard_test.py``, 19 assertions without a GPU or a model (the medium splice to low, the xhigh splice byte-identical to the old algorithm, the sentence-run firing on the production shape while the coverage look is still under its line, silence on short code pieces, numbered enumerations and interleaved text, a Chinese sentence run counted by characters, the dense-loop coverage regression, and both sides of the system-turn gate: bare recovery allowed with a system turn, recovery stayed skipped without one); ``python -m unittest serve.test_reasoning_loop_recovery``: the suite's 18 recovery tests pass, including the contract test above; two further assertions on a fake tokenizer: ``focused_recovery_prompt`` returns None for a low-effort render (the trigger of the no-splice branch) and the xhigh splice is unchanged. - failed: none. - not run: a live looped session end to end (the changed behavior is the live resume; it needs a real looped generation with a server). **Limits.** The sentence-run trigger can mistake a deliberately repeated useful sentence for a loop, as the coverage look can; the opt-in keys are unchanged and nothing fires when they are off. At low there is no sentence left to splice down to, so the sampling raise is the only lever; if it does not break the loop the request finishes as "length" as it does today. **Environment**: e88ee39d (v0.2.0-beta-12), Windows 11 Pro, Python 3.12.0, RTX 5090.
Mehr auf der Site
Links zu Install, Modellen, Releases.