Pull requests / #3234
#3234 [Engine] loop guard: recover on the sampling raise when no effort sentence is there to splice
closed · @Yunado · 0 Kommentare · Auf GitHub
Server & APINVIDIA / CUDADocumentationWindows
Beschreibung
**The problem and the result.** ``"reasoning_loop_recovery": "recover"`` detects a repeated reasoning and then does nothing when the prompt carries no effort sentence to swap: a low-effort request, or a client that sends no effort sentence at all, leaves ``focused_recovery_prompt`` returning None and the caller (``if recover_prompt is not None: break``) silently keeps generating. A production loop at low burned 29,476 output tokens with ``reasoning_recoveries=0`` while the detector itself reported coverage 1.0. With this change such a run recovers on the sampling raise the resume block already performs (temperature to at least 1.0, presence penalty to at least 1.5): it resumes from the same ids with no splice. **The approach.** ``serve/server.py``, three hunks in the guarded block: a ``spliced`` init next to ``looped``; when ``focused_recovery_prompt`` returns None, ``recover_prompt`` becomes ``ids + generated``, ``spliced`` goes False, and the loop breaks the same way a splice does; the recovery line in the server window says "on the sampling raise alone" instead of the low-effort-instruction wording. ``docs/DETAILS.md``: the recover bullet says the splice needs the exact xhigh sentence, and without it the resume is the sampling raise alone. The xhigh splice, ``"stop"`` and the default path (both keys off) are byte-identical; requests with images never took the recovery and still do not. **Validation.** - passed: ``py_compile`` on serve/server.py; two assertions on a fake tokenizer (one id per character, one special token): ``focused_recovery_prompt`` returns None for a low-effort render (the trigger of the new branch), and the xhigh splice is unchanged (low in, xhigh out). - failed: none. - not run: a live looped session end to end (the changed behavior is the live resume itself; it needs a real looped generation with a server). **Limits.** At low there is no sentence left to splice down to, so the sampling raise is the only lever; if it does not break the loop the request finishes as "length" as it does today. The change is behind the existing opt-in key and cannot fire when it is off. **Environment**: e88ee39d (v0.2.0-beta-12), Windows 11 Pro, Python 3.12.0, RTX 5090.
Mehr auf der Site
Links zu Install, Modellen, Releases.