Pull requests / #454
#454 serve: honor stop and stop_sequences
closed · @zuraiz-anjum · 0 评论 · 在 GitHub 查看
描述
The server ignored OpenAI `stop` and Anthropic `stop_sequences` without an error, so a client got the text past its stop string and stop_reason "end_turn". With the mock engine scripted to say "alpha END beta gamma": Before: OpenAI stop=['END'] -> 'alpha END beta gamma' stop Anthropic stop_sequences=['END'] -> 'alpha END beta gamma' end_turn None After: OpenAI stop=['END'] -> 'alpha ' stop Anthropic stop_sequences=['END'] -> 'alpha ' stop_sequence END Service.run now cuts the answer text at the first stop string and leaves it out. Only the end of the text that could still be the start of a stop string is held back, so a stop split across tokens or streamed chunks is caught, and anything that turns out not to be a stop goes out as normal. On a match run() breaks out of the generation loop the same way a stop token does, so gen.close() sends STOP and the engine does not keep decoding. No engine change was needed. OpenAI takes a string or a list of up to 4 and Anthropic a list; a bad value is a 400. Anthropic sets stop_reason "stop_sequence" and stop_sequence to the matched string, streamed and not. Only answer text is matched, not thinking or tool call arguments. docs/DETAILS.md lists stop strings with the other honored fields. New tests in serve/test_server.py (StopStrings) cover both APIs, streamed and not, several stops, a stop split across tokens, a near miss prefix, no stop, an empty list, bad values, and that the engine stopped early. python -m unittest serve.test_server serve.test_mcp serve.test_lifecycle serve.test_monitor serve.test_detok serve.test_structured serve.test_winjob plus all tools.test_setup_* modules: 270 tests OK (7 skipped). One edge I left alone: if a reply hits a stop string and also has a finished tool call, OpenAI still reports finish_reason "tool_calls". Easy to change if you'd rather stop win.
站内延伸阅读
链到安装、模型与版本说明,便于 SEO/GEO,非官方 issue 正文。