反馈 / #2277

#2277 Feature: legacy completions on /v1/completions (raw-prompt clients)

open · @kossum · 0 评论 · 去 GitHub 看

Server & API

说明

clients that did their own templating send a prompt as a string and want the old shape back, text_completion, cmpl- ids, usage. old openai sdk, eval harnesses, promptfoo, the usual. llama.cpp and vllm answer those with POST /v1/completions, here it's a 404

wanting: raw passthrough (no template, caller already did it), stream + non-stream, usage in the last chunk like the other dialects do

built it already, running with tests. can send a PR

本站相关内容

相关页面的快捷入口。