Issues / #893

#893 serve/server.py reads POST bodies as empty when sent with Transfer-Encoding: chunked (no Content-Length fallback) — breaks requests through some reverse-proxy/relay agents

closed · @oscar-investmatic · 1 comentarios · En GitHub

Server & APILinux

Descripción

**Environment:** Linux, engine 0.1.39, IQ2_XS, behind a reverse-proxy relay agent (Model Uplink) that tunnels requests over a WebSocket and re-issues them to the local server without a known `Content-Length`, using `Transfer-Encoding: chunked` instead.

**Bug:** Every POST handler in `serve/server.py` reads the request body the same way:

```python
self.rfile.read(int(self.headers.get("Content-Length", 0)))
```

When a client/proxy sends `Transfer-Encoding: chunked` instead of `Content-Length` (valid per RFC 7230, and common for proxies streaming a body of unknown length), `Content-Length` is absent, so this reads `0` bytes. The JSON body then parses as empty/`{}`, and the API returns a misleading `400 invalid_request_error: "No messages provided"` instead of actually seeing the request.

**Repro (no proxy needed):**
```bash
curl -s http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Transfer-Encoding: chunked" \
  --data-binary '{"model":"X","messages":[{"role":"user","content":"hi"}],"max_tokens":10}'
# -> 400 "No messages provided", even though a normal (Content-Length) request with the
#    identical payload returns 200
```

**Impact:** Any reverse proxy, relay, or load balancer that forwards requests without a precomputed `Content-Length` (common — many proxies stream the body through before they know its final size) will see every request fail with a confusing "no messages" error, with no indication it's a transport-framing issue.

**Fix:** added a `_read_body()` helper to the request `Handler` that falls back to manually decoding a chunked body (reading `<size-hex>\r\n<data>\r\n` segments until the terminating `0`-size chunk) when `Content-Length` is missing and `Transfer-Encoding: chunked` is present, used in `do_POST`'s main dispatch (covers `/v1/chat/completions`, `/v1/messages`, etc.). PR incoming. Three other spots (`_control_body`, `_config_post`, `_settings`) read bodies the same unsafe way and would benefit from the same helper for consistency - left them alone here to keep the fix minimal and scoped to the reported bug.

En el sitio

Enlaces a install, modelos, releases.