Issues / #3171
#3171 [Bug]: elastic vision — a tool-returned image crashes the request (400 "malformed", AttributeError on a None encoder proc)
open · @Lithivm · 1 comments · View on GitHub
Server & APINVIDIA / CUDAModels & quantsWindows
Description
## What happened
With the elastic image encoder (`vram_elastic: true` + vision `"elastic": true`), **any request that carries an image a tool returned fails with a 400** and an internal `AttributeError`, because the encoder is not running yet and nothing starts it on that path.
```
{"error": {"type": "invalid_request_error",
"message": "the request is malformed (AttributeError: 'NoneType' object has no attribute 'stdin')"}}
```
An agent hits this continuously: a tool that reads an image file (Claude Code's `Read` of a screenshot, an MCP tool returning a plot) breaks the whole turn — and since clients resend history, the same picture can break the next turns too.
## Reproduction — no waiting involved
The encoder has not started in a fresh elastic session (it starts with the *first picture*), so this reproduces on the first request:
1. A model config with `"vram_elastic": true` and `"vision": {"gpu": true, "elastic": true, ...}`; start the server. `strata-vision.exe` is **not** running (expected for elastic).
2. Send one request whose messages contain an image in a **tool** message — Anthropic dialect, e.g.:
```json
POST /v1/messages
{"model": "...", "max_tokens": 120, "messages": [
{"role": "user", "content": "read this"},
{"role": "assistant", "content": [{"type": "tool_use", "id": "t1", "name": "Read", "input": {}}]},
{"role": "user", "content": [{"type": "tool_result", "tool_use_id": "t1",
"content": [{"type": "image", "source": {"type": "base64", "media_type": "image/png", "data": "<any png>"}}]}]}]}
```
3. **400** with the message above; the engine log shows nothing, the traceback only appears on the server's stdout.
The same request succeeds with `"elastic": false` (the encoder is started with the model, so `Vision.proc` is never `None`), which is the current workaround. A *user*-sent image works too — only the tool-returned path fails.
## Traceback
```
[strata] 400 malformed request (AttributeError: 'NoneType' object has no attribute 'stdin'):
Traceback (most recent call last):
File "D:\Strata\serve\server.py", line 5508, in _post
self._responses(req)
File "D:\Strata\serve\server.py", line 5899, in _responses
ids, thinking, tools, max_new, asm, validator, req = self._responses_prepare(req, cancel)
File "D:\Strata\serve\server.py", line 6019, in _responses_prepare
ids, thinking, max_new = svc.prepare(messages, shown, kw, req.get("max_output_tokens") or 0, req=req, ...)
File "D:\Strata\serve\server.py", line 3854, in prepare
fetched = self._note_unreadable_tool_images(messages, cancel)
File "D:\Strata\serve\server.py", line 3840, in _note_unreadable_tool_images
self.vision.encode(src)
File "D:\Strata\serve\server.py", line 2426, in encode
self.proc.stdin.write(f"ENC {img.name} {out.name}\n") # relative to the encoder's cwd (#480)
AttributeError: 'NoneType' object has no attribute 'stdin'
```
(line numbers are 0.1.44.1; it arrives through `/v1/responses` here and through `/v1/messages` in the repro)
## Cause
The two places that encode a request's pictures are not symmetric. `prepare()` starts an elastic encoder before encoding:
```python
# prepare(), the request's own images
with self._turn(cancel):
self._vision_up() # ← present
encoded = self.vision.encode_all(images, cancel) ...
```
`_note_unreadable_tool_images()` (called earlier from `prepare()`, for pictures a tool returned) encodes without it:
```python
# _note_unreadable_tool_images()
with self._turn(cancel): # the encoder takes its turn with the requests (see below)
self.vision.encode(src) # ← no _vision_up(): Vision.proc is None while an elastic encoder is down
```
`Vision.encode()` writes to `self.proc.stdin` unconditionally, so with an elastic encoder that has not started (or has stopped after `elastic_idle_s`) it raises `AttributeError`. That function's `except (ValueError, OSError)` does not catch it, so the exception escapes `prepare()` and lands in `_post`'s generic `except (TypeError, KeyError, AttributeError, IndexError, UnicodeError)`, which reports **any** such internal error as `400 invalid_request_error` ("the request is malformed") — hence a server bug that looks like a client bug. This is present on `main` (76e51103): `prepare()` has the `_vision_up()` call at line 4345-4346, `_note_unreadable_tool_images()` has none.
## Suggested fix
Start the encoder there too, exactly as the sibling path does (PR follows):
```diff
with self._turn(cancel): # the encoder takes its turn with the requests (see below)
+ self._vision_up()
self.vision.encode(src)
```
`_vision_up()` is a no-op when the encoder is not elastic or is already running, so this changes nothing for non-elastic setups.
Two smaller things seen on the way, both separate from the fix above:
- `_note_unreadable_tool_images()` exists so that an unreadable tool picture becomes `[image omitted: ...]` instead of a 400. Widening its `except` to `AttributeError` (or to `Exception`) would keep that promise for unexpected failures as well — right now an internal error on this path defeats the function's own purpose.
- `_post`'s generic handler maps **any** `AttributeError` raised while preparing a request to `400 invalid_request_error`. That is how this surfaced as "the request is malformed": the message sent the reporter looking at their own client. An internal error is a 500.
## Environment
- Strata 0.1.44.1, engine 0.1.44.1 (release zip), Windows 11 Pro 10.0.26200
- RTX 5070 Ti 16 GiB, driver 616.64; Ryzen 7 9700X, 31.6 GiB RAM
- Model Q2_0, context 131072, `--kv q4_0`, `--mmap-experts`, `--vram-reserve-mib 1600`, `--vram-elastic`
- `"vision": {"gpu": true, "max_tokens": 1024, "elastic": true, "elastic_mib": 3584, "elastic_idle_s": 120}`, `"vram_elastic": true`
Related but different: #2454 (an elastic encoder re-starts on every turn whose history re-sends a cached picture — a churn/wall-time issue, not a crash).
Related on this site
Quick links to related pages.