#582 Serve tool result images

本文

## What this is

The Anthropic dialect flattened a tool's result through `_text_of()`, which keeps the text parts
only, so an image a tool returned — Claude Code's `Read` on a picture, a screenshot tool, a browser
tool — was dropped with no error, no log line and no refusal. The model received an empty tool
result and answered anyway: **a confident description of a picture it had never been given.**

This is the shape Claude Code uses for every image its `Read` tool returns, which is why images
could work in one client and come back wrong in another.

## Measured, before the fix

4-way IQ3_S rig, one image, three request shapes:

| request shape | prompt tokens | answer |
|---|---|---|
| Anthropic, image inside `tool_result` | 380 | "B — blue shape on a white background" — **wrong** |
| Anthropic, plain user image block | 890 | correct |
| OpenAI, image in the `"tool"` message | 956 | correct |

The image contributes ~576 tokens, so 380 is the proof that it never arrived.

## The fix

`_parts_of()` keeps it, and the pack's chat template already renders one image item per image for a
message of any role (`chat_template.jinja:8`), so the rows land inside `<tool_response>` where the
picture belongs. A message holding an image beside a `tool_result` was dropped the same way and now
keeps it too.

A text-only result is unchanged: `_parts_of()` returns a plain string when there is no image.

**After:** 959 prompt tokens and "The letter Q inside a green circle on a purple background."

## A bad image source, folded in

The second commit is the same API surface: `Vision.load` let `OSError` out of every fetch — a 404
(`HTTPError`), a refused connection or an unknown host (`URLError`) are all `OSError` — and a `data:`
URL with no comma raised `IndexError`. The dispatcher maps only `ValueError` to a status, so all of
those closed the client's connection with **no reply at all** (`curl: (52) Empty reply from server`)
instead of a 400. Every branch of `load` now raises `ValueError` saying what went wrong, like the last
line already did. It sat in the Monitor's commit on the old branch; it is an image-API fix, so it is
here rather than there.

## Testing

`serve/test_server.py`'s `ToolResultImages`: the image survives and `images_of()` finds it, a
text-only result is unchanged, an image beside a tool result survives, and the rendered prompt gets
the encoder's rows. `BadImageSource` pins the 400: a 404, a refused connection, an unknown host, a
`data:` URL that is not base64, and a file that cannot be read all raise `ValueError`, and a readable
source still loads. `serve`'s 126 tests pass on `v0.1.38` (`python3 -m unittest serve.test_server`).

🤖 Generated with [Claude Code](https://claude.com/claude-code)