#5897 [Engine] serve: let the "vision" section carry its own "env" for the image encoder

open · @Vop3p · 0 评论 · 去 GitHub 看

Server & APIAMD / HIPNVIDIA / CUDAModels & quantsDocumentationWindows

说明

## Problem and result

The image encoder inherits the engine's environment. `child_env` applies the config's top-level `"env"`, and `vision_env` only adds `HIP_VISIBLE_DEVICES` / `CUDA_VISIBLE_DEVICES`. So nothing can be set for the encoder alone.

That matters on AMD gfx1030 with the encoder on an engine card. KFD gets 8 compute queues per card: MEC1 has 16, and amdgpu keeps 8 for its kernel compute rings (`kfd/hqds` lists `CP Pipe 0–3, Queue 2–3`). On a 2× RX 6900 XT layer split, the engine uses 5–6 compute queues on the encoder's card and the encoder adds 4. The kernel then logs `Runlist is getting oversubscribed due to too many queues`, and every #267 verify hang I reproduced came with that line (25 of 25 hanging runs; 0 of 13 runs without it hung). Details are in #4668.

Capping only the encoder with `GPU_MAX_HW_QUEUES=1` fixed it here: 81 pictures plus a 12-question text gate, 0 oversubscription lines, 0 hangs, OCR unchanged. Today that takes a wrapper script as `vision.exe`. Putting `GPU_MAX_HW_QUEUES` in the top-level `"env"` caps the engine too, and on this config capping the engine at 2 caused a different hang in 1 of 2 runs (`no progress for 60 s (CPU experts of layer 19)`).

After this change:

```json
"vision": { "...": "...", "env": { "GPU_MAX_HW_QUEUES": "1" } }
```

reaches the encoder only. The engine's environment is unchanged. Configs without `"vision.env"` behave exactly as before: `vision_env` still returns the engine's mapping itself.

## Related discussion

#4668 (the #267 / MEC firmware thread), #5892 (a single RX 6800 with `--vision gpu` logging the same oversubscription before a #267), ROCm/TheRock#9306. This adds one config key and changes no default, so I've opened it as a draft for the maintainer to decide whether the key, or an encoder default on gfx1030, is the preferred shape.

## Approach

- `serve/server.py` `vision_env`: after the existing card selection, the section's `"env"` is applied on top (values are stringified, as for the top-level `"env"`). It keeps the early return when there is neither a card nor a section env.
- The CPU fallback reader shares the spawn environment. A queue cap is harmless there.
- `Vision.reader_args` only reads known keys, so `"env"` never reaches the encoder's command line.
- `docs/DETAILS.md`: one row in the vision settings table.
- No default changes. An automatic `GPU_MAX_HW_QUEUES=1` for the encoder on gfx103x would be a further step; I haven't proposed it because I can only measure gfx1030.

## Validation

**Passed:**
- `python -m unittest tests.terminal.test_no_debugger_children` on this branch (base `13eedf552`), Ubuntu 26.04, Python 3.14: 18 tests OK, including the new `test_the_vision_section_env_reaches_the_reader_alone`. It covers the section env winning over the top-level env for the encoder only, numbers stringified, combined with `cuda_device`, and the no-op path returning the same mapping.
- Control: the same new test against unmodified `serve/server.py` fails with `KeyError: 'GPU_MAX_HW_QUEUES'`.
- `python -m unittest serve.test_vision_*` (13 modules): 100 tests OK.
- The setting itself, applied through a wrapper script with the same effect, on 2× RX 6900 XT (gfx1030), kernel 7.0.0-43, ROCm 10.0, a local engine build based on 0.2.11 (engine 0.1.57): 81 picture requests plus a 12-question gate, 0 oversubscription lines, 0 #267 (numbers in #4668). `/proc/<pid>/environ` of the encoder showed `GPU_MAX_HW_QUEUES=1`.

**Failed:**
- none observed

**Not run:**
- An end-to-end GPU run of this exact server revision with `"vision.env"`. The measurement above used the wrapper, which sets the same variable for the same process.
- The full `serve/test_server.py` suite and the GUI/TUI tests. The change is confined to `vision_env`.
- CUDA, Metal and Windows: no such machines. The code path is backend-independent apart from the existing `cuda_device` branch, which is unchanged.

## Limits

- The key passes any variable through. It doesn't validate names.
- It doesn't fix the underlying firmware/runtime issue (ROCm/TheRock#9306). It only lets a config keep the encoder within the card's queue budget.

## Update 2026-11-21

Rebased onto current `main` (6e38f588a, 0.2.17 plus later commits). The code moved with the 0.2.16 layout change and is now in `terminal/serve/server.py` and `terminal/tests/terminal/test_no_debugger_children.py`, unchanged otherwise. The one conflict was the `docs/DETAILS.md` table: I kept upstream's newer `"elastic"` row, added the `"env"` row, and dropped the old `--flash-attn` row that upstream had already removed. `tests/terminal/test_no_debugger_children.py`: 18 of 18 pass. On 0.2.17, `vision_env` still only sets the encoder's card, so a per-encoder `GPU_MAX_HW_QUEUES=1` still needs this change.

Developed with an AI coding assistant; every number above was measured on a 2× RX 6900 XT machine.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

本站相关内容

相关页面的快捷入口。