Pull requests / #1946

#1946 fix(vision): scale GPU warm-up for image caps above 4096

open · draft · @xszwow · 0 comentarios · En GitHub

BenchmarksSetup & installAMD / HIPNVIDIA / CUDAModels & quantsWindows

Descripción

Issue: Fixes #1945

## Summary

With `--gpu --max-tokens 8192`, `strata-vision` still warms a fixed 2048 × 2048 bitmap, which produces only 4096 image tokens. On a Radeon 8060S/Vulkan run, encoding an 8160-token 4K image subsequently allocated another 605,757,440 bytes after `READY`, when the language engine may already have sized its expert cache.

Scale the dummy bitmap for explicitly configured caps above 4096. The tested 8192 cap now warms at 8100 tokens and reaches its observed 2.1474 GiB encoder allocation before `READY`; the same 4K image adds no further allocation in this test.

## What changed

Only `tools/vision/strata_vision.cpp` changes:

- Add `<cmath>` and use `32 * ceil(sqrt(max_tokens))` for caps above 4096, matching the Qwen merged-token pixel size.
- Retain the 2048-pixel square for an omitted cap and caps up to 4096. CPU mode continues to skip warm-up.
- Correct the comment: mtmd rounds the square's patch grid, so prewarm approaches the cap rather than guaranteeing its exact token count.

The model, encoder math, CLI defaults, language engine, installer and server settings remain unchanged.

## Validation

Same-day source builds against `main` at `61b3fb5dd3f1e8ec09cf7e4e05208bc6d3c46406`, with llama.cpp pinned at `3cf03257f219afbe7334045ff7c6a06ac68c627d`. Tested on Windows 11 Home build 26200, Ryzen AI Max+ 395 / Radeon 8060S (gfx1151), 128 GB RAM / 96 GB GPU assignment, AMD driver 32.0.31032.1003, LLVM MinGW 23.1.3 (20261006), Vulkan SDK 1.4.350.0. Both builds used native CPU kernels.

- CPU-only and Vulkan Release build/link passed for baseline and patch.
- Vulkan: caps 300, 4096 and 8192 all reached `READY 2560`, encoded valid finite SVE embeddings and exited cleanly.
- CPU: cap 8192 kept the no-warm-up path and produced valid finite embeddings.
- All six paired SVE files were byte-identical: a 1200 × 720 Chinese-text fixture at all three Vulkan caps and CPU 8192; a synthetic 3840 × 2160 fixture at Vulkan 4096 and 8192.
- The published source blob matches the locally compiled file; `git diff --check` passed.

| GPU image cap | Baseline warm-up tokens | Patched warm-up tokens |
| --- | ---: | ---: |
| 300 | 289 | 289 |
| 4096 | 4096 | 4096 |
| 8192 | 4096 | 8100 |

For cap 8192, Windows dedicated-GPU-memory counters were:

| Snapshot | Baseline bytes | Patched bytes |
| --- | ---: | ---: |
| At READY | 1,699,991,552 | 2,305,748,992 |
| After the 8160-token 4K image | 2,305,748,992 | 2,305,748,992 |

Measured prewarm was 8.2 s before and 33.2 s after at cap 8192. This intentionally moves the larger allocation/encode cost to startup. These are single functional observations, not a throughput benchmark or an established end-to-end speed improvement.

Reproduction:

```text
strata-vision --mmproj <mmproj-Qwen3.8-Flash-Next-BF16.gguf> --model <first-text-GGUF-shard> --gpu --threads 16 --min-tokens 1024 --max-tokens 8192
ENC <3840x2160-image.png> <output.sve>
QUIT
```

## Extra Notes

Draft because CUDA encoder build/runtime validation is still outstanding under the repository's backend-validation requirement; this AMD machine has no CUDA toolkit. HIP/SYCL language-engine files are untouched. Other GPUs, caps above 8192 and unusual aspect ratios were not tested, and no OOM/paging failure is claimed to have been reproduced.

Both local builds needed an explicit `<functional>` include in the external pinned llama.cpp Vulkan header when using libc++; that identical build prerequisite is excluded from this Strata PR.

Related Windows AMD image work: #1155, #1287, #1789. This change can accompany those build/installer changes.

En el sitio

Enlaces a install, modelos, releases.