Issues / #2301

#2301 calibrate: configured GPU vision encoder is absent during auto expert-cache sizing

open · @Tsury · 0 Kommentare · Auf GitHub

Setup & installServer & APIMulti-GPUNVIDIA / CUDAModels & quantsLinux

Beschreibung

GPU vision configurations are calibrated without the configured encoder resident. Auto expert-cache sizing therefore differs from normal serving.

At `v0.1.43.1` (`a0116b2782a5b82622358c04a06496485d12b2d6`), [`tools/calibrate.py`](https://github.com/Niko1221/Strata/blob/a0116b2782a5b82622358c04a06496485d12b2d6/tools/calibrate.py#L138) constructs only `StrataEngine`. Normal [`serve/server.py`](https://github.com/Niko1221/Strata/blob/a0116b2782a5b82622358c04a06496485d12b2d6/serve/server.py#L6392) starts `Vision` before the engine. That order explicitly lets the encoder occupy VRAM before cache allocation.

I checked this on Linux with an RTX 5090 Laptop 24 GiB, Core Ultra 9 275HX, and 64 GB RAM. The model is SC117 Swift 1.5 GSQ-RCO IQ3_S. Both starts used the same configuration and effective controls: context 262144, INT8 KV/resident32768, automatic cache, workers7, PCIe0.55, and `--vram-reserve-mib 1600`.

| Initialization | Expert slots | Expert-cache MiB | Free VRAM MiB |
| --- | ---: | ---: | ---: |
| Stock calibration initialization: engine only | 8343 | 16202 | 1148 |
| Configured GPU encoder first, then the same engine | 7615 | 14786 | 1143 |
| Normal image-enabled serving | 7615 | 14786 | 1143 |

The calibration allocation has 728 additional slots and 1416 MiB additional cache. Neither diagnostic startup generated tokens. The vision-resident start matches the existing serving measurements.

To reproduce with an existing absolute-path configuration:

1. Load its JSON, then construct `StrataEngine` using `CAL.apply(CAL.engine_args(cfg), {})` and `child_env(cfg)`.
2. Read `engine.info`, then close the engine completely.
3. Construct the configured `Vision` with `vision_env(cfg, child_env(cfg))`. Repeat the same engine initialization and read `info`.

The installed `calibrate.py` is byte-identical to the release source. The native engine is our compatible combined b11582 build; the calibration source is unchanged. The configured CUDA vision helper is b11580. The same native executable/configuration was used for both diagnostic starts.

Expected: calibration keeps the configured GPU encoder resident during all tuner engine restarts. It should use serving's vision configuration, environment, path handling, and final cleanup.

Established impact is different cache geometry. It can change the workload being tuned, including miss handling and adaptive-cache behavior. I have not measured a resulting settings, quality, or speed penalty. CPU vision, other GPUs, and multi-GPU configurations were not tested here.

I checked existing calibration/vision reports and found no focused report of this initialization mismatch.

Mehr auf der Site

Links zu Install, Modellen, Releases.