Pull requests / #2691
#2691 tools tests: the golden compare takes either platform's engine names, on both sides
open · @clothespin · 0 comentarios · En GitHub
BenchmarksSetup & installServer & APIWindowsLinux
Descripción
## Title Issue: none. No open issue or pull request covers this (searched `test_setup_golden`, `strata-vision.exe` and `golden`). It is a test-only bug, separate from #2041, which is where I hit it. ## Summary `tools/test_setup_golden.py` cannot match the committed baseline on Linux. The baseline was recorded on Windows, so its five unsloth UD-Q4_K_XL configs hold `"<T>/engine/strata-vision.exe"`, while on Linux setup writes `strata-vision` (`setup.py:498`, `VEXE = "strata-vision.exe" if WIN else "strata-vision"`). `normalize()` replaced only `setup.EXE`, and only in the config it had just written - the baseline read back in `setUpClass` was compared as it was. Every UD-Q4_K_XL case therefore differs on Linux: 8 failures, on a clean checkout of `origin/main` (`296b59d2`) as much as on a branch. One commit, `tools/test_setup_golden.py` only; `tools/test_setup_golden.json` is untouched. Re-recording with `--write` is not the fix. On Linux it changes exactly those five lines and nothing else, which makes the baseline Linux-spelled and moves all five onto Windows. ## What changed `canonical()` maps either platform's spelling of an engine name to one placeholder (`<EXE>`, `<VEXE>`), and `canonical_all()` runs it through the baseline in `setUpClass`, so a file recorded on either platform compares on both. Both names come from `setup.EXE` and `setup.VEXE` with the extension stripped, so they cannot drift away from setup. `normalize()` now calls `canonical()`. `#973`'s rule is kept: only the whole name matches, so `strata-device.exe`, `strata-x.log` and `strata-vision-bench` are left alone. `tools/test_setup_golden.json` is untouched, so nothing about what the baseline asserts changes - only how the two sides are compared. Left alone: `setup.py`, the baseline JSON, every other test. ### How I checked Linux, no GPU used, `python tools/test_setup_golden.py`: - before, on a clean checkout of `origin/main` (`296b59d2`): `FAILED (failures=8)` - 5 in `test_yes`, 3 in `test_enter_for_every_question` (the other two UD-Q4_K_XL profiles are skipped by the existing `ram < 48` guard); - after: `OK`, 4 tests; - the mapping is symmetric: `"<T>/engine/strata-vision.exe"` and `"<T>/engine/strata-vision"` both reduce to `"<T>/engine/<VEXE>"`, as do `strata.exe` and `strata` to `<EXE>`, so Windows - where both sides already agree - behaves as it did; - the negatives hold: `strata-device.exe`, `strata-x.log`, `strata-vision-bench` and an already-replaced `<EXE>` come back unchanged. - every other `tools/test_setup_*.py` and `tools/test_strata_mcp.py` passes as it did before the change. ### Not tested - Windows. There is no Windows machine here. On Windows `canonical()` maps `strata-vision.exe` to `<VEXE>` on both sides of the compare, which is what `normalize()` already did to the live side, so the change should be a no-op there - but that is reasoning, not a run. - A baseline recorded on Linux. `--write` would produce one; I reverted it rather than commit a platform-specific baseline. ## Extra Notes - `tools/test_setup_engine_hash.py` fails 2 tests on `main`, unrelated to this and left as it is: `AssertionError: SystemExit not raised` in `test_a_first_install_stops_because_there_is_nothing_to_fall_back_to`, and `AssertionError: 'keeping the engine that is installed' not found in ''` in `test_updating_keeps_the_installed_engine_instead_of_stopping`. Both are `get_prebuilt()`'s SHA-256 refusal. Worth its own issue. - Until this lands, `python tools/test_setup_golden.py` reports those 8 failures on #2041's branch too, which makes that PR's "tests pass" line need a caveat it would rather not have.
En el sitio
Enlaces a install, modelos, releases.