Issues / #3325
#3325 [Other] test suite is not clean on a Linux source-build machine: 48 setup-golden failures, and one module needs pytest
open · @Dmitry-B · 0 Kommentare · Auf GitHub
Setup & installServer & APINVIDIA / CUDALinux
Beschreibung
Two independent reasons why `python -m unittest` on a normal Linux install cannot finish clean. Neither
is an engine or server problem: both are in the test side. Both reproduced in a clean worktree of the
same commit, with no local files.
## Problem 1: `tools/test_setup_golden` expects `strata-vision.exe`, so every golden entry fails on Linux
Problem: on Linux, `python -m unittest tools.test_setup_golden` fails 48 subtests. The only differing
field in every one of them is the vision binary name: `setup.py` writes `strata-vision` (correct for
Linux), the recorded baseline in `tools/test_setup_golden.json` has `strata-vision.exe`.
Environment: Strata `e88ee39d` (0.2.0-beta, 12 commits past the `v0.2.0-beta` tag), Ubuntu 26.04.1,
kernel 7.0.0-38, Python 3.14.4, RTX 4090 (the GPU is not involved in these tests), machine with a
source-built engine. Reproduced in `git worktree /tmp/gold-check e88ee39d` - a clean tree, without our
local configs and scripts.
Reproduce (in the clean worktree, with the project venv):
```bash
python -m unittest tools.test_setup_golden
# Ran 4 tests in 3.018s
# FAILED (failures=48)
```
Expected and actual, from the assertion with `maxDiff = None` (the whole config matches except these
two lines):
```text
- 'vision': {'exe': '<T>/engine/strata-vision',
+ 'vision': {'exe': '<T>/engine/strata-vision.exe',
```
All 25 entries of `tools/test_setup_golden.json` carry `"exe": "<T>/engine/strata-vision.exe"` in their
`vision` block (`grep -c strata-vision.exe` on that file: 25).
Suspected path: `normalize()` in `tools/test_setup_golden.py:52` removes the platform engine name with
`re.sub(... setup.EXE ..., "<EXE>", ...)`, so the main binary compares platform-independently - but it
does nothing about `setup.VEXE` (`setup.py:668`: `VEXE = "strata-vision.exe" if WIN else "strata-vision"`).
The same treatment of `VEXE` (and `HVEXE`) as of `EXE` would make the baseline platform-independent the
way it already is for the engine binary.
Reproduction status: reproduced locally, 48/48 subtests, in a clean worktree of the same commit. Note:
the same run on engine 0.1.44.1 in October gave 8 failures from this module; the number grew with the
fixture, the cause is the same.
## Problem 2: `tools/test_nvfp4_local_pack.py` imports pytest, which is not in `requirements.txt`
Problem: running the suite the way the repository runs it (`python -m unittest`) produces an import
error instead of a skip, so a full-suite run always has at least one red item on a fresh install.
Reproduce:
```bash
python -m unittest discover -s tools -p "test_nvfp4_local_pack.py"
# ERROR: test_nvfp4_local_pack (unittest.loader._FailedTest.test_nvfp4_local_pack)
# ImportError: Failed to import test module: test_nvfp4_local_pack
# File ".../tools/test_nvfp4_local_pack.py", line 6, in <module>
# import pytest
# ModuleNotFoundError: No module named 'pytest'
```
Expected: either the module skips itself without pytest (as other optional-dependency tests do), or
pytest is listed in the development requirements.
Actual: after `pip install pytest` (9.1.1) into the project venv, the same module passes: 9 passed,
2 skipped. So nothing in the module is broken on Linux - only the dependency is not declared.
Reproduction status: reproduced locally (before and after installing pytest).
## Not investigated
The other failures of a full run on this machine are machine state, not code, and are listed here only
so that the counts above can be read: 4 tests of `tools.test_strata_mcp` start their own server on port
8080, which on this machine is the running production service; 2 of
`tools.test_setup_engine_hash` assume a downloaded engine (`engine/BUILD.json` has `"source": "local"`
on a machine that always compiles the engine); one `serve/test_keepawake` timing test passes on re-run.
Mehr auf der Site
Links zu Install, Modellen, Releases.