Pull requests / #428

#428 tests: test_setup_golden passes on Linux (<EXE> only as a whole path part)

closed · @homeofe · 0 commentaires · Sur GitHub

Setup & installAMD / HIPNVIDIA / CUDAModels & quantsWindowsLinux

Description

## Problem

`tools.test_setup_golden` (added in 07ffa4b) fails on Linux on `main`, with 46 failures across `test_yes` and `test_enter_for_every_question`, for every profile. The only key that differs is `log`:

```
AssertionError: ... 'log': '<T>/<EXE>-iq3_xxs.log' != '<T>/strata-iq3_xxs.log'
```

## Cause

`normalize()` takes the platform-specific names out before comparing against `tools/test_setup_golden.json`:

```python
return v.replace(str(t), "<T>").replace("\\", "/").replace(setup.EXE, "<EXE>")
```

`setup.EXE` is `strata.exe` on Windows, where the baseline was recorded, and `strata` on Linux. The run config's log path is `ROOT / f"strata-{tag.lower()}.log"` (`setup.py`, in the config writer). On Linux the plain substring replace also rewrites the `strata` in `strata-iq3_xxs.log`, so every config's `log` value differs from the baseline. On Windows `strata.exe` doesn't occur in the log name, so the test passes there, which is why this wasn't noticed.

## Fix

Replace `EXE` only where it is a whole path part, i.e. between `/` separators or at the start/end of the string. Backslashes are already turned into `/` the line before:

```python
v = v.replace(str(t), "<T>").replace("\\", "/")
return re.sub(rf"(?<![^/]){re.escape(setup.EXE)}(?![^/])", "<EXE>", v)
```

The baseline contains `<EXE>` only as `<T>/engine/<EXE>` (23 occurrences), so the JSON doesn't change.

| input | `EXE = strata` (Linux) | `EXE = strata.exe` (Windows) |
|---|---|---|
| `<T>/engine/strata` / `<T>/engine/strata.exe` | `<T>/engine/<EXE>` | `<T>/engine/<EXE>` |
| `<T>/strata-iq3_s.log` | unchanged | unchanged |
| bare `strata` / `strata.exe` | `<EXE>` | `<EXE>` |
| `<T>/engine/stratax` / `<T>/engine/strata.exex` | unchanged | unchanged |

`re.escape` keeps the `.` in `strata.exe` literal.

## Testing

### Test machine (Linux)

| | |
|---|---|
| CPU | AMD Ryzen Threadripper 3960X (Zen 2), 24 cores / 48 threads, up to 4.57 GHz, 128 MiB L3 |
| CPU features | `sse4_2 avx2 fma f16c bmi2`; **no AVX-512, no AVX-VNNI**. This is the class of CPU the 0.1.32 encoder crashed on (#411 i9-9900KF, #412 Ryzen 9 5950X, #419 i7-13700KF) |
| RAM | 128 GB DDR4-2133, 8 × 16 GB (125 GiB usable), 8 GiB swap |
| GPU | NVIDIA GeForce RTX 2080 Ti, 11 GiB (sm_75), driver 595.84, PCIe 3.0 x16 |
| OS | Ubuntu 24.04.4 LTS, kernel 6.8.0 |
| Toolchain | GCC 13.3.0, CMake 3.28.3, Ninja 1.11.1, GNU binutils/objdump 2.42, Python 3.12.3, CUDA 12.0 (nvcc 12.0.140) |

The suite runs fully mocked (`setup.main()` on simulated PCs), so the hardware doesn't affect the results. The tests ran beside a live Strata model server on this PC (about 58 of 125 GiB RAM and 10.5 of 11 GiB VRAM in use).


- Linux: `python3 -m unittest tools.test_setup_golden` goes from 46 failures to OK.
- I checked the regex against both platform names (table above). I couldn't run the Windows suite from here. The Windows path only changes in that `strata.exe` is now matched as a path part rather than a substring, and the baseline only has it as a path part.

Only the test file changes (one `import re` plus the two lines above). Found while verifying #427.

Sur le site

Liens install, modèles, releases.