Issues / #2792

#2792 tools/strata_bench.py: UnicodeDecodeError reading CMakeLists.txt on non-UTF-8 Windows (GBK/cp932), before any measurement

open · @Lithivm · 0 Kommentare · Auf GitHub

BenchmarksSetup & installWindows

Beschreibung

`tools/strata_bench.py:564` (`project_version`) reads `CMakeLists.txt` with `Path.read_text()` and **no `encoding=`**, so it decodes with the OS locale code page. On a Chinese or Japanese Windows (GBK / cp932) the file's UTF-8 BOM makes the read fail and `BENCH.bat` dies **before any measurement**:

```
$ python tools/strata_bench.py
Starting strata-iq2_xs.json on port 8080 (loading takes 1-3 minutes; the PC may be slow meanwhile) ...
  still loading (2 s) ...
Traceback (most recent call last):
  File "D:\Strata\tools\strata_bench.py", line 1028, in <module>
    sys.exit(main())
  File "D:\Strata\tools\strata_bench.py", line 989, in main
    pre = collect_machine(base, key, cfg, a.gpu)
  File "D:\Strata\tools\strata_bench.py", line 639, in collect_machine
    "strata_version": project_version(), "strata_commit": git_commit(),
  File "D:\Strata\tools\strata_bench.py", line 564, in project_version
    m = re.search(r"project\(strata VERSION ([\d.]+)", (ROOT / "CMakeLists.txt").read_text())
  File "C:\Users\...\lib\pathlib.py", line 1135, in read_text
    return f.read()
UnicodeDecodeError: 'gbk' codec can't decode byte 0xbf in position 2: illegal multibyte sequence
```

`CMakeLists.txt` starts with the UTF-8 BOM `EF BB BF`; `0xbf` at offset 2 is the third BOM byte. Note it leaves the server it started behind, and the process exits with a traceback rather than a message.

`setup.py:4649` reads the same file with `encoding="utf-8"`:

```python
m = re.search(r"project\(strata VERSION ([\d.]+)", (ROOT / "CMakeLists.txt").read_text(encoding="utf-8"))
```

so this looks like a `strata_bench.py` omission rather than an intentional locale dependency. Other `read_text()` calls in the file that read repository files are likely to hit the same thing on those systems.

**Workaround** (no source change needed): run with UTF-8 mode on.

```
set PYTHONUTF8=1
BENCH.bat
```

Verified: the suite then runs to completion (420 s) and writes a normal report.

**Suggested fix:** pass `encoding="utf-8"` at that call (and audit the other `read_text()` calls in the file), or read with `errors="replace"` — the value only needs to match `project(strata VERSION x.y.z)`.

Environment: Windows 11 Pro 10.0.26200, Python 3.10.11, Strata 0.1.44.1 (release engine). A benchmark report produced on this machine is in #2791, which mentions this bug.

Mehr auf der Site

Links zu Install, Modellen, Releases.