Pull requests / #2879

#2879 tools: read CMakeLists.txt as UTF-8, not through the OS code page (#2792)

open · @ctr-137 · 0 comentários · No GitHub

BenchmarksSetup & installWindowsLinux

Descrição

`tools/strata_bench.py:564` read this install's own `CMakeLists.txt` with no `encoding=`, so the decode went
through `locale.getpreferredencoding()`.  That is GBK or cp932 on a Chinese or Japanese Windows, where the
file's UTF-8 BOM is not a valid sequence - the third byte `0xbf` - and `BENCH.bat` died before any
measurement:

```
  File "tools/strata_bench.py", line 564, in project_version
    m = re.search(r"project\(strata VERSION ([\d.]+)", (ROOT / "CMakeLists.txt").read_text())
UnicodeDecodeError: 'gbk' codec can't decode byte 0xbf in position 2: illegal multibyte sequence
```

`setup.py` reads the same file with `encoding="utf-8"`, so this is an omission here rather than an
intentional locale dependency.

A cp1252 machine decodes the BOM to mojibake and says nothing, which is why only those two locales ever saw
it - and why this needs a test rather than a reproducer.

### What I changed

`tools/strata_bench.py`: read it as `utf-8-sig`.  That matches how this file already reads its other repo and
config files (lines 657, 688, 971), and it strips the BOM the file actually has rather than leaving a
`` in front of the string the regex scans.

### What I left alone

- `/proc/meminfo` and `/proc/cpuinfo` are still read without `encoding=`.  They are the Linux-only branch
  and their content is ASCII whatever the locale, so I did not touch them.
- `project_version()` still catches only `OSError`.  A repo file that is unreadable for any other reason
  still reaches the top of `main()` as a traceback and leaves a started server behind, which the report also
  notes.  `UnicodeDecodeError` is a `ValueError`, so widening that except would cover it - but what the
  bench should do with an unreadable install is a separate decision and I did not want to bundle it here.

### How I checked

Three tests in `tools/test_strata_bench.py`, no GPU or server needed:

- the encoding `project_version()` passes is captured and has to be a UTF-8 one, so a future call that
  silently goes back to the code page fails;
- the file really is undecodable as GBK, which pins why the BOM is the thing to survive and documents the
  reported failure;
- the version it returns is the same as a strict `utf-8-sig` read of the file.

With the fix reverted the first of them fails (`encoding=''`), with it all 20 tests in
`tools/test_strata_bench.py` pass (`python -m unittest tools.test_strata_bench`).

The reporter's workaround (`set PYTHONUTF8=1`) still works and is no longer needed.

Fixes #2792.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

No site

Links install, modelos, releases.