Issues / #2864
#2864 tools/strata_bench.py: NameError: name 'port' is not defined — bench cannot start a server (regression from 69fe4055 / #2330)
open · @yusheng227507 · 0 コメント · GitHub で見る
BenchmarksServer & APIMulti-GPUNVIDIA / CUDAModels & quantsLinux
本文
## Summary
`tools/strata_bench.py` is broken in **v0.1.45 (477ac13c)**. Any run that starts its own server (i.e. `--config` without `--url`) dies immediately with `NameError: name 'port' is not defined`. Passing `--url` (pointing at an already-running server) still works, so the failure is confined to the "start the server" path.
## Environment
- Strata: `v0.1.45` / commit `477ac13c`
- OS: Ubuntu (Linux), Python from the project `.venv`
- Hardware: 2x RTX 2080 Ti (sm_75)
- Config: the standard `strata-*.json` with `"port": 8080`, `"host": "0.0.0.0"`
- No server running on the port at the time (so the "start our own server" branch is taken).
## Steps to reproduce
1. Make sure nothing is listening on the config's port (e.g. `8080`).
2. Run the bench without `--url`:
```
.venv/bin/python tools/strata_bench.py --config strata-iq3_s.json --out bench/results/x
```
## Actual result
It fails before loading the model (and therefore before any measurement):
```
Traceback (most recent call last):
File "/home/yusheng/strata/tools/strata_bench.py", line 1037, in <module>
sys.exit(main())
^^^^^^
File "/home/yusheng/strata/tools/strata_bench.py", line 990, in main
say(f"Starting {cfg_path.name} on port {port} (loading takes 1-3 minutes; the PC may be slow meanwhile) ...")
^^^^
NameError: name 'port' is not defined
```
Exit code 1.
## Expected result
The bench starts the server on the configured port, waits for readiness, and runs the suite (as it did in v0.1.44.1 and earlier).
## Root cause
Commit **`69fe4055`** ("tools: poll the server address the config binds, not loopback (#2330)") replaced
```python
port = a.port or int(cfg.get("port", 8080))
base = f"http://127.0.0.1:{port}"
```
with
```python
base = server_base(cfg.get("host"), a.port or int(cfg.get("port", 8080)))
```
That removed the only assignment of the local name `port`, but three later uses in the same `else` branch still reference it (the "port is used by something that is not Strata" message, the "Starting ... on port {port}" message, and `start_server(cfg_path, port, ...)`). The `--url` path skips the branch, so it never trips.
The new test added by that commit (`tools/test_bench_host.py`) only exercises `server_base()` in isolation, so it does not catch the missing local.
## Suggested fix
Restore the local name (keeping the new `server_base` host handling):
```python
else:
port = a.port or int(cfg.get("port", 8080))
base = server_base(cfg.get("host"), port)
```
## Workaround (no source change)
Start the server yourself and point the bench at it, which bypasses the broken branch:
```
.venv/bin/python serve/server.py --engine strata --config <cfg>.json --port 8080 &
.venv/bin/python tools/strata_bench.py --url http://127.0.0.1:8080 --out bench/results/x
```関連リンク
インストール・モデル・リリースへの站内リンク。