改动 / #3320
#3320 [Benchmark] bind the port so strata_bench.py can start its own server
open · @victorgabr · 0 评论 · 去 GitHub 看
BenchmarksSetup & installAMD / HIPNVIDIA / CUDAModels & quantsWindowsLinux
说明
Closes #3188. Also covers the crash reported in #2864, #2872, #2908 and #3130.
## Problem and resulting behavior
`tools/strata_bench.py` dies with `NameError: name 'port' is not defined` on every run that has to **start the server itself** - the default path, and the one `BENCH.bat` and the setup "Run a quick benchmark now?" prompt use:
```
File "tools/strata_bench.py", line 990, in main
say(f"Starting {cfg_path.name} on port {port} (loading takes 1-3 minutes; the PC may be slow meanwhile) ...")
NameError: name 'port' is not defined
```
After this change the tool starts the server on the config's port (or `--port`), waits for the model, runs the suite and writes the results folder; the "port is used by something that is not Strata" message also prints the real port instead of crashing.
## Why this approach
Commit 69fe4055 (#2330) replaced `port = a.port or int(cfg.get("port", 8080))` + `base = f"http://127.0.0.1:{port}"` with `base = server_base(cfg.get("host"), a.port or int(cfg.get("port", 8080)))` and left the three remaining uses of `port` (lines 987, 990, 991) unbound. The fix restores the assignment the line used to carry and uses it in all three places - the same shape suggested in #3188. The `--url` path is untouched.
The suite never caught it because every `main()` test passes `--url`, so the start-server branch is never executed; that gap is called out in #3188. One new test (`test_the_start_server_branch_knows_its_port`) reaches the branch with a stubbed `start_server`: it asserts the config's port reaches the starter and the not-ready path returns 1, and that a non-Strata port occupant returns 2 with the port named in the message.
## Validation
On `upstream/main` `e88ee39d`, Linux, Python 3.14.7, branch exactly as submitted:
**Passed**
- `python tools/test_strata_bench.py` with the fix: `Ran 18 tests ... OK` (17 existing + the new one).
- The new test fails **without** the fix (fix reverted, test kept): `FAILED (errors=1)`, with the real traceback `NameError: name 'port' is not defined`, `line 990, in main`.
- End-to-end with the fix, no CI, on hardware: `python tools/strata_bench.py --config strata-iq3_s.json` on Linux (TITAN RTX, engine 0.1.45.3) started its own server, ran the suite and wrote a full results folder - the path the change unblocks. Two paired runs this way produced the A/B evidence posted alongside this bug report's thread dates.
**Failed**
- none.
**Not run**
- Windows path (`BENCH.bat`) - the changed code is platform-independent Python and reproduces identically on Windows per #3188 and #3130, but I ran it only on Linux.
- No CUDA/HIP/SYCL builds needed: no compiled file is touched.
## Limits
The new test stubs `start_server`, so it proves the branch binds and passes the port, not that a real engine boots. The port-free check still relies on `/health` probing (unchanged behavior).
本站相关内容
相关页面的快捷入口。