改动 / #2924
#2924 fix(sycl): pause GPU telemetry while the model is unloaded
open · draft · @pankleks · 0 评论 · 去 GitHub 看
BenchmarksSetup & installServer & APIAMD / HIPNVIDIA / CUDAModels & quantsLinux
说明
## Summary Fixes #2921 Stop the SYCL Monitor's once-per-second GPU sensor reads while the model is unloaded. The HTTP server stays up, CPU/RAM/disk/server metrics continue, and the next inference still loads the model normally. - `Telemetry` accepts an optional `gpu_active` predicate. Only SYCL supplies it (`Service.loaded`); CUDA/HIP/Metal and callers that omit it retain their existing sampling path. - Paused GPU fields are absent/unknown, not a stale value or a fabricated zero-watt reading. The existing Monitor renders unavailable values as a dash. - `_XeGpu.pause()` clears only its energy-delta baseline, so the first resumed power sample is not averaged over the unloaded interval. - The explicit pre-load/free-VRAM reader is unchanged. No model/config/API/engine/kernel/firmware/power-policy changes and no new dependency. ## Validation - Four focused regression tests cover service wiring (SYCL only), two-card pause/resume without hardware reads, CPU/RAM/server metrics while paused, backward compatibility with readers without `pause()`, and Intel energy-baseline reset. - Linux: `python -m unittest serve.test_server serve.test_monitor serve.test_prometheus tools.test_telemetry_idle` — **357 tests passed**. - Setup and existing AMD/Mac telemetry tests plus the focused tests — **30 passed**. - Live Intel Arc Pro B70 / xe, Strata 0.1.45, IQ3_XXS / 262,144 context: streaming text, arithmetic, tool-call JSON, synthetic tool result and final answer passed; explicit unload followed by a fresh inference successfully auto-reloaded the model and resumed GPU telemetry. Config hash and 600-second idle policy stayed unchanged. - Polling `/metrics` once per second for 120 seconds while unloaded: all 120 responses remained unloaded, GPU power/temperature absent, and CPU/RAM metrics available. Simultaneous passive PM counters recorded **+104,619 ms suspended / +15,384 ms active** over ~120 seconds, comparable to stopping the server in the original isolation test. An earlier equivalent run recorded +105,566 ms suspended / +14,437 ms active. The card still has transient wakeups (the final interval ended active/usage=0 and a later check was suspended); this is not a claim of uninterrupted suspend or that every wake source has been isolated. This is not a claim of a measured wattage reduction: no independent power meter was used, and no GPU sensor reads were made by the PM observation. No CUDA/HIP/Metal GPU hardware run, engine rebuild, long-context benchmark, or sustained-load test was performed for this Python-only change. Inference sources and binaries are unchanged. Draft pending maintainer cross-backend review/build policy; the live Intel fix and Python tests are complete. No review request has been sent.
本站相关内容
相关页面的快捷入口。