Pull requests / #3552

#3552 [Engine] skip file-tier experts while a helper GPU is attached

open · @sakuumomo · 0 Kommentare · Auf GitHub

AMD / HIPNVIDIA / CUDASecurityDocumentationLinux

Beschreibung

Problem and result: The file-tier tail skip turned off whenever a helper GPU was attached. A file-tier miss is skipped again while a helper is attached. A row the helper holds, or a row whose bytes are already in the host copy, is still computed. With no helper, a missed tail is skipped even when those bytes are already in RAM. A peer GPU still turns the skip off. The startup line and `docs/DETAILS.md` say the same thing.

Related discussion: none for this helper case. The skip itself is the existing `STRATA_ROUTE_TAIL_SKIP` default.

Approach: The dispatch no longer treats `remote_count > 0` as a reason to disable the skip. A helper still refuses the skip for a row it holds and for a row `host_copy()` reports as already in a host copy (a resident complement, or an arena that is not a demand-paged file mapping). Peer GPUs keep the old disable. The default rank is unchanged. The SYCL port compiles its own `generate.cpp` and `expert_source.cpp`, so this pull request does not change that backend.

Validation:
  Passed: HIP and CUDA builds of `4c1f0dda` on Linux x86-64, through Nix, with `STRATA_BUILD_TESTS=OFF` and `STRATA_PORTABLE=ON`.
    HIP: `STRATA_ENABLE_HIP=ON`, `STRATA_ENABLE_CUDA=OFF`, `CMAKE_HIP_ARCHITECTURES=gfx1030`, `STRATA_PREFILL_MMQ=ON`, HIP 7.2.3. Output `/nix/store/c6zp3p826fryl6x7cgp3lsw16z56him8-strata-hip-0.2.1` (`strata` and `strata-device`). The `strata` binary contains the `gfx1030` name.
    CUDA: `STRATA_ENABLE_CUDA=ON`, `STRATA_ENABLE_HIP=OFF`, `STRATA_HYBRID` left off, `CMAKE_CUDA_ARCHITECTURES=89-real`, `STRATA_PREFILL_MMQ=OFF`, CUDA 12.9.86, gcc 14.4.0. Output `/nix/store/yr652vys91l94sj7dvxigsz6a2ynjgpy-strata-cuda-0.2.1` (`strata` and `strata-device`).
    The author served on two Linux machines, each with a helper GPU, and reports that both function as intended. The desktop is an RX 6800 and an RX 6900 XT on the HIP build. The laptop is an RTX 4060 and a Radeon 890M on the CUDA and HIP hybrid build. No prompt log or timing was attached to that report.
  Failed: none observed.
  Not run: a SYCL build of this commit, because the port does not compile these files. A new measurement of the skip.

Limits: The startup line no longer prints the +10..20% figure while a helper is attached, because that figure was measured without one.

Mehr auf der Site

Links zu Install, Modellen, Releases.