Issues / #2014
#2014 SYCL resident RAM mode: resident_stage_swaps stages the exchange without copying the evicted expert back
open · @kuhlsnu · 0 comentários · No GitHub
Descrição
**The SYCL port of `resident_stage_swaps` has no device-to-host copy where the CUDA version has one.** I found this reading the code while porting an exchange check; I have no Intel GPU, so nothing here ran.
CUDA, `src/program/generate.cpp:292-299` (7695859a):
```cpp
const SwapHome home = locate(s.layer);
const strata::core::OnDevice on(home.dev);
if (const cudaError_t e = cudaMemcpyAsync(src.exchange_buffer(q), home.cache->device_slot(slot),
(size_t) strata::kernels::cpu::expert_layout().blob_bytes(s.layer), cudaMemcpyDeviceToHost,
home.stream); e != cudaSuccess) {
```
SYCL, `sycl/src/program/generate.cpp` (`resident_stage_swaps` at :318), same spot:
```cpp
const SwapHome home = locate(s.layer);
const strata::core::OnDevice on(home.dev);
/*
DPCT1124: cudaMemcpyAsync is migrated to asynchronous memcpy API.
...
*/
bool seen = false;
```
The migration comment is there but the memcpy itself is not, and the swap is still staged (`staged.push_back`, then `stage_exchange`). If this path runs, the RAM copy would take whatever is already in exchange buffer `q` as the evicted expert, so a later request routed to it on the CPU reads wrong weights, with no error.
**What I could not check:**
- whether SYCL ever reaches this function (resident RAM mode with adaptive swaps on a SYCL build)
- whether anything else fills the exchange buffer first
If it is reachable, this could be related to the silent wrong answers in #1484, though that one points at the expert kernels. A quick check: run resident RAM mode with adaptive swaps on SYCL and compare a swapped expert's RAM bytes with the model file.
No site
Links install, modelos, releases.