#3074 Fix crash (0xC0000005) when Qwen3.6's Q2_0 experts are offloaded to the CPU pool

Descrição

# Overview

The 2-bit GSQ quant of Qwen3.6-35B-A3B (`chitsgit/Qwen3.6-35B-A3B-2Bit-GSQ-hybrid-GGUF`) runs great as long as every routed expert fits into VRAM. However, whenever I tried running it in hybrid mode—offloading some of the experts to the CPU pool—it crashed immediately during warm-up with an access violation (`0xC0000005`) across all worker threads.

### Whats wrong

I traced the root cause to the CPU pool activations: native 2-bit (Q2_0) CPU kernels don't read standard activations directly. Instead, they expect activations pre-quantized into a specialized layout (`ActQ`) that stores the quantized data alongside scaling factors.

The `q36` engine set up the standard activation buffer for hybrid layers, but left the `ActQ` pointer uninitialized (`nullptr`). When worker threads picked up a Q2_0 expert job, they tried to read scale factors from this null pointer and crashed immediately. The Flash-Next engine already handles this case properly, but `q36` never got the equivalent logic.

### How I fixed it

I mirrored the approach used in Flash-Next: before dispatching expert jobs to the pool, I check the layer's quantization format. 
- If the layer uses native Q2_0 kernels, I quantize the hidden states into an `ActQ` buffer on the fly and hand that pointer to the worker job.
- For all other formats, I keep the standard activation path without adding extra overhead.

With this in place, hybrid mode runs completely stable. On my RX 7800 XT with 155 of 256 experts per layer in VRAM (and the rest in system RAM), decode speed sits at **41.8 tok/s**—on par with UD-Q4_K_XL in the same scenario, where it previously crashed on startup.

---

### What Changed

All edits `src/qwen35/q36_engine.cu` inside the CPU pool logic:

1. **Included headers** for the `ActQ` structures and activation quantization helpers.
2. **Added an `ActQ` buffer:** I added a vector of `ActQ` structures to the engine class and resized it alongside the existing activation allocations during expert placement.
3. **Branched by layer quantization type:**
   - I check if the current layer uses native Q2 kernels.
   - If yes: I quantize activations into the `ActQ` buffer and give the worker job the `ActQ` pointer. If no: I pass the standard activation buffer pointer as before.

I didn't need to touch the pool scheduler or compute kernels: the down-projection pass already quantizes its own intermediate buffers between phases, and Qwen3.6’s layer dimensions (embedding dimension 2048, expert intermediate size 512) divide cleanly into the kernel's 64-element block size.

---

### Benchmarks

**My test setup:** AMD Radeon RX 7800 XT (16 GB), RAM 32GB, Ryzen 7 7700, Strata 0.1.45 + this patch.

I benchmarked three configurations:
1. **UD-Q4_K_XL (Hybrid, 32K context):** At 20.2 GiB, the model exceeds 16 GB VRAM, so it always runs in hybrid mode.
2. **GSQ 2-bit (Full GPU, 32K context):** At 13.2 GiB, the model and KV cache fit entirely in VRAM with no CPU offloading.
3. **GSQ 2-bit (Hybrid, 64K context):** Doubling the context pushes the KV cache over budget, forcing 101 of 256 experts per layer into system RAM (8.44 GiB in the CPU pool). **This is the scenario that used to crash.**

| Scenario | UD-Q4_K_XL<br>*(Hybrid, 32K ctx)* | GSQ 2-bit<br>*(Hybrid, 64K ctx)* |
| :--- | :---: | :---: |
| **Chat decode**, tok/s *(median)* | 38.8 | **41.8** |
| Chat TTFT, s | 0.31 | **0.18** |
| **4K prompt**, tok/s | 256 | **520** |
| 4K TTFT, s | 15.7 | **7.7** |
| **16K prompt**, tok/s | 242 | **453** |
| 16K TTFT, s | 66.3 | **35.5** |
| **Agent turn decode**, tok/s | **39.9** | 39.0 |
| Agent turn TTFT, s | 38.8 | **20.7** |

### Key Takeaways:

1. **Prompt processing is ~2× faster on 2-bit GSQ.** Prefill throughput jumps from 256 to **520 tok/s** on 4K prompts and from 242 to **453 tok/s** on 16K prompts. This cuts time-to-first-token in half across both short and long contexts (**7.7s** vs 15.7s at 4K; **35.5s** vs 66.3s at 16K).
2. **Hybrid decode remains faster even with double the context.** Despite holding a 64K context window (which forces 101 of 256 experts per layer into system RAM), 2-bit GSQ decodes at **41.8 tok/s** versus 38.8 tok/s on Q4 hybrid at only 32K context, all while using a much smaller CPU pool memory footprint (8.44 GiB).
3. **Decode throughput is bound by RAM bandwidth.** In both hybrid setups, generation speed is limited by how fast offloaded expert weights are transferred across the system bus per token. With the patch eliminating the previous crash and startup stalls, the Q2_0 hybrid path demonstrates smooth, expected scaling.