#2091 sm_75: the hyper-connection's xn16 written as the FP16 its tensor-core GEMMs read (bitwise, prompt +3 %)

Descrição

The follow-up I mentioned in #655. With `STRATA_BF16_TC=1` the prompt path converts `xn16` (T × 10240) from BF16 to FP16 before each of its two GEMMs (the down and inject projections), in 16M-element slices: on a 130K prompt read that was ~2.8 s of `bf16_to_f16_kernel` (nsys), 3.2 % of the read.

Now the four `xn16` writers (`gr_norm`, `gr_norm_rs`, `gr_write_norm_rs`, `gr_write_cvec_norm_rs`) can store `gemm.cu`'s conversion of the BF16 value directly (`bf_to_hf_conv`: the same clamp of finite values beyond ±65504 and the same rounding as `bf16_to_f16_kernel`), and `Gemm::bf16(..., x_f16)` skips the conversion on the tensor-core path - the same slices, so the same GEMMs. Any other path (one output row, no room for the buffers) converts the image back to BF16 a slice at a time (exact: every value is a BF16 that fits FP16, or was clamped as the tensor-core path clamps it).

It is on only where it changes nothing: the stage runs BF16 on FP16 tensor cores (`Gemm::bf16_converts_x()`), without the BF16X2 low parts and with the unpadded `xn16` stride; the MTP draft's own read keeps BF16. `STRATA_XN_F16=0` turns it off.

## Setup

RTX 2080 Ti 22 GB (sm_75), i7-6700, CUDA 13.0, driver 595, SM 1650 MHz, 240 W; `v0.1.43`. Model Q2_0, `--resident-experts`, `--max-context 524288`, `--kv int8 --kv-resident 32768`, `STRATA_BF16_TC=1`.

## Results

| | `STRATA_XN_F16=0` | `=1` |
|---|---:|---:|
| 98K prompt read, run 1 | 1431.6 tok/s | 1473.4 |
| 98K prompt read, run 2 | 1430.4 | 1474.0 |

First logits after the 98K prompt (`STRATA_DUMP_FIRST_LOGITS`): bitwise identical. A 4K one-shot run writes the same tokens.

🤖 Generated with [Claude Code](https://claude.com/claude-code)