Pull requests / #391

#391 iq_avx2: build the IQ2 sign table without a variable shift (BMI2 \shlx\ traps on pre-Haswell CPUs)

closed · @demetree · 0 comments · View on GitHub

Server & APINVIDIA / CUDA

Description

## The bug

`src/kernels/cpu/iq_avx2.cpp` built a static lookup table with a **variable** shift:

```cpp
r |= (uint64_t) (((ksigns_iq2xs[i] >> k) & 1) ? 0xFF : 0x01) << (8 * k);
```

MSVC compiling this file with `/arch:AVX2` emits **BMI2 `shlx`** for that shift — it assumes anything
with AVX2 also has BMI2, which is true from Haswell (2013) on and false of every CPU before it.

Because `even_signs` is a namespace-scope `static`, its constructor runs **before `main()`**. So this is
not a fault at first use, it is a fault at **process start**:

```
strata.exe --help
Exception code: 0xc000001d      (STATUS_ILLEGAL_INSTRUCTION)
(terminates with no output at all)
```

One `shlx` in the entire linked binary — that single instruction was enough to make the engine
unstartable on any pre-Haswell CPU built with MSVC. It is reached from `expert_layout_load()` →
`native_fmt()` → `iq_avx2.cpp`, so it is on the path for every native (IQ) pack.

## The fix

Store the byte instead of shifting it into place:

```cpp
uint8_t b[8];
for (int k = 0; k < 8; ++k) b[k] = ((ksigns_iq2xs[i] >> k) & 1) ? 0xFF : 0x01;
uint64_t r;
std::memcpy(&r, b, sizeof(r));
v[i] = r;
```

Identical 64-bit value: the OR of `0xFF << 8k` / `0x01 << 8k` contributes only byte *k*, so a direct
byte store is the same result on the little-endian targets this file is compiled for. It leaves no
variable shift for the compiler to widen.

## Verification

Built with MSVC 19.44 / CUDA 12.6, `CMAKE_CUDA_ARCHITECTURES=61-virtual`,
`STRATA_EXPERIMENTAL_SM60=ON`, for a **GTX 1080 Ti (sm_61, Pascal)**.

```
shlx / shrx / sarx / mulx / andn / bzhi / pext / pdep in strata.exe :  none
strata.exe --help                                                    :  exit 0
```

Before the fix the same binary listed `shlx x1` and exited `0xC000001D`. After it, no BMI2 remains and
the engine starts.

This is independent of any AVX1 work — it is a plain startup crash, and it reproduces on **pristine
0.1.31** with no other patch applied.

## Why it matters even though 0.1.31 refuses these CPUs

0.1.31 added the startup refusal for CPUs without AVX2 (`d4279d5`), and that refusal is *after* this
static initializer — so on MSVC the engine dies with an illegal instruction before it ever reaches the
message that would explain why. Anyone hitting this sees a crash rather than the intended diagnostic.

It also means the AVX2 baseline is real in a way the message does not convey: the file needs AVX2 for
its 256-bit kernel body, but the table build was silently requiring **BMI2**, which is a different
feature and arrived with the same generation only by coincidence.

Related on strata.com

Editorial links to help you install, pick models, or read release notes — not part of the upstream thread.