Pull requests / #2041

#2041 setup: agentionai's Signal 3.8 Flash Next as --family signal (AP-Q4_K_XL, AP-IQ4_XS), images with its own encoder

open · @clothespin · 0 commentaires · Sur GitHub

BenchmarksSetup & installServer & APIAMD / HIPNVIDIA / CUDAModels & quantsDocumentationWindowsLinux

Description

## Title
Issue: none. No open issue or pull request covers Signal (searched "Signal", "agentionai" and "AP-Q4_K_XL"). This
builds on #290 (`--embd-gguf`).

## Summary
Adds agentionai's [Signal 3.8 Flash Next](https://huggingface.co/agentionai/Signal-3.8-Flash-Next-GGUF) to setup as
`--family signal`, with its two ~4-bit sizes, AP-Q4_K_XL (101 GB) and AP-IQ4_XS (90 GB), and images through Signal's
own encoder. The engine runs the file unchanged. What was missing: a pack built with `--compat-bf16` (its dense weights
are Q6_K), a BF16 copy of its Q6_K token embedding (the stock engine has no GPU dequantizer for it), and the installer
plumbing. No engine source changes, no default changes, and the other families' configs are byte-identical
(`tools/test_setup_golden` passes).

## What changed
Three commits, one feature. The first two stand on their own if you would rather take them separately.

1. `tools/embd_bf16_pack.py`: `--gguf PATH` (and `--tensor`) dequantizes a GGUF's token embedding to BF16 for
   `--embd-gguf`, rounding to nearest even as `--compat-bf16` does. On Signal AP-Q4_K_XL: 248,320 x 2,560 Q6_K, a
   1,271,398,784-byte file, worst rounding error 0.00036. `--model` (from the checkpoint) writes the same header byte for
   byte as before.
2. `docs/SIGNAL_AP_Q4_K_XL.md` (new) and a pointer in `docs/MODELS.md`: the manual workflow, in the style of
   `docs/ORCA.md`, with a config calibrated on one machine and what was and was not checked.
3. `setup.py`: the `signal` family and the AP-Q4_K_XL and AP-IQ4_XS models, pinned at revision `eba565be` with each
   file's size and SHA-256.
   - setup writes `<pack>/token-embd-bf16.gguf` and adds `--embd-gguf` to the config, for this family only;
   - with images on, it downloads `mmproj-F16.gguf` from the repository and saves it as
     `mmproj-Signal-3.8-Flash-Next-F16.gguf`;
   - `budget_model()` and `--gguf-dir`'s name check now handle a one-file name with the `AP-` prefix (before, they
     took Signal for UD-Q4_K_XL, or for a GGUF Strata cannot run);
   - AP-Q4_K_XL is NVIDIA only, and setup asks on AMD, for the same reason as UD-Q4_K_XL: no HIP prompt kernels for
     Q4_K experts.

   Also: `tools/strata_mcp.py` (its fallback model table), `tools/test_setup_signal.py` (10 tests: the pins, the
   budget, the file and config names, the pack and embedding commands and `--embd-gguf` in a mocked install, the
   encoder's download name), and docs: English README, `MODELS.md`, `AI_SETUP.md`.

Left alone: the engine, every default, the other families' configs, the translated READMEs.

### Measured
Linux, RTX 4090 (24 GB), Ryzen 9 7950X3D, 93 GB RAM, engine 0.1.43 at `7695859a`, 2026-10-11, with the file already
downloaded:
`./setup.sh --family signal --model AP-Q4_K_XL --gguf-dir <folder> --vision gpu --build --yes --no-start`

- It took 5 minutes, including the SHA-256 check of the 101 GB file and compiling `strata-vision`. The pack, tokenizer
  and embedding file it wrote are byte for byte the ones built by hand in the doc.
- The config it wrote (32K context, 8-bit KV, 63 GiB RAM budget, images on) was ready in 86 s. `tools/strata_bench.py`,
  3 runs per test, thinking off, temperature 0:

| test | median | range |
|---|---|---|
| short chat, decode | 59.9 tokens/s | 48.1-70.8 |
| short chat, first token | 0.38 s | 0.33-0.45 |
| 4K prompt | 1400 tokens/s | 960-1402 |
| 16K prompt | 2301 tokens/s | 1834-2317 |
| agent session, decode | 71.2 tokens/s | 44.4-89.8 |
| agent session, prompt reused from the cache | 68.7% | 68.6-68.8% |

- Images: a drawn test picture got "7341 red circle blue square lighthouse" (all four right);
  `docs/media/pagoda-preview.webp` got an accurate two-sentence description.
- The text checks from `docs/ORCA.md`: arithmetic (391), a Python function, a translation, and a cancelled stream
  followed by a correct answer (19). One answer was wrong: asked for the second-largest number in a list, it said 100
  (it is 99).

### Not tested
- AP-IQ4_XS was not downloaded or run. I read only its header: it differs from AP-Q4_K_XL only in the experts'
  gate/up type, IQ3_S, the same pair UD-IQ4_XS uses.
- Setup's own download of the files (the run above used `--gguf-dir`).
- Windows, AMD cards, several GPUs, and the image encoder on the CPU.
- Output quality against the base model, and the draft head's acceptance rate on this fine-tune.
- `embd_bf16_pack.py --gguf` has no unit test of its own; it is covered by the run above. `--model` was not run end to
  end (no safetensors checkpoint on the test machine).

## Extra Notes
- Neither size is marked experimental. Their experts (67.9 and 57.3 GB) fall between UD-IQ4_XS (regular) and
  UD-Q4_K_XL (experimental). Happy to flag either one if you prefer.
- Tests: all `tools/test_setup_*.py` and `tools/test_strata_mcp.py` pass, except the two tests in
  `tools/test_setup_engine_hash.py`, which fail the same way on `main`.

Sur le site

Liens install, modèles, releases.