Strata 0.2.12

# Strata 0.2.12: a tuned DFlash2 drafter, faster prompts on 12 GB cards, Strix Halo and Mac, and tool-call fixes

The app and the Terminal are 0.2.12. Control and every engine package are 0.1.58. The update feed is sequence 122. The DGX Spark (linux-aarch64) gets 0.2.12 too.

Highlights

• 12 GB cards (RTX 3060 class): UD-IQ4_XS-v31 prompts read about 3 times faster, 87 to 264 tokens per second at 2K.
• Strix Halo (Windows): prompts x1.24 to x2.14 on UD-IQ4_XS-v31 and UD-Q4_K_XL.
• Mac (M5 Max): output +10.3% to +12.7% and prompts +11.7% to +13.9% on UD-Q4_K_XL and UD-IQ4_XS-v31.
• Tuned DFlash2 drafter: Strata's own copy of the helper model that guesses the next words; decode +2.4% to +5.0% on the RTX PRO 5000 and the DGX Spark.
• Radeon R9700: 32K prompts +8.6% to +11.6%.
• Tool calls: Claude Desktop's model name is accepted (#5312), plus fixes for #5409, #5276, #5281, #4622 and PRs #4828 and #4832.

Speed against 0.2.11

Tokens per second, 0.2.11 to 0.2.12. "Output" is the geometric mean over the kinds we tried where we have one; the worst kind is shown when it is a loss.

| Machine, model | Prompt | Output | |---|---|---| | Mac M5 Max, UD-Q4_K_XL | 4K 1,406 to 1,605; 32K 1,350 to 1,524 | +12.7% (every kind up) | | Mac M5 Max, UD-IQ4_XS-v31 | 4K 1,440 to 1,649; 32K 1,373 to 1,543 | +10.3% (agent -0.9%) | | Strix Halo, UD-IQ4_XS-v31 | 4K 690 to 1,220; 32K 1,107 to 1,377 | code +10%, chat +3%; tool call -9.5% | | Strix Halo, UD-Q4_K_XL | 4K 597 to 1,267; 32K 1,034 to 1,370 | code +24%, chat +6%; tool call -5.8%, agent -2.5% | | RTX 3060 12 GB, UD-IQ4_XS-v31 | 2K 87 to 264; 32K 361 to 981 | code +37%, agent +34%; JSON -18.9% | | RTX PRO 5000, IQ3_S | 8K level (x1.002) | +2.4%; agent -1.5%, chat -1.0% | | DGX Spark, IQ3_S | see Known problems | +3.1%; chat -3.3% | | DGX Spark, UD-IQ4_XS-v31 | see Known problems | +5.0%; chat -0.4% | | Radeon R9700, IQ3_S (Auto now MTP) | 32K 3,076 to 3,304 | code -10.5%, tool call -9.3%, JSON -2.3% | | Radeon R9700, UD-IQ4_XS-v31 (Auto now MTP) | 32K 2,817 to 3,015 | code -8.5%, JSON -4.8%, tool call -3.2% | | 2 x RTX A6000, IQ3_S | level | split engine -0.40%, layer split +0.38% |

How it was measured: each machine ran 0.2.11 and the 0.2.12 candidate side by side, 2 to 3 rounds, a fresh engine start per run, the arms swapped in order each round, greedy answers of 300 tokens. Each version used the settings it writes for that machine, so the 0.2.12 column includes the new drafter choices. Speeds were not measured again on the final 0.1.58 packages; the engine source after the check changed only in the version number and two build fixes for old NVIDIA cards.

What changed for you

• Tuned DFlash2 drafter (0.34 GB, 0.52 GB on Intel Arc). Auto now uses it on: Mac (UD-Q4_K_XL, UD-IQ4_XS-v31; DFlash2 is now the Mac default for these, confidence 0.7); NVIDIA cards with every expert on the card and the DGX Spark (IQ3_S, UD-IQ4_XS-v31); Strix Halo (IQ3_S, UD-Q4_K_XL, UD-IQ4_XS, UD-IQ4_XS-v31); Intel Arc A-series (IQ3_XXS); 12 GB NVIDIA cards (UD-IQ4_XS-v31). Back: strata launch --drafting mtp.
• Radeon gfx1201 (R9700, RX 9070): Auto starts MTP, not DFlash2, because 11 of 50 DFlash2 engines on 0.2.11 stopped on the prompt. Line shown: "DFlash2 is off by default on this graphics card for now; choose it in Drafting to use it." Back: --drafting dflash2.
• 12 GB cards (Linux): the second start-up check no longer counts the engine's own memory as free, so it stops switching to the slower file cache. When it still switches it says: strata: expert reads now go through the file cache (X GiB of RAM free for them, Y GiB limit). On the RTX 3060, 0.2.11 switched in 3 of 3 starts and 0.2.12 in 0 of 3. Auto now starts MTP for IQ3_S and the tuned DFlash2 for UD-IQ4_XS-v31 on these cards. Back: STRATA_FILE_TIER_HONEST=0.
• A download with the wrong checksum is kept as `<name>.bad`, not deleted: <name> does not match the pinned SHA-256 (it is <got>, Strata expects <want>); the file was kept as <name>.bad. Setup checks it again on the next run.
• Strix Halo (Windows): the fused prompt kernels for UD-Q4_K_XL and the Q5_0 layers of UD-IQ4_XS-v31, and the grouped decode kernels, are on. Back: STRATA_GFX1151_DEFAULTS=0.
• Mac: the prompt path overlaps host work with the GPU (same bits). Back: STRATA_METAL_PROMPT_HOSTPIPE=0, STRATA_PROMPT_MERGEH=0, STRATA_PROMPT_V6=0. Mac line: "On your Mac, DFlash2 was faster than MTP on this model in our tests."
• Radeon gfx12, prompts of 30,000 tokens or more use the chunked GDN scan (32K +6.1%, 64K +6.7% on an R9700): strata: RDNA4 (gfx12): prompt chunks of 30,000+ tokens use the chunked GDN scan (STRATA_GDN_CHUNKED=0 turns it off).

Fixed

• Tool calls:
• #5312: Claude Desktop's claude-5-5-sonnet on /v1/messages now uses the one loaded model.
• #5409: <function=tools NAME> and a bare <function=tools> are read as the declared tool.
• #5276: <toolcall> and <tool-call> openers are read when a declared tool follows.
• #5281: an MCP tool written by its last segment is matched when exactly one tool ends that way.
• #4828 (gdevenyi): forced tool_choice with several tools; 24 of 24 runs now, 0 of 24 before.
• #4832 (gdevenyi): Anthropic tool results follow tool_use_id.
• #4622: new setting undeclared_tool_calls (text by default, call is opt-in).
• Engine fixes from pull requests:
• Copy ordering and lost adaptive swaps (#5096, #4844).
• Row-table over-read that could fault long CUDA prompts (#5108).
• A first-request hang with --pipeline-windows 2 on two AMD cards (#5149); not run on hardware.
• Speed and time figures in the server reports (#5318, #5307, #4006); a short request admitted into a slot now reads 125, not 215, tokens per second because admission time is counted.
• Hyper-connection rows copied per stage: 414 MiB less per card on two A6000 (#3559).
• A window of slots kept in every expert pool, so 12, 20 and 30 GiB RAM splits start (#5158).
• File tier defaults stay off when every expert is on the cards (#4972).
• Mac install (#5390, cwf75): a fresh install of UD-Q4_K_XL or UD-IQ4_XS-v31 from the Mac app or Terminal stopped at "Preparing your model" ("the desktop app's installer only runs its bundled preparation tools"); the installer now runs the bundled Mac tools in metal/tools. On an M5 Max a fresh Terminal install of UD-Q4_K_XL prepared the model, fetched the tuned drafter and answered plain, two-turn and tool chats.
• Builds: the CUDA 12 and 13 engines build again for old NVIDIA cards (sm_60 to 75); MSVC and the Spark link failures; the Docker image build.

Opt-in switches (all off unless set)

• STRATA_PF_FUSED_Q50=1 (Radeon gfx12): Q5_0 layers on a fused kernel; prompts 2K +44%, 8K +40% on an R9700.
• STRATA_IQ4_W4A8=1|2 (NVIDIA, cctan): int8 kernel for IQ4_XS and IQ4_NL experts, same output as MMQ.
• STRATA_SELECT_TWO_PASS=1 (NVIDIA, bitaccesscomau): exact two-pass selection for long prompts; 128K +1.65%.
• STRATA_ADAPT_FETCH=1|2 (CUDA, sergqwer): fetched misses go into the card's cache; decode +10.2% with =2 in one setup.
• STRATA_ADAPT_OVERLAP=1 with --peer-device (bulhenvaster): the peer's copies land while the next window runs.
• STRATA_PF_PEER_CPU=1 (bulhenvaster): the CPU helps beside a peer GPU; on two A6000, 12 follow-up turns took 14.28 s against 20.50 s.
• STRATA_VNNI_TIER=1 (Michel Pollet): 512-bit expert rows on CPUs without AVX-512-VBMI.
• undeclared_tool_calls: "call" (or STRATA_UNDECLARED_TOOL_CALLS=call): return a call of an undeclared tool as a tool call.

Known problems

• Peer mode (`--peer-device`, two cards) can fault on its first request, in about 4% of fresh starts (also in 0.2.11); the all-cards split engine is not affected. Cause not found.
• Radeon gfx1201: DFlash2 chosen by hand can stall on the prompt (11 of 50 engines on 0.2.11). The engine stops itself and the next