Strata 0.1.44 Mais recente

0.1.44: Unsloth's 4- to 8-bit files and NVFP4 packs in setup, Qwen3.6-35B-A3B, faster multi-GPU on AMD and Arc A-series cards, and a long list of fixes. Answers on the default models are the same as 0.1.43.1 (byte-identical in our identity tests on every machine we tested), with the exceptions listed under "Defaults that changed" (RDNA4, and the Unsloth files setup configures). Update with UPDATE.bat (Linux: ./update.sh); setup replaces the engine with 0.1.44.

New

• Unsloth's UD-Q4_K_XL, UD-Q5_K_XL, UD-Q6_K_XL and Q8_0 are regular choices in setup, next to UD-IQ4_XS (docs/QUANTS.md). UD-Q4_K_XL is no longer experimental or NVIDIA-only; the three larger files and Q8_0 are new, each pinned by size and SHA-256. Where the experts fit in RAM with 16 GiB to spare, setup keeps every expert in RAM (UD-Q4 on a 64 GB PC with the old budget rule against this rule, 5 interleaved pairs: decode x1.34-1.40, a 4K prompt x2.07). Images are on by default for these files.
• Quality against Q8_0 (4,995 teacher-forced positions): UD-Q6_K_XL 94.6% same top token (mean KL 0.033), UD-Q5_K_XL 94.6% (0.038), UD-Q4_K_XL 93.1% (0.056), UD-IQ4_XS 88.9% (0.101), IQ3_S 87.7% (0.144), IQ3_XXS 84.5% (0.202), Q2_0 80.6% (0.315). tools/quant_quality.py reruns it on your own files.
• Speed (one R9700, every expert in RAM): UD-Q4_K_XL 63 tok/s on a story and 85-88 on code; UD-Q5_K_XL 57-60 and 81-83; UD-Q6_K_XL 54 and 76; Q8_0 53-57 and 66-74. llama.cpp on the same box: 16-20 tok/s decode.
• The K-quant prompt kernels are built by default; on NVIDIA they are a library of their own (strata_kq, shipped in the Windows zips), opened only when a model has these formats, so no other model pays for them. They are opt-in at run time (STRATA_MMQ_KQ=1): prompts x1.3-1.7 on AMD with every expert in RAM, with a prompt-path KL change we have not yet judged on enough text to make it the default.
• Dense Q8_0 matrices read one row per block where that measured faster (RDNA4 and sm_86), the same bits as before: decode +2.8 to +3.8% on UD-Q4 (5/5 pairs).
• NVFP4 packs (docs/NVFP4.md; sergiywith's port of sergqwer's strata-nvfp4 work). The original model with NVFP4 experts, as ready-made packs (GPTQ by default, NVIDIA's own checkpoint second), pinned and SHA-256-checked, with the original's image encoder. RTX 50 cards have FP4 tensor cores; older RTX cards run the experts at 8-bit precision and UD-IQ4_XS is faster there, which is what setup's menu now says. Quality against Q8_0: 92.1% same top token (KL 0.070), between UD-IQ4_XS (88.9%) and UD-Q4_K_XL (93.1%). NVIDIA only; AMD and Intel setups do not list it. The route tail skip stays off for NVFP4 experts (it costs more quality there); STRATA_ROUTE_TAIL_SKIP=7 turns it on.
• Qwen3.6-35B-A3B (UD-Q4_K_XL, UD-Q6_K_XL, Q8_0), with the file's own MTP block for speculative decode. One strata binary now runs both families (the file's architecture decides). Setup offers it as the last family in the model menu and sizes it by RAM (32 / 44 / 52 GB). On an R9700 UD-Q4_K_XL: 3,466 tok/s on a 4K prompt and 83 tok/s decode (llama.cpp on the same card: 3,054 and 82), about 161 tok/s with --mtp 2; exact against the plain decode on our 10 parity prompts. On an RTX 3060 with 46 GB of RAM (hybrid mode, experts in RAM): 4K prompt 1.47x llama.cpp's best split. NVIDIA and AMD only; the Intel (SYCL) engine and the Mac engine refuse it with a clear message.
• Image generation is in preview (POST /v1/images/generations, off unless a config asks for it): setup does not offer it yet.
• A 780M iGPU joins the gfx11 WMMA kernels (decode x1.53 on a Radeon 780M, same answers), and the Vulkan image decoder works in tiles (a 1024x1024 image's VAE: 23 s instead of 45 s).

Faster

• AMD multi-GPU (RDNA4): the draft layer sits on the first card in the pipelined decode, the pipeline is the default from two stages, prompts read in 8192-token chunks across a split, and the PCIe share is measured in the pipelined loop as well. R9700 decode +2.5 to +4% from STRATA_HC_Q8 on gfx12 (see "Defaults that changed").
• Intel Arc A-series (Alchemist), by aslater3: the fused hyper-connection read without the private copy plus large-GRF mode (A750 decode 11.8 -> 14.5 tok/s, 5 of 5 pairs), more and smaller work-groups (13.3 -> 15.9 tok/s on top of it, 5 of 5), the prompt GDN recurrence with one barrier per token (a 4K prompt +4.6%; the kernel is 6x faster), and an expert cache sized from the driver's fdinfo when xe reports no free memory (Arc A770, #1545). All four are bit-identical to 0.1.43.1 on the A750 and on a B70, where they are off or unchanged. Unsloth's K-quant and Q8_0 expert formats decode each weight word once per call on SYCL, bitwise the same.
• Commit graph in one launch per layer kind (STRATA_COMMIT_BATCH, on by default, =0 keeps the per-layer loop) and the draft stream at high priority on gfx1200/gfx1201 (STRATA_MTP_HIP_PRIO=0 turns it off): the engine-wide latency per decode window, measured and bitwise the same.
• Two opt-ins for the expert arena, by RubenTorney (#1904, #1905); both are off by default and the default path is byte-identical to 0.1.43.1.
• --adapt-async 1 now also works in the arena mode (one GPU): the adaptive tier's swaps run on a job thread instead of stopping the decode. On an RTX 5070 under Windows (Q2_0, arena mode, 5 interleaved pairs, three batches) decode is +4.2 to +5.2% (13 of 15 pairs faster), all of it on code and document prompts (+7 to +10%), the story prompt level; on an R9700 with 20K cache slots it is level except on a cold prompt (+12.9%). Answers with the tier on differ slightly from the blocking tier (deterministic run to run, not equal to it), which is why it stays opt-in. It also fixes a real race for the RAM-resident async tier: a K/V grow could lend a slot a swap was still writing.
• STRATA_PIN_BY_NEED=1 (a Windows / WDDM feature, CUDA and HIP) registers the experts the GPU will not hold as pinned memory instead of a prefix of the arena, in the order of the expert profile. We added a second tier for the experts the adaptive tier has swapped out: on the 5070 the prompt read is +4.4% (5 of 5 pairs), +8.5% with a pin budget of half the arena (STRATA_ARENA_PIN_GIB=16); decode is within +-1%. STRATA_PCIE_MIN1=1 (send one miss over PCIe as soon as a layer has two, where the share otherwise rounds to 0) did not help on this card and is a separate switch.
• A server start that dies inside the GPU runtime's own set-up (a ROCm hsa_exception one start in twenty after a quick restart) is retried 2, 4 and 8 seconds later (STRATA_START_RETRIES).

Fixes

• A picture in a long conversation no longer loses the cached state (#2228). The server sends each picture's byte hash and the engine matches pictures by it (plus the grid), not by their embedding floats. If a conversation still re-reads its history, the engine now says why.
• Intel A770 under xe: decode never started on the first window (#2194). Reproduced on an A750 with STRATA_WINDOW_SYNC=0; the drain of the other queues is now the default on every Alchemist card (Battlemage is unchanged).
• Two-card Intel and NVIDIA splits: the auto split takes the placement with the most spare VRAM on the fullest card among near-ties (#1440); a refused SYCL start exits cleanly and setup writes NO_HOST=0 for a split whose experts exceed 80% of the cards' VRAM (#1980); the start-up VRAM check covers every card of a split (#2192); a card behind a slow link is penalised only against the fastest link and from a 4x gap on (#2007).
• The prompt path gives way to a short request at the next chunk (SuperDistance, #2170).
• A literal `<tool_call>`, `<tool_response>` or `</tool_response>` in a message is text (47Hunter47, #2026).
• **Foresight exchanges keep the experts they move (#2134); the gfx12 16384 prompt chunk stays at 8192 when RAM is short (#2090); setup --update with an MTP folder sh