Strata 0.2.14

下载

版本说明

# Strata 0.2.14: the prompt runs on Strata's own kernels on NVIDIA and AMD

The app and the Terminal are 0.2.14. Control and every engine package are 0.1.60. The update feed is sequence 125. The Intel Arc engine stays at 0.1.59 in this release. The DGX Spark (linux-aarch64) engine stays at 0.1.59 too; its feed stays at sequence 123.

0.2.14.1 (11-15): drafting fixes in the Terminal

Control 0.1.60.1 for Windows, Linux and the Mac, with Terminal 0.2.14.1. To get it, run strata update in the Terminal, or press Update on the Strata row in the app's Settings > Updates. The engines and the app stay at 0.2.14. The update feed is sequence 126. The first two fixes below are in the Terminal; the app gets the third.

  • Mac: choosing drafting Off no longer downloads the 5.2 GB MTP layer. M5 Max, Q2_0, first launch with --drafting off: 13 s (0.2.14: 272 s, with the download).
  • Without internet, on a card where Auto picks the MTP drafter, the launch now says why it starts without a drafter: "There is no internet connection, so the draft layer can't be downloaded now. The model starts without a drafter." The next launch with internet downloads the draft layer.
  • "Preparing the draft layer" appears only while the draft layer is prepared. Before, every launch showed it at the end of the model's preparation, also when nothing was downloaded.

From 0.2.14 on Linux (RTX PRO 5000 72 GB, IQ3_S), strata update took 7 s: Control 0.1.60 to 0.1.60.1, the engine unchanged, and a launch with the DFlash2 drafter and chats passed afterwards.

What changes for you

  • Nothing to do. Update as usual (see Updating below). Your models, chats and settings stay.
  • The engine reads your prompt with Strata's own code on NVIDIA and AMD cards, on Windows and Linux. Before, it used code taken from llama.cpp for this step. Answers can differ from 0.2.13 in some words; the tables below show how much.
  • There are no new settings.

Highlights

  • Prompt reading on Strata's own kernels on NVIDIA and AMD, Windows and Linux. A source build through setup builds the same way. -DSTRATA_MMQ=ON still builds the old llama.cpp path.
  • Strix Halo (Windows), UD-Q4_K_XL, a 4,096-token prompt: 2,621 ms (0.2.13: 2,689 ms). An 8,192-token prompt: 4,910 ms (0.2.13: 5,091 ms).
  • RTX 4090, IQ3_S, a 20,000-token prompt: 5,656 and 5,651 ms in two runs (0.2.13: 6,110 and 6,150 ms).
  • RTX 5090, UD-Q4_K_XL, a 32,000-token prompt: 7,870 tokens per second (0.2.13: 6,851).
  • Mac (M5 Max), IQ3_S, a 7,755-token prompt: 4,488 ms (0.2.13: 4,581 ms). Output, a 256-token story: 94.7 tokens per second (0.2.13: 92.8), the same text.

Speed against 0.2.13

Prompt times in milliseconds (ms, fewer is quicker) or prompt tokens per second, and output in tokens per second (more is quicker). Numbers in brackets are 0.2.13 on the same PC. Each row names the engine build it was measured on: most rows ran on builds of 11-14, and "build of 11-15" is the 0.1.60 build of that morning. The NVIDIA and AMD engines that ship were built once more after it, with a fix that changes only a model split across two or more cards; the Mac engine that ships is the build of 11-15.

| Machine, model | Prompt | Output | Engine measured | |---|---|---|---| | Strix Halo (Windows, 96 GB graphics memory), UD-Q4_K_XL | 4,096 tokens: 2,621 ms (0.2.13: 2,689). 8,192: 4,910 ms (0.2.13: 5,091). 32,000: 19,151 ms (0.2.13: 20,023) | all six kinds: 54.47 with the DFlash2 drafter. 0.2.13 on this PC still ran the MTP drafter from a setup of 11-04: 46.97. The 0.2.13 notes gave 49.1 with DFlash2 | build of 11-15 (4K, 8K); 32K and output on the build before it, same answers token for token | | Strix Halo (Windows), UD-Q4_K_XL, short prompts at 0.2.13's start line | 256 tokens: 408 ms (0.2.13: 2,010). 1,000: 877 ms (0.2.13: 3,112) | not measured | build of 11-14 (before Path A) | | Strix Halo (Windows), Gyro-S | 256 tokens: 485 ms (0.2.13: 736). 1,000: 1,098 ms (1,676). 4,096: 3,233 ms (4,323). 8,192: 6,598 ms (7,404). 32,000: 24,247 ms (28,027). Prompts of 26 to 64 tokens, 11 prompts together: 4,490 ms (4,436) | code and chat: 49.84 (0.2.13: 49.49) | build of 11-15 with the Gyro change, before the final | | RTX 4090 24 GB, IQ3_S | 4,096: 1,863 / 1,858 ms (0.2.13: 1,917 / 2,000). 8,192: 2,233 / 2,255 (2,380 / 2,404). 8,279: 2,695 / 2,757 (2,734 / 2,750). 16,884: 4,818 / 4,899 (5,212 / 5,050). 20,000: 5,656 / 5,651 (6,110 / 6,150). 32,000: 7,779 / 7,782 (8,295 / 8,300) | not measured | build of 11-14 with the prompt-cut fix, two runs each | | RTX 5090 32 GB, IQ3_S | tokens per second: 256: 1,347 (0.2.13: 1,196). 2,048: 3,800 (3,730). 8,192: 6,901 (6,517). 32,000: 7,899 (7,178) | chat 254.2 (274.0), code 304.8 (300.2), json 320.1 (316.0) | build of 11-14 | | RTX 5090 32 GB, UD-Q4_K_XL | tokens per second: 256: 988 (868). 8,192: 6,350 (5,945). 20,000: 7,710 (6,707). 32,000: 7,870 (6,851) | chat 148.6 (147.7), code 183.5 (183.9), json 160.5 (160.2) | build of 11-14 | | RTX 5070 Ti 16 GB, IQ3_S | tokens per second: 1,000: 372 (380). 8,192: 1,126 (1,125). 20,000: 1,408 (953) | chat 17.5 (19.4), code 21.8 (23.8) | build of 11-14 | | Mac M5 Max 128 GB, Q2_0 | 7,755 tokens: 3,673 ms (3,660) | 256-token story: 117.0 (116.0), same text | build of 11-15 | | Mac M5 Max 128 GB, IQ3_S | 7,755 tokens: 4,488 ms (4,581), the second long prompt after a start; the first one of the new engine took 11,878 ms, 7,346 ms of it preparing the prompt kernels once | 256-token story: 94.7 (92.8), same text | build of 11-15 | | Mac mini M4 32 GB, Gyro-S read from the SSD | 7,715 tokens: 57,294 ms (54,204) | 128-token story: 11.92 (12.73), same text | build of 11-14 (before the final) |

Not measured against 0.2.13: RTX 5070 12 GB, RTX 3060 Ti 8 GB, Titan RTX, Tesla P100, two-card PCs, Radeon R9700 on Linux, Radeon 780M. On these the new engine was measured against the same version built with llama.cpp's kernels; see the details below.

How it was measured: the engine alone on the users' launch line, the same prompts in each arm, 2 or 3 interleaved rounds, a fresh engine start per round, greedy answers. Strix Halo 4K / 8K: 2 rounds, 3 prompts per size; the 0.2.13 figures come from a separate run on the same PC the same day. Mac: one launch per arm.

What changed for you

  • Prompt reading. The prompt experts run on Strata's own kernels on NVIDIA and AMD, Windows and Linux. Setup's source builds build the same way. -DSTRATA_MMQ=ON still builds the old path.
  • Still from llama.cpp / ggml after this release: its picture reader as the fallback and as a reader you can choose (Strata's own reader is the default where it was measured), the CPU expert kernels on the Mac, and stable-diffusion.cpp for the opt-in image preview. Intel Arc and the DGX Spark keep the 0.1.59 engine.
  • Strix Halo (Windows): Path A is the default for the prompt's experts on UD-Q4_K_XL (43 of 48 layers). 4,096-token prompt: 2,623 ms with Path A, 2,739 ms without; 32,000 tokens: 19,151 ms with, 20,047 ms without. Output, all six kinds: 54.47 with, 54.72 without tokens per second; agent 60.86 with, 63.32 without; agent copy 91.64 with, 90.44 without. Quality, measured as KL against a run of the same engine with the prompt's experts in FP16 (lower is closer): 4,096 tokens 0.015488 with, 0.012877 without (ratio 1.2027, 95% range 1.0360 to 1.3995); 8,192 tokens ratio 0.9661 (0.8813 to 1.0563).
  • Long prompts on one card are cut into pieces of equal size instead of full pieces plus a short rest. When an equal piece would be short, the engine keeps the old cut: with a prompt piece of 8,192 tokens or more, two equal pieces need 5,400 tokens each and three or more need 6,144 each. RTX 4090, IQ3_S, 8,279 tokens: 2,695 / 2,757 ms with this rule, 3,501 / 3,534 ms with equal pieces of 4,140. RTX 4090, IQ3_S, 11,450 tokens: 3,698 / 3,734 ms with two pieces of 5,725, 3,874 / 4,053 ms with the old cut. RTX 5070 12 GB, Q2_0, 11,000 tokens: 3,592 ms with two pieces of 5,500, 4,172 ms w

去 GitHub 看全文

较新版本: v0.2.15较旧版本: v0.2.13