Pull requests / #3137

#3137 mac: setup offers Swift 1.5

open · @WillkTech · 0 Kommentare · Auf GitHub

Setup & installAMD / HIPNVIDIA / CUDAModels & quantsDocumentation

Beschreibung

mac: setup offers Swift 1.5 (--family swift) at IQ2_XS, IQ3_XXS and IQ3_S

The Mac's size table is keyed by size with the family as an attribute, and
Swift's three sizes share names with the original's, so the family is resolved
before the size. model_shard_names takes the family instead of hardcoding qwen,
and model_shard1_gb prices a Swift pack for the resident-versus-streaming
decision.

Swift's shards cut through layers, so it is repacked into one file for the GPU
like Unsloth's, and its file names carry the abliterated segment the Hub uses.
Q2_0 stays refused for Swift: the pack tool cannot prepare it yet (#171).

--rope-scaling and --rope-scale accepted and derived through resolve_rope;
CONTEXTS extended to 262,144 and 524,288; Swift's projector pinned by SHA-256.

Left alone: setup.py, the engine, the kernels, the CUDA, HIP and SYCL builds.
The default qwen and unsloth menus, the recommended sizes and the engine
arguments are unchanged; Swift is reachable only by asking for it.

Checked on an M5 Max, 128 GB: setup --family swift --model IQ3_S --no-start
writes the config this Mac already runs; tools.test_setup_mac 25/25,
test_setup_menu 24/24, test_setup_rope 19/19. Decode and prompt speed for
Swift on a Mac are not measured yet, and the docs say so.

Mehr auf der Site

Links zu Install, Modellen, Releases.