Pull requests / #1961

#1961 Add opt-in experimental DFlash2 runtime

open · draft · @j-luwierski · 0 comments · View on GitHub

Setup & installAMD / HIPNVIDIA / CUDAModels & quantsDocumentationLinux

Description

## Summary

Adds an opt-in experimental DFlash2 extension on top of the standalone DFlash runtime in #1426. The DFlash prerequisite has been updated to main at `7695859a`, with merge conflicts resolved separately in `dcd07e9b`. DFlash2 remains a separate commit, `6ac9c8cb`, changing 14 files relative to that prerequisite.

- Dynamic causal convolutions around attention and MLP sublayers.
- Predecessor-conditioned greedy top-K selection and anchor-row handling.
- GGUF conversion, metadata validation and extension-weight preservation.
- Synthetic tests and documentation in `docs/DFLASH2.md`.

## Dependency and scope

This PR depends on #1426 and must merge after it. Since this upstream PR targets main, Files changed includes the prerequisite until #1426 merges. The extension-only comparison is [feat/dflash...codex/dflash2-main](https://github.com/j-luwierski/Strata/compare/feat/dflash...codex/dflash2-main).

DFlash2 is not added to the DFlash PR. Its build flag, `STRATA_ENABLE_DFLASH2`, defaults to OFF. No trained weights or DLoop changes are included. Installer, calibration and server files are unchanged relative to the updated DFlash prerequisite.

## Validation

Revalidated on Linux, RTX 4070 Ti SUPER 16 GB, CUDA 13.4, GCC 14.2, sm89:

| Check | Result |
| --- | --- |
| DFlash and DFlash2 CUDA engine builds | Passed |
| DFlash CTest checks | 7 passed |
| DFlash2 CTest checks | 8 passed |
| Converter/quantization tests | 6 passed |
| Server tests on the shared DFlash prerequisite | 297 passed |
| DFlash installer and calibration tests | 39 passed |
| Older CUDA attention-path compilation | Passed |

## Experimental limitations

No compatible trained checkpoint was available for end-to-end generation, default-path byte equality, acceptance-rate or performance testing. No speed improvement is claimed. HIP was not built because ROCm is unavailable; DFlash2 with SYCL is explicitly rejected. Single-GPU, serial requests and the documented Flash-Next drafter geometry only.

I am also working on a compatible DFlash2 drafter checkpoint for testing. Once it is available, I plan to validate end-to-end generation correctness, acceptance rates and performance against a same-day target-only baseline on the same hardware.

Related on strata.com

Editorial links to help you install, pick models, or read release notes — not part of the upstream thread.