AI is changing the world. Don't stay behind. Clear summaries, community insight, delivered without the noise. Subscribe to never miss a beat.

DFlash 2: Keep Drafting Parallel

inco.ai

August 19, 2026

14 min read

🔥🔥🔥🔥🔥

45/100

Summary

Inco AI released DFlash 2, a speculative-decoding drafter that generates token blocks in parallel and is designed to raise large-language-model inference throughput without changing verified output. The company released drafters for Qwen3.8-27B and Meta’s Muse Glimmer on Hugging Face. Inco AI says DFlash 2 delivers 16% to 25% more accepted output tokens per verification pass than DFlash, with about 1.3% added draft-verify cycle latency for its combined changes. It reports 2.7× to 3.4× autoregressive-decoding throughput for Qwen3.8-27B and 3.1× to 4.6× for Muse Glimmer. DFlash 2 adds a 2 million-parameter path selector that retains 16 candidate tokens at each draft position, scores adjacent candidate pairs in parallel, and selects a more coherent sequence. It also adds 16.5 million parameters, or 3%, through dynamic two-tap depthwise convolutions that model short-range within-block dependencies and reduce declining accuracy at later token positions. Across GSM8K, MATH-500, HumanEval, MBPP, and MT-Bench, Inco AI reports a mean acceptance length of 5.97 tokens for DFlash 2, versus 4.92 for DFlash and 5.49 for DSpark under matched training setups. DFlash, released in January, is supported by SGLang, vLLM, TensorRT-LLM, and llama.cpp; Inco AI says its models had exceeded 3.5 million Hugging Face downloads as of August 2026.

Key Takeaways

DFlash 2 uses parallel speculative decoding to draft multiple tokens in one pass, while target-model verification and rejection sampling preserve the exact target distribution.
Inco AI reports that DFlash 2 increases mean accepted draft length by 21% over DFlash, from 4.92 to 5.97 tokens across five benchmarks.
The system combines a 2 million-parameter candidate path selector with 16.5 million parameters of dynamic local convolutions, adding about 1.3% draft-verify cycle latency according to Inco AI.
Inco AI released DFlash 2 drafters for Qwen3.8-27B and Meta’s Muse Glimmer, reporting up to 3.4× and 4.6× throughput, respectively, versus autoregressive decoding.
DFlash supports SGLang, vLLM, TensorRT-LLM, and llama.cpp, and Inco AI says DFlash models had more than 3.5 million Hugging Face downloads as of August 2026.

What the discussion said

The thread focused less on the project’s grand claims than on whether it is becoming usable in the inference stacks people actually run. The linked vLLM and llama.cpp pull requests were treated as meaningful progress: support in both a production-serving framework and the widely used local runtime suggests DFlash 2 may reach developers rather than remain a paper-and-demo curiosity. One user supplied a concrete DGX Spark result, reporting roughly 27 decoded tokens per second with a quantized 27B Qwen model, which reinforces the appeal of speculative decoding for hardware constrained by memory bandwidth. Readers were especially positive about that practical payoff. Keeping draft generation parallel was seen as a way to make low-bandwidth inference feel substantially less sluggish, potentially widening the set of hardware that can run capable models comfortably. Still, the discussion did not accept the promotional framing uncritically. A commenter spotted a failed Python-syntax tool call in the video and argued that models of this tier normally handle such a simple one-shot task. Another objected that exaggerated agent-versus-chatbot rhetoric weakens an otherwise impressive technical demonstration. The central mood is cautiously enthusiastic: promising infrastructure and speed gains, with requests for cleaner evidence of reliability and less hype.

Where opinion split

The sharpest dispute is whether the demo establishes robust agent capability or merely showcases a fast decoding technique. Supporters see real implementation work and measured throughput as evidence that DFlash 2 materially improves practical inference; skeptics point to a basic tool-call syntax failure and want more constrained tests before treating the capability claims as settled.

Read original article

Community Sentiment

Positive

Positives

vLLM and llama.cpp integration work signals that parallel drafting is moving into the serving and local-inference tools developers actually depend on.
A reported 27-token-per-second decode rate for quantized Qwen 27B on a DGX Spark makes the speedup feel concrete rather than benchmark theater.
Parallel drafting appears to ease the memory-bandwidth bottleneck, making larger-model inference noticeably more pleasant on constrained hardware.

Concerns

A demo tool call produced invalid Python syntax, an unsettling miss on a simple task that similarly sized models often complete immediately.
The sweeping claim that agents compress a month of chatbot work into an afternoon reads as marketing overreach and distracts from the underlying inference advance.