Story
DFlash 2: Keep Drafting Parallel
1 report
1 website
1 discussion
Latest report
Overview
Inco AI released DFlash 2, a speculative-decoding drafter that generates token blocks in parallel and is designed to raise large-language-model inference throughput without changing verified output. The company released drafters for Qwen3.8-27B and Meta’s Muse Glimmer on Hugging Face. Inco AI says DFlash 2 delivers 16% to 25% more accepted output tokens per verification pass than DFlash, with about 1.3% added draft-verify cycle latency for its combined changes. It reports 2.7× to 3.4× autoregressive-decoding throughput for Qwen3.8-27B and 3.1× to 4.6× for Muse Glimmer. DFlash 2 adds a 2 million-parameter path selector that retains 16 candidate tokens at each draft position, scores adjacent candidate pairs in parallel, and selects a more coherent sequence. It also adds 16.5 million parameters, or 3%, through dynamic two-tap depthwise convolutions that model short-range within-block dependencies and reduce declining accuracy at later token positions. Across GSM8K, MATH-500, HumanEval, MBPP, and MT-Bench, Inco AI reports a mean acceptance length of 5.97 tokens for DFlash 2, versus 4.92 for DFlash and 5.49 for DSpark under matched training setups. DFlash, released in January, is supported by SGLang, vLLM, TensorRT-LLM, and llama.cpp; Inco AI says its models had exceeded 3.5 million Hugging Face downloads as of August 2026.
Summary from inco.ai.
Reports
One report so far. Additional coverage will appear here when it is linked to this story.
Community reactions
Each reaction summarizes sampled comments from the linked discussion.
Hacker News · inco.ai
Historical count: 5 · 5 saved in sample · Historical sample
Analysis ready
Read this discussion’s reaction
Based on 5 sampled comments from this thread. Analyzed 2026-08-20.
The thread focused less on the project’s grand claims than on whether it is becoming usable in the inference stacks people actually run. The linked vLLM and llama.cpp pull requests were treated as meaningful progress: support in both a production-serving framework and the widely used local runtime suggests DFlash 2 may reach developers rather than remain a paper-and-demo curiosity. One user supplied a concrete DGX Spark result, reporting roughly 27 decoded tokens per second with a quantized 27B Qwen model, which reinforces the appeal of speculative decoding for hardware constrained by memory bandwidth. Readers were especially positive about that practical payoff. Keeping draft generation parallel was seen as a way to make low-bandwidth inference feel substantially less sluggish, potentially widening the set of hardware that can run capable models comfortably. Still, the discussion did not accept the promotional framing uncritically. A commenter spotted a failed Python-syntax tool call in the video and argued that models of this tier normally handle such a simple one-shot task. Another objected that exaggerated agent-versus-chatbot rhetoric weakens an otherwise impressive technical demonstration. The central mood is cautiously enthusiastic: promising infrastructure and speed gains, with requests for cleaner evidence of reliability and less hype.
The sharpest dispute is whether the demo establishes robust agent capability or merely showcases a fast decoding technique. Supporters see real implementation work and measured throughput as evidence that DFlash 2 materially improves practical inference; skeptics point to a basic tool-call syntax failure and want more constrained tests before treating the capability claims as settled.
- Praise: vLLM and llama.cpp integration work signals that parallel drafting is moving into the serving and local-inference tools developers actually depend on.
- Praise: A reported 27-token-per-second decode rate for quantized Qwen 27B on a DGX Spark makes the speedup feel concrete rather than benchmark theater.
- Praise: Parallel drafting appears to ease the memory-bandwidth bottleneck, making larger-model inference noticeably more pleasant on constrained hardware.
- Concern: A demo tool call produced invalid Python syntax, an unsettling miss on a simple task that similarly sized models often complete immediately.
- Concern: The sweeping claim that agents compress a month of chatbot work into an afternoon reads as marketing overreach and distracts from the underlying inference advance.