Story
We got 207 tok/s with Qwen3.5-27B on an RTX 3090
1 report
1 website
1 discussion
Latest report
Overview
Lucebox is an optimization hub for hand-tuned LLM inference, specifically designed for individual consumer hardware. It features kernels, speculative decoding, and quantization tailored for each target, with the first megakernel for hybrid DeltaNet/Attention LLMs achieving 1.87 tokens per joule on a 2020 GPU.
Summary from github.com.
Reports
One report so far. Additional coverage will appear here when it is linked to this story.
Community reactions
Each reaction summarizes sampled comments from the linked discussion.
Hacker News · github.com
Hacker News discussion ↗
Historical count: 5 · 5 saved in sample · Historical sample
Analysis ready
Read this discussion’s reaction
Based on 5 sampled comments from this thread. Analyzed 2026-04-21.
- Praise: Running Qwen3.5-27B on an RTX 3090 demonstrates the potential for local AI deployment, allowing users to leverage existing hardware without vendor lock-in.
- Praise: The ability to run models on Vulkan and Apple's hardware indicates a growing flexibility in AI infrastructure, which could enhance accessibility for developers.
- Praise: The reported peak performance of 207.6 tok/s showcases significant advancements in model efficiency, potentially enabling faster inference for real-time applications.
- Concern: The claim of achieving 207 tok/s is misleading, as it relies on speculative decoding, which may compromise output quality compared to traditional methods.
- Concern: The focus on a specific GPU like the RTX 3090 raises concerns about accessibility, as it may not represent the average developer's hardware capabilities.
- Concern: The implementation appears to prioritize speed over quality, which could lead to inferior outputs and undermine the reliability of the model's performance.