Story
Running Gemma 4 26B at 5 tokens/sec on a 13-year-old Xeon with no GPU
1 report
1 website
1 discussion
Latest report
Overview
Gemma 4, a 26-billion-parameter open-weights mixture-of-experts model, runs on a 13-year-old Xeon server without a GPU at a speed of approximately five tokens per second. The server is a repurposed HP StoreVirtual storage box originally designed for disk storage.
Summary from neomindlabs.com.
Reports
One report so far. Additional coverage will appear here when it is linked to this story.
Community reactions
Each reaction summarizes sampled comments from the linked discussion.
Hacker News · neomindlabs.com
Hacker News discussion ↗
Historical count: 5 · 5 saved in sample · Historical sample
Analysis ready
Read this discussion’s reaction
Based on 5 sampled comments from this thread. Analyzed 2026-07-16.
- Praise: Running advanced models like Qwen3.6-35B locally at 7-9 tokens/second on a 16GB Mac is a game changer for accessibility — it shows just how far we’ve come in AI efficiency.
- Praise: The emergence of models like Prism's Bonsai 27B, running at 44+ tokens/sec on a consumer laptop, suggests we're on the brink of a new era in AI capabilities.
- Praise: With local models processing tokens at impressive speeds, many are excited about the potential for rapid advancements in AI applications without the need for expensive cloud services.
- Concern: The cost comparison of running models locally versus using inference providers raises eyebrows — local setups can be 30x more expensive, which is a significant barrier for many.
- Concern: Some commenters express skepticism about the feasibility of running >200B MoE models on basic hardware, pointing to challenges in parameter compression and speed limitations.
- Concern: Concerns about the actual performance of older hardware when running these models highlight the potential pitfalls of relying on outdated technology for cutting-edge AI tasks.