Hetzner is working on LLM Inference
sliplane.io
July 24, 2026
6 min read
🔥🔥🔥🔥🔥
55/100
Summary
Hetzner is conducting experiments with large language model (LLM) inference, currently offering only one model without billing, service level agreements, or production guarantees. The company aims to assess user interest, system scalability, essential features, and load handling capabilities.
Key Takeaways
- Hetzner is experimenting with LLM inference using an OpenAI-compatible API on its own infrastructure.
- The only available model for this experiment is Qwen/Qwen3.6-35B-A3B-FP8, a 35-billion-parameter Mixture-of-Experts model.
- The API currently has no billing, no service level agreement (SLA), and is not intended for production use.
- Initial tests showed a median time to first token of 153 ms and an output rate of 224 tokens per second, but these results are not indicative of future performance under load.
Community Sentiment
Positives
- Having a respected EU-native inference provider like Hetzner could ease regulatory concerns and provide more options for developers in Europe.
- The potential for cost-effective inference for smaller models is a game-changer, making AI more accessible for smaller companies.
- More players in the LLM inference space is a win for competition, which can lead to better services and pricing.
Concerns
- Some commenters are skeptical about Hetzner's capabilities, fearing they may not support larger models or could limit their offerings.
- Concerns about rising costs for Hetzner's services hint at a potential profit-driven approach that could undermine accessibility.
- Criticism of other providers like Infomaniak highlights issues with support and reliability, raising doubts about the overall quality in the market.
