AirLLM 70B inference with single 4GB GPU

Run Kimi K3 using 29 GB of RAM at 0.50 tok/s

Kimi K3 Now Available via Telnyx Inference API

Hetzner is working on LLM Inference

Flash-MoE: Running a 397B Parameter Model on a Laptop

How Taalas “prints” LLM onto a chip?

Nano-vLLM: How a vLLM-style inference engine works

David Patterson: Challenges and Research Directions for LLM Inference Hardware