vllm
Latest stories covering vllm, with summaries and links to related coverage.
3 articles · page 1 of 1
Connected topics
Inside vLLM: Anatomy of a High-Throughput LLM Inference System (2025)
tool
36 min read
vLLM is a high-throughput LLM inference system that utilizes components such as paged attention, continuous batching, prefix caching, and specdec. It supports multi-GPU and multi-node dynamic serving to enhance performance at scale.
AMD Strix Halo RDMA Cluster Setup Guide
tool
10 min read
This guide provides instructions for configuring a two-node AMD Strix Halo cluster using Intel E810 (RoCE v2) for distributed vLLM inference with Tensor Parallelism. It covers hardware prerequisites, host configuration for Fedora 43, toolbox installation, network verification, cluster operation, and troubleshooting steps.
Eagle 3.1: Collaboration Between the EAGLE Team, vLLM Team, and TorchSpec Team
tool
1 min read
The vLLM Blog provides technical articles, release announcements, model guides, and community updates related to the vLLM project. The latest update includes details about the Eagle 3.1 model release.