Model optimization
Source reports covering Model optimization. Stories will appear when analysis and community reactions are ready.
7 reports · page 1 of 1
Connected topics are temporarily unavailable. You can still browse stories below.
Source reports
Micro-Agent: Beat Frontier Models with Collaboration Inside Model API
tool
9 min read
Micro-Agent enables collaboration within model APIs to enhance AI inference efficiency. Routers serve as the control plane, optimizing requests by directing them to appropriate models, thereby reducing costs associated with using frontier models versus open-source or local alternatives.
VibeThinker: 3B param model that beats Opus 4.5 on reasoning with novel SFT+GRPO
research
2 min read
VibeThinker-3B is a compact dense model with 3 billion parameters designed to advance verifiable reasoning in small language models. It utilizes the Spectrum-to-Signal post-training paradigm for systematic enhancement.
MiMo-v2.5-Pro-UltraSpeed: 1T model with 1000 tokens per second
tool
8 min read
Xiaomi has released the MiMo-V2.5-Pro-UltraSpeed, capable of generating 1 trillion parameter models at a speed of 1000 transactions per second (TPS). This advancement enhances real-time AI reasoning and collaboration, making it more responsive and integrated into human thought processes.
A sleep-like consolidation mechanism for LLMs
research
2 min read
Transformer-based large language models struggle with long-context tasks due to poor scaling of their attention mechanism. Implementing a sleep-like consolidation mechanism allows models to convert recent context into persistent fast weights while clearing their key-value cache.
Quantization from the Ground Up
tool
26 min read
Qwen-3-Coder-Next is an 80 billion parameter model that requires 159.4GB of RAM to run. Techniques exist to reduce the size of large language models by 4x and increase their speed by 2x.
Run a 1T parameter model on a 32gb Mac by streaming tensors from NVMe
tool
6 min read
Hypura is a storage-tier-aware LLM inference scheduler designed for Apple Silicon, allowing users to run large models that exceed their Mac's memory. It optimally distributes model tensors across GPU, RAM, and NVMe storage based on access patterns and hardware capabilities to prevent system crashes.
Scaling Karpathy's Autoresearch: What Happens When the Agent Gets a GPU Cluster
research
12 min read
Claude Code was given access to 16 GPUs on a Kubernetes cluster and submitted approximately 910 experiments over 8 hours. It determined that scaling model width was more significant than any single hyperparameter and achieved a 2.87% improvement in validation performance, reducing val_bpb from 1.003 to 0.974.