Micro-Agent enables collaboration within model APIs to enhance AI inference efficiency. Routers serve as the control plane, optimizing requests by directing them to appropriate models, thereby reducing costs associated with using frontier models versus open-source or local alternatives.

Xiaomi has released the MiMo-V2.5-Pro-UltraSpeed, capable of generating 1 trillion parameter models at a speed of 1000 transactions per second (TPS). This advancement enhances real-time AI reasoning and collaboration, making it more responsive and integrated into human thought processes.

Qwen-3-Coder-Next is an 80 billion parameter model that requires 159.4GB of RAM to run. Techniques exist to reduce the size of large language models by 4x and increase their speed by 2x.

Hypura is a storage-tier-aware LLM inference scheduler designed for Apple Silicon, allowing users to run large models that exceed their Mac's memory. It optimally distributes model tensors across GPU, RAM, and NVMe storage based on access patterns and hardware capabilities to prevent system crashes.