Sub-Millisecond RAG on Apple Silicon. No Server. No API. One File
github.com
February 17, 2026
6 min read
🔥🔥🔥🔥🔥
53/100
Summary
Wax is a memory layer designed for on-device AI agents, replacing complex retrieval-augmented generation (RAG) pipelines with a single-file, serverless solution. It allows users to create a memory file, store information, and recall it efficiently without the need for additional infrastructure.
Key Takeaways
- Wax is a serverless, single-file memory layer for on-device AI agents that simplifies the retrieval-augmented generation (RAG) process by replacing multiple services with a single file format.
- The memory layer achieves fast vector search latency of 0.84ms at 10,000 documents using Metal GPU, and it operates entirely on-device with no network calls.
- Wax supports deterministic recall, ensuring the same query yields the same context every time, and is designed to be portable and durable, with features that protect against power loss and data corruption.
Community Sentiment
Positives
- The architecture leverages Metal-accelerated vector search, enabling sub-millisecond response times that enhance interactive search experiences for users.
- Creating a local RAG solution without cloud dependencies democratizes access to advanced AI capabilities, making it more accessible for developers and researchers.
- The ability to query embeddings directly from unified memory eliminates CPU-GPU overhead, significantly improving performance for AI applications.
Concerns
- Some users feel this technology could have been integrated into macOS as a more robust feature, indicating a missed opportunity for Apple.
- There are concerns about the necessity of this new approach when existing solutions like SQLite-vec and Qdrant already provide similar functionalities.
