embeddings
Latest stories covering embeddings, with summaries and links to related coverage.
2 articles · page 1 of 1
Connected topics
RAG Is Simpler Than You Think
tool
13 min read
Retrieval-augmented generation systems can begin with BM25 or other full-text search rather than embeddings and vector databases, particularly for keyword-heavy queries, exact identifiers, proprietary terminology, low query volumes, and frequently changing content. The source recommends measuring a baseline for two to four weeks, then adding complexity only when user feedback and evaluation data show that simpler retrieval fails. Full-text search avoids embedding API costs, document chunking, re-indexing, and model-deprecation risks; it can return results in under 10 milliseconds, according to the source. LLM-based query rewriting can convert conversational requests into keyword searches, add synonyms, preserve internal product names, and decompose complex questions at an estimated cost of about $0.001 per query using GPT-4o-mini. Hybrid retrieval uses BM25 to select 50 to 100 candidates and embeddings to rerank the top results, trading stronger semantic matching for roughly 200–500 milliseconds of added latency when documents are embedded on demand. On-the-fly embedding suits high-churn data, while hot/cold tiers pre-embed commonly accessed documents and process less-used material dynamically. Full pre-embedding is presented as appropriate for stable corpora, broad access patterns, more than 10,000 daily queries, and sub-50-millisecond latency requirements. The source estimates that 60% of systems can stop at full-text search plus query rewriting.
14× faster embeddings: how we rebuilt the ONNX path in Manticore
tool
14 min read
Manticore has rebuilt the ONNX path for Auto Embeddings, achieving a 14× increase in speed for converting text columns into vectors. The previous method using SentenceTransformers and Candle limited throughput, resulting in low document processing rates and serialized concurrent calls.