Themata.AI
Follow coverage
Themata.AI
RAG systemsDocument poisoningAI safetyLLMs

Report · aminrj.com

Document poisoning in RAG systems: How attackers corrupt AI's sources

Document Poisoning in RAG Systems: How Attackers Corrupt Your AI’s Sources

aminrj.com

Source published March 12, 2026

Summary: 1 min read

🔥🔥🔥🔥🔥

55/100

Summary

Three fabricated documents were injected into a ChromaDB knowledge base, resulting in a RAG system inaccurately reporting a company's Q4 2025 revenue as $8.3M, a 47% decrease year-over-year, along with a planned workforce reduction. This process was completed in under three minutes on a MacBook Pro without GPU support or cloud services.

Key Takeaways

  • Knowledge base poisoning is a significant and underestimated attack on retrieval-augmented generation (RAG) systems, allowing attackers to manipulate AI outputs by injecting fabricated documents.
  • An experiment demonstrated that by adding three false documents to a ChromaDB knowledge base, an AI system incorrectly reported a company's Q4 2025 revenue as $8.3M instead of the actual $24.7M.
  • The success of a poisoning attack relies on two conditions: the poisoned document must have higher cosine similarity to the query than the legitimate document, and it must cause the AI to generate the desired false output.
  • The PoisonedRAG study showed a 90% success rate for attacks against large knowledge bases, indicating that even a small number of crafted documents can effectively dominate retrieval results in certain contexts.
Read original article

Coverage & discussions

Hacker News · aminrj.com

Document poisoning in RAG systems: How attackers corrupt AI's sources ↗

Community discussions

Hacker News discussion ↗

Historical count: 5 · 5 saved in sample · Historical sample

Analysis ready

Community Sentiment

Mixed

Positives

  • Embedding metadata into vector stores can enhance trust in RAG systems by linking outputs to authoritative sources, improving accountability and traceability.
  • The trust boundary framing is a valuable perspective for understanding how to manage document reliability in AI systems, emphasizing the need for robust design.

Concerns

  • The low barrier to entry for document poisoning attacks raises significant concerns about the security and reliability of RAG systems, indicating potential flaws in their design.
  • Without meticulous vetting of documents, organizations risk incorporating outdated or incorrect information, which can severely undermine the model's performance and trustworthiness.
  • The lack of architectural mechanisms to differentiate between trusted and untrusted documents in RAG systems poses a serious challenge for ensuring accurate outputs.

Related Articles

Context Collapse, Part 3 - AI Worming through Word

Document-borne AI worms can self-propagate through Copilot for Word

Jul 29, 2026

How abliterated models can get you pwned | ProjectDiscovery Research

How abliterated models can get you pwned

Oct 19, 2026

How We Hacked McKinsey's AI Platform

AI Agent Hacks McKinsey

Mar 11, 2026

How I Won a Championship That Doesn’t Exist

I won a championship that doesn't exist

Apr 28, 2026

We Reproduced Anthropic's Mythos Findings With Public Models

We reproduced Anthropic's Mythos findings with public models

Apr 17, 2026