Publish fake stats, get cited in LLMs

Based on 15 sampled comments from this thread. Analyzed 2026-10-20.

Commenters treated the stunt less as a novel LLM failure than as a loud demonstration of an old internet disease: fabricated claims, SEO gaming, citation cascades, and astroturfing have always fooled people. What changes with AI is scale and authority. A made-up statistic can be lifted into search-grounded answers, academic work, or investment research, then repeated in the polished, confident tone of an expert. That repetition launders the claim for readers who would have questioned the original sketchy page. Several readers argue that the practical answer is better provenance rather than magical truth detection: query official statistical databases directly, rank sources by demonstrated reliability, cross-check claims, and build models or retrieval systems that distinguish generated sludge from trusted material. Others warn that training data may already contain the poison, while grounding can merely narrow the model to a small, still-gameable set of web sources. A few pushed back on treating the experiment as uniquely damning for AI, since humans and search engines have long been vulnerable to lies; they see public demonstrations as useful pressure to improve information systems. The sharp concern is that indiscriminate web ingestion turns ordinary misinformation into a durable, automated reference layer, especially as AI-generated pages increasingly fill search results and training corpora.

The core dispute is whether this is an inherent LLM reliability crisis or a general misinformation problem that better source handling can contain. Skeptics say web-scale models amplify falsehoods into authoritative-looking answers and may absorb poisoned claims permanently; optimists argue that first-party data access, reputation systems, cross-checking, and generated-text detection can preserve AI usefulness without sealing it off from the open web.

• Praise: The stunt usefully makes invisible data-poisoning risks concrete, forcing the public to see how a bogus web statistic can migrate into AI answers and institutional research.
• Praise: Direct connections to authoritative datasets such as government statistics could replace fragile web search for factual queries and make AI answers meaningfully more auditable.
• Praise: Source scoring, claim cross-checking, and detection of machine-generated content look feasible to some commenters, offering a path beyond blindly trusting whatever ranks highly.
• Concern: A fake number can acquire institutional credibility after AI repeats it with expert-like confidence, leaving downstream readers unable to trace that it began as fabrication.
• Concern: Search-grounded systems increasingly encounter AI-written pages among top results, creating a feedback loop where models train on and retrieve synthetic misinformation.
• Concern: Better reasoning cannot fully rescue a model whose training corpus already contains planted falsehoods or whose grounding set is too narrow and easy to manipulate.
• Concern: Indiscriminate harvesting of the public web rewards misinformation publishers and makes poisoned data a predictable consequence of AI companies treating everything online as usable evidence.