Stay discoverable in search while disallowing AI training

blog.cloudflare.com
September 16, 2026
12 min read
48/100
Summary
Have it both ways: stay discoverable in search while disallowing AI training Without proper controls, website owners have long faced a difficult tradeoff: allow your content to be used for AI training, or risk losing discoverability in search. That tradeoff exists because some of the largest organizations on the Internet use mixed-use crawlers: a single crawler serving both search and AI training....
Coverage & discussions
Hacker News · blog.cloudflare.com
Community discussions
Historical count: 15 · 15 saved in sample · Historical sample
Awaiting analysis