AI is changing the world. Don't stay behind. Clear summaries, community insight, delivered without the noise. Subscribe to never miss a beat.

I prompted ChatGPT, Claude, Perplexity, and Gemini and watched my Nginx logs

AI traffic vs referral traffic: what nginx logs prove | SurfacedBy

surfacedby.com

April 20, 2026

9 min read

🔥🔥🔥🔥🔥

53/100

Summary

Nginx logs reveal that "AI traffic" can indicate either provider-side fetching or referral traffic. The logs provide measurable data on whether AI assistants fetch live pages or rely on previously indexed information.

Key Takeaways

AI traffic can be categorized into two types: provider-side fetches by assistants and real clickthrough visits by humans.
Five AI assistants used distinct user-agent signals to indicate retrieval requests, while three assistants did not announce themselves, making it difficult to differentiate their fetches from human visits.
ChatGPT consistently fetched pages from multiple IPs, while Claude always checked the robots.txt file before fetching.
Gemini did not make any live fetch requests during the testing period, indicating it answered from an index instead.

Read original article

Community Sentiment

Mixed

Positives

Claude's adherence to rules demonstrates a commitment to ethical AI usage, which could set a precedent for responsible AI development.
The test results indicate that Gemini may prioritize its own cached data over live fetching, suggesting a focus on efficiency in response times.

Concerns

The unclear writing style of the article detracts from its credibility, making it difficult for readers to grasp key points about AI model performance.
The implication that Gemini may not be fetching live data raises concerns about the accuracy and reliability of its responses.