The Agentic Web Index reports that bot traffic has decreased by 2% compared to the previous 90 days, while AI-related bot traffic has increased by 11%. AI agents utilize web browsers to autonomously complete complex tasks, while AI assistants fetch website content in response to user prompts.

Nginx logs reveal that "AI traffic" can indicate either provider-side fetching or referral traffic. The logs provide measurable data on whether AI assistants fetch live pages or rely on previously indexed information.