TIME serves two different versions of its website: one for human readers and a stripped-down markdown version for AI crawlers that includes embedded ads. GPT-5.6 Sol xhigh utilizes more than double the tokens per session compared to GPT-5.5 xhigh in Codex workflows.

The new /crawl endpoint in Browser Rendering allows users to crawl entire websites with a single API call. It discovers and renders pages in a headless browser, returning results in formats such as HTML, Markdown, and structured JSON.

Crawling 1.005 billion web pages took just over 25.5 hours and cost $462. Advances in technology, such as multi-core CPUs and NVMe solid-state drives, have significantly improved web crawling efficiency since 2012.