Story
Cloudflare Crawl Endpoint
1 report
1 website
1 discussion
Latest report
Overview
The new /crawl endpoint in Browser Rendering allows users to crawl entire websites with a single API call. It discovers and renders pages in a headless browser, returning results in formats such as HTML, Markdown, and structured JSON.
Summary from developers.cloudflare.com.
Reports
One report so far. Additional coverage will appear here when it is linked to this story.
developers.cloudflare.com
Cloudflare Crawl Endpoint
Read report →
Community reactions
Each reaction summarizes sampled comments from the linked discussion.
Hacker News · developers.cloudflare.com
Hacker News discussion ↗
Historical count: 5 · 5 saved in sample · Historical sample
Analysis ready
Read this discussion’s reaction
Based on 5 sampled comments from this thread. Analyzed 2026-03-11.
- Praise: The Cloudflare Crawl endpoint utilizes a headless Chrome instance, enabling efficient rendering and fetching of publicly accessible URLs, which can enhance web scraping capabilities.
- Concern: Concerns arise that Cloudflare's dual role as a provider of scraping countermeasures and a scraping service creates a conflict of interest, potentially undermining trust in their services.
- Concern: There is skepticism about Cloudflare's intentions, with some viewing their scraping service as a way to control and monetize access to web content, likening it to a 'mob outfit'.
- Concern: The pricing structure for Browser Rendering on the Workers Paid plan raises accessibility concerns, as it limits usage to paying customers, potentially excluding smaller developers.