What Happened
Cloudflare will implement a new policy in September that requires web crawlers serving AI companies to either pay publishers for content or be blocked from accessing it. The company is pushing for AI firms to compensate publishers for the data scraped from websites, particularly ad-supported content. Publishers will have the option to opt-in to allow these crawlers access to their data.
This initiative by Cloudflare addresses concerns about data sovereignty and the use of copyrighted material by AI companies for training purposes. The company's stance suggests a move towards establishing a framework where the value derived from publisher content by AI models is recognized and compensated.
The policy change will affect AI companies that utilize web crawlers to gather vast amounts of data for training their large language models and other AI systems. By setting a deadline, Cloudflare is signaling a firm commitment to enforcing this new standard for data access and compensation.
Key Facts
- 1
Cloudflare will block AI web crawlers starting in September.
- 2
AI crawlers will be denied access to ad-supported content.
- 3
Publishers must opt-in to allow AI crawlers access.
- 4
The policy aims to make AI companies pay publishers for content.
- 5
This addresses data sovereignty and compensation for AI training data.
