Cloudflare CEO Matthew Prince is pushing UK regulator to unbundle Google’s search and AI crawlers

After launching a marketplace earlier this year Which allows websites to charge AI bots To scrape their content, web infrastructure provider Cloudflare is pushing for increased regulation in the AI ​​sector.

The company’s chief executive, Matthew Prince, said he was in London to speak to the UK’s Competition and Markets Authority (CMA), where he is proposing tougher rules on how Google will be able to compete in the AI ​​race due to its search dominance.

The CMA earlier this month designated Google with a special status in the search and advertising market because of its “theoretical and entrenched” position. The move will allow the regulator to impose stricter regulations beyond just search and advertising, including Google’s AI overview and areas like AI Mode, Discover Feed, Top Stories and News tab.

According to Prince, Cloudflare is in a good position to recommend because it’s not in the AI ​​business itself, but they have a lot of relationships with AI companies.

“We don’t have dogs in direct combat. We’re not an AI company,” Prince said. Bloomberg Tech conference in London this week. “We are not a media publisher, but we are this network that sits within them. 80% of AI companies are our customers,” he added.

The Cloudflare boss believes Google should compete on the same footing as other AI companies, which is not what it’s doing today, he said. Rather, Google uses its existing web crawlers to crawl content for its AI products and services in addition to its search engine. Prince said this gives Google an unfair advantage.

“Google is saying, ‘We have absolute God-given rights to all the content in the world, even if we’re not paying for it, because look what we’ve done for the last 27 years,'” Prince explained. “And, they’re saying we can take that and use the crawlers we use for search to power our AI systems. And if you want to opt out of one, you have to opt out of both,” he noted.

That, obviously, isn’t possible for most, especially in the media business where losing search means losing about 20% of your revenue, the executive said.

“But it gets worse. If you block Google’s crawler, it blocks their ad protection team, which means your ads stop working on all your platforms, which is just a non-starter.”

Because Google bundles its crawlers, it is able to get access to content that others, such as Anthropic, OpenAI, and Perplexity, have to pay for.

“The problem is that we then effectively hand the game over to Google,” he said.

The solution, Prince said, is to increase a lot of competition in the marketplace, where potentially thousands of AI companies could compete to buy content from thousands of media businesses and millions of small businesses. He suggested that the UK CMA’s flagging of Google as a potential regulatory target was a thoughtful move and one that indicated they were aware of Google’s unique advantages.

Cloudflare also provided CMA with data that shows how Google’s crawler works and why it’s nearly impossible for other players to replicate Google’s same success.

The prince is not the only one to share these views in recent days. last month, Neil Vogel, CEO of People, Inc. The largest digital and print publisher in the US, which operates more than 40 media brands, said essentially the same thing. In an interview, he called Google a “bad actor”, saying media companies had no choice but to let Google crawl their sites for AI content as crawlers converged.

Vogel, whose company took over Cloudflare’s solution to block AI crawlers Who will not pay, claims the system is working, as he said contract negotiations are underway with several major LLM providers.