Cloudflare launches a marketplace that lets websites charge AI bots for scraping

Cloudflair, a cloud infrastructure supplier that serves 20% of the web, announced to launch a new marketplace on Tuesday that revises relations between website owners and AI companies – ideally gives more control over their content.

For the past year, the AI ​​crewr has launched tools to solve the widespread emergence of AI crewers, including a One-click solution to block all AI botAs well as an Dashboard to see how AI Creollers are watching on their siteThe In an interview in 2021, Mathew Prince, CEO of Claudflair, told Prince TechCrunch that these products were laid on a new type of marketplace where publishers could distribute their contents to AI companies and compensate for it.

Now, Cloudflair is making the market lively.

This is called the crawl per salary and the CloudFlair is launching “test” on the private bit on Tuesday. Owners of the website in the examination may choose to give their site to scrap a set rate on a separate basis – a micrope for each single “crawl”. Alternatively, website owners AI crawlers can choose to scrap their site free or block them completely. Cloudflare claims that its tools will allow website owners to see that crewers are scraping their site for AI training data, to appear in response to AI search, or other purposes.

On the scale, Cloudflare’s marketplace is a big idea that publishers can provide a potential business model for the AI ​​era – and it also puts the cloudflair at the center of it. The introduction of the marketplace came at the time of the Google search traffic away and the popularity of AI chatbots as the news publishers are facing the question of how they will reach readers.

There is no clear answer to how news publishers will survive in the AI ​​era. Has filed something like the New York Times Suit against technology agencies To train their AI models in news articles without permission. Meanwhile, there are other publishers Many years of deals hit their content licenses AI model training and their content to appear in AI chatboat reactions.

Nevertheless, only large publishers have hit AI licensing deals and whether they still provide meaningful sources of revenue is still unclear. The goal of the cloudflair is to create a sustainable system where publishers can set prices on their own terms.

The company also announced on Tuesday that new websites set up with Cloudflair will now block all AI crawl by default. The owners of the site need to allow specific AI crawler to access their site – a change in the cloudfliers has said that each new domain will “default” to “control”.

Conde nest, time, associated press, Atlantic, Edwick and Fortune have signed with Cloudflair to block AI crawlalars with Cloudflare “Creaking permit-based approach” by default by default.

The business model that many of these publishers relied for decades have been gradually becoming incredible. ICally, online publishers have allowed Google to scrap their sites in exchange for referrals in Google search, which translate traffic on their sites and finally earn advertising.

However, Cloudflair’s new data suggests that publishers can get a worse deal in the AI ​​era than the Google search era. While Some websites quoted the Chatzipte as a major traffic sourceIt doesn’t seem to be a great case.

This June, the CloudFlair says it has shown that Google’s Craoller has scraped his website 14 times for each referral given to them. Meanwhile, the Open Craoller scraped the websites 17,000 times for each referral, while the ethnographic scraped websites are 73,000 times for each referral.

এদিকে, ওপেনএআই এবং গুগল এআই এজেন্ট তৈরি করছে যা ডিজাইন করা হয়েছে View websites on behalf of usersCollect the information and provide it directly to the users. In the future where these tools are mainstream, there is a huge impact for publishers that depend on readers to visit their sites.

Cloudflair notes that the crawling “true possibility” of per pay “may arise in the future.

“If an agent can fully work on the program on the network edge of the Paywal Network, if an agent can help you with the latest cancer research or legal synthesis, or to help you find the best restaurants to help you find the best and most relevant content,” said a bloudflare blog post.

To participate in the experimental marketplace in the cloudflare, both AI companies and publishers need to set up with the CloudFlair Accounts. In their accounts, both sides can determine the rate where they want to buy and sell a “crawl” of the publisher’s content. Claudflare acts as an intermediary in these transactions, charges the AI ​​company and distributes the ear to the publisher.

Cloudflare spokesman Ripley Park told TechCrunch that there is no stabicon or cryptocurrency associated with the wages at the moment, though many have suggested Digital currency would be suitable for something like thisThe

Cloudflair’s Marketplace feels like a bold outlook for the future for which a lot of publishers and AI companies need to be on the board. Nevertheless, there is no guarantee that publishers will get a good deal, and it can be hard to see the AI ​​companies to participate, they are currently scraping free content.

Nevertheless, the CloudFlair seems to be one of the few companies in the position to be made in this national market.