Reddit could block Google Search, Bing, if the AI companies don’t pay up to use its data

According to a report, Reddit may block search crawlers from Google and Bing which will result in it not being discoverable in search results. However, that will happen if a deal can’t be reached with these AI companies to receive payments for data.

Reddit is trying to negotiate deals with companies working with AI such as Google and Microsoft and asking them to make payments for using its data for training the large language models. (Pexels)

Of late, many platforms that manage a large amount of data, especially social media platforms, news websites, and large blog sites have expressed their frustrations at bots being used by companies working with AI to crawl these platforms and scrape their data to train large language models (LLM). Many companies are asking to be paid by AI firms now. Among them is Reddit, whose senior officials recently met with AI companies to strike a deal, however, they have not yet been able to reach any agreement. A report claims that if Reddit cannot reach an agreement soon, it can block bots from crawling its site, even if it means that the platform will get delisted from Google and Bing search engines.

According to a report by The Washington Post, a person familiar with the matter said that Reddit “has met with top generative AI companies about being paid for its data” and that if a deal can't be reached “Reddit is considering blocking search crawlers from Google and Bing, which would prevent the forum from being discovered in searches and reduce the number of visitors to the site”. The feeling within the company is that Reddit can survive without search, The Washington Post's source mentioned.

Reddit could block Google, Bing from crawling its site

Reddit has also not denied the reports. Reddit spokesperson Tim Rathschmidt told The Verge, after being asked whether Reddit could block Google, Bing, and other search engine crawl bots, “In terms of crawlers, we don't have anything to share on that topic at the moment”.

If the worst-case scenario is played out, people searching for Reddit posts or looking for results from Reddit pages will not get any search results.

This is not the first time Reddit has gone after AI companies. Earlier this year, it put a limit to API requests that third-party apps and platforms could make and increased the prices of requests. As a result, many third-party Android and iOS apps were forced to shut down.

This move against being scraped by AI companies is not limited to Reddit. Twitter did the same earlier, and now users who aren't logged in cannot see comments on a post. Elon Musk has also spoken about measures taken to prevent extreme data scraping from bots. As per The Washington Post, as many as 535 news organizations have opted to block their content from being scraped by companies like OpenAI.