Google is now the only search engine that can surface results from Reddit, making one of the web’s most valuable repositories of user generated content exclusive to the internet’s already dominant search engine.
If you use Bing, DuckDuckGo, Mojeek, Qwant or any other alternative search engine that doesn’t rely on Google’s indexing and search Reddit by using “site:reddit.com,” you will not see any results from the last week. DuckDuckGo is currently turning up seven links when searching Reddit, but provides no data on where the links go or why, instead only saying that “We would like to show you a description here but the site won’t allow us.” Older results will still show up, but these search engines are no longer able to “crawl” Reddit, meaning that Google is the only search engine that will turn up results from Reddit going forward. Searching for Reddit still works on Kagi, an independent, paid search engine that buys part of its search index from Google.
The news shows how Google’s near monopoly on search is now actively hindering other companies’ ability to compete at a time when Google is facing increasing criticism over the quality of its search results. And while neither Reddit or Google responded to a request for comment, it appears that the exclusion of other search engines is the result of a multi-million dollar deal that gives Google the right to scrape Reddit for data to train its AI products.
Kagi can do it … for now, I assume. Maybe they’ll get to keep the privilege if they decide to throw some of that pay-for search money at Spez, but chances are that’s going to get stopped. Or their prices will go up. Again.
From what I recall their in-house solution combines results from a bunch of search engines, including Google. So I think as long as Google can index it, and as long as Kagi works that way and continues to, it shouldn’t be affected.
Ah. Paid-for search aggregation, not paid-for search. I hadn’t picked up on that aspect of their model for some reason.