I’m willing to bet the lack of api access going forward will make all reddit posts disappear from crawler results anyways. I’m no expert, but I imagine the crawler is picking up on all of the interconnected references to reddit that are all due to free api access. As soon as those connections disappear, so dies the value to the entire community. It will be just like the garbage results we get from every single source now. This is the path of neo digital feudalism.
API calls are almost always private between the caller and the endpoint (think telegram bots or mobile apps). There isn’t really a technically feasible way for a crawler to somehow “infer” any kind of knowledge of how api calls are being used unless the result has some kind of publically visible side effect (E. G. The program using the api is generating a web page and uploading it somewhere crawlable). Google et Al go by how many links from other pages to the page of interest exist (inbound links) and multiply by a smattering of other things like quality of keywords, length of content etc.
That said, if you’re implying that the api changes mean that:
people are less likely to use reddit because they can’t access it via RIF/Apollo
less useful content is added to the site to be indexed,
fewer inbound links will be generated that point to existing posts
I’m willing to bet the lack of api access going forward will make all reddit posts disappear from crawler results anyways. I’m no expert, but I imagine the crawler is picking up on all of the interconnected references to reddit that are all due to free api access. As soon as those connections disappear, so dies the value to the entire community. It will be just like the garbage results we get from every single source now. This is the path of neo digital feudalism.
API calls are almost always private between the caller and the endpoint (think telegram bots or mobile apps). There isn’t really a technically feasible way for a crawler to somehow “infer” any kind of knowledge of how api calls are being used unless the result has some kind of publically visible side effect (E. G. The program using the api is generating a web page and uploading it somewhere crawlable). Google et Al go by how many links from other pages to the page of interest exist (inbound links) and multiply by a smattering of other things like quality of keywords, length of content etc.
That said, if you’re implying that the api changes mean that:
That is a plausible concern.