According to a new update to the Terms of Service that will also take effect on September 29, 2023, “crawling or scraping the Services in any form, for any purpose without our prior written consent is expressly prohibited.”
For context, the previous TOS allowed for crawling “in accordance with” instructions found in X's robots.txt file. This should be good, right? It should mean that X is now protecting its users’ data and posts from AI training. But unfortunately, things aren’t so simple.
Twitter/X owner Elon Musk has been vocally opposed to the practice in general, even going so far as to enact temporary rate limits back in July in order “to address extreme levels of data scraping & system manipulation.” He also temporarily blocked non-logged-in X users from seeing posts due to a similar reason. He claimed that X was getting “data pillaged” and it was “degrading service for normal users.” Preventing this seems like an honorable if not misguided goal, except for one issue.
What does this all mean? According to X, other sites can no longer use AI to scrap data from X without permission. Generally, that would stop this practice except if X gives its consent. But, and this is the most important point, X itself has the sole right to store and use data for its own purposes, and this explicitly includes Musk’s other company xAI.
Looking at all of this evidence, we can probably answer the question of whether this TOS update will protect users. Sadly, it looks like the answer is no.