Usually "obtaining" is just making a bunch of HTTP requests - which is kind of how the web is designed to work. The "consent" (and perhaps "desired payment" when there is no paywall) issue is the important bit and ultimately boils down to the use case. Is it a human viewing the page, a search engine updating its index, or OpenAI collecting data for training? It is annoying when things like robots.txt are simply ignored, even if they are not legally or technically binding.
The legal situation is unsurprisingly murky at the moment. Copyright law was designed for a different use case, and might not be the right tool or regulatory framework to address GenAI.
But as I think you are suggesting, it may be an example of regulatory entrepreneurship, where (AI) companies try to move forward quickly before laws and regulations catch up with them, while simultaneously trying to influence new laws and regulations in their favor.
[Copyright law itself also has many peculiarities, for example not applying to recipes, game rules, or fashion designs (hence fast fashion, knockoffs, etc.) Does it, or should it, apply to AI training and GenAI services? Time will tell.]
Usually "obtaining" is just making a bunch of HTTP requests - which is kind of how the web is designed to work. The "consent" (and perhaps "desired payment" when there is no paywall) issue is the important bit and ultimately boils down to the use case. Is it a human viewing the page, a search engine updating its index, or OpenAI collecting data for training? It is annoying when things like robots.txt are simply ignored, even if they are not legally or technically binding.
The legal situation is unsurprisingly murky at the moment. Copyright law was designed for a different use case, and might not be the right tool or regulatory framework to address GenAI.
But as I think you are suggesting, it may be an example of regulatory entrepreneurship, where (AI) companies try to move forward quickly before laws and regulations catch up with them, while simultaneously trying to influence new laws and regulations in their favor.
[Copyright law itself also has many peculiarities, for example not applying to recipes, game rules, or fashion designs (hence fast fashion, knockoffs, etc.) Does it, or should it, apply to AI training and GenAI services? Time will tell.]