The Italian DPA said it’s concerned that the ChatGPT maker is breaching the European Union’s General Data Protection Regulation (GDPR), and is opening an investigation.
Specifically, the Garante said it has issued the order to block ChatGPT over concerns OpenAI has unlawfully processed people’s data as well as over the lack of any system to prevent minors from accessing the tech.
It’s worth noting that since OpenAI does not have a legal entity established in the EU, any data protection authority is empowered to intervene, under the GDPR, if it sees risks to local users. (So where Italy steps in, others may follow.)
Suite of GDPR issues
The GDPR applies whenever EU users’ personal data is processed. And it’s clear OpenAI’s large language model has been crunching this kind of information, since it can, for example, produce biographies of named individuals in the region on-demand (we know; we’ve tried it).
Although OpenAI declined to provide details of the training data used for the latest iteration of the technology, GPT-4, it has disclosed that earlier models were trained on data scraped from the Internet, including forums such as Reddit. So if you’ve been reasonably online, chances are the bot knows your name.
Moreover, ChatGPT has been shown producing completely false information about named individuals, apparently making up details its training data lacks. That potentially raises further GDPR concerns, since the regulation provides Europeans with a suite of rights over their data, including the right to rectification of errors. It’s not clear how/whether people can ask OpenAI to correct erroneous pronouncements about them generated by the bot, for example.
The Garante‘s statement also highlights a data breach the service suffered earlier this month, when OpenAI admitted a conversation history feature had been leaking users’ chats, and said it may have exposed some users’ payment information.
The GDPR allows for a number of possibilities — from consent to public interest — but the scale of processing to train these large language models complicates the question of legality. As the Garante notes (pointing to the “mass collection and storage of personal data”), with data minimization being another big focus in the regulation, which also contains principles that require transparency and fairness. Yet, at the least, the (now) for-profit company behind ChatGPT does not appear to have informed people whose data it has repurposed to train its commercial AIs. That could be a pretty sticky problem for it.
If OpenAI has processed Europeans’ data unlawfully, DPAs across the bloc could order the data to be deleted, although whether that would force the company to retrain models trained on data unlawfully obtained is one open question as an existing law grapples with cutting edge tech.
“The Privacy Guarantor notes the lack of information to users and all interested parties whose data is collected by OpenAI but above all the absence of a legal basis that justifies the mass collection and storage of personal data, for the purpose of ‘training’ the algorithms underlying the operation of the platform,” the DPA wrote in its statement today [which we’ve translated from Italian using AI].
“As evidenced by the checks carried out, the information provided by ChatGPT does not always correspond to the real data, thus determining an inaccurate processing of personal data,” it added.
The authority added that it is concerned about the risk of minors’ data being processed by OpenAI since the company is not actively preventing people under the age of 13 from signing up to use the chatbot, such as by applying age verification technology.
So if OpenAI can’t definitively confirm the age of any users it’s signed up in Italy, it could, at the very least, be forced to delete their accounts and start again with a more robust sign-up process.
OpenAI was contacted for a response to the Garante‘s order.
Lilian Edwards, an expert in data protection and Internet law at Newcastle University who has been ahead of the curve in conducting research on the implications of “algorithms that remember,” told TechCrunch: “What’s fascinating is that it more or less copy-pasted Replika in the emphasis on access by children to inappropriate content. But the real time-bomb is denial of lawful basis, which should apply to ALL or at least many machine learning systems, not just generative AI.”
“Large language models don’t offer those remedies and it’s not entirely clear they would, could or what the consequences would be,” Edwards added, suggesting that enforced retraining of models may be one potential fix.
Or, well, that technologies like ChatGPT may simply have broken data protection law
