Does processing personal data without human involvement breach privacy?
I recently came across an interesting OpenAI and MIT Media Lab study exploring how interactions with AI chatbots affect people’s social and emotional well-being (https://openai.com/index/affective-use-study/). However, one claim caught my attention immediately:
“The team at OpenAI conducted a large-scale, automated analysis of nearly 40 million ChatGPT interactions without human involvement in order to ensure user privacy.”
Their footnote explains:
“To protect user privacy, we designed our conversation analysis pipeline to run entirely via automated classifiers. This allowed us to analyze user conversations without humans in the loop, preserving the privacy of our users.”
This suggests that privacy is protected simply because no human directly reads the content. Is this accurate?
Short answer: No.
Privacy encompasses individual control over data, contextual integrity, and protection of dignity and autonomy. These dimensions remain relevant whether or not humans are directly involved. Automated processing can breach privacy by depriving individuals of control, violating contextual expectations, or undermining autonomy—even without human eyes directly viewing the data. Also, importantly, privacy/data protection laws still apply when there is no human in the loop and sometimes even specifically address automated data operations (e.g. Art. 22 GDPR).
I’m not asserting OpenAI has violated the law. However, their statement reflects a misunderstanding—or at least a misrepresentation—of privacy concepts and possibly their ethical implications (consent).
This matters because OpenAI significantly influences AI practices globally, also those related to automated data processing (take, for instance, Agentic AI). If their researchers misunderstand or downplay these privacy implications, the broader impacts could be substantial.
Perhaps alongside AI literacy, OpenAI should prioritise privacy literacy. 😉
Leave a Reply