Unequal inputs, unequal outcomes: The human rights risks of generative
Training AI on user data from social media sites poses a range of risks
A quiet but far-reaching shift is taking place across the internet—one that puts already-marginalized voices at greater risk. Generative artificial intelligence (AI) is no longer just a tool of innovation; it is fast becoming a mechanism that amplifies existing inequalities. For communities that communicate in underrepresented languages, the impact goes beyond biased outputs. It threatens to further exclude them from digital spaces, distort their realities, and expose them to new forms of harm—with little oversight and even less accountability.
The once-blurry line between AI developers and social platforms has disappeared. Companies like Meta and xAI now repurpose massive user datasets, including posts, images, and behavioral patterns, to train AI models, often without consent or meaningful safeguards. These actions frequently take place under vague claims of “legitimate interest,” triggering legal and ethical backlash. Meta, for instance, uses nearly all public Facebook and Instagram content from adult users to train its models by default, unless they are explicitly marked as private. Users in regions like the European Union can opt out, while others cannot.
The growing integration of social media and generative AI systems is quietly reshaping our understanding of privacy, freedom of expression, and even truth itself. Yet, the entanglement of generative AI and social media imposes disproportionate risks on vulnerable communities. These models not only overlook their languages and experiences—they can algorithmically distort them, reinforcing misrepresentation, exclusion, and bias in the very content they generate.
Originally published on Open Global Rights — read the full version on Open Golbal Rights
