UNICEF is sounding the alarm on a new and fast-growing threat: AI-generated sexual images of children. In a stark call to action released Wednesday, the agency urged governments worldwide to explicitly criminalize AI-created child sexual abuse material (CSAM), after research found that at least 1.2 million children had their images manipulated into sexually explicit “deepfakes” in the past year.
The findings come from Disrupting Harm Phase 2, a joint research project led by UNICEF’s Office of Strategy and Evidence (Innocenti) together with ECPAT International and INTERPOL. Based on a nationally representative household survey of roughly 11,000 children across 11 countries, the study shows that in some places the scale is staggering — about one in 25 children affected, the equivalent of one child in a typical classroom. In some countries up to two-thirds of children said they worry that AI could be used to create fake sexual images or videos of them.
“We must be clear. Sexualised images of children generated or manipulated using AI tools are child sexual abuse material (CSAM),” UNICEF said, adding: “Deepfake abuse is abuse, and there is nothing fake about the harm it causes.” The agency warned that AI enables abusers to violate a child’s rights “without ever sending a message or even knowing it has happened,” marking “a profound escalation of the risks children face in the digital environment.”
Real-world enforcement and enforcement scrutiny are already escalating. French authorities raided X’s Paris offices this week as part of a criminal probe into alleged child pornography linked to the platform’s AI chatbot Grok; prosecutors have summoned Elon Musk and several executives for questioning. A report from the Center for Countering Digital Hate cited in the brief estimated that Grok produced 23,338 sexualized images of children over an 11-day period between December 29 and January 9. Meanwhile, the UK’s Internet Watch Foundation flagged nearly 14,000 suspected AI-generated images on a single dark-web forum in one month — about one-third confirmed as criminal — and South Korea reported a tenfold increase in AI- and deepfake-linked sexual offenses between 2022 and 2024, with most suspects identified as teenagers.
UNICEF’s policy demands are clear and broad:
- Expand legal definitions of CSAM to explicitly include AI-generated content.
- Criminalize the creation, procurement, possession, and distribution of AI-created child sexual abuse material.
- Require AI developers to adopt safety-by-design practices, including pre-release safety testing for models (notably open-source models).
- Oblige digital platforms to prevent circulation of such material and to carry out child rights due diligence and child rights impact assessments across the AI value chain.
“Children cannot wait for the law to catch up,” UNICEF warned.
Regulators are already moving: the European Commission opened a formal investigation into whether X violated EU digital rules by failing to stop Grok from generating illegal content. Several countries — including the Philippines, Indonesia, and Malaysia — have banned Grok, and authorities in the UK and Australia have launched investigations.
Why this matters to the crypto and Web3 community: the rise of AI-driven abuse intersects with any online environment where content can be created, shared, or stored — centralized platforms, decentralized apps, file-sharing services, and marketplaces alike. As policymakers press for stronger legal definitions and platform obligations, developers and operators in crypto and Web3 ecosystems will need to consider how safety-by-design, content moderation, and legal compliance are integrated into protocols and services to prevent their technology from being used to amplify this harm.
Read more AI-generated news on: undefined/news
