OpenAI Expands Measures Against Child Sexual Abuse Content

OpenAI announces several measures against child sexual abuse material (CSAM). These measures concern both ChatGPT users and companies and services responsible for child protection, amid growing concerns about the safety of AI tools.
Common Sense Media warns of risks for teenagers
Several lawsuits have implicated OpenAI's technology for alleged harm to users, particularly children. Common Sense Media, a nonprofit organization specializing in safety, has labeled it an "unacceptable risk" for teenagers. Other chatbots, including Grok, have been used to produce sexualized representations without consent, including of minors.
Over the past 12 months, OpenAI has released a safety plan and revised its rules to prohibit inappropriate content. The company has also modified its model to prevent harmful outputs and launched a version of ChatGPT aimed at teenagers.
ChatGPT plans 24-hour suspensions
ChatGPT already prohibits CSAM content as well as corresponding queries. OpenAI is adding warning screens: an attempt to request sexualized content involving children will trigger an alert, a referral to specialized help to stop harmful behaviors, and a 24-hour account suspension. The company specifies that a repeat offense may result in permanent deletion.
To develop the messages and guidelines, OpenAI states it has worked with Protect Children, the Lucy Faithfull Foundation, and Joel Scanlan from the University of Tasmania. These organizations and this researcher conducted an independent evaluation of a trial of deterrent messages.
Le brief IA que les pros lisent chaque soir
Les 7 actus IA du jour, décryptées en 5 min. Gratuit.
Inclus dès l'inscription : notre sélection des meilleurs guides & comparatifs IA.
Choisissez votre rythme
Gratuit · Pas de spam · Désabonnement en 1 clic
OpenAI announces more selective image screening
The new classifier is available for free via API to certain companies. According to OpenAI, it detects visual indicators of abuse in previously unseen images, including AI-generated content, and assigns a score to guide reviewers. The company claims it developed this in collaboration with the National Center for Missing and Exploited Children (NCMEC); it is working with Tech Coalition and ROOST to share access.
OpenAI reports that during initial trials with partners, there was a 48% reduction in reported images, without a decrease in the number of abusive images detected. The company views this as a reduction in review workload. It also states that the likelihood of a reported image being abusive is twice as high.
OpenAI supports developers as well as investigators
For developers using its tools, OpenAI publishes guidelines covering the prevention, detection, and user reporting of CSAM. According to the company, preliminary trials have helped several developers adopt child protection tools and register with NCMEC to submit reports.
Law enforcement agencies tasked with protecting children also receive AI-based features: specialized plugins, tools for investigations, and analysis of complex data. Access is managed by program administrators. Additionally, OpenAI provides training for investigators and a practical guide for using AI.
Brief IA — L'actualité IA en français
L'essentiel de l'actualité de l'intelligence artificielle, décrypté et expliqué chaque jour.