Brief IA
›

OpenAI Expands Measures Against Child Sexual Abuse Content

AI Tools·Tom Levy·

OpenAI Expands Measures Against Child Sexual Abuse Content

OpenAI Expands Measures Against Child Sexual Abuse Content
Key Takeaways
1ChatGPT plans a 24-hour suspension for requests sexualizing children.
2Some companies may access the image classifier for free via API.
3According to OpenAI, initial trials resulted in a 48% reduction in reports with the same number of abusive images detected.
4OpenAI publishes guidelines for developers and offers tools to law enforcement.
Why it matters : these measures add warnings and suspensions to the blocks already implemented by ChatGPT.

OpenAI announces several measures against child sexual abuse material (CSAM). These measures concern both ChatGPT users and companies and services responsible for child protection, amid growing concerns about the safety of AI tools.

Common Sense Media warns of risks for teenagers

Several lawsuits have implicated OpenAI's technology for alleged harm to users, particularly children. Common Sense Media, a nonprofit organization specializing in safety, has labeled it an "unacceptable risk" for teenagers. Other chatbots, including Grok, have been used to produce sexualized representations without consent, including of minors.

Over the past 12 months, OpenAI has released a safety plan and revised its rules to prohibit inappropriate content. The company has also modified its model to prevent harmful outputs and launched a version of ChatGPT aimed at teenagers.

ChatGPT plans 24-hour suspensions

ChatGPT already prohibits CSAM content as well as corresponding queries. OpenAI is adding warning screens: an attempt to request sexualized content involving children will trigger an alert, a referral to specialized help to stop harmful behaviors, and a 24-hour account suspension. The company specifies that a repeat offense may result in permanent deletion.

To develop the messages and guidelines, OpenAI states it has worked with Protect Children, the Lucy Faithfull Foundation, and Joel Scanlan from the University of Tasmania. These organizations and this researcher conducted an independent evaluation of a trial of deterrent messages.

Le brief IA que lisent les pros

Le brief IA que les pros lisent chaque soir

Les 7 actus IA du jour, décryptées en 5 min. Gratuit.

Inclus dès l'inscription : notre sélection des meilleurs guides & comparatifs IA.

Choisissez votre rythme

Gratuit · Pas de spam · Désabonnement en 1 clic

OpenAI announces more selective image screening

The new classifier is available for free via API to certain companies. According to OpenAI, it detects visual indicators of abuse in previously unseen images, including AI-generated content, and assigns a score to guide reviewers. The company claims it developed this in collaboration with the National Center for Missing and Exploited Children (NCMEC); it is working with Tech Coalition and ROOST to share access.

OpenAI reports that during initial trials with partners, there was a 48% reduction in reported images, without a decrease in the number of abusive images detected. The company views this as a reduction in review workload. It also states that the likelihood of a reported image being abusive is twice as high.

OpenAI supports developers as well as investigators

For developers using its tools, OpenAI publishes guidelines covering the prevention, detection, and user reporting of CSAM. According to the company, preliminary trials have helped several developers adopt child protection tools and register with NCMEC to submit reports.

Law enforcement agencies tasked with protecting children also receive AI-based features: specialized plugins, tools for investigations, and analysis of complex data. Access is managed by program administrators. Additionally, OpenAI provides training for investigators and a practical guide for using AI.

Brief IA — L'actualité IA en français

L'essentiel de l'actualité de l'intelligence artificielle, décrypté et expliqué chaque jour.