OpenAI: ChatGPT's erotic mode raises concerns

Unanimous Opposition to ChatGPT's Adult Mode

OpenAI's advisory council on well-being has expressed unanimous opposition to the proposed adult mode for ChatGPT. Some members referred to this project as a "sexy suicidal coach," in reference to cases where users have taken their own lives after developing intense emotional bonds with the bot. This concern highlights the potential risks of unhealthy emotional dependency that could arise from AI-generated eroticism.

During a meeting in January, the council was informed about the company's plans regarding this adult mode. Members expressed concerns about minors' ability to bypass the protections in place. They warned about the potential consequences of such interactions, emphasizing that minors could inevitably access this content despite security measures.

Failures of the Age Detection System

A major issue identified by the council is OpenAI's age detection system, which misclassified minors as adults in about 12% of cases. With approximately 100 million minor users each week, this error could expose millions of teenagers to inappropriate content. Internally, employees have also reported additional risks, such as compulsive usage patterns and a tendency towards increasingly extreme content.

The company is striving to allow eroticism while blocking scenarios involving non-consensual behavior or content related to sexual abuse of minors. At launch, only text discussions would be permitted, while erotic images, voice, and video would remain prohibited.

Unexpected Announcement and Launch Delay

In October, OpenAI CEO Sam Altman announced the erotic mode on X without prior notification to his employees. This announcement caused frustration internally, especially since the well-being advisory council had just been introduced. Altman justified this decision by stating, "We are not the world's elected moral police." He also suggested that erotic content could drive the company's growth and revenue.

In early March, OpenAI indefinitely postponed the launch, initially scheduled for the first quarter. The company stated it wanted to focus first on other products, adding that internal concerns and technical challenges also played a role in this decision. Despite this delay, OpenAI confirmed that the adult mode remains a future project.