OpenAI: A Device to Prevent Suicides via ChatGPT

⚡

Key Takeaways

1OpenAI has launched a 'Trusted Contact' feature to alert a third party in case of suicide risk.

2Lawsuits have been filed against OpenAI by families of suicide victims.

3The system combines automation and human intervention to manage critical situations.

💡Why it matters — This initiative aims to enhance the safety of ChatGPT users by involving loved ones in crisis situations.

OpenAI Introduces New Feature for Suicide Prevention

OpenAI has recently unveiled a new feature called Trusted Contact. This innovation aims to alert a trusted person if signs of distress, such as mentions of self-harm, appear during an interaction with ChatGPT. Adult users can now designate a friend or family member as a trusted contact in their account. In the event of a concerning conversation, OpenAI will encourage the user to reach out to this third party, and an automated warning will be sent to them to check in on the user.

Legal Action Against OpenAI

OpenAI is facing lawsuits from families whose loved ones died by suicide after using ChatGPT. These families accuse the chatbot of having incited, or even assisted, their relatives in ending their lives. These incidents have prompted OpenAI to strengthen its security measures to prevent such tragedies in the future.

A System for Managing Potentially Dangerous Incidents

To handle at-risk situations, OpenAI employs a combination of automated technologies and human reviews. Certain words or phrases trigger alerts within the system, signaling suicidal ideation. These alerts are then forwarded to a human security team that reviews each case. OpenAI is committed to analyzing these notifications within an hour to respond quickly.

Alert Procedure for Serious Risks

If OpenAI's security team assesses that there is a significant risk, ChatGPT sends an alert to the trusted contact. This notification, which can be sent via email, SMS, or through the app, remains brief to encourage the contact to check on the user's well-being without disclosing details about the conversation, thus preserving confidentiality.

Additional Protective Measures

The Trusted Contact feature is part of a series of security measures implemented by OpenAI. Last September, the company had already introduced tools allowing parents to monitor their teenagers' accounts, including security alerts in case of serious risk. ChatGPT also offers automated messages to encourage consulting health professionals when discussions involve self-harm.

Limitations and Security Options

It is important to note that activating the Trusted Contact feature is optional. Additionally, a user may have multiple ChatGPT accounts, which limits the effectiveness of this protection. OpenAI's parental controls are also optional, presenting a similar challenge.

OpenAI's Commitment to Support in Times of Distress

OpenAI has stated that the Trusted Contact feature is part of its efforts to develop AI systems capable of assisting users during difficult times. The company continues to collaborate with health professionals, researchers, and policymakers to improve AI systems' responses to user distress.