Meta Strengthens Safety for Youth Against Self-Harm Risks

⚡

Key Takeaways

1Meta introduces parental alerts for self-harm discussions with its AI chatbot.

2Parents can supervise their children's Instagram, Facebook, and Meta Horizons accounts.

3Meta collaborates with experts to identify dangerous conversations and alert parents.

💡Why it matters — These measures aim to protect young users from potential dangers related to self-harm and suicide, reinforcing the accountability of social platforms.

Meta Strengthens Youth Safety Against Self-Harm Risks

In a context where technology giants are increasingly scrutinized for protecting vulnerable users, Meta announces new features for its AI chatbot. These additions aim to provide enhanced support and notify parents when signs of self-harm or suicidal thoughts are detected among young users.

Expanded Parental Supervision System

Meta has implemented a set of supervision tools for parents, allowing them to monitor their children's Instagram, Facebook, and Meta Horizons accounts. To activate these features, parents must sign up and select the accounts they wish to supervise. Details of this procedure are available on Meta's dedicated family center page. Currently, these settings are accessible to users residing in the United States, Canada, the United Kingdom, and Australia.

Alerts for Risky Conversations

When a teenager engages in a conversation with Meta's AI chatbot on potentially dangerous topics, an alert is sent to the supervising parent. This notification can be received via SMS, email, or directly within the app. While the exact content of the exchanged messages is not disclosed, Meta provides parents with resources and advice to help them manage the situation.

Partnership with Mental Health Experts

Meta claims it is already sharing information about crisis hotlines with teenagers and encouraging them to turn to trusted adults. Additionally, parents are notified if their child conducts frequent searches related to self-harm or suicide on Instagram within a short period.

An AI System to Detect Dangerous Conversations

To identify potentially dangerous discussions, Meta relies on an AI system developed in collaboration with human experts, including wellness panels and external mental health clinicians. At launch, all alerts will be manually reviewed, and Meta assures that it will act cautiously when alerting parents.

Towards Emergency Service Alerts

Meta is also considering alerting emergency services in cases of credible suicide risk. Although this feature is not yet available, it could draw inspiration from the existing alert system on Facebook and Instagram, which has already facilitated over 19,000 emergency interventions for wellness checks.

Context and Challenges

The parent company of Facebook and Instagram is aware of the negative impacts that technology can have on mental health. Earlier this year, Meta was found guilty by two juries of designing intentionally addictive social media platforms, thereby facilitating the exploitation of children.

These initiatives come as concerns grow among tech observers and safety advocates. The latter have criticized, and at times sued, tech and AI companies for the risky behaviors that chatbots can encourage, including self-harm and suicide.