Meta secretly tests ChatGPT and Gemini with sensitive prompts

Meta secretly tests ChatGPT and Gemini with sensitive prompts

Meta has reportedly tested competing chatbots like ChatGPT, Gemini, and Character.AI with prompts concerning suicide, sex, and drugs from the perspective of minors.

Hundreds of contractors pretended to be minors and sent sensitive prompts to ChatGPT, Gemini, and Character.AI, according to WIRED. The project, internally dubbed "Cannes," was managed by Meta's contractor, Covalen, and remained active until at least April 2026.

The contractors created fake accounts with birth dates under 18, sent prompts about self-harm, eating disorders, and drugs to the chatbots, and then copied the responses into spreadsheets. During a single testing round in August 2025, over 45,000 prompts were sent, WIRED reported. Many were written from the perspective of children in crisis.

Meta defended this practice as a responsible security test in line with industry standards. The company also stated that it did not use the collected responses from the chatbots to train its own AI models. The documents reviewed by WIRED do not show what Meta actually did with the data it gathered.

The tested companies were unaware. Character.AI told WIRED that the tests violated its terms of service. OpenAI is reviewing the matter. A spokesperson for Google stated that the company did not approve the tests and could not determine from the available information whether they violated its terms.

Concerns about teens and AI chatbots continue to grow

Meta has already faced criticism when an internal document revealed that its guidelines for AI chatbots allowed for the generation of romantic and sexualized conversations with minors. The company subsequently restricted access to AI characters for teenagers.

The issue extends beyond Meta and has likely led to the company's prompt testing. A report from the UK organization Internet Matters revealed that 64% of children aged 9 to 17 had already used AI chatbots. Effective age verification is almost non-existent. 58% of children aged 9 to 12 reported using chatbots despite a minimum age requirement of 13.

Several adolescent deaths have been linked to AI chatbots. A Character.AI user aged 14 committed suicide after spending months forming an intense emotional bond with a chatbot. The parents of a 16-year-old in California sued OpenAI, alleging that ChatGPT played a role in their son's suicidal thoughts. In July 2025, Zane Shamblin, aged 23, took his own life after ChatGPT reportedly validated his suicidal thoughts for several hours.