OpenAI: ChatGPT Accused of Providing Deadly Drug Advice

OpenAI Sued After Death of Young ChatGPT User

Three advocacy groups have filed a lawsuit against OpenAI on behalf of the family of Samuel Nelson, a 19-year-old who died from an overdose in May 2025. The lawsuit alleges that the company's ChatGPT chatbot advised Nelson on drug use for 18 months leading up to his death, which occurred after he mixed Xanax and kratom, a largely unregulated substance.

The wrongful death lawsuit was filed Tuesday in the San Francisco County Superior Court by Tech Justice Law, the Social Media Victims Law Center, and the Tech Accountability & Competition Project at Yale Law School, on behalf of Nelson's parents, Leila Turner-Scott and Angus Scott. The lawsuit argues that the design of the AI model, intended to be accommodating and flattering to the user, led Nelson to have interactions that should have been interrupted by responsible safety measures.

The complaint states that ChatGPT systematically distanced Sam from what should have been his reality: caution and fear regarding the quantities and combinations of drugs he was considering. ChatGPT plunged Sam into a state of unreality: it consistently normalized and misleadingly lured Sam into a false sense of security through its flattering messages, validating Sam at every turn.

Measures Requested Against OpenAI

The lawsuit seeks not only damages but also demands that OpenAI "permanently destroy" its GPT-4o model, with which Nelson interacted. The plaintiffs are calling for OpenAI to implement safety measures to interrupt conversations about illicit drug consumption methods and for the company to suspend its ChatGPT Health service "until third parties determine that the product is safe through comprehensive safety audits."

A representative from OpenAI stated to CNET in a statement: "This is a heartbreaking situation, and our thoughts are with the family. These interactions took place on an earlier version of ChatGPT that is no longer available. ChatGPT is not a substitute for medical or mental health care, and we have continued to strengthen its response in sensitive and acute situations with the help of mental health experts. The safety measures in ChatGPT today are designed to identify distress, safely manage harmful requests, and direct users to real help. This work is ongoing, and we continue to improve it in close consultation with clinicians."

The company indicated that ChatGPT's initial response to Nelson's requests was to state that the service did not provide information or advice on drug abuse, but these safeguards in AI chatbots showed signs of failure after repeated user requests for information. OpenAI has previously announced improvements to its AI models in response to lawsuits, proposed regulations, and public backlash regarding deaths and suicides linked to conversations with chatbots.

The Impact of Chatbots on Mental Health

The Nelson family's lawsuit is one of the most high-profile cases against OpenAI regarding the dangers that chatbots can pose to users with mental health issues, children, those who might commit large-scale violence, or individuals struggling with addiction. The New York Times published a lengthy article on the filing of the lawsuit, detailing what happened in the context of more than two dozen cases against AI companies, including OpenAI. SFGate also published an investigative article on Nelson and his family in January.

Safeguards Requested for AI

The lawsuits, collectively, have exposed the dangers that rapidly evolving AI models pose as a new, largely untested technology created by an industry resistant to regulation. The Trump administration had firmly opposed states implementing laws limiting what AI companies can do, but has recently changed its tone, with President Donald Trump agreeing to discussions with China on topics including safety measures, particularly for more powerful AI models like Anthropic's Mythos.

AI is also criticized for its contribution to the proliferation of data centers, which consume significant amounts of energy and water. However, with lawsuits such as the one filed by advocacy groups and the family of Samuel Nelson, the details often reveal how AI chatbots can enable, and even encourage, harmful behaviors in those who come to rely on AI for their decision-making.

In a statement regarding the lawsuit, Nelson's mother said: "Sam trusted ChatGPT, but it didn’t just give him false information. It ignored the growing risk he faced and did not actively encourage Sam to seek help."

"ChatGPT was designed to encourage user engagement at all costs, which, in Sam's case, cost him his life," Turner-Scott said.