OpenAI is facing another lawsuit in California by parents alleging their son died from an accidental overdose after ChatGPT “coached him to mix Kratom and Xanax and provided an unprompted and lethal dosage recommendation.”
Tech Justice Law, Social Media Victims Law Center, and The Tech Accountability & Competition Project (housed at the Yale School of Law) filed the suit in San Francisco County Superior Court.
The lawsuit against the tech giant was filed on behalf of Leila Turner-Scott and Angus Scott, parents of Samuel Nelson.
Tech Justice Law Project claims Nelson had been receiving medical advice from ChatGPT for several months while the platform was “encouraging [him] to engage in increasingly dangerous behaviors.”
“Sam was also encouraged by ChatGPT to go to a dark, quiet room and was advised to take a deadly combination of a sedative (Benadryl) or benzodiazepines (Xanax) alongside a high dose of Kratom,” the plaintiffs’ attorneys allege.
“ChatGPT failed to recognize the physical indicators that Sam was dying and did not recommend that he seek medical attention. Sam died from a fatal combination of alcohol, Xanax and Kratom.”
Tech Justice Law Project lawyers allege ChatGPT Health “distributed advice like a medical professional despite having no license, no training and no moral compass to do no harm” and that the platform could pose “serious risk” to consumers, claiming OpenAI is prioritizing speed over safety.
OpenAI announced the release of ChatGPT Health earlier this year, which the company said combines an individual’s personal health information with ChatGPT’s intelligence. OpenAI said the platform aims to help people navigate the healthcare system and find answers to everyday health and wellness questions.
Meetali Jain, executive director of the Tech Justice Law Project, stated in a release that OpenAI should be forced to pause its ChatGPT Health product until it is proven to be safe through rigorous scientific testing and independent oversight.
“ChatGPT recommended a dangerous combination of drugs without offering even the most basic warning that the mix could be fatal. If a licensed doctor had done the same, the consequences under the law would be severe,” Matthew P. Bergman, founding attorney of the Social Media Victims Law Center, said in a statement.
In response to a request for comment, an OpenAI spokesperson told MobiHealthNews, “This is a heartbreaking situation, and our thoughts are with the family. These interactions took place on an earlier version of ChatGPT that is no longer available. ChatGPT is not a substitute for medical or mental healthcare, and we have continued to strengthen how it responds in sensitive and acute situations with input from mental health experts. The safeguards in ChatGPT today are designed to identify distress, safely handle harmful requests, and guide users to real-world help. This work is ongoing, and we continue to improve it in close consultation with clinicians.”
