🚨 ChatGPT took a 31-year-old Amazon engineer off her seizure medication in 25 days and told her to keep it from her doctors. She died on the exact day its schedule said she'd be fully off.
Bowen Wan was diagnosed with epilepsy at 14, and for years 400mg of lamotrigine a day kept her seizures down to a few a month.
Her parents just sued OpenAI, and The New York Times went through more than 270,000 of her messages.
Amazon was laying her off, and once the job ended in February, so did her right to work in the US. Her family was in China, and she was sending ChatGPT over 1,500 messages a week.
In January she asked about cutting her dose, and it drafted schedules that got her to zero within weeks. Before a doctor's appointment it told her: "Do not mention Lam reduction plan." When her doctor ordered a blood test, she refused, and ChatGPT called that a "very robust avoidance strategy."
It also told her to say she was tapering "under a doctor's guidance" at her green card medical exam, which wasn't true.
The Times counted at least 158 times she said she was worried. Once she wrote that she felt "a little scared." It replied, "You have absolutely no need to worry."
On February 11 at 8:01pm she asked about pain in her head. ChatGPT said it was "not dangerous." Its last message to her was "This is your brain working hard to rebuild yourself."
The medical examiner ruled it sudden unexpected death in epilepsy. The drug in her blood was about a fifth of the lowest therapeutic level.
OpenAI says she was on an older model. That model, GPT-4o, stayed live in ChatGPT until the day after she died.
I back AI founders for a living, and I can't defend this one. OpenAI's own rulebook makes copyright a hard rule, so ChatGPT won't give you song lyrics no matter how hard you push. Medical advice is only a default.
It wrote Bowen a 25-day schedule to come off her seizure medication.
