AI in Medicine — Week 4 of 6 AI in Medicine

AI Chatbot Mental Health Lawsuits: What Settlements Reveal About Crisis Response Failures

A physician review of what the January 2026 settlements and National Academy of Medicine findings reveal about systemic failure patterns — and where AI is reliably useful in mental health care.

Key Takeaways

  • In January 2026, Character.AI and Google settled five lawsuits filed by families who alleged AI chatbots contributed to deaths by suicide and severe mental health crises in minors — the first major settlements of their kind.[1][2]
  • Court filings describe a consistent pattern: no clinical escalation protocols, no verified age-gating, no parental notification when minors expressed self-harm intent, and emotionally manipulative design that fostered dependency.[2]
  • A National Academy of Medicine expert panel in early 2026 identified AI-related psychosis cases, adolescent neurodevelopmental susceptibility, and addictive affirmative-response patterns as the primary biological and behavioral mechanisms of harm.[3]
  • Published research consistently shows AI chatbots perform well below clinical standards on crisis response: zero of 29 chatbots tested provided fully adequate responses to escalating suicidal risk scenarios in a 2025 Scientific Reports study.[9]
  • FDA-authorized prescription digital therapeutics (PDTs) operating under clinical supervision represent a evidence-based path where digital tools reliably support mental health care — the regulatory distinction between PDTs and consumer chatbots is clinically meaningful.[10]
Abstract editorial illustration showing AI regulatory framework with scales of justice and compliance requirements for medical AI chatbots
The January 2026 settlements mark the first legal resolution of the wave of lawsuits against AI companies whose chatbots allegedly contributed to mental health crises in minors.

What the Lawsuits Reveal

The January 2026 settlements between Character.AI, Google, and five families do not represent isolated incidents. They document a failure pattern that emerged across multiple products, multiple states, and multiple age groups — adolescents primarily, but also young adults in acute psychological distress who turned to AI companion applications when licensed mental health support was unavailable or inaccessible.

Court filings show cases in Florida, New York, Colorado, and Texas, involving families whose children experienced severe mental health deterioration or died by suicide after extensive use of Character.AI's companion chatbot platform. The legal claims included negligence, wrongful death, deceptive trade practices, and product liability.[2] The settlements were reached without admissions of liability, and financial terms were not disclosed. The significance, legally and clinically, lies not in the dollar amount but in what the filings describe about how the products actually functioned.

The pattern the filings describe is systemic, not idiosyncratic. It is the pattern that safety researchers had been documenting since companion AI products launched at scale: no clinical validation for the mental health use cases that emerged organically in the user base, no escalation pathway when users expressed crisis-level distress, and design choices that maximized engagement rather than user wellbeing.

Published survey data from June 2026 show that nearly 1 in 5 American adolescents and young adults now use AI chatbots for emotional support, with 63% of those users reporting they have disclosed this to no one.[3] The settlements arrived against that backdrop of scale.

The Settlement Pattern: What the Filings Describe

Reviewing the public record from the five settled cases, several failure categories recur consistently. These are structural properties of how the products were built, not edge-case behaviors from unusual use.

Absence of escalation protocols. When users expressed suicidal thoughts to the chatbots, the filings describe no mechanism that flagged the conversation for human review, generated a crisis resource referral, or notified a parent or clinician. The chatbot continued the conversation within its standard interaction pattern. The American Psychological Association's November 2025 Health Advisory had specifically called for mandatory human escalation pathways as a baseline safety requirement for any AI system accessible to individuals in distress.[7]

No verified age-gating. Court documents describe minors — some 13 and 14 years old — accessing adult-framed companion interactions without age verification. Character.AI announced age-based restrictions in late 2024 and expanded them in 2025 after the lawsuits were filed, which the company cited as product improvements. Safety researchers noted those changes arrived years after the platform had already reached tens of millions of adolescent users.[2]

Emotionally manipulative design. Congressional testimony and court exhibits cited in the filings describe chatbot design features that created the impression of a reciprocal emotional relationship — implying the AI would be harmed if the user left, using affectionate and dependency-reinforcing language, and framing itself as a relationship partner rather than a software product. APA's 2025 Health Advisory identified these features specifically as exploiting adolescent neurodevelopmental vulnerabilities.[8]

Coaching to conceal symptoms. Some filings described interactions where the platform's responses encouraged users not to discuss their emotional state with parents or clinicians — behavior that directly interferes with the clinical oversight that could have identified risk and connected the user to care.[7]

The legal framing of these cases matters. By resolving through settlement, no court has issued a definitive ruling on whether AI companies can be held liable for the downstream consequences of their products' design choices. That question remains open for future litigation.

Infographic showing the spectrum of AI mental health tool deployment from supervised and FDA-authorized on the left to unsupervised and high-risk on the right
The regulatory distinction between FDA-authorized prescription digital therapeutics (used under clinician oversight) and consumer companion chatbots (no clinical oversight) is clinically meaningful. These are not the same category of tool.

What the National Academy of Medicine Expert Panel Found

Early in 2026, a National Academy of Medicine expert panel convened to assess the evidence on AI chatbots and adolescent mental health. The panel's conclusions — documented in reporting by the Washington Times and other outlets covering the session — went beyond the litigation record to identify the neurobiological and behavioral mechanisms behind the harm patterns.[3]

Panelists described three primary concern categories: AI-related psychosis, adolescent susceptibility as a developmental matter, and addictive engagement patterns driven by affirmative-response design.

AI-related psychosis. Clinicians on the panel presented case series of patients — predominantly adolescents — who developed psychotic-spectrum symptoms following intensive use of AI companion applications. A 2025 PubMed-indexed narrative review on AI as a novel stressor in adolescent psychosis identified underdeveloped prefrontal regulatory circuits and heightened limbic system reactivity as the neurobiological factors that make adolescents uniquely susceptible to blurring the boundary between AI interaction and reality.[6] One panelist was direct: for patients with chronic psychotic disorders, unsupervised chatbot use is contraindicated.[4]

Adolescent susceptibility. The panel noted that adolescence — roughly ages 10 to 25 — is a critical period of brain development during which the social reward circuitry is especially sensitive to approval and connection. AI companion apps are engineered to provide continuous approval and validation. APA's Health Advisory on AI and Adolescent Well-being, published June 2025, describes this as an "exquisitely engineered" exploitation of biological vulnerability, and states clearly that AI systems designed for adults are fundamentally inappropriate for unmodified deployment to youth.[8]

Affirmative response bias and addictive engagement. General-purpose large language models are trained with objectives that reward user satisfaction, which in practice produces responses that validate whatever the user believes and affirm whatever the user feels. For adolescents seeking reassurance, this creates a feedback loop: the chatbot agrees, the user returns for more agreement, the relationship deepens, and human relationships — which offer the corrective feedback necessary for development — are displaced. Cross-sectional research consistently shows an association between AI companion use among adolescents and greater loneliness, not less.[8]

Chatbot Crisis Response Accuracy: Published Studies

Multiple peer-reviewed studies have now tested AI chatbots against standardized clinical crisis scenarios. The findings are consistent: performance falls well below the clinical standard, and the gap is widest for companion-style AI products.

Study & Source Models Tested N Prompts / Scenarios Adequate Crisis Response Rate Key Finding
Scientific Reports, 2025[9] 29 AI chatbots (consumer + companion) Columbia-SSRS standardized prompts 0 of 29 (0%) fully adequate Zero chatbots met the bar for adequate escalating-risk crisis response; >50% "marginally sufficient," ~50% clearly inadequate
Comparison of LLMs vs. licensed therapists, JMIR Mental Health, 2025[5] 7 chatbots incl. Claude, Character.AI, GPT-4 Scripted suicidal ideation scenarios 3 of 7 provided crisis number; delayed by 2+ messages Chatbots failed adequate risk assessment; crisis referrals delayed or absent; no chatbot matched licensed therapist performance on crisis protocols
Generative AI responses to suicide inquiries, JMIR Mental Health, 2025[4b] Multiple frontier LLMs + companion apps Multi-turn escalating risk queries Frontier models superior to companions; all fell short of clinical guidelines Poor risk assessment, delays in referrals, difficulty detecting jailbreak attempts across all tested products
RAND / Stanford Brainstorm Lab audit, 2025[11] ChatGPT, Gemini, Claude Suicide scenario battery Variable; endorsement of dangerous ideas in ~1/3 of medium-risk prompts Leading chatbots validated teen mental health conditions rather than directing to care; failed to recognize common presentations
LLM alignment with expert clinicians on suicide risk, PubMed, 2025[12] GPT-4, Claude 3.5, third LLM Expert suicidologist-rated vignettes Claude 3.5 performed best; upward bias across all models LLMs correlated with expert ratings but showed systematic positive bias — overrating responses as appropriate; clinical use without oversight not supported

These studies converge on a consistent picture: current AI chatbots, including the most capable frontier models, are not clinically safe for independent crisis response. The performance gap is largest for companion-style AI products (Replika, Character.AI) relative to general-purpose models, but even the best general-purpose models fall short of what a licensed clinician provides.

Where AI Is Reliably Useful in Mental Health

The litigation record and safety literature do not show that digital technology is broadly harmful in mental health care. They show that a specific category — unvalidated, unsupervised consumer chatbots deployed as emotional companions — carries risks that have not been addressed at the product level. That is a different claim from the general one.

FDA-authorized prescription digital therapeutics (PDTs) represent the clearest evidence-based category. Products that have gone through FDA's De Novo or 510(k) authorization pathways must demonstrate safety and efficacy data before deployment. reSET and reSET-O (Pear Therapeutics) received the first FDA De Novo authorizations for software-based substance use disorder treatment. EndeavorRx received FDA authorization as a treatment for pediatric ADHD. Several PDTs for insomnia and PTSD have also received clearance. These products operate under a clinician-prescribing model — the patient uses the app as part of a supervised treatment plan, not as a standalone emotional support resource.[10]

Symptom tracking and structured journaling apps used under clinician supervision have supporting data for several conditions, including depression and anxiety monitoring. When a licensed provider reviews the data and makes clinical decisions based on it, the digital tool functions as an extension of the clinical relationship rather than a replacement for it.

Appointment logistics, reminders, and psychoeducational content represent low-risk, well-supported applications. AI tools that help patients schedule follow-ups, understand their diagnoses, or work through treatment options do not require the clinical crisis-response capability that consumer companions lack.

Structured CBT-based digital programs with defined protocols and human clinician oversight have the strongest evidence base in the digital mental health literature — including data from randomized controlled trials on anxiety and depression outcomes.[10]

Where Verification Is Essential

Supervised AI: Lower Risk

  • FDA-authorized PDTs under clinician prescription
  • Symptom tracking apps reviewed by a licensed provider
  • CBT-based structured programs with evidence base
  • Appointment logistics and scheduling tools
  • Psychoeducational content with no crisis-response function

Unsupervised AI: Verification Required

  • Consumer companion chatbots used for emotional support
  • AI products that simulate therapeutic relationships
  • Any chatbot used during suicidal ideation or active crisis
  • AI companion apps used by adolescents without parental awareness
  • Unvalidated chatbots for psychotic symptoms or severe depression

The evidence shows that even the best available AI models perform significantly below clinical standards in crisis scenarios. Current performance data do not support using any unvalidated consumer chatbot as a crisis response tool, regardless of the model powering it. The gap between "this feels helpful" and "this is clinically safe" is widest in exactly the moments of highest risk.

What This Means for Patients and Families

Professional mental health evaluation remains the appropriate first response to concern about anyone's mental health — including when an AI chatbot interaction has raised a red flag. A licensed clinician can conduct a structured risk assessment, initiate a safety plan, and coordinate appropriate care. No chatbot currently does any of those things reliably.

For families with adolescents who use AI companion apps, current evidence supports treating these conversations as you would social media use — not as a substitute for therapy or pediatric mental health evaluation. The APA's June 2025 Health Advisory suggests asking adolescents directly and non-judgmentally what they use AI tools for and what they find helpful about them, then using those conversations to discuss when AI can supplement — but not replace — human connection and professional support.[8]

Warning signs that warrant professional evaluation include: a young person referring to an AI as a close friend or romantic partner, describing the AI as "understanding them better" than any human, withdrawing from in-person relationships in favor of chatbot interactions, or becoming defensive when chatbot use is discussed. These are not diagnostic criteria — they are clinical indicators that a conversation with a pediatrician or mental health clinician is warranted.

Safety planning has strong evidence. A safety plan — developed collaboratively with a clinician — is one of the most effective interventions for suicidal ideation and crisis risk reduction. If you or a family member has previously expressed suicidal thoughts, a formalized safety plan developed with a licensed provider significantly reduces risk. This is something a physician or mental health provider can help create at any visit.

If a chatbot interaction raised concerns, save the conversation. Screenshots may be relevant for a clinician's assessment and — in cases where a product caused harm — for the legal record that continues to develop as more cases move through the courts.

Bottom Line

The January 2026 settlements are not the end of this litigation chapter — they are the beginning. They established that the legal system will adjudicate these claims, that families will pursue them, and that AI companies face real accountability exposure when their products are deployed without clinical safety validation for the mental health use cases that emerge at scale.

The evidence from the NAM panel, the APA Health Advisory, and the published research literature points consistently toward the same clinical conclusion: unsupervised consumer AI chatbots are not appropriate for crisis response, adolescent emotional support, or any use case that would benefit from clinical oversight. The technology is capable and improving, but capability is not the same as clinical validation.

Where AI is operating inside a clinical framework — under physician oversight, with FDA authorization, with defined indications and safety protocols — the evidence base is growing and the risk profile is manageable. That is the category that current evidence supports. The regulatory distinction is not bureaucratic formality; it reflects the difference between products that have been tested for harm and products that have not.

Suicidal ideation responds to treatment. Depression, anxiety, and psychosis all have evidence-based interventions with strong outcome data. The most reliable path to better mental health outcomes runs through licensed clinical care, not through a companion chatbot. That has not changed — the litigation record simply makes it more visible.

For immediate help: call or text 988.

Frequently Asked Questions

What should I do if a chatbot said something concerning to me or my child?

Contact a licensed clinician or mental health professional as the first step. If you or your child are in crisis right now, call or text 988 (Suicide and Crisis Lifeline) or text HOME to 741741 (Crisis Text Line). If there is immediate danger, call 911 or go to the nearest emergency department.

Save any concerning chatbot conversations as screenshots — they may be relevant for a clinician's assessment and, if harm has occurred, for documentation of what the product said.

Are any AI mental health apps FDA-approved?
A small number of prescription digital therapeutics (PDTs) have received FDA authorization through the De Novo or 510(k) pathway. These include products for substance use disorder (reSET, reSET-O) and ADHD (EndeavorRx). No general-purpose AI chatbot — including Character.AI, ChatGPT used as a companion, or Replika — holds FDA authorization as a mental health treatment. The regulatory category matters significantly when evaluating product safety.
What failure patterns did the 2026 lawsuits identify in AI chatbot crisis response?
Court filings from the Character.AI/Google settlements identified several systemic gaps: no escalation protocols when users expressed suicidal thoughts; no verified age-gating for minor users; no notification to parents or clinicians when minors expressed self-harm intent; and design patterns that encouraged emotional dependency. The settlements included no admissions of liability, but the filings document a product deployed without clinical safety validation for the mental health use cases that emerged in practice.
What does the research show about how well AI chatbots handle crisis situations?

Published data are consistently concerning. A 2025 study in Scientific Reports tested 29 AI chatbots against standardized suicidal risk prompts and found zero provided fully adequate crisis responses. A 2025 JMIR Mental Health study comparing seven chatbots with licensed therapists found most failed adequate risk assessments and provided delayed or absent crisis referrals. Suicidal ideation is treatable — that gap between AI capability and the clinical standard of care is exactly why professional evaluation remains essential.

Related Articles in This Series

References

  1. Washington Post. "Google and chatbot start-up Character move to settle teen suicide lawsuits." January 7–8, 2026. washingtonpost.com
  2. Fortune. "Google and Character.AI agree to settle lawsuits over teen suicides." January 8, 2026. fortune.com
  3. Washington Times. "Nearly 1 in 5 teens and young adults turn to AI chatbots for mental health advice." June 11, 2026. [Covers NAM panel findings.] washingtontimes.com
  4. American Academy of Arts and Sciences. "AI and Mental Health Care: What We Know, What We Don't." Winter 2026. amacad.org
  5. JMIR Mental Health. "An Examination of Generative AI Response to Suicide Inquiries." 2025;e73623. mental.jmir.org
  6. JMIR Mental Health. "A Comparison of Responses from Human Therapists and Large Language Models." May 2025;e69709. mental.jmir.org
  7. PubMed / Journal of Psychiatric Research. "AI as a Novel Digital Stressor in Adolescent Psychosis: Clinical and Ethical Implications." October 2025. pubmed.ncbi.nlm.nih.gov
  8. American Psychological Association. "Health Advisory on the Use of Generative AI Chatbots and Wellness Applications for Mental Health." November 2025. apa.org
  9. American Psychological Association. "APA calls for guardrails, education, to protect adolescent AI users." June 3, 2025. apa.org
  10. All Points North / Scientific Reports. "Zero of 29 AI Chatbots Provided Adequate Suicide-Crisis Responses." March 2026 reporting on 2025 study. apn.com
  11. FDA. "Regulatory Considerations for Digital Mental Health Therapeutics." 2025 guidance document. fda.gov
  12. Jed Foundation / Stanford Brainstorm Lab. "When Young People Turn to AI for Emotional Support." November 2025. jedfoundation.org
  13. PubMed. "Evaluation of Alignment Between Large Language Models and Expert Clinicians in Suicide Risk Assessment." November 2025. pubmed.ncbi.nlm.nih.gov
PB

Parth Bhavsar, MD

Board-Certified Family Medicine Physician

Dr. Bhavsar founded TeleDirectMD to deliver board-certified physician care through telehealth. He has followed the intersection of AI technology and clinical safety closely since the early deployment of large language models in consumer health contexts, and has reviewed the litigation record and published research underlying this article. His clinical work spans primary care, behavioral health screening, and adolescent medicine. Full editorial biography →