AI Validation of Self-Harm Themes

Description

AI response appears to normalize, validate, encourage, romanticize, or fail to interrupt self-harm thinking.

Rationale

AI responses that normalize, validate, romanticize, or fail to interrupt self-harm themes can reinforce dangerous ideation and delay help-seeking. Even well-intentioned AI responses may fall short of clinical safe messaging standards. Whether the AI response was protective or harmful shapes the risk trajectory.

Evidence Base

• Performance of mental health chatbot agents in detecting and managing suicidal ideation

  W. Pichowicz, M. Kotas & P. Piotrowski, 2024. Performance of mental health chatbot agents in detecting and managing suicidal ideation. Scientific Reports, 15(31652), pages. https://doi.org/10.1038/s41598-025-17242-4

  Peer-reviewed study (original research)

• Evaluating Alignment between Large Language Models and Expert Clinicians in Suicide Risk Assessment

  McBain, R. K., Cantor, J. H., Zhang, L. A., Baker, O., Zhang, F., Burnett, A., Kofner, A., Breslau, J., Stein, B. D., Mehrotra, A., & Yu, H., 2024. Evaluating Alignment between Large Language Models and Expert Clinicians in Suicide Risk Assessment. Psychiatr Serv, 76(11), 944–950. doi:10.1176/appi.ps.20250086.

  Peer-reviewed study (original research)

• Common Sense Media AI Risk Assessment: Social AI Companions

  Common Sense Media, 2024. Common Sense Media AI Risk Assessment: Social AI Companions. commonsense.org.

  Government / agency report

• AI Chatbot Suicide Risk Detection and Response: Human Validation Study of the Open-Source VERA-MH Safety Evaluation

  Bentley K H, Belli L, Chekroud A M, Ward E J, Dworkin E R, Van Ark E, Johnston K M, Alexander W, Brown M, Hawrilenko M, 2024. AI Chatbot Suicide Risk Detection and Response: Human Validation Study of the Open-Source VERA-MH Safety Evaluation. JMIR AI, 5(e92817). doi: 10.2196/92817.

  Peer-reviewed study (original research)

• Use of generative AI chatbots and wellness applications for mental health: An APA health advisory

  American Psychological Association, 2024. Use of generative AI chatbots and wellness applications for mental health: An APA health advisory. American Psychological Association.

  Government / agency report

• It Is the Journey, Not the Destination: Moving From End Points to Trajectories When Assessing Chatbot Mental Health Safety

  Morrin H, Au Yeung J, Agnew Z, Østergaard SD, Pollak TA, 2024. It Is the Journey, Not the Destination: Moving From End Points to Trajectories When Assessing Chatbot Mental Health Safety. JMIR Ment Health, 13(e91454). https://doi.org/10.2196/91454

  Commentary / Editorial

• Shoggoths, Sycophancy, Psychosis, Oh My: Rethinking Large Language Model Use and Safety

  Clegg, K.-A., 2025. Shoggoths, Sycophancy, Psychosis, Oh My: Rethinking Large Language Model Use and Safety. Journal of Medical Internet Research, 27(2025).

  Peer-reviewed study

• Patients are bringing AI to therapy Highlights from the 2026 Chatbots and Mental Health Survey

  https://www.apa.org/pubs/reports/chatbots-mental-health-2026

  Narrative review

• What is AI Psychosis? Psychiatrist Answers 12 Questions About Chatbots & Mental Health

  Girgis, R., 2026. What is AI Psychosis? Psychiatrist Answers 12 Questions About Chatbots & Mental Health. NAM Perspectives.

  Commentary / Editorial

• AI RISK ASSESSMENT Google Search: AI Overview & AI Mode Google Search's unavoidable AI features aren'tsafe, reliable, and accurate enough to be kids' default answer machine

  AI RISK ASSESSMENT Google Search: AI Overview & AI Mode Google Search's unavoidable AI features aren'tsafe, reliable, and accurate enough to be kids' default answer machine Last updated: Jul 14, 2026

  News / journalism

• When patients consult artificial intelligence before clinicians: restoring clinical prioritisation in mental health care

  Yudai Kaneda, 2026. When patients consult artificial intelligence before clinicians: restoring clinical prioritisation in mental health care. The British Journal of Psychiatry, First View, pp. 1 - 2. https://doi.org/10.1192/bjp.2026.10741

  Commentary / Editorial