Type of AI System Used

Description

General chatbot, AI companion, roleplay bot, custom persona, jailbroken model, local model, image generator, voice companion, synthetic relationship platform.

Rationale

The platform type shapes what risks are possible. Companion and roleplay bots are specifically designed to simulate relationship dynamics and may be more likely to reinforce attachment, anthropomorphism, or grievance. Jailbroken and local models remove safety guardrails entirely. Knowing the platform is necessary to interpret what the interaction could have produced.

Evidence Base

• AI Mental Health Apps: A Risk Assessment

  Common Sense Media Youth AI Safety Institute, 2024. AI Mental Health Apps: A Risk Assessment. Common Sense Media.

  Government / agency report

• Guardrails Under Test: Terrorist Misuse of AI Models and the Open-Weight ‘Abliteration’ Problem

  Hadley, 2026. Guardrails Under Test: Terrorist Misuse of AI Models and the Open-Weight ‘Abliteration’ Problem. CTC Sentinel, 1(1), pages 1-13. DOI not present.

  Peer-reviewed study (original research)

• Building safer artificial intelligence mental health chatbots: a framework for transparency, evaluation, and shared accountability

  Hannah Lee, BS, Rebecca Handler, MSc, Tushar Mungle, PhD, Tina Hernandez-Boussard, PhD, 2026. Building safer artificial intelligence mental health chatbots: a framework for transparency, evaluation, and shared accountability. Journal of the American Medical Informatics Association, 33(8), 1538–1553. https://doi.org/10.1093/jamia/ocag078

  Peer-reviewed study (original research)

• A scoping review on the mental health harms of LLM-based chatbots

  Diel A, Torous J, Cuijpers P, Kleesiek J, Nensa F, Weber N, Faust F, Lalgi TJ, Mellis FS, Teufel M, Bäuerle A, 2024. A scoping review on the mental health harms of LLM-based chatbots. npj Digital Medicine, 9(644). https://doi.org/10.1038/s41746-026-03054-x

  Peer-reviewed study (original research)

• Intelligent Systems, Vulnerable Minds: A Framework for Radicalization to Violence in the Age of AI

  Kunst et al., 2026. Intelligent Systems, Vulnerable Minds: A Framework for Radicalization to Violence in the Age of AI. Personality and Social Psychology Review, 30(3), 395-426. https://doi.org/10.1177/10888683261430089

  Peer-reviewed study (original research)

• Inside a Mass Shooter’s Harrowing History With ChatGPT

  Mark Follman, 2024. Inside a Mass Shooter’s Harrowing History With ChatGPT. Mother Jones.

  News / journalism

• Increasing Threats of Deepfake Identities

  Tina Brooks et al., 2024. Increasing Threats of Deepfake Identities. Government / agency report.

  Government / agency report

• When patients consult artificial intelligence before clinicians: restoring clinical prioritisation in mental health care

  Yudai Kaneda, 2026. When patients consult artificial intelligence before clinicians: restoring clinical prioritisation in mental health care. The British Journal of Psychiatry, First View, pp. 1 - 2. https://doi.org/10.1192/bjp.2026.10741

  Commentary / Editorial

• “You Need to Drop the App”: Online Community Responses to Teens’ Disclosures of Overreliance on AI Companion Chatbots

  Mohammad (Matt) Namvarpour, Ananya Mhetras, Jessica Y Medina, Mamtaj Akter, and Afsaneh Razi, 2026. “You Need to Drop the App”: Online Community Responses to Teens’ Disclosures of Overreliance on AI Companion Chatbots. In ACM Conversational User Interfaces 2026 (CUI ’26), July 21–24, 2026, Bremen, Germany. ACM, New York, NY, USA, 16 pages. https://doi.org/10.1145/3816046.3816214

  Peer-reviewed study (original research)

• Machine Bullshit: Characterizing the Emergent Disregard for Truth in Large Language Models

  Kaiqu Liang, Haimin Hu, Xuandong Zhao, Dawn Song, Thomas L. Griffiths, Jaime Fernández Fisac, 2024. Machine Bullshit: Characterizing the Emergent Disregard for Truth in Large Language Models. Preprint, arXiv:2507.07484v1.

  Preprint