D12 — Data Reliability, Ethics, and Assessment Limits
Source Reliability
Description
Information comes from direct interview, witness report, chat logs, forensic review, screenshots, public posts, third-party summary, or unknown source.
Rationale
The source of information about AI interactions directly determines how much confidence the evaluator can place in the assessment. Direct chat logs obtained with lawful authority are qualitatively different from third-party summaries or self-report. Documenting source type is necessary for calibrating confidence and identifying what additional information is needed.
Evidence Base
Characterizing Delusional Spirals through Human-LLM Chat Logs
Moore J, Mehta A, Agnew W, Anthis JR, Louie R, Mai Y, Yin P, Cheng M, Paech SJ, Klyman K, Chancellor S, Lin E, Haber N, Ong D, 2024. Characterizing Delusional Spirals through Human-LLM Chat Logs. arXiv, 2603.16567v1.
Peer-reviewed study (original research)“God has helped us, and so will AI”: How the Terrorist Group Boko Haram Uses Frontier AI
Antonia Juelich, 2026. “God has helped us, and so will AI”: How the Terrorist Group Boko Haram Uses Frontier AI. Frontier AI Working Paper Series, No. 1/2026.
AI company report