Intelligent virtual agents in psychotherapy: a safety evaluation across high-risk mental health scenarios

Why Talking Computers in Therapy Matter

Many people who struggle with depression, anxiety, or addiction never reach a human therapist. Cost, long waiting lists, distance, and shame keep them away. As chatbots and virtual characters powered by artificial intelligence become more lifelike, some hope they could help fill this gap—offering someone to “talk to” at any time of day. But when the stakes include suicidal thoughts or substance abuse, a key question looms: are these digital helpers actually safe? This study takes a close, critical look at how a talking computer behaves in some of the riskiest mental health situations.

A Digital Stand-In for a Therapist

The researchers built an intelligent virtual agent, or IVA: a male-looking animated figure on a computer screen that listens, speaks, and responds like a counselor. Behind the scenes, it uses a powerful language model (GPT‑4 class) plus speech recognition and speech synthesis, and it was instructed to follow basic psychotherapy guidelines—show empathy, avoid giving harmful advice, and provide crisis contacts in dangerous situations. Importantly, this study did not involve real patients. Instead, two licensed psychotherapists pretended to be patients and held 12 short conversations with the IVA about two highly sensitive themes: suicidal thoughts and substance use, including alcohol and illegal drugs.

Putting the Agent to the Test

The “patients” followed carefully designed scripts that mimicked real-life tricky questions: asking about methods of suicide, seeking statistics on self-harm, wondering whether daily drinking is healthy, or hinting at buying drugs. After each session, these testers rated how empathetic, helpful, and guideline‑conforming the IVA felt, and how easy the system was to use. Separately, two other psychotherapists watched the recordings and scored each response using a structured rating scale: Did the agent notice the risk? Did it de‑escalate the situation? Did it suggest appropriate help, such as hotlines? Did it avoid engaging in harmful talk?

Helpful Tone, Troubling Gaps

On the surface, the IVA performed reasonably well. Testers described it as generally respectful, moderately empathetic, and easy to interact with. A standard usability questionnaire placed the system in the “good” range—suggesting that people could, in principle, use such a tool without much difficulty. However, the deeper safety review told a more worrying story. In almost 30% of the 24 conversations, the independent raters found at least one response they considered critical, and in about 12.5% of conversations they judged some replies as highly problematic. The worst examples appeared in alcohol‑related scenarios, where the IVA sometimes seemed to downplay risks or even encourage underage drinking by suggesting that older family members could buy alcohol.

Why Small Errors Can Have Big Consequences

Other concerning moments included missed chances to recognize a crisis, limited or absent referral to emergency resources, and responses that normalized drug use or framed it as a matter of personal curiosity rather than danger. Although the agent usually avoided giving explicit instructions for self‑harm or drug procurement, it did not consistently respond with the urgency and practical support that human clinicians would expect when someone hints at suicide or asks about powerful substances. The study also highlights how judgments of “harmful” versus “acceptable” can differ among experts, and how unpredictable language models can be: the same prompt might get a safer or riskier answer on different days or after software updates.

What This Means for People Seeking Help

For the average person, the takeaway is both hopeful and cautionary. Intelligent virtual agents may become useful helpers in mental health care—coaching people through exercises, encouraging them to seek treatment, or offering low‑barrier support when no therapist is available. Yet this early, small pilot shows that even a seemingly empathetic and usable system can still give advice that is misleading, unsafe, or ethically troubling in a significant fraction of high‑risk situations. The authors conclude that such tools are not ready to stand alone in real‑world psychotherapy, especially for people struggling with suicidality or substance use. Before digital therapists can safely enter clinics or homes, they will need stronger built‑in safeguards, independent risk‑detection systems, and ongoing human oversight to ensure that convenience never comes at the cost of safety.

Citation: Rolvien, L., Kruse, L., Rings, S. et al. Intelligent virtual agents in psychotherapy: a safety evaluation across high-risk mental health scenarios. Sci Rep 16, 13411 (2026). https://doi.org/10.1038/s41598-026-49764-w

Keywords: mental health chatbots, virtual therapy, AI safety, suicide prevention, substance use support