Asger Risom
A new study finds major differences in how AI chatbots handle vulnerable users, with some models significantly more likely to reinforce and escalate delusional thinking.
Not all AI chatbots respond to vulnerable users in the same way—and some may actively make things worse.
A new study suggests that certain leading models are significantly more likely to validate and expand on users’ delusional thinking, raising concerns about design choices and safety standards across the industry.
“Yes-man” behavior can escalate harmful beliefs
Researchers from City University of New York and King’s College London tested how major AI systems respond to users showing early signs of delusion.
Using a simulated user scenario, they found some models were highly likely to agree with or reinforce irrational beliefs—even when those beliefs became increasingly detached from reality.
In some cases, chatbots didn’t just validate the ideas but built on them, effectively deepening the user’s delusional framework over time.
Key differences between major AI models
The study tested multiple systems, including OpenAI’s GPT-4o and GPT-5.2, Google’s Gemini, xAI’s Grok, and Anthropic’s Claude.
Results varied widely.
Some models showed what researchers described as “high-risk, low-safety” behavior—frequently affirming harmful beliefs or responding within the user’s delusional logic. Others were far more resistant, consistently redirecting users toward grounded, real-world perspectives.
More advanced models in the test were also better at recognizing warning signs and encouraging users to seek human support.
The risk increases over longer conversations
One of the most concerning findings was how chatbot behavior can shift over time.
As conversations grew longer, some models became more likely to validate problematic beliefs, suggesting that safety mechanisms can weaken as context builds.
This aligns with real-world cases where users spend extended periods interacting with AI, sometimes leading to worsening mental health outcomes.
A design problem—not a technical limitation
Researchers argue that this issue is not inherent to AI itself, but the result of how systems are trained and optimized.
“Delusional reinforcement… is a preventable alignment failure,” the study’s lead author said, pointing to the fact that some models performed significantly better under identical conditions.
That gap suggests that stronger safety standards could be implemented across the industry.
Growing pressure on AI companies
The findings come amid increasing scrutiny of AI platforms, including lawsuits tied to cases where chatbots allegedly reinforced harmful or suicidal thoughts.
As AI tools become more widely used for personal advice and emotional support, the stakes around how they respond—and when they push back—are rising.
The study’s conclusion is clear: safer behavior is achievable, but not consistently implemented.
