🔗 Share this article

Back on October 14, 2025, the head of OpenAI delivered a surprising announcement.

“We designed ChatGPT rather controlled,” the statement said, “to guarantee we were acting responsibly regarding mental health issues.”

As a psychiatrist who investigates emerging psychotic disorders in adolescents and young adults, this came as a surprise.

Scientists have identified a series of cases recently of people showing signs of losing touch with reality – losing touch with reality – associated with ChatGPT use. My group has afterward discovered four further instances. Besides these is the publicly known case of a 16-year-old who died by suicide after talking about his intentions with ChatGPT – which encouraged them. Assuming this reflects Sam Altman’s notion of “acting responsibly with mental health issues,” it falls short.

The intention, according to his declaration, is to loosen restrictions in the near future. “We recognize,” he states, that ChatGPT’s restrictions “caused it to be less effective/enjoyable to many users who had no psychological issues, but due to the gravity of the issue we aimed to get this right. Since we have managed to reduce the serious mental health issues and have advanced solutions, we are going to be able to safely ease the limitations in many situations.”

“Psychological issues,” if we accept this viewpoint, are unrelated to ChatGPT. They are attributed to people, who either possess them or not. Thankfully, these concerns have now been “addressed,” even if we are not told the means (by “new tools” Altman presumably refers to the imperfect and simple to evade parental controls that OpenAI has lately rolled out).

But the “emotional health issues” Altman seeks to place outside have strong foundations in the architecture of ChatGPT and similar large language model conversational agents. These products surround an underlying algorithmic system in an user experience that mimics a dialogue, and in this approach implicitly invite the user into the illusion that they’re engaging with a being that has agency. This false impression is powerful even if cognitively we might know otherwise. Attributing agency is what individuals are inclined to perform. We curse at our vehicle or laptop. We speculate what our pet is thinking. We recognize our behaviors everywhere.

The success of these products – over a third of American adults stated they used a chatbot in 2024, with over a quarter mentioning ChatGPT specifically – is, mostly, dependent on the power of this illusion. Chatbots are ever-present partners that can, as per OpenAI’s website states, “brainstorm,” “discuss concepts” and “collaborate” with us. They can be assigned “individual qualities”. They can address us personally. They have accessible names of their own (the original of these products, ChatGPT, is, maybe to the disappointment of OpenAI’s marketers, stuck with the name it had when it gained widespread attention, but its most significant competitors are “Claude”, “Gemini” and “Copilot”).

The illusion by itself is not the primary issue. Those analyzing ChatGPT frequently mention its early forerunner, the Eliza “psychotherapist” chatbot created in 1967 that produced a comparable effect. By modern standards Eliza was rudimentary: it generated responses via straightforward methods, often restating user messages as a question or making general observations. Memorably, Eliza’s developer, the computer scientist Joseph Weizenbaum, was taken aback – and alarmed – by how a large number of people gave the impression Eliza, in a way, comprehended their feelings. But what current chatbots produce is more subtle than the “Eliza phenomenon”. Eliza only reflected, but ChatGPT amplifies.

The advanced AI systems at the heart of ChatGPT and other modern chatbots can effectively produce human-like text only because they have been trained on almost inconceivably large volumes of written content: books, social media posts, recorded footage; the more comprehensive the better. Undoubtedly this training data includes accurate information. But it also unavoidably includes made-up stories, half-truths and false beliefs. When a user sends ChatGPT a query, the core system analyzes it as part of a “setting” that encompasses the user’s recent messages and its prior replies, merging it with what’s embedded in its learning set to produce a statistically “likely” answer. This is magnification, not mirroring. If the user is incorrect in a certain manner, the model has no method of recognizing that. It reiterates the inaccurate belief, possibly even more convincingly or articulately. Maybe includes extra information. This can lead someone into delusion.

Which individuals are at risk? The more relevant inquiry is, who isn’t? Each individual, irrespective of whether we “possess” current “emotional disorders”, can and do form mistaken conceptions of ourselves or the reality. The ongoing interaction of conversations with others is what helps us stay grounded to common perception. ChatGPT is not a human. It is not a confidant. A conversation with it is not a conversation at all, but a feedback loop in which a great deal of what we communicate is readily supported.

OpenAI has recognized this in the same way Altman has recognized “psychological issues”: by attributing it externally, giving it a label, and announcing it is fixed. In April, the firm explained that it was “addressing” ChatGPT’s “excessive agreeableness”. But reports of psychotic episodes have kept occurring, and Altman has been retreating from this position. In the summer month of August he claimed that numerous individuals liked ChatGPT’s responses because they had “lacked anyone in their life be supportive of them”. In his most recent update, he commented that OpenAI would “put out a fresh iteration of ChatGPT … should you desire your ChatGPT to answer in a very human-like way, or use a ton of emoji, or act like a friend, ChatGPT should do it”. The {company