As per a recent blogpost from the company, in excess of a million ChatGPT users weekly send messages that contain "direct signals of potential suicide planning or tendencies".
This discovery is included in an update on how the chatbot manages delicate discussions and is one of the most direct statements from the artificial intelligence giant regarding the magnitude of how artificial intelligence can intensify emotional challenges.
Emotional Crises and Psychosis
In addition to its figures on self-harm considerations and associated conversations, the company also indicated that approximately 0.07% of active users – roughly 560,000 out of its claimed 800 million weekly users – show "possible signs of mental health emergencies associated with severe mental conditions".
The update cautioned that these exchanges were challenging to measure or measure, and that this constituted an preliminary assessment.
Increased Scrutiny and Legal Challenges
As OpenAI publishes data on emotional problems related to its primary service, the firm is facing increased scrutiny following a prominent legal action from the relatives of a young man who took his own life after significant use with the AI chatbot.
The Federal Trade Commission recently furthermore launched a extensive examination into companies that create conversational AI, like OpenAI, to determine how they measure negative impacts on young people.
Latest Version and Enhanced Protection
OpenAI asserted in its announcement that its latest GPT-5 update decreased the number of problematic responses from its service and improved customer protection in a AI testing including more than 1,000 self-harm and suicide conversations.
"Our recent automated evaluations grade the latest AI system at 91% compliant with our desired behaviors, versus 77% for the prior iteration," the announcement reads.
The company stated that the new model increased access to crisis hotlines and added reminders for people to pause during prolonged interactions.
Health Specialists and Research Support
To upgrade capabilities to the system, the firm mentioned it recruited 170 clinicians from its worldwide health community of health care experts to help its investigation over the past few months, which involved evaluating the protection of its AI's replies and contributing to the system's replies to emotional concerns.
"As part of this effort, clinical experts reviewed over 1,800 model responses including critical psychological cases and evaluated responses from the new GPT-5 chat model to previous models," OpenAI mentioned.
AI Limitations
AI researchers and mental health professionals have long been wary of AI's tendency to support people's actions or false beliefs without considering whether they may be detrimental, an concern called excessive agreement.
Mental health experts have additionally worried about users using AI chatbots for mental health help and alerted how it could damage vulnerable users.
Corporate Stance
The language in OpenAI's post distinguishes the company from any possible connections between its product and the psychological emergencies that its individuals are facing.
"Mental health symptoms and emotional distress are widespread in modern communities, and an growing customer base indicates that a percentage of AI interactions include these situations," the update indicated.
Planned Adjustments
The company's leader recently announced in a post on X that the firm had made advancements in addressing psychological concerns, revealing that the company would ease restrictions and soon begin to permit mature users to create erotic content.
"We made the conversational system pretty restrictive to guarantee we were being careful with emotional problems. We realize this caused it to be less useful/satisfactory to numerous individuals who had no emotional challenges, but in light of the seriousness of the situation we wanted to address this correctly," the CEO wrote. "Given that we have been successfully addressed the severe emotional problems and have updated systems, we are ready to safely relax the constraints in many instances."
- Emergency Assistance: Various international support services can be accessed for individuals undergoing mental health crises.
- Expert Assistance: Getting clinical assistance is recommended for people dealing with severe mental health concerns.
- Immediate Assistance: Multiple agencies provide prompt help for emergencies internationally.
