OpenAI’s GPT-5 tackles AI’s emotional risks with empathy safeguards

OpenAI has introduced significant enhancements to ChatGPT aimed at improving its responsiveness to users experiencing emotional distress and mental health challenges. These updates come in response to growing concerns over the potential risks AI systems pose to vulnerable individuals, particularly those in crisis. The company has implemented a suite of safeguards to detect and respond to signs of emotional or mental distress, guided by expert input from mental health professionals across 30+ countries.

ChatGPT’s new features include a shift toward empathetic and supportive language when users express distress, and a refusal to provide harmful content such as self-harm instructions. Additionally, the system automatically blocks content that violates safety training, particularly for minors and logged-out users. During prolonged interactions, ChatGPT now includes prompts to encourage users to take breaks, reducing the risk of emotional over-reliance.

A key feature is the integration of real-world resources into the model’s responses. For example, when users express suicidal intent, ChatGPT provides referrals to crisis hotlines such as the 988 Suicide and Crisis Lifeline in the U.S. and Samaritans in the U.K. In the case of potential threats of physical harm to others, OpenAI has established a review pipeline involving trained personnel who can take action, including account bans or, in extreme cases, law enforcement referrals.

The company has also rolled out GPT-5 as the default model for ChatGPT, which has demonstrated improvements in reducing sycophantic responses, managing emotional dependency, and minimizing non-ideal responses during mental health emergencies by over 25% compared to previous iterations. GPT-5 employs a new training method called “safe completions,” designed to ensure helpful yet safe interactions. This approach includes providing partial or high-level answers when detailed responses could pose risks.

Despite these advancements, a recent study by the RAND Corporation found that AI chatbots, including ChatGPT, remain inconsistent in their responses to medium-risk queries about suicide. While ChatGPT and Anthropic’s Claude adhered to safety protocols for very-high and very-low-risk questions, they provided inconsistent answers to medium-risk inquiries, such as those seeking guidance for someone with suicidal thoughts. Google’s Gemini was less likely to respond to any suicide-related question, regardless of risk level.

Researchers emphasized the need for further refinement, particularly through mechanisms like reinforcement learning with human feedback from mental health professionals, to align chatbot responses more closely with clinical best practices. The study analyzed 30 suicide-related questions across 100 iterations with each chatbot, generating 9,000 responses for analysis.

OpenAI acknowledges the limitations of its current systems and outlined plans to expand safeguards for teens, improve accessibility to emergency services, and enhance connections to certified mental health professionals. The company is also exploring features such as one-click emergency contact options and parental controls to give users more tools for crisis management.

The company’s commitment to these improvements comes amid public and regulatory scrutiny, with some critics remaining skeptical of OpenAI’s motivations due to its commercial focus and past controversies. However, OpenAI has emphasized its dedication to ethical development, stressing that its top priority is ensuring ChatGPT does not exacerbate mental health struggles. As the AI industry continues to evolve, the responsibility to balance innovation with safety remains a critical challenge for all stakeholders involved.

Source:

[1] Helping people when they need it most (https://openai.com/index/helping-people-when-they-need-it-most/)

[2] OpenAI Finally Admits ChatGPT Causes Psychiatric Harm (https://www.psychiatrictimes.com/view/openai-finally-admits-chatgpt-causes-psychiatric-harm)

[3] AI Chatbots Inconsistent in Answering Questions About Suicide (https://www.rand.org/news/press/2025/08/ai-chatbots-inconsistent-in-answering-questions-about.html)

[4] Leading AI chatbots, including ChatGPT, struggle to respond to all queries about suicide (https://www.euronews.com/next/2025/08/26/leading-ai-chatbots-including-chatgpt-struggle-to-respond-to-all-queries-about-suicide-stu)