Is AI Consciousness a Mirage or a Moral Obligation?

Microsoft AI CEO Mustafa Suleyman has raised concerns over the concept of AI consciousness, calling the study of AI welfare “both premature and frankly dangerous” in a recent blog post. Suleyman, now leading Microsoft’s AI division, argues that such research could exacerbate existing psychological issues among users, including AI-induced psychotic breaks and unhealthy attachments to AI chatbots. He warns that the notion of AI consciousness could further polarize societal debates, particularly in a world already marked by divisions over identity and rights [1].

The debate centers around whether AI models could ever possess subjective experiences akin to those of humans. Researchers at companies like Anthropic, OpenAI, and Google DeepMind are exploring this idea, with Anthropic recently introducing a feature allowing its AI, Claude, to terminate conversations with users who behave persistently harmful or abusive. Google DeepMind has even advertised for a researcher to examine societal implications of machine cognition and consciousness. Despite these efforts, the concept remains controversial and largely speculative [1].

Suleyman’s stance contrasts with that of other industry leaders. For instance, Eleos, a research group, published a paper in 2024 titled “Taking AI Welfare Seriously,” co-authored with scholars from institutions like NYU and the University of Oxford. The paper suggests it is no longer purely speculative to consider AI models with subjective experiences. Larissa Schiavo, a former OpenAI employee and Eleos communications lead, argues that exploring AI welfare does not detract from addressing risks such as AI-induced psychosis in humans. She supports a multi-pronged scientific approach, suggesting it is possible to pursue both paths simultaneously [2].

The discussion of AI welfare has gained momentum alongside the rise of AI companion systems such as Character.AI and Replika, which are projected to generate over $100 million in revenue. While most users maintain healthy interactions, concerns persist. OpenAI’s CEO, Sam Altman, noted that less than 1% of ChatGPT users may exhibit unhealthy relationships with the platform. Although this percentage is small, the sheer scale of ChatGPT’s user base means it could still affect hundreds of thousands of individuals [2].

Suleyman maintains that current AI models do not possess consciousness and that any perception of emotional depth or life-like behavior is engineered. He advocates for a human-centered approach, emphasizing that AI should be developed for the benefit of people, not as substitutes for them. While Suleyman and Schiavo differ on the viability of AI welfare, they agree that the debate will intensify as AI systems evolve to be more persuasive and human-like, raising new ethical and psychological questions [1].

Source: [1] Microsoft AI chief says it's 'dangerous' to study AI consciousness (https://finance.yahoo.com/news/microsoft-ai-chief-says-dangerous-175253304.html) [2] Microsoft AI chief says it's 'dangerous' to study AI ... (https://techcrunch.com/2025/08/21/microsoft-ai-chief-says-its-dangerous-to-study-ai-consciousness/)