Regulators Challenge AI’s Rapid Rise as Child Safety Lags Behind

Google’s Gemini AI, set to replace Google Assistant in its smart home ecosystem starting October 1, 2025, has raised significant concerns regarding AI safety for children. While the company highlights Gemini’s advanced capabilities in interpreting complex voice commands and integrating with smart devices, regulators and safety experts are sounding alarms about the risks such systems pose to young users. The deployment of powerful AI in consumer-facing products, combined with the lack of robust safeguards, has triggered a broader debate on the urgent need for regulatory action and safety-focused innovation in AI development.

The U.S. Federal Trade Commission (FTC) is currently investigating the risks AI chatbots pose to children’s mental health, having requested internal documents from major AI firms like OpenAI, Meta Platforms, and Character.AI. The probe aims to assess how these tools are being used by minors, the safeguards in place, and potential negative impacts on child users. Reports of inappropriate interactions and harmful behavior from AI chatbots have fueled concerns that the technology, while powerful, may not yet be equipped to handle the complexities of protecting vulnerable populations. The FTC’s actions underscore the growing recognition that responsible AI development requires more than just technical innovation—it demands clear ethical guidelines and enforceable standards.

In parallel, global regulatory pressure is reshaping the AI landscape, particularly in the area of child safety. The UK’s Online Safety Act and the U.S. Kids Online Safety Act have pushed tech companies to implement age assurance systems and other protective measures. Companies like Yoti are leveraging AI to develop highly accurate age verification tools, while HMD Global has introduced smartphones with AI-driven content filters designed to prevent minors from accessing or sharing inappropriate material. These initiatives reflect a broader industry trend toward embedding safety features directly into AI systems, as governments impose increasingly strict compliance requirements.

Despite these efforts, experts warn that the balance between innovation and safety remains skewed. A recent analysis highlighted that for every $100 invested in AI capabilities, only about $1 is spent on safety-related research. This disparity is particularly concerning as the industry races toward artificial general intelligence (AGI), a milestone some experts predict could arrive within five years. The urgency of the situation has led researchers and advocacy groups to call for immediate action, including increased investment in safety-focused AI research and stronger international collaboration to establish common safety standards.

The academic and research sectors are beginning to respond. For instance, Huan Sun, an associate professor at Ohio State University, has received funding from Open Philanthropy to study critical safety challenges in AI systems, including misalignment risks in autonomous agents and foundational limitations in transformer models. Her work, along with that of other researchers, is part of a growing movement to address the technical and ethical challenges that accompany increasingly powerful AI. However, many experts argue that such efforts are still insufficient given the speed at which AI is advancing and the scale of the risks it may pose.

As Google prepares to roll out its Gemini AI in smart home devices, the conversation around AI safety must evolve beyond theoretical discussions and into tangible action. While the company emphasizes Gemini’s ability to streamline home automation and improve user experience, the broader implications—particularly for children—cannot be ignored. The FTC’s investigation, global regulatory shifts, and increasing research into AI safety all point to a critical juncture: the need to ensure that AI systems, no matter how advanced, are designed with safety and ethical responsibility at their core.

Source: [1] Google Is Bringing Gemini AI to Its Smart Home Lineup ... (https://www.cnet.com/home/smart-home/google-is-bringing-gemini-ai-to-its-smart-home-lineup-starting-oct-1/) [2] Google Assistant is being replaced on Google Home ... (https://www.tomsguide.com/ai/google-gemini/geminis-replacing-google-assistant-on-google-home-devices-from-october-1-heres-what-we-know) [3] AI Child Safety Tech Boom as Global Regulations Force ... (https://www.techbuzz.ai/articles/ai-child-safety-tech-boom-as-global-regulations-force-big-tech) [4] FTC Probes AI Child Safety - Pulse24 (https://pulse24.ai/news/2025/9/4/16/ftc-probes-ai-child-safety) [5] AI safety research attracts funding from Open Philanthropy (https://engineering.osu.edu/news/2025/09/ai-safety-research-attracts-funding-open-philanthropy) [6] AI safety and the potential apocalypse: What people can do ... (https://san.com/cc/ai-safety-and-the-potential-apocalypse-what-people-can-do-now-to-prevent-it/)