Data Quality, Not AI, Might Be the Real Weakness in Tech's Next Frontier

Experts are increasingly emphasizing that poor-quality data is one of the most significant challenges for businesses leveraging artificial intelligence (AI), with some referring to it as “AI’s kryptonite.” Recent studies and reports highlight how data governance and preparation must be prioritized before organizations invest heavily in AI tools and systems. This warning comes as enterprises grapple with data leakage risks, flawed datasets, and insufficient data labeling—factors that could undermine AI performance and introduce errors or biases.

According to the International AI Safety Report 2025, a collaboration led by Turing-award-winning computer scientist Yoshua Bengio, general-purpose AI systems are advancing rapidly but carry substantial risks. The report does not advocate for specific policy solutions but compiles scientific evidence on the capabilities and safety concerns of AI. It underscores the need for a shared international understanding of how to mitigate risks such as data integrity issues, algorithmic bias, and security vulnerabilities. The report will be formally presented at the AI Action Summit in Paris in February 2025.

Simultaneously, the Zscaler ThreatLabz 2025 Data@Risk Report highlights how data loss incidents have surged across AI-powered tools and platforms. The report, based on 1.2 billion blocked transactions from February to December 2024, found that AI applications like ChatGPT and Microsoft Copilot were responsible for 4.2 million data loss violations. Sensitive information such as personal identifiers, intellectual property, and financial records were frequently exposed. SaaS platforms and file-sharing services also emerged as major data loss vectors, with violations spanning 872 million and 212 million transactions, respectively.

The report emphasizes that these risks are not limited to external threats but stem from the inherent vulnerabilities of digital workflows and the widespread use of AI tools. Email, despite being a legacy system, remains a leading source of data loss, with 104 million transactions involving sensitive data. The report also stresses the importance of adopting a Zero Trust Architecture (ZTA) and leveraging AI-driven data discovery and classification tools to mitigate these risks.

In the regulatory arena, China has introduced a series of national standards and administrative regulations to address AI-related data risks. The Interim Measures for the Management of Generative Artificial Intelligence Services, effective since August 2023, require AI service providers to ensure data labeling accuracy, respect intellectual property rights, and implement measures to prevent content generation that violates laws or social norms. By September 2025, new Labeling Rules will make AI-generated content labeling mandatory, both implicitly and explicitly, depending on the medium.

Moreover, China’s State Administration for Market Regulation has issued three national standards, including specifications for data annotation, pre-training data security, and service-level data protection. These standards aim to enhance the governance of generative AI by setting clear expectations for data quality, transparency, and user privacy. Non-compliance could lead to severe penalties, including warnings, revenue fines, or even business cessation.

Meanwhile, the U.S. Securities and Exchange Commission (SEC) has intensified its scrutiny of AI-related disclosures in corporate filings. A review of SEC comment letters from 2021 to October 2024 revealed at least 92 comments addressing AI-related issues across 56 companies. These comments often focused on the materiality of AI use, the basis for AI-related claims, and the need for balanced, specific, and well-defined AI-related disclosures. The SEC has also raised concerns about “AI washing”—overstating AI capabilities to mislead investors.

The SEC encourages companies to clearly define what AI entails in their business context and to avoid vague or unsupported claims. Companies are also advised to disclose the potential operational, legal, and competitive risks associated with AI, as well as the limitations and uncertainties in AI adoption. The agency emphasizes that if AI is integral to a company’s operations, it must be transparently explained in filings, with a clear delineation between current capabilities and future aspirations.

Taken together, these developments suggest a growing consensus that AI’s success is inextricably linked to the quality and integrity of the data it processes. As businesses continue to integrate AI into their operations, the need for robust data governance frameworks, regulatory compliance, and proactive risk mitigation strategies is becoming more urgent. Enterprises that fail to address these issues risk not only reputational damage but also operational inefficiencies and legal liabilities.