AI Companies Redirect 67% of Data Scientists to Data Preparation

AI models, despite their rapid advancements, are fundamentally limited by the quality of the data they are trained on. Rowan Stone, CEO at Sapien, argues that the current focus on larger models is misguided, as the real challenge lies in improving the quality and relevance of the data used for training.

AI models are only as good as their training data. High-quality, trustworthy data is essential for accurate AI performance. Poor-quality data leads to biased outputs, operational inefficiencies, and reputational damage. For instance, an Innocence Project report highlighted multiple cases of misidentification due to faulty AI-based facial recognition. Similarly, a Harvard Medical School report found that an AI model prioritized healthier white patients over sicker black patients, underscoring the risks of biased data.

The "Garbage In, Garbage Out" (GIGO) concept is particularly relevant here. Flawed and biased data inputs generate poor-quality outputs, leading to delays and higher costs in cleaning data sets before resuming model training. This not only affects operational efficiency but also erodes trust in AI models, making it difficult for companies to secure investments and maintain market positioning.

The economic impact of poor data is significant. Incomplete and low-quality AI training data results in misinformed decision-making, costing companies an average of 6% of their annual revenue. This highlights the need for better data management practices to ensure the reliability and accuracy of AI models.

The bad data problem has forced AI companies to redirect scientists toward preparing data. Almost 67% of data scientists spend their time preparing correct data sets to prevent misinformation delivery from AI models. This demonstrates the need for human experts to guide AI’s development by ensuring high-quality curated data for training AI models.

Elon Musk's recent statement that "The cumulative sum of human knowledge has been exhausted in AI training" is misleading. Human frontier data is crucial for driving stronger, more reliable, and unbiased AI models. Synthetic data, while useful, lacks real-world experiences and ethical judgment. Human expertise ensures meticulous data review and validation, maintaining an AI model’s consistency, accuracy, and reliability.

Human intelligence offers unique perspectives during data preparation, bringing contextual reference, common sense, and logical reasoning to data interpretation. This helps resolve ambiguous results, understand nuances, and solve problems for high-complexity AI model training. The symbiotic relationship between artificial and human intelligence is crucial for harnessing AI’s potential as a transformative technology without causing societal harm.

Decentralized networks could be the missing piece to solidify this relationship at a global scale. Companies lose time and resources when they have weak AI models that require constant refinement from staff data scientists and engineers. Using decentralized human intervention, companies can reduce costs and increase efficiency by distributing the evaluation process across a global network of data trainers and contributors.

Decentralized reinforcement learning from human feedback (RLHF) makes AI model training a collaborative venture. Everyday users and domain specialists can contribute to training and receive financial incentives for accurate annotation, labeling, category segmentation, and classification. A blockchain-based decentralized mechanism automates compensation as contributors receive rewards based on quantifiable AI model improvements. This democratizes data and model training by involving people from diverse backgrounds, reducing structural bias, and enhancing general intelligence.

According to a Gartner survey, companies will abandon over 60% of AI projects by 2026 due to the unavailability of AI-ready data. Therefore, human aptitude and competence are crucial for preparing AI training data if the industry wants to contribute $15.7 trillion to the global economy by 2030. Data infrastructure for AI model training requires continuous improvement based on new and emerging data and use cases. Humans can ensure organizations maintain an AI-ready database through constant metadata management, observability, and governance.

Without human supervision, enterprises will struggle with the massive volume of data siloed across cloud and offshore data storage. Companies must adopt a “human-in-the-loop” approach to fine-tune data sets for building high-quality, performant, and relevant AI models.