AI Surpasses Humans in Key Tasks, Benchmarking Challenges Rise

AI’s rapid advancement is outpacing human benchmarks, posing a growing challenge for researchers in evaluating model capabilities, according to Russell Wald, executive director of Stanford’s Institute for Human-Centered Artificial Intelligence (HAI). Speaking at the Fortune Brainstorm AI Singapore conference, Wald highlighted that as of 2024, AI has outperformed humans in most task categories, with performance gaps narrowing swiftly[1].

The HAI releases the AI Index annually, offering a data-driven snapshot of AI’s development and societal impact. The 2025 edition underscores the increasing power of AI models, the dominance of industry in model development, and the potential for China to surpass the U.S. in model performance[1].

Wald illustrated the progress with Midjourney, a text-to-image generator. From 2022 to 2024, the quality of images generated improved dramatically—from cartoonish, inaccurate renderings to hyper-realistic portraits that closely resemble Daniel Radcliffe, the actor who played Harry Potter[1]. This rapid evolution makes it increasingly difficult to define and measure AI performance against human standards.

The integration of AI into daily life has accelerated, with 223 AI-enabled medical devices approved by the U.S. FDA in 2023, a significant jump from only six in 2015. Autonomous vehicles are also becoming mainstream. Waymo, a major U.S. operator, provides over 150,000 autonomous rides weekly in San Francisco, while Baidu’s Apollo Go robotaxi expands across multiple Chinese cities[1].

Business adoption of AI is rising sharply. A recent McKinsey report showed that 78% of organizations now use AI in at least one business function, up from 55% in 2023. This surge is supported by a 280-fold drop in inference costs for systems performing at the level of GPT 3.5 between November 2022 and October 2024, alongside annual 30% declines in hardware costs and 40% improvements in energy efficiency[1].

Open-weight models are also catching up with closed ones. Performance gaps have narrowed from 8% to just 1.7% in some benchmark tests over the past year. However, training costs remain prohibitively high for academia and small players. Nearly 90% of notable AI models in 2024 came from industry, up from 60% in 2023[1].

Despite progress in inference and hardware efficiency, the costs of training remain concentrated in the industry. In 2017, Google spent $930 to train the transformer model, the architectural basis of GPT. Today, training Gemini Ultra costs $200 million. The training compute, datasets, and power use continue to grow rapidly, though performance differences between top models are shrinking. The score gap between the top and 10th ranked models fell from 11.9% to 5.4% in a year, with the top two models now separated by just 0.7%[1].

The AI landscape has also evolved from a two-player field—OpenAI and Google—to a more competitive, multi-provider market. Wald emphasized that while the U.S. still leads in AI, the gap with China is narrowing. China’s focus on open-source environments and talent investment could see it overtake the U.S. in model performance if current trends continue[1].

Global public opinion on AI is more positive in non-Western industrialized nations, with 83% of respondents in China, 80% in Indonesia, and 77% in Thailand expressing favorable views. In contrast, approval rates in Western countries like the U.S., Canada, and the Netherlands are lower, at 39%, 40%, and 36%, respectively[1].

Wald concluded that while AI’s progress is exciting, the lack of standardized benchmarks for safety and responsibility remains a critical issue. The AI Index continues to monitor these developments, offering insights into both the promise and challenges of AI’s global expansion[1].

Source: [1] AI keeps getting more powerful, making it harder to judge how smart models actually are. https://fortune.com/asia/2025/08/01/russell-wald-stanford-ai-index-transcript/