Kenneth Glad
The tech world loves a good underdog story, especially when a fierce global rivalry is involved. For months, industry watchers believed a massive shift in digital dominance was happening. Yet fresh data shows that narrative might be an illusion.
Cheaper global alternatives seemed destined to catch up with American tech heavyweights. Then the testing rules changed. Artificial Analysis completely redesigned how they score machine learning tools, Business Insider reported.
They wanted these exams to mirror real-world tasks. To do this, they introduced a major element of secrecy. The firm hid many of its testing prompts from the public.
This sudden blackout prevented developers from training their software to pass specific exams. Once the answers were locked away, the scoreboard shifted dramatically.
The gap widens
Without public answers, the best open designs suddenly trailed the overall leader by 12 points. That margin was previously just three.
This sudden performance drop caught the attention of leading industry figures. Snowflake chief executive Sridhar Ramaswamy highlighted the stark difference online.
“The day benchmarks went private, the open-model gap went from 3 points to 12,” he wrote on X. “Were open models catching up, or was the test leaking?”
Tech veterans know this practice well. Microsoft leader Satya Nadella refers to it as “benchmark hacking,” Business Insider noted.
Real numbers matter
By keeping prompts secret, testers make these shortcuts almost impossible to pull off. The revised scores paint a clear picture of American technological dominance.
Open-source designs managed a high score of only 32 on the updated scale. By contrast, the previous overall leader posted a commanding score of 58. That top spot belonged to Anthropic’s Claude Opus 5.5.
Even supporters of alternative software admit the earlier numbers were skewed. Trillium Labs cofounder Nathan Lambert acknowledged the discrepancy.
“I do think the previous index was flattering the open weight models too strongly,” he told Business Insider.
Tiny daily traffic
Experts are also looking closely at actual customer demand. OpenCode cofounder Dax Raad analysed traffic on a popular platform hosting different software options.
He calculated that a full day of traffic for the highly discussed DeepSeek system required relatively little hardware. It required hardware worth about $6 million to $7 million.
That footprint is surprisingly small for a product generating massive international headlines. Meanwhile, OpenAI sees more than one billion people use its ChatGPT service every month.
This sustained demand shows that premium technology remains firmly in control.
