The Best AI Coder on the Leaderboard Is Only the Best at Python
There's a single number meant to tell you which AI to trust with your code. It's been quietly measuring the wrong thing — and the gap is wide enough to pick the wrong tool.
`★ Insight ─────────────────────────────────────` The piece is clean — only two phrases trip the hushed-register rule. Both are assertions of subtlety the prose doesn't need: "almost insultingly simple" (the hedge softens a punch that lands harder plain) and "quietly handing the crown" (the dek already owns the one permitted "quietly"; a second recurrence is the tell itself). `─────────────────────────────────────────────────`
There is a number that is supposed to end the argument. You want an artificial intelligence to help write your software, so you open a leaderboard, find the system with the highest coding score, and trust it. That is what the score is for — to spare you the guessing. The trouble is hidden in what the score actually counts. It counts one language. Python. Every leading AI has spent the last few years racing to ace the same single-subject exam, while the certificate on the wall says it passed all of them.
The exam is real, and widely cited. The dominant benchmark for judging machine-written code draws its problems from programming contests, and those problems are posed in Python. Researchers quote the resulting scores as a general verdict on coding skill. But Python sits inside these systems the way English sits inside a translation app: it is simply everywhere. A model learns from the public internet, and the internet is thick with Python, so the model swims in it. Score it on Python and you may not be measuring a good coder at all. You may be measuring a very good Python student. It is like ranking a translator on her French alone, in a world where most books happen to be written in French.
Then a group of researchers took the mask off. They did something insultingly simple: they took the exact same contest problems and rewrote them in eleven more languages — the same puzzles, no harder, just posed in Go or Rust or C# instead of Python. When today's strongest models ran the full set, the leaderboard came apart. Qwen3-235B, one of the largest systems available, scores 74 percent on the Python version. On Go it drops to 57. On C# it falls below half. Its Python polish, it turned out, did not travel.
The sharpest result is a reversal. Take a second model, GPT-OSS-120B, a middleweight — smaller, and genuinely worse at Python, scoring 71 to Qwen's 74. By the only number anyone had been watching, the big model wins, no contest. But across six other languages — Go, JavaScript, TypeScript, Rust, Ruby, Kotlin — the smaller model beats it. One model was better at the test. The other was better at coding. These are not the same thing, and the old benchmark had no way to tell them apart. It had been handing the crown to good test-taking the whole time.
A fairer ruler is not the same as a straight one. The new benchmark still builds itself from contest puzzles — tidy problems with clean inputs and a single right answer, graded on whether the program runs. Real software is messier than that. Wiring up a payment system, chasing a memory leak, untangling someone else's decade-old code: none of it looks like a contest. So we have learned something true and unsettling — these systems are far less universal than their headline number suggests — and we have learned how little that headline number was ever telling us. How they handle the actual work is still sitting there, unscored.
The most pressing question left unanswered is whether the performance variations observed on Multi‑LCB translate to real‑world, library‑heavy software development tasks across those twelve languages.