Where AI already beats the best humans in 2026, and where it still doesn't
• 8 min read • By Will Schulz
Key findings
- At the 2025 ICPC World Finals, OpenAI reported solving 12 of 12 problems against 11 for the best human team; at IMO 2025, IMO coordinators certified Gemini Deep Think's gold-medal 35 of 42.
- The Jacobian conjecture was disproved for every dimension n ≥ 3 in July 2026 with a counterexample credited to Claude Fable 5, which anyone can check with computer algebra; the two-dimensional case is still open.
- On ARC-AGI-2 the best verified score is 95.0% against roughly 60% for an average human tester, and ARC-AGI-3 went from under 1% at launch in March 2026 to 81.2% in November with the standard harness.
- METR's measured gap between 50% and 80% reliability is about eightfold: the public frontier completes 12-hour tasks half the time but is reliable only to about 1.5 hours.
- IMO 2026 perfect scores, the IOI 2026 result and vendor SWE-bench Pro figures are company-reported or AI-graded; no IMO statement certifying a 2026 AI score was found.

“Superhuman” used to mean a board game. AlphaGo beat Lee Sedol 4–1 in 2016, AlphaZero taught itself chess, shogi and Go, and Pluribus beat elite professionals at six-player poker. Even there the word needs care. Adversarial policies beat a superhuman Go program more than 97% of the time, and a human amateur learned the trick and used it too. Superhuman in a game has never meant robust.
Outside games, the settled cases are narrow and scientific. AlphaFold2 led the blind CASP14 assessment by a wide margin, though it still struggles with protein motion and disordered regions. DeepMind's GenCast beat the European Centre's ensemble forecast on 97.2% of 1,320 targets in Nature, a comparison with the best operational system rather than with human forecasters. Two other famous claims did not hold up. Outside chemists found that few of GNoME's 2.2 million predicted crystals were both new and useful, and independent re-evaluations of AlphaChip's “superhuman” chip placement did not beat strong baselines.
Bounded intellectual contests have now mostly fallen. At IMO 2025, IMO coordinators certified Gemini Deep Think's 35 of 42, a gold medal, written in natural language within the time limit. At the 2025 ICPC World Finals, OpenAI reported solving all 12 problems and Google's Gemini solved 10, against 11 for the best human team, both in the official AI track with the same five hours. On the ARC Prize verified leaderboard the best ARC-AGI-2 score is 95.0%, against roughly 60% for an average individual tester, and the third-best, 93.8%, cost under a dollar a task. ARC-AGI-3, a set of interactive games that every human tester could solve, was designed in March 2026 to be the gap; the best AI then scored under 1%. By November 9, Claude Fable 5.2 scored 81.2% with the standard harness and 97.7% with another. Official ARC Prize results are due December 4.
The newest evidence is real research. In July 2026 the Jacobian conjecture was disproved for every dimension three and above, with a counterexample credited to Claude Fable 5: a polynomial map with constant Jacobian that is not one-to-one, which anyone can check with computer algebra. The two-dimensional case, the one most mathematicians cared about, is still open, and how the model found the map has not been disclosed. Erdős problem #728 was solved with GPT-5.2 Pro and its proof checked in Lean, as Quanta reported, while other announced “first solutions” turned out to be already in the literature. METR's August 2026 audit of discovery rates found clear acceleration in only one area, security bugs: cURL went from 9 reported vulnerabilities in 2025 to 36 by late June 2026, and OpenSSL from 6 to 39. It found no AI results on the Hilbert or Millennium lists and no visible acceleration in algorithmic optimization.
Medicine shows how strong results and weak evidence can sit together. Microsoft's MAI-DxO, published in Science in October 2026, reached the right diagnosis on 83.2% of hard New England Journal of Medicine cases, against 36.1% for generalist physicians with references. Every author works for Microsoft, the cases were text, not patients, and at its most accurate setting it cost more per case than the physicians. Independent trials point the same way and add a twist: in a JAMA Network Open trial of 50 physicians, the model alone beat both the doctors given the model and the doctors without it. Giving people a stronger tool did not make them as strong as the tool.
Where AI still falls short is less about intelligence than about reliability, time and the world. METR's May 2026 risk report put the public frontier's 50% time horizon at about 12 hours but its 80% horizon at about 1.5, and found that at least 16% of successes on tasks over eight hours were cheating or gaming the specification. In homes, Figure reported its humanoid succeeding on 56% of three chores, a company figure with no independent benchmark. And no publicly documented frontier model learns from its own deployment: it improves between training runs, not on the job. Our own work on updating models shows why that is hard. At 1.5B parameters, rehearsal rules that worked on a toy model reversed, and outdated consequences survived pruning.
Who checked matters as much as what was claimed. Huawei and Xiaohongshu each announced perfect IMO 2026 scores graded by the organisers, but we found no statement from the IMO itself. NVIDIA's model beat the top human at IOI 2026 in a run the paper says IOI did not supervise. Vendors report around 80% on SWE-bench Pro with their own scaffolds, while Scale's standardized harness tops out at 61.5%. The fair summary for November 2026: AI exceeds the best humans on many bounded, checkable problems, has started to contribute to open mathematics, and remains short of human reliability on long, open-ended work. That uneven pattern, not a single line crossed, is what the evidence shows.
Sources
- Adversarial Policies Beat Superhuman Go AIs — Wang et al., ICML 2023
- Highly accurate protein structure prediction with AlphaFold — Jumper et al., Nature 2021
- Probabilistic weather forecasting with machine learning (GenCast) — Price et al., Nature 2025
- Gemini Deep Think officially achieves gold-medal standard at the IMO — Google DeepMind, July 2025
- Gemini achieves gold-medal level at the ICPC World Finals — Google DeepMind, September 2025
- Claude Fable 5.2 verified results — ARC Prize, November 9, 2026
- Why the legendary Erdős problems are falling to AI — Quanta Magazine, August 3, 2026
- Have we seen an acceleration in discoveries? — METR, August 14, 2026
- Sequential diagnosis with language models — Science, October 22, 2026
- Large language model influence on diagnostic reasoning: a randomized clinical trial — Goh et al., JAMA Network Open 2024
- Frontier Risk Report, February–March 2026 — METR, May 19, 2026
- SWE-bench Pro public leaderboard — Scale AI
Related research
- Research Desk • Machine LearningSuperintelligence has many definitions and no test. Here is what a test would need.Labs now say superintelligence is in sight, but their definitions compare AI with different humans, and none comes with a way to check. Benchmarks saturate, ceilings are set by label errors, and the best long-task metric stops working at 16 hours.
- Research Desk • Machine LearningThe case for a 2027 intelligence explosion now rests on evidence the public can't seeForecasters moved their dates for automated AI research back to 2027 this week, on the strength of METR's finding that Anthropic is improving about twice as fast as a year ago. Most of the evidence behind that finding is redacted, and the gains may be limited to tasks that grade themselves.
- Research Desk • Machine LearningInvestors are paying for agents that learn on the job. The hard part is knowing what they forgot.Trajectory just raised $45M at a $300M valuation to make deployed models keep learning. But on CL-Bench, the best system gained only 25.4% over a stateless baseline, and plain long context beat dedicated memory.
- Publication • Continual LearningCatastrophic forgetting in class-incremental learning is a verifier-exploitability problem
- Publication • Continual LearningEscaping the saturated game: plasticity, harder benchmarks, and retrieval in continual learning
