The research we build on.
1,514 papers by other researchers that our work cites, from continual learning and verification to language models, agents and hardware. Each entry links to the paper on arXiv and to where we use it. 301 are cited on published pages of this site; the rest in the research notes behind our studies. Our own papers are in the archive.
- 2026-09arXiv 2609.02849Post-Training Language Models for Gold-Medal Performance in Coding Competitions
Aleksander Ficek, Sean Narenthiran, Mehrzad Samadi et al. · Machine learning
Competitive programming has become a key test of large language model reasoning, with international competitions such as IOI and ICPC representing its most challenging settings.
Cited inStudy: Superintelligence - 2026-03arXiv 2603.24621ARC-AGI-3: A New Challenge for Frontier Agentic Intelligence
ARC Prize Foundation · AI & agents
We introduce ARC-AGI-3, an interactive benchmark for studying agentic intelligence through novel, abstract, turn-based environments in which agents must explore, infer goals, build internal models of environment dynamics, and plan effective action sequences without explicit instructions.
Cited inStudy: Superintelligence - 2026-01arXiv 2601.07421Resolution of Erdős Problem #728: a writeup of Aristotle's Lean proof
Nat Sothanaphan · Statistics & theory
We provide a writeup of a resolution of Erdős Problem #728; this is the first Erdős problem (a problem proposed by Paul Erdős which has been collected in the Erdős Problems website) regarded as fully resolved autonomously by an AI system.
Cited inStudy: Superintelligence - 2025-10arXiv 2510.18212A Definition of AGI
Dan Hendrycks, Dawn Song, Christian Szegedy et al. · AI & agents
The lack of a concrete definition for Artificial General Intelligence (AGI) obscures the gap between today's specialized AI and human-level cognition.
Cited inStudy: Superintelligence - 2025-07arXiv 2507.09089Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity
Joel Becker, Nate Rush, Elizabeth Barnes et al. · AI & agents
Despite widespread adoption, the impact of AI tools on software development in the wild remains understudied. 16 developers with moderate AI experience complete 246 tasks in mature projects on which they have an average of 5 years of prior experience.
- 2025-06arXiv 2506.22405Sequential Diagnosis with Language Models
Harsha Nori, Mayank Daswani, Christopher Kelly et al. · Language models & NLP
Artificial intelligence holds great promise for expanding access to expert medical knowledge and reasoning.
Cited inStudy: Superintelligence - 2025-05arXiv 2505.11831ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systems
Francois Chollet, Mike Knoop, Gregory Kamradt et al. · AI & agents
The Abstraction and Reasoning Corpus for Artificial General Intelligence (ARC-AGI), introduced in 2019, established a challenging benchmark for evaluating the general fluid intelligence of artificial systems via a set of unique, novel tasks only requiring minimal prior knowledge.
Cited inStudy: Superintelligence - 2025-04arXiv 2504.01848PaperBench: Evaluating AI's Ability to Replicate AI Research
Giulio Starace, Oliver Jaffe, Dane Sherburn et al. · AI & agents
We introduce PaperBench, a benchmark evaluating the ability of AI agents to replicate state-of-the-art AI research. Agents must replicate 20 ICML 2024 Spotlight and Oral papers from scratch, including understanding paper contributions, developing a codebase, and successfully executing experiments.
Cited inStudy: Superintelligence - 2025-03arXiv 2503.14499Measuring AI Ability to Complete Long Software Tasks
Thomas Kwa, Ben West, Joel Becker et al. · NeurIPS 2025 · AI & agents
Despite rapid progress on AI benchmarks, the real-world meaning of benchmark performance remains unclear. To quantify the capabilities of AI systems in terms of human capabilities, we propose a new metric: 50%-task-completion time horizon.
- 2025-02arXiv 2502.18864Accelerating scientific discovery with Co-Scientist
Juraj Gottweis, Wei-Hung Weng, Alexander Daryin et al. · AI & agents
Scientific discovery is driven by scientists generating novel hypotheses for complex problems that undergo rigorous experimental validation. To augment this process, we introduce Co-Scientist, a multi-agent AI system built on Gemini for structured scientific thinking and hypothesis generation.
- 2024-11arXiv 2411.15114RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts
Hjalmar Wijk, Tao Lin, Joel Becker et al. · Machine learning
Frontier AI safety policies highlight automation of AI research and development (R&D) by AI agents as an important capability to anticipate. However, there exist few evaluations for AI R&D capabilities, and none that are highly realistic and have a direct comparison to human performance.
- 2024-10arXiv 2410.07095MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering
Jun Shern Chan, Neil Chowdhury, Oliver Jaffe et al. · Language models & NLP
We introduce MLE-bench, a benchmark for measuring how well AI agents perform at machine learning engineering.
Cited inStudy: Superintelligence - 2024-06arXiv 2406.04127Are We Done with MMLU?
Aryo Pradipta Gema, Joshua Ong Jun Leang, Giwon Hong et al. · Language models & NLP
Maybe not. We identify and analyse errors in the popular Massive Multitask Language Understanding (MMLU) benchmark. Even though MMLU is widely adopted, our analysis demonstrates numerous ground truth errors that obscure the true capabilities of LLMs.
- 2024-05arXiv 2405.00332A Careful Examination of Large Language Model Performance on Grade School Arithmetic
Hugh Zhang, Jeff Da, Dean Lee et al. · Language models & NLP
Large language models (LLMs) have achieved impressive success on many benchmarks for mathematical reasoning.
- 2024-01arXiv 2401.02843Thousands of AI Authors on the Future of AI
Katja Grace, Harlan Stewart, Julia Fabienne Sandkühler et al. · Journal of Artificial Intelligence Research 84:9 (2025) · Society, software & economics
In the largest survey of its kind, 2,778 researchers who had published in top-tier artificial intelligence (AI) venues gave predictions on the pace of AI progress and the nature and impacts of advanced AI systems The aggregate forecasts give at least a 50% chance of AI systems achieving several milestones by 2028,…
- 2023-11arXiv 2311.12022GPQA: A Graduate-Level Google-Proof Q&A Benchmark
David Rein, Betty Li Hou, Asa Cooper Stickland et al. · AI & agents
We present GPQA, a challenging dataset of 448 multiple-choice questions written by domain experts in biology, physics, and chemistry., the questions are "Google-proof").
Cited inStudy: Superintelligence - 2023-11arXiv 2311.02462Levels of AGI for Operationalizing Progress on the Path to AGI
Meredith Ringel Morris, Jascha Sohl-Dickstein, Noah Fiedel et al. · Proceedings of ICML 2024 · AI & agents
We propose a framework for classifying the capabilities and behavior of Artificial General Intelligence (AGI) models and their precursors.
- 2023-02arXiv 2302.11014An Updated Assessment of Reinforcement Learning for Macro Placement
Chung-Kuan Cheng, Andrew B. Kahng, Sayak Kundu et al. · Machine learning
We provide an improved assessment of Google Brain's deep reinforcement learning approach to macro placement and its updated Circuit Training (CT) implementation in GitHub.
Cited inStudy: Superintelligence - 2022-11arXiv 2211.00241Adversarial Policies Beat Superhuman Go AIs
Tony T. Wang, Adam Gleave, Tom Tseng et al. · Machine learning
We attack the state-of-the-art Go-playing AI system KataGo by training adversarial policies against it, achieving a >97% win rate against KataGo running at superhuman settings.
- 2019-11arXiv 1911.01547On the Measure of Intelligence
François Chollet · AI & agents
To make deliberate progress towards more intelligent and more human-like artificial systems, we need to be following an appropriate feedback signal: we need to be able to define and evaluate intelligence in a way that enables comparisons between two systems, as well as comparisons with humans.
Cited inStudy: Superintelligence - 2007-12arXiv 0712.3329Universal Intelligence: A Definition of Machine Intelligence
Shane Legg, Marcus Hutter · Minds & Machines, 17:4 (2007) pages 391-444 · AI & agents
A fundamental problem in artificial intelligence is that nobody really knows what intelligence is. The problem is especially acute when we need to consider artificial systems which are significantly different to humans.
Cited inStudy: Superintelligence
Built from every arXiv paper cited in our papers, posts, topic hubs, reports and research notes, with titles, authors, dates and abstracts from the arXiv API (updated 2026-11-12). Summaries are the opening sentences of each paper's own abstract, not our assessment; a citation means we used or checked the work, not that we endorse it. Fields follow each paper's primary arXiv category. Papers cited only outside arXiv (journals, proceedings) are not listed yet. Entries are generated from our citations, so a mistyped ID in a note would list the wrong paper; we remove those as we find them. Report a wrong entry.
