NeuroGenesysThe research we build on.
1,514 papers by other researchers that our work cites, from continual learning and verification to language models, agents and hardware. Each entry links to the paper on arXiv and to where we use it. 301 are cited on published pages of this site; the rest in the research notes behind our studies. Our own papers are in the archive.
- 2023-01arXiv 2301.13196Looped Transformers as Programmable Computers
Angeliki Giannou, Shashank Rajput, Jy-yong Sohn et al. · Machine learning
We present a framework for using transformer networks as universal computers by programming them with specific weights and placing them in a loop. Our input sequence acts as a punchcard, consisting of instructions and memory for data read/writes.
- 2023-01arXiv 2301.12131Restricted Orthogonal Gradient Projection for Continual Learning
Zeyuan Yang, Zonghan Yang, Peng Li et al. · Machine learning
Continual learning aims to avoid catastrophic forgetting and effectively leverage learned experiences to master new knowledge.
- 2023-01arXiv 2301.04104Mastering Diverse Domains through World Models
Danijar Hafner, Jurgis Pasukonis, Jimmy Ba et al. · AI & agents
Developing a general algorithm that learns to solve tasks across a wide range of applications has been a fundamental challenge in artificial intelligence.
Cited inStudy: Genesys program prior art - 2023-01arXiv 2301.04213Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Models
Peter Hase, Mohit Bansal, Been Kim et al. · Machine learning
Language models learn a great quantity of factual information during pretraining, and recent work localizes this information to specific model weights like mid-layer MLP weights.
- 2022-12arXiv 2212.10511When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories
Alex Mallen, Akari Asai, Victor Zhong et al. · Language models & NLP
Despite their impressive performance on diverse tasks, large language models (LMs) still struggle with tasks requiring rich world knowledge, implying the limitations of relying solely on their parameters to encode a wealth of world knowledge.
- 2022-12arXiv 2212.09849Dataless Knowledge Fusion by Merging Weights of Language Models
Xisen Jin, Xiang Ren, Daniel Preotiuc-Pietro et al. · Language models & NLP
Fine-tuning pre-trained language models has become the prevalent paradigm for building downstream NLP models. Oftentimes fine-tuned models are readily available but their training data is not, due to data privacy or intellectual property concerns.
- 2022-12arXiv 2212.07677Transformers learn in-context by gradient descent
Johannes von Oswald, Eyvind Niklasson, Ettore Randazzo et al. · Machine learning
At present, the mechanisms of in-context learning in Transformers are not well understood and remain mostly an intuition. In this paper, we suggest that training Transformers on auto-regressive objectives is closely related to gradient-based meta-learning formulations.
- 2022-12arXiv 2212.08073Constitutional AI: Harmlessness from AI Feedback
Yuntao Bai, Saurav Kadavath, Sandipan Kundu et al. · Language models & NLP
As AI systems become more capable, we would like to enlist their help to supervise other AIs. We experiment with methods for training a harmless AI assistant through self-improvement, without any human labels identifying harmful outputs.
- 2022-12arXiv 2212.05055Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpoints
Aran Komatsuzaki, Joan Puigcerver, James Lee-Thorp et al. · Machine learning
Training large, deep neural networks to convergence can be prohibitively expensive. As a result, often only a small selection of popular, dense models are reused across different contexts and tasks.
- 2022-12arXiv 2212.04089Editing Models with Task Arithmetic
Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman et al. · Machine learning
, improving their performance on a downstream task or mitigating biases learned during pre-training -- is a common practice when developing machine learning systems.
- 2022-12arXiv 2212.03827Discovering Latent Knowledge in Language Models Without Supervision
Collin Burns, Haotian Ye, Dan Klein et al. · Language models & NLP
Existing techniques for training language models can be misaligned with the truth: if we train models with imitation learning, they may reproduce errors that humans make; if we train them to generate text that humans rate highly, they may output errors that human evaluators can't detect.
- 2022-12arXiv 2212.02872A 64-core mixed-signal in-memory compute chip based on phase-change memory for deep neural network inference
Manuel Le Gallo, Riduan Khaddam-Aljameh, Milos Stanisavljevic et al. · Nature Electronics 6, 680-693 (2023) · Hardware, photonics & physics
The need to repeatedly shuttle around synaptic weight values from memory to processing units has been a key source of energy inefficiency associated with hardware implementation of artificial neural networks.
- 2022-11arXiv 2211.15944The Effectiveness of World Models for Continual Reinforcement Learning
Samuel Kessler, Mateusz Ostaszewski, Michał Bortkiewicz et al. · Machine learning
World models power some of the most efficient reinforcement learning algorithms. In this work, we showcase that they can be harnessed for continual learning - a situation when the agent faces changing environments.
Cited inStudy: Genesys program prior art - 2022-11arXiv 2211.15259A Call to Reflect on Evaluation Practices for Failure Detection in Image Classification
Paul F. Jaeger, Carsten T. Lüth, Lukas Klein et al. · ICLR 2023 (oral) · Vision
Reliable application of machine learning-based decision systems in the wild is one of the major challenges currently investigated by the field. A large portion of established approaches aims to detect erroneous predictions by means of assigning confidence scores.
- 2022-11arXiv 2211.15661What learning algorithm is in-context learning? Investigations with linear models
Ekin Akyürek, Dale Schuurmans, Jacob Andreas et al. · Machine learning
Neural sequence models, especially transformers, exhibit a remarkable capacity for in-context learning. They can construct new predictors from sequences of labeled examples presented in the input without further parameter updates.
- 2022-11arXiv 2211.13218CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learning
James Seale Smith, Leonid Karlinsky, Vyshnavi Gutta et al. · Vision
Computer vision models suffer from a phenomenon known as catastrophic forgetting when learning novel concepts from continuously shifting training data.
- 2022-11arXiv 2211.12485HyperTuning: Toward Adapting Large Language Models without Back-propagation
Jason Phang, Yi Mao, Pengcheng He et al. · Language models & NLP
Fine-tuning large language models for different tasks can be costly and inefficient, and even methods that reduce the number of tuned parameters still require full gradient-based optimization.
- 2022-11arXiv 2211.11875Enhancing Self-Consistency and Performance of Pre-Trained Language Models through Natural Language Inference
Eric Mitchell, Joseph J. Noh, Siyan Li et al. · Language models & NLP
While large pre-trained language models are powerful, their predictions often lack logical consistency across test inputs.'.
Cited inCoherence is selected, anchored, then pruned: a literature synthesis on logical coherence in LLM world modelsTrust the premises, not the confidence: which belief to drop when an LLM contradicts itself, and why the solver is not the bottleneckStudy: Logical coherence of LLM world modelsStudy: Orchestrated specialists and bridge synthesis - 2022-11arXiv 2211.11031Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors
Thomas Hartvigsen, Swami Sankaranarayanan, Hamid Palangi et al. · Machine learning
Deployed language models decay over time due to shifting inputs, changing user needs, or emergent world-knowledge gaps. When such problems are identified, we want to make targeted edits while avoiding expensive retraining.
- 2022-11arXiv 2211.09760VeLO: Training Versatile Learned Optimizers by Scaling Up
Luke Metz, James Harrison, C. Daniel Freeman et al. · Machine learning
While deep learning models have replaced hand-designed features across many domains, these models are still trained with hand-designed optimizers. In this work, we leverage the same scaling approach behind the success of deep learning to learn versatile optimizers.
- 2022-11arXiv 2211.07725Hierarchically Structured Task-Agnostic Continual Learning
Heinke Hihn, Daniel A. Braun · Machine learning
One notable weakness of current machine learning algorithms is the poor ability of models to solve new problems without forgetting previously acquired knowledge.
- 2022-11arXiv 2211.02193Benchmarking Quality-Diversity Algorithms on Neuroevolution for Reinforcement Learning
Manon Flageat, Bryan Lim, Luca Grillotti et al. · Neuroscience & neural computation
We present a Quality-Diversity benchmark suite for Deep Neuroevolution in Reinforcement Learning domains for robot control. The suite includes the definition of tasks, environments, behavioral descriptors, and fitness.
- 2022-11arXiv 2211.02633A Theoretical Study on Solving Continual Learning
Gyuhak Kim, Changnan Xiao, Tatsuya Konishi et al. · Machine learning
Continual learning (CL) learns a sequence of tasks incrementally. There are two popular CL settings, class incremental learning (CIL) and task incremental learning (TIL).
Cited inThe Capacity AuditInvestors are paying for agents that learn on the job. The hard part is knowing what they forgot.Penalising wrong answers only works if someone else is gradingStudy: Forgetting as selection at scaleStudy: Genesys program prior artInternal notesStudy: Orchestrated specialists and bridge synthesisStudy: Stopping confident wrong expert votes - 2022-11arXiv 2211.00241Adversarial Policies Beat Superhuman Go AIs
Tony T. Wang, Adam Gleave, Tom Tseng et al. · Machine learning
We attack the state-of-the-art Go-playing AI system KataGo by training adversarial policies against it, achieving a >97% win rate against KataGo running at superhuman settings.
- 2022-10arXiv 2210.17223Accelerating Distributed MoE Training and Inference with Lina
Jiamin Li, Yimin Jiang, Yibo Zhu et al. · Other
Scaling model parameters improves model quality at the price of high computation overhead.
Cited inThe Wrong Phase - 2022-10arXiv 2210.16955Learning to Defer to Multiple Experts: Consistent Surrogate Losses, Confidence Calibration, and Conformal Ensembles
Rajeev Verma, Daniel Barrejón, Eric Nalisnick · Machine learning
We study the statistical properties of learning to defer (L2D) to multiple experts. In particular, we address the open problems of deriving a consistent surrogate loss, confidence calibration, and principled ensembling of experts.
- 2022-10arXiv 2210.13382Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task
Kenneth Li, Aspen K. Hopkins, David Bau et al. · Machine learning
Language models show a surprising range of capabilities, but the source of their apparent competence is unclear. Do these networks just memorize a collection of surface statistics, or do they rely on internal representations of the process that generates the sequences they see?
- 2022-10arXiv 2210.12217Entailer: Answering Questions with Faithful and Truthful Chains of Reasoning
Oyvind Tafjord, Bhavana Dalvi Mishra, Peter Clark · AI & agents
Our goal is a question-answering (QA) system that can show how its answers are implied by its own internal beliefs via a systematic chain of reasoning. Such a capability would allow better understanding of why a model produced the answer it did.
- 2022-10arXiv 2210.11442Augmentative Topology Agents For Open-Ended Learning
Muhammad Umair Nasir, Michael Beukman, Steven James et al. · AI & agents
In this work, we tackle the problem of open-ended learning by introducing a method that simultaneously evolves agents and increasingly challenging environments.
- 2022-10arXiv 2210.11466Surgical Fine-Tuning Improves Adaptation to Distribution Shifts
Yoonho Lee, Annie S. Chen, Fahim Tajwar et al. · Machine learning
A common approach to transfer learning under distribution shift is to fine-tune the last few layers of a pre-trained model, preserving learned features while also adapting to the new task.
- 2022-10arXiv 2210.10760Scaling Laws for Reward Model Overoptimization
Leo Gao, John Schulman, Jacob Hilton · Machine learning
In reinforcement learning from human feedback, it is common to optimize against a reward model trained to predict human preferences. Because the reward model is an imperfect proxy, optimizing its value too much can hinder ground truth performance, in accordance with Goodhart's law.
Cited inReal judges did not break Best-of-N at N ≤ 32, and measured ε did not predict the budgetThurstone Bounds Best-of-N: A 1927 Identity, Calibrated for Modern VerifiersStudy: Generational competitive continual learningStudy: Genesys program prior artInternal notesStudy: Judge exploitability on real Best-of-NStudy: Logical coherence of LLM world modelsStudy: Stopping confident wrong expert votes - 2022-10arXiv 2210.07229Mass-Editing Memory in a Transformer
Kevin Meng, Arnab Sen Sharma, Alex Andonian et al. · Language models & NLP
Recent work has shown exciting promise in updating large language models with new memories, so as to replace obsolete information or add specialized knowledge. However, this line of work is predominantly limited to updating single associations.
Cited inEWC protects what its Fisher sees: a literature synthesis on elastic weight consolidation, scale and coherenceStudy: Consequence consistent updatesStudy: Decision signatures for conflict detectionStudy: Elastic weight consolidation deep diveStudy: Fortifying knowledge in continual learningStudy: LLMs modifying open model weights - 2022-10arXiv 2210.05643A Kernel-Based View of Language Model Fine-Tuning
Sadhika Malladi, Alexander Wettig, Dingli Yu et al. · Machine learning
It has become standard to solve NLP tasks by fine-tuning pre-trained language models (LMs), especially in low-data settings., why fine-tuning a model with or more parameters on a couple dozen training points does not result in overfitting.
- 2022-10arXiv 2210.02406Decomposed Prompting: A Modular Approach for Solving Complex Tasks
Tushar Khot, Harsh Trivedi, Matthew Finlayson et al. · Language models & NLP
Few-shot prompting is a surprisingly powerful way to use Large Language Models (LLMs) to solve various tasks.
- 2022-10arXiv 2210.01948Game-theoretic statistics and safe anytime-valid inference
Aaditya Ramdas, Peter Grünwald, Vladimir Vovk et al. · Statistics & theory
Safe anytime-valid inference (SAVI) provides measures of statistical evidence and certainty -- e-processes for testing and confidence sequences for estimation -- that remain valid at all stopping times, accommodating continuous monitoring and analysis of accumulating data and optional stopping or continuation for…
- 2022-09arXiv 2209.15189Learning by Distilling Context
Charlie Snell, Dan Klein, Ruiqi Zhong · Language models & NLP
Language models significantly benefit from context tokens, such as prompts or scratchpads. They perform better when prompted with informative instructions, and they acquire new reasoning capabilities by generating a scratch-pad before predicting the final answers.
- 2022-09arXiv 2209.15558Out-of-Distribution Detection and Selective Generation for Conditional Language Models
Jie Ren, Jiaming Luo, Yao Zhao et al. · Language models & NLP
Machine learning algorithms typically assume independent and identically distributed samples in training and at test time.
- 2022-09arXiv 2209.13085Defining and Characterizing Reward Hacking
Joar Skalse, Nikolaus H. R. Howe, Dmitrii Krasheninnikov et al. · Machine learning
We provide the first formal definition of reward hacking, a phenomenon where optimizing an imperfect proxy reward function leads to poor performance according to the true reward function.
- 2022-09arXiv 2209.11379Do Current Multi-Task Optimization Methods in Deep Learning Even Help?
Derrick Xin, Behrooz Ghorbani, Ankush Garg et al. · Machine learning
Recent research has proposed a series of specialized optimization algorithms for deep multi-task models. It is often claimed that these multi-task optimization (MTO) methods yield solutions that are superior to the ones found by simply optimizing a weighted average of the task losses.
- 2022-09arXiv 2209.10652Toy Models of Superposition
Nelson Elhage, Tristan Hume, Catherine Olsson et al. · Machine learning
Neural networks often pack many unrelated concepts into a single neuron - a puzzling phenomenon known as 'polysemanticity' which makes interpretability much more challenging.
Built from every arXiv paper cited in our papers, posts, topic hubs, reports and research notes, with titles, authors, dates and abstracts from the arXiv API (updated 2026-11-12). Summaries are the opening sentences of each paper's own abstract, not our assessment; a citation means we used or checked the work, not that we endorse it. Fields follow each paper's primary arXiv category. Papers cited only outside arXiv (journals, proceedings) are not listed yet. Entries are generated from our citations, so a mistyped ID in a note would list the wrong paper; we remove those as we find them. Report a wrong entry.
