NeuroGenesysThe research we build on.
1,514 papers by other researchers that our work cites, from continual learning and verification to language models, agents and hardware. Each entry links to the paper on arXiv and to where we use it. 301 are cited on published pages of this site; the rest in the research notes behind our studies. Our own papers are in the archive.
- 2024-06arXiv 2406.07814Collective Constitutional AI: Aligning a Language Model with Public Input
Saffron Huang, Divya Siddarth, Liane Lovitt et al. · Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency. 1395-1417 · AI & agents
There is growing consensus that language model (LM) developers should not be the sole deciders of LM behavior, creating a need for methods that enable the broader public to collectively shape the behavior of LM systems that affect them.
- 2024-06arXiv 2406.06560Inverse Constitutional AI: Compressing Preferences into Principles
Arduin Findeis, Timo Kaufmann, Eyke Hüllermeier et al. · Language models & NLP
Feedback data is widely used for fine-tuning and evaluating state-of-the-art AI models. Pairwise text preferences, where human or AI annotators select the "better" of two options, are particularly common.
- 2024-06arXiv 2406.00392Artificial Generational Intelligence: Cultural Accumulation in Reinforcement Learning
Jonathan Cook, Chris Lu, Edward Hughes et al. · AI & agents
Cultural accumulation drives the open-ended and diverse progress in capabilities spanning human history. It builds an expanding body of knowledge and skills by combining individual exploration with inter-generational information transmission.
- 2024-05arXiv 2405.18432Unsupervised Model Tree Heritage Recovery
Eliahu Horwitz, Asaf Shul, Yedid Hoshen · Machine learning
The number of models shared online has recently skyrocketed, with over one million public models available on Hugging Face. Sharing models allows other users to build on existing models, using them as initialization for fine-tuning, improving accuracy, and saving compute and energy.
- 2024-05arXiv 2405.05904Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?
Zorik Gekhman, Gal Yona, Roee Aharoni et al. · Language models & NLP
When large language models are aligned via supervised fine-tuning, they may encounter new factual information that was not acquired through pre-training.
Cited inStable wrong beliefs evade every self-check: a literature synthesis on model self-knowledge and confident errorsTo stop a model forgetting, rehearse what it knew. New context won't do it.Study: Consequence consistent updatesStudy: Continual learning order and companion dataStudy: Fortifying knowledge in continual learningStudy: Generational competitive continual learningStudy: Graded succession by earned authorityStudy: Logical coherence of LLM world modelsStudy: Model self knowledge and confident errorsStudy: Orchestrated specialists and bridge synthesisStudy: Stopping confident wrong expert votes - 2024-05arXiv 2405.00332A Careful Examination of Large Language Model Performance on Grade School Arithmetic
Hugh Zhang, Jeff Da, Dean Lee et al. · Language models & NLP
Large language models (LLMs) have achieved impressive success on many benchmarks for mathematical reasoning.
- 2024-04arXiv 2404.04286Bias Amplification in Language Model Evolution: An Iterated Learning Perspective
Yi Ren, Shangmin Guo, Linlu Qiu et al. · Language models & NLP
With the widespread adoption of Large Language Models (LLMs), the prevalence of iterative interactions among these models is anticipated to increase. Notably, recent advancements in multi-round self-improving methods allow LLMs to generate new examples for training subsequent models.
- 2024-04arXiv 2404.01413Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data
Matthias Gerstgrasser, Rylan Schaeffer, Apratim Dey et al. · Machine learning
The proliferation of generative models, combined with pretraining on web-scale data, raises a timely question: what happens when these models are trained on their own generated outputs?
- 2024-03arXiv 2403.18886Self-Expansion of Pre-trained Models with Mixture of Adapters for Continual Learning
Huiyi Wang, Haodong Lu, Lina Yao et al. · Machine learning
Continual learning (CL) aims to continually accumulate knowledge from a non-stationary data stream without catastrophic forgetting of learned knowledge, requiring a balance between stability and adaptability.
- 2024-03arXiv 2403.13187Evolutionary Optimization of Model Merging Recipes
Takuya Akiba, Makoto Shing, Yujin Tang et al. · Nat Mach Intell (2025) · Neuroscience & neural computation
Large language models (LLMs) have become increasingly capable, but their development often requires substantial computational resources.
- 2024-03arXiv 2403.04132Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference
Wei-Lin Chiang, Lianmin Zheng, Ying Sheng et al. · AI & agents
Large Language Models (LLMs) have unlocked new capabilities and applications; however, evaluating the alignment with human preferences still poses significant challenges.
- 2024-03arXiv 2403.01315Near-optimal Per-Action Regret Bounds for Sleeping Bandits
Quan Nguyen, Nishant A. Mehta · Machine learning
We derive near-optimal per-action regret bounds for sleeping bandits, in which both the sets of available arms and their losses in every round are chosen by an adversary.
- 2024-02arXiv 2402.14800Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models
Xudong Lu, Qi Liu, Yuhui Xu et al. · Language models & NLP
A pivotal advancement in the progress of large language models (LLMs) is the emergence of the Mixture-of-Experts (MoE) LLMs. Compared to traditional LLMs, MoE LLMs can achieve higher performance with fewer parameters, but it is still hard to deploy them due to their immense parameter sizes.
- 2024-02arXiv 2402.14992tinyBenchmarks: evaluating LLMs with fewer examples
Felipe Maia Polo, Lucas Weber, Leshem Choshen et al. · Language models & NLP
The versatility of large language models (LLMs) led to the creation of diverse benchmarks that thoroughly test a variety of language models' abilities. These benchmarks consist of tens of thousands of examples making evaluation of LLMs very expensive.
- 2023-12arXiv 2312.05934Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs
Oded Ovadia, Menachem Brief, Moshik Mishaeli et al. · AI & agents
Large language models (LLMs) encapsulate a vast amount of factual information within their pre-trained weights, as evidenced by their ability to answer diverse questions across different domains.
- 2023-11arXiv 2311.03099Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch
Le Yu, Bowen Yu, Haiyang Yu et al. · Language models & NLP
In this paper, we unveil that Language Models (LMs) can acquire new capabilities by assimilating parameters from homologous models without retraining or GPUs.
- 2023-10arXiv 2310.13798Specific versus General Principles for Constitutional AI
Sandipan Kundu, Yuntao Bai, Saurav Kadavath et al. · Language models & NLP
Human feedback can prevent overtly harmful utterances in conversational models, but may not automatically mitigate subtle problematic behaviors such as a stated desire for self-preservation or power.
- 2023-10arXiv 2310.07064Large Language Models can Learn Rules
Zhaocheng Zhu, Yuan Xue, Xinyun Chen et al. · AI & agents
When prompted with a few examples and intermediate steps, large language models (LLMs) have demonstrated impressive performance in various reasoning tasks.
- 2023-09arXiv 2309.14316Physics of Language Models: Part 3.1, Knowledge Storage and Extraction
Zeyuan Allen-Zhu, Yuanzhi Li · Language models & NLP
")., cheating), or by genuinely learning to extract knowledge from sources like Wikipedia? In this paper, we investigate this issue using a controlled biography dataset.
- 2023-06arXiv 2306.11644Textbooks Are All You Need
Suriya Gunasekar, Yi Zhang, Jyoti Aneja et al. · Language models & NLP
5 (1B tokens).5% on MBPP.
- 2023-06arXiv 2306.04529Git-Theta: A Git Extension for Collaborative Development of Machine Learning Models
Nikhil Kandpal, Brian Lester, Mohammed Muqeeth et al. · Machine learning
Currently, most machine learning models are trained by centralized teams and are rarely updated. In contrast, open-source software development involves the iterative development of a shared artifact through distributed collaboration using a version control system.
- 2023-06arXiv 2306.01708TIES-Merging: Resolving Interference When Merging Models
Prateek Yadav, Derek Tam, Leshem Choshen et al. · Machine learning
, further fine-tuning a pre-trained model on a downstream task - can confer significant advantages, including improved downstream performance, faster convergence, and better sample efficiency.
Cited inStudy: Decision signatures for conflict detectionStudy: Fortifying knowledge in continual learningStudy: Generational competitive continual learningStudy: Hierarchical and T shaped MoEStudy: LLMs modifying open model weightsStudy: Merging and splitting MoE expertsStudy: Weight editing to prevent forgetting - 2023-05arXiv 2305.16291Voyager: An Open-Ended Embodied Agent with Large Language Models
Guanzhi Wang, Yuqi Xie, Yunfan Jiang et al. · AI & agents
We introduce Voyager, the first LLM-powered embodied lifelong learning agent in Minecraft that continuously explores the world, acquires diverse skills, and makes novel discoveries without human intervention.
- 2023-05arXiv 2305.12281Lifelong Language Pretraining with Distribution-Specialized Experts
Wuyang Chen, Yanqi Zhou, Nan Du et al. · Language models & NLP
Pretraining on a large-scale corpus has become a standard method to build general language models (LMs). Adapting a model to new data distributions targeting different downstream tasks poses significant challenges.
- 2023-02arXiv 2302.03648Class-Incremental Learning: A Survey
Da-Wei Zhou, Qi-Wei Wang, Zhi-Hong Qi et al. · Vision
, CNNs and Vision Transformers, have achieved impressive achievements in many vision tasks in the closed world. However, novel classes emerge from time to time in our ever-changing world, requiring a learning system to acquire new knowledge continually.
- 2022-12arXiv 2212.08073Constitutional AI: Harmlessness from AI Feedback
Yuntao Bai, Saurav Kadavath, Sandipan Kundu et al. · Language models & NLP
As AI systems become more capable, we would like to enlist their help to supervise other AIs. We experiment with methods for training a harmless AI assistant through self-improvement, without any human labels identifying harmful outputs.
- 2022-11arXiv 2211.09760VeLO: Training Versatile Learned Optimizers by Scaling Up
Luke Metz, James Harrison, C. Daniel Freeman et al. · Machine learning
While deep learning models have replaced hand-designed features across many domains, these models are still trained with hand-designed optimizers. In this work, we leverage the same scaling approach behind the success of deep learning to learn versatile optimizers.
- 2022-11arXiv 2211.02193Benchmarking Quality-Diversity Algorithms on Neuroevolution for Reinforcement Learning
Manon Flageat, Bryan Lim, Luca Grillotti et al. · Neuroscience & neural computation
We present a Quality-Diversity benchmark suite for Deep Neuroevolution in Reinforcement Learning domains for robot control. The suite includes the definition of tasks, environments, behavioral descriptors, and fitness.
- 2022-10arXiv 2210.11442Augmentative Topology Agents For Open-Ended Learning
Muhammad Umair Nasir, Michael Beukman, Steven James et al. · AI & agents
In this work, we tackle the problem of open-ended learning by introducing a method that simultaneously evolves agents and increasingly challenging environments.
- 2022-10arXiv 2210.10760Scaling Laws for Reward Model Overoptimization
Leo Gao, John Schulman, Jacob Hilton · Machine learning
In reinforcement learning from human feedback, it is common to optimize against a reward model trained to predict human preferences. Because the reward model is an imperfect proxy, optimizing its value too much can hinder ground truth performance, in accordance with Goodhart's law.
Cited inReal judges did not break Best-of-N at N ≤ 32, and measured ε did not predict the budgetThurstone Bounds Best-of-N: A 1927 Identity, Calibrated for Modern VerifiersStudy: Generational competitive continual learningStudy: Genesys program prior artInternal notesStudy: Judge exploitability on real Best-of-NStudy: Logical coherence of LLM world modelsStudy: Stopping confident wrong expert votes - 2022-09arXiv 2209.15189Learning by Distilling Context
Charlie Snell, Dan Klein, Ruiqi Zhong · Language models & NLP
Language models significantly benefit from context tokens, such as prompts or scratchpads. They perform better when prompted with informative instructions, and they acquire new reasoning capabilities by generating a scratch-pad before predicting the final answers.
- 2022-09arXiv 2209.13085Defining and Characterizing Reward Hacking
Joar Skalse, Nikolaus H. R. Howe, Dmitrii Krasheninnikov et al. · Machine learning
We provide the first formal definition of reward hacking, a phenomenon where optimizing an imperfect proxy reward function leads to poor performance according to the true reward function.
- 2021-12arXiv 2112.00861A General Language Assistant as a Laboratory for Alignment
Amanda Askell, Yuntao Bai, Anna Chen et al. · Language models & NLP
Given the broad capabilities of large language models, it should be possible to work towards a general-purpose, text-based assistant that is aligned with human values, meaning that it is helpful, honest, and harmless.
- 2021-02arXiv 2102.13202Online Multi-Armed Bandits with Adaptive Inference
Maria Dimakopoulou, Zhimei Ren, Zhengyuan Zhou · Machine learning
During online decision making in Multi-Armed Bandits (MAB), one needs to conduct inference on the true mean reward of each arm based on data collected so far at each step.
- 2020-07arXiv 2007.04965A Study on Encodings for Neural Architecture Search
Colin White, Willie Neiswanger, Sam Nolen et al. · Advances in Neural Information Processing Systems 2020 · Machine learning
Neural architecture search (NAS) has been extensively studied in the past few years.
- 2020-07arXiv 2007.02686Meta-Learning through Hebbian Plasticity in Random Networks
Elias Najarro, Sebastian Risi · Advances in Neural Information Processing Systems (2020) · Neuroscience & neural computation
Lifelong learning and adaptability are two defining aspects of biological agents.
- 2020-02arXiv 2002.06673Performative Prediction
Juan C. Perdomo, Tijana Zrnic, Celestine Mendler-Dünner et al. · Machine learning
When predictions support decisions they may influence the outcome they aim to predict. We call such predictions performative; the prediction influences the target.
- 2020-02arXiv 2002.01365Compositional Languages Emerge in a Neural Iterated Learning Model
Yi Ren, Shangmin Guo, Matthieu Labeau et al. · ICLR-2020 · Language models & NLP
The principle of compositionality, which enables natural language to represent complex concepts via a structured combination of simpler ones, allows us to convey an open-ended set of messages using a limited vocabulary.
- 2019-08arXiv 1908.04734Reward Tampering Problems and Solutions in Reinforcement Learning: A Causal Influence Diagram Perspective
Tom Everitt, Marcus Hutter, Ramana Kumar et al. · AI & agents
Can humans get arbitrarily capable reinforcement learning (RL) agents to do their bidding? Or will sufficiently capable RL agents always find ways to bypass their intended objectives by shortcutting their reward signal?
- 2019-06arXiv 1906.04358Weight Agnostic Neural Networks
Adam Gaier, David Ha · Machine learning
Not all neural network architectures are created equal, some perform much better than others for certain tasks. But how important are the weight parameters of a neural network compared to its architecture?
Built from every arXiv paper cited in our papers, posts, topic hubs, reports and research notes, with titles, authors, dates and abstracts from the arXiv API (updated 2026-11-12). Summaries are the opening sentences of each paper's own abstract, not our assessment; a citation means we used or checked the work, not that we endorse it. Fields follow each paper's primary arXiv category. Papers cited only outside arXiv (journals, proceedings) are not listed yet. Entries are generated from our citations, so a mistyped ID in a note would list the wrong paper; we remove those as we find them. Report a wrong entry.
