NeuroGenesysThe research we build on.
1,514 papers by other researchers that our work cites, from continual learning and verification to language models, agents and hardware. Each entry links to the paper on arXiv and to where we use it. 301 are cited on published pages of this site; the rest in the research notes behind our studies. Our own papers are in the archive.
- 2024-01arXiv 2401.01335Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models
Zixiang Chen, Yihe Deng, Huizhuo Yuan et al. · Machine learning
Harnessing the power of human-annotated data through Supervised Fine-Tuning (SFT) is pivotal for advancing Large Language Models (LLMs). In this paper, we delve into the prospect of growing a strong LLM out of a weak one without the need for acquiring additional human-annotated data.
- 2023-12arXiv 2312.09390Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision
Collin Burns, Pavel Izmailov, Jan Hendrik Kirchner et al. · Language models & NLP
Widely used alignment techniques, such as reinforcement learning from human feedback (RLHF), rely on the ability of humans to supervise model behavior - for example, to evaluate whether a model faithfully followed instructions or generated safe outputs.
- 2023-12arXiv 2312.06585Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models
Avi Singh, John D. Co-Reyes, Rishabh Agarwal et al. · Machine learning
Fine-tuning language models~(LMs) on human-generated data remains a prevalent practice. However, the performance of such models is often limited by the quantity and diversity of high-quality human data.
- 2023-10arXiv 2310.06762TRACE: A Comprehensive Benchmark for Continual Learning in Large Language Models
Xiao Wang, Yuansen Zhang, Tianze Chen et al. · Language models & NLP
Aligned large language models (LLMs) demonstrate exceptional capabilities in task-solving, following instructions, and ensuring safety. However, the continual learning aspect of these aligned LLMs has been largely overlooked.
- 2023-10arXiv 2310.01798Large Language Models Cannot Self-Correct Reasoning Yet
Jie Huang, Xinyun Chen, Swaroop Mishra et al. · Language models & NLP
Large Language Models (LLMs) have emerged as a groundbreaking technology with their unparalleled text generation capabilities across various applications. Nevertheless, concerns persist regarding the accuracy and appropriateness of their generated content.
Cited inCoherence is selected, anchored, then pruned: a literature synthesis on logical coherence in LLM world modelsStable wrong beliefs evade every self-check: a literature synthesis on model self-knowledge and confident errorsStudy: Civilization as a superintelligence modelStudy: Cross generational teaching and examinationStudy: Logical coherence of LLM world modelsStudy: Model self knowledge and confident errorsStudy: Orchestrated specialists and bridge synthesis - 2023-09arXiv 2309.17167DyVal: Dynamic Evaluation of Large Language Models for Reasoning Tasks
Kaijie Zhu, Jiaao Chen, Jindong Wang et al. · AI & agents
Large language models (LLMs) have achieved remarkable performance in various evaluation benchmarks. However, concerns are raised about potential data contamination in their considerable volume of training corpus.
- 2023-09arXiv 2309.00267RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedback
Harrison Lee, Samrat Phatale, Hassan Mansoor et al. · Proceedings of the 41st International Conference on Machine Learning, PMLR 235:26874-26901, 2024 · Language models & NLP
Reinforcement learning from human feedback (RLHF) has proven effective in aligning large language models (LLMs) with human preferences, but gathering high-quality preference labels is expensive.
- 2023-07arXiv 2307.09009How is ChatGPT's behavior changing over time?
Lingjiao Chen, Matei Zaharia, James Zou · Language models & NLP
5 and GPT-4 are the two most widely used large language model (LLM) services. However, when and how these models are updated over time is opaque.
- 2023-07arXiv 2307.02762PRD: Peer Rank and Discussion Improve Large Language Model based Evaluations
Ruosen Li, Teerth Patel, Xinya Du · Language models & NLP
Nowadays, the quality of responses generated by different modern large language models (LLMs) is hard to evaluate and compare automatically. Recent studies suggest and predominantly use LLMs for reference-free evaluation of open-ended question answering.
- 2023-07arXiv 2307.01850Self-Consuming Generative Models Go MAD
Sina Alemohammad, Josue Casco-Rodriguez, Lorenzo Luzi et al. · Machine learning
Seismic advances in generative AI algorithms for imagery, text, and other data types has led to the temptation to use synthetic data to train next-generation models.
- 2023-06arXiv 2306.04181Benchmarking Foundation Models with Language-Model-as-an-Examiner
Yushi Bai, Jiahao Ying, Yixin Cao et al. · Language models & NLP
Numerous benchmarks have been established to assess the performance of foundation models on open-ended question answering, which serves as a comprehensive test of a model's ability to understand and generate language in a manner similar to humans.
- 2023-06arXiv 2306.01711OMNI: Open-endedness via Models of human Notions of Interestingness
Jenny Zhang, Joel Lehman, Kenneth Stanley et al. · AI & agents
Open-ended algorithms aim to learn new, interesting behaviors forever. That requires a vast environment search space, but there are thus infinitely many possible tasks.
- 2023-05arXiv 2305.20050Let's Verify Step by Step
Hunter Lightman, Vineet Kosaraju, Yura Burda et al. · Machine learning
In recent years, large language models have greatly improved in their ability to perform complex multi-step reasoning. However, even state-of-the-art models still regularly produce logical mistakes.
- 2023-05arXiv 2305.17493The Curse of Recursion: Training on Generated Data Makes Models Forget
Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao et al. · Machine learning
Stable Diffusion revolutionised image creation from descriptive text.5) and GPT-4 demonstrated astonishing performance across a variety of language tasks. ChatGPT introduced such language models to the general public.
- 2023-05arXiv 2305.13281LM vs LM: Detecting Factual Errors via Cross Examination
Roi Cohen, May Hamri, Mor Geva et al. · Language models & NLP
A prominent weakness of modern language models (LMs) is their tendency to generate factually incorrect text, which hinders their usability. A natural question is whether such factual errors can be detected automatically.
- 2022-07arXiv 2207.10397CodeT: Code Generation with Generated Tests
Bei Chen, Fengji Zhang, Anh Nguyen et al. · Language models & NLP
The task of generating code solutions for a given programming problem can benefit from the use of pre-trained language models such as Codex, which can produce multiple diverse samples.
- 2022-03arXiv 2203.14465STaR: Bootstrapping Reasoning With Reasoning
Eric Zelikman, Yuhuai Wu, Jesse Mu et al. · Machine learning
Generating step-by-step "chain-of-thought" rationales improves language model performance on complex reasoning tasks like mathematics or commonsense question-answering.
- 2022-02arXiv 2202.03286Red Teaming Language Models with Language Models
Ethan Perez, Saffron Huang, Francis Song et al. · Language models & NLP
Language Models (LMs) often cannot be deployed because of their potential to harm users in hard-to-predict ways. Prior work identifies harmful behaviors before deployment by using human annotators to hand-write test cases.
- 2021-07arXiv 2107.01057Backward-Compatible Prediction Updates: A Probabilistic Approach
Frederik Träuble, Julius von Kügelgen, Matthäus Kleindessner et al. · Machine learning
When machine learning systems meet real world applications, accuracy is only one of several requirements. In this paper, we assay a complementary perspective originating from the increasing availability of pre-trained and regularly improving state-of-the-art models.
- 2021-06arXiv 2106.05945Does Knowledge Distillation Really Work?
Samuel Stanton, Pavel Izmailov, Polina Kirichenko et al. · Machine learning
Knowledge distillation is a popular technique for training a small student network to emulate a larger teacher model, such as an ensemble of networks.
- 2021-05arXiv 2105.03048Regression Bugs Are In Your Model! Measuring, Reducing and Analyzing Regressions In NLP Model Updates
Yuqing Xie, Yi-an Lai, Yuanjun Xiong et al. · Language models & NLP
Behavior of deep neural networks can be inconsistent between different versions. Regressions during model update are a common cause of concern that often over-weigh the benefits in accuracy or efficiency gain.
- 2020-12arXiv 2012.02096Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Design
Michael Dennis, Natasha Jaques, Eugene Vinitsky et al. · Machine learning
A wide range of reinforcement learning (RL) problems - including robustness, transfer learning, unsupervised RL, and emergent complexity - require specifying a distribution of tasks or environments in which a policy will be trained.
- 2020-11arXiv 2011.09161Positive-Congruent Training: Towards Regression-Free Model Updates
Sijie Yan, Yuanjun Xiong, Kaustav Kundu et al. · Vision
Reducing inconsistencies in the behavior of different versions of an AI system can be as important in practice as reducing its overall error.
- 2020-05arXiv 2005.04118Beyond Accuracy: Behavioral Testing of NLP models with CheckList
Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin et al. · Association for Computational Linguistics (ACL), 2020 · Language models & NLP
Although measuring held-out accuracy has been the primary approach to evaluate generalization, it often overestimates the performance of NLP models, while alternative approaches for evaluating models either focus on individual tasks or on specific behaviors.
- 2020-01arXiv 2001.00689A Neural Dirichlet Process Mixture Model for Task-Free Continual Learning
Soochan Lee, Junsoo Ha, Dongsu Zhang et al. · Machine learning
Despite the growing interest in continual learning, most of its contemporary works have been studied in a rather restricted setting where tasks are clearly distinguishable, and task boundaries are known during training.
- 2019-10arXiv 1910.07224Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environments
Rémy Portelas, Cédric Colas, Katja Hofmann et al. · Machine learning
We consider the problem of how a teacher algorithm can enable an unknown Deep Reinforcement Learning (DRL) student to become good at a skill over a wide range of diverse environments.
- 2019-10arXiv 1910.01348On the Efficacy of Knowledge Distillation
Jang Hyun Cho, Bharath Hariharan · ICCV 2019 · Machine learning
In this paper, we present a thorough evaluation of the efficacy of knowledge distillation and its dependence on student and teacher architectures.
- 2019-09arXiv 1909.03329LAMOL: LAnguage MOdeling for Lifelong Language Learning
Fan-Keng Sun, Cheng-Hao Ho, Hung-Yi Lee · Language models & NLP
Most research on lifelong learning applies to images or games, but not language. We present LAMOL, a simple yet effective method for lifelong language learning (LLL) based on language modeling.
- 2019-02arXiv 1902.03393Improved Knowledge Distillation via Teacher Assistant
Seyed-Iman Mirzadeh, Mehrdad Farajtabar, Ang Li et al. · Machine learning
Despite the fact that deep neural networks are powerful models and achieve appealing results on many tasks, they are too large to be deployed on edge devices like smartphones or embedded sensor nodes.
- 2019-01arXiv 1901.01753Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutions
Rui Wang, Joel Lehman, Jeff Clune et al. · Neuroscience & neural computation
While the history of machine learning so far largely encompasses a series of problems posed by researchers and algorithms that learn their solutions, an important question is whether the problems themselves can be generated by the algorithm at the same time as they are being solved.
- 2018-05arXiv 1805.04770Born Again Neural Networks
Tommaso Furlanello, Zachary C. Lipton, Michael Tschannen et al. · Machine learning
Knowledge Distillation (KD) consists of transferring âknowledgeâ from one machine learning model (the teacher) to another (the student). Commonly, the teacher is a high-capacity model with formidable performance, while the student is more compact.
- 2018-05arXiv 1805.03643Learning to Teach
Yang Fan, Fei Tian, Tao Qin et al. · Machine learning
Teaching plays a very important role in our society, by spreading human knowledge and educating our next generations. A good teacher will select appropriate teaching materials, impact suitable methodologies, and set up targeted examinations, according to the learning behaviors of the students.
- 2018-01arXiv 1801.05927An Overview of Machine Teaching
Xiaojin Zhu, Adish Singla, Sandra Zilles et al. · Machine learning
In this paper we try to organize machine teaching as a coherent set of ideas. Each idea is presented as varying along a dimension. The collection of dimensions then form the problem space of machine teaching, such that existing teaching problems can be characterized in this space.
- 2017-08arXiv 1708.01547Lifelong Learning with Dynamically Expandable Networks
Jaehong Yoon, Eunho Yang, Jeongtae Lee et al. · Machine learning
We propose a novel deep network architecture for lifelong learning which we refer to as Dynamically Expandable Network (DEN), that can dynamically decide its network capacity as it trains on a sequence of tasks, to learn a compact overlapping knowledge sharing structure among tasks.
- 2017-07arXiv 1707.00183Teacher-Student Curriculum Learning
Tambet Matiisen, Avital Oliver, Taco Cohen et al. · Machine learning
We propose Teacher-Student Curriculum Learning (TSCL), a framework for automatic curriculum learning, where the Student tries to learn a complex task and the Teacher automatically chooses subtasks from a given set for the Student to train on.
- 2017-04arXiv 1704.03003Automated Curriculum Learning for Neural Networks
Alex Graves, Marc G. Bellemare, Jacob Menick et al. · Neuroscience & neural computation
We introduce a method for automatically selecting the path, or syllabus, that a neural network follows through a curriculum so as to maximise learning efficiency.
- 2017-03arXiv 1703.05407Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Play
Sainbayar Sukhbaatar, Zeming Lin, Ilya Kostrikov et al. · Machine learning
We describe a simple scheme that allows an agent to learn about its environment in an unsupervised manner. Our scheme pits two versions of the same agent, Alice and Bob, against one another.
Built from every arXiv paper cited in our papers, posts, topic hubs, reports and research notes, with titles, authors, dates and abstracts from the arXiv API (updated 2026-11-12). Summaries are the opening sentences of each paper's own abstract, not our assessment; a citation means we used or checked the work, not that we endorse it. Fields follow each paper's primary arXiv category. Papers cited only outside arXiv (journals, proceedings) are not listed yet. Entries are generated from our citations, so a mistyped ID in a note would list the wrong paper; we remove those as we find them. Report a wrong entry.
