NeuroGenesysThe research we build on.
1,514 papers by other researchers that our work cites, from continual learning and verification to language models, agents and hardware. Each entry links to the paper on arXiv and to where we use it. 301 are cited on published pages of this site; the rest in the research notes behind our studies. Our own papers are in the archive.
- 2024-07arXiv 2407.21787Large Language Monkeys: Scaling Inference Compute with Repeated Sampling
Bradley Brown, Jordan Juravsky, Ryan Ehrlich et al. · Machine learning
Scaling the amount of compute used to train language models has dramatically improved their capabilities. However, when it comes to inference, we often limit models to making only one attempt at a problem.
- 2024-06arXiv 2406.16437Theory on Mixture-of-Experts in Continual Learning
Hongbo Li, Sen Lin, Lingjie Duan et al. · Machine learning
Continual learning (CL) has garnered significant attention because of its ability to adapt to new tasks that arrive over time. Catastrophic forgetting (of old tasks) has been identified as a major issue in CL, as the model adapts to new tasks.
- 2024-06arXiv 2406.12045-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains
Shunyu Yao, Noah Shinn, Pedram Razavi et al. · AI & agents
Existing benchmarks do not test language agents on their interaction with human users or ability to follow domain-specific rules, both of which are vital for deploying them in real world applications.
Cited inStudy: Genesys program prior artInternal notesStudy: Model self knowledge and confident errors - 2024-05arXiv 2405.11157Towards Modular LLMs by Building and Reusing a Library of LoRAs
Oleksiy Ostapenko, Zhan Su, Edoardo Maria Ponti et al. · Machine learning
The growing number of parameter-efficient adaptations of a base large language model (LLM) calls for studying whether we can reuse such trained adapters to improve performance for new tasks.
- 2024-04arXiv 2404.13076LLM Evaluators Recognize and Favor Their Own Generations
Arjun Panickssery, Samuel R. Bowman, Shi Feng · Language models & NLP
Self-evaluation using large language models (LLMs) has proven valuable not only in benchmarking but also methods like reward modeling, constitutional AI, and self-refinement.
- 2024-04arXiv 2404.04475Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators
Yann Dubois, Balázs Galambosi, Percy Liang et al. · Machine learning
LLM-based auto-annotators have become a key component of the LLM development process due to their cost-effectiveness and scalability compared to human-based evaluation.
Cited inStudy: Genesys program prior art - 2024-03arXiv 2403.11549Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapters
Jiazuo Yu, Yunzhi Zhuge, Lu Zhang et al. · Vision
Continual learning can empower vision-language models to continuously acquire new knowledge, without the need for access to the entire historical dataset.
- 2024-03arXiv 2403.07714StableToolBench: Towards Stable Large-Scale Benchmarking on Tool Learning of Large Language Models
Zhicheng Guo, Sijie Cheng, Hao Wang et al. · Language models & NLP
Large Language Models (LLMs) have witnessed remarkable advancements in recent years, prompting the exploration of tool learning, which integrates LLMs with external tools to address diverse real-world challenges.
Cited inStudy: Genesys program prior art - 2024-01arXiv 2401.16650Augmenting Replay in World Models for Continual Reinforcement Learning
Luke Yang, Levin Kuhlmann, Gideon Kowadlo · Machine learning
Continual RL requires an agent to learn new tasks without forgetting previous ones, while improving on both past and future tasks.
Cited inStudy: Genesys program prior art - 2024-01arXiv 2401.10191Divide and not forget: Ensemble of selectively trained experts in Continual Learning
Grzegorz Rypeść, Sebastian Cygert, Valeriya Khan et al. · Machine learning
Class-incremental learning is becoming more popular as it helps models widen their applicability while not forgetting what they already know. A trend in this area is to use a mixture-of-expert technique, where different models work together to solve the task.
- 2023-10arXiv 2310.14152Orthogonal Subspace Learning for Language Model Continual Learning
Xiao Wang, Tianze Chen, Qiming Ge et al. · Language models & NLP
Benefiting from massive corpora and advanced hardware, large language models (LLMs) exhibit remarkable capabilities in language understanding and generation. However, their performance degrades in scenarios where multiple tasks are encountered sequentially, also known as catastrophic forgetting.
Cited inEWC protects what its Fisher sees: a literature synthesis on elastic weight consolidation, scale and coherenceStudy: Continual learning order and companion dataStudy: Elastic weight consolidation deep diveStudy: Fortifying knowledge in continual learningStudy: Genesys program prior artStudy: Hardware guardian core for continual learningStudy: Merging and splitting MoE expertsStudy: Orchestrated specialists and bridge synthesisStudy: Seven continual learning methods tested - 2023-10arXiv 2310.07234Hierarchical Decomposition of Prompt-Based Continual Learning: Rethinking Obscured Sub-optimality
Liyuan Wang, Jingyi Xie, Xingxing Zhang et al. · Machine learning
Prompt-based continual learning is an emerging direction in leveraging pre-trained knowledge for downstream continual learning, and has almost reached the performance pinnacle under supervised pre-training.
- 2023-10arXiv 2310.03128MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Use
Yue Huang, Jiawen Shi, Yuan Li et al. · Society, software & economics
Large language models (LLMs) have garnered significant attention due to their impressive natural language processing (NLP) capabilities. Recently, many studies have focused on the tool utilization ability of LLMs.
- 2023-09arXiv 2309.15048Class Incremental Learning via Likelihood Ratio Based Task Prediction
Haowei Lin, Yijia Shao, Weinan Qian et al. · ICLR 2024 · Machine learning
Class incremental learning (CIL) is a challenging setting of continual learning, which learns a series of tasks sequentially. Each task consists of a set of unique classes.
- 2023-08arXiv 2308.00675Tool Documentation Enables Zero-Shot Tool-Usage with Large Language Models
Cheng-Yu Hsieh, Si-An Chen, Chun-Liang Li et al. · Language models & NLP
Today, large language models (LLMs) are taught to use new tools by providing a few demonstrations of the tool's usage. Unfortunately, demonstrations are hard to acquire, and can result in undesirable biased usage if the wrong demonstration is chosen.
- 2023-07arXiv 2307.16789ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs
Yujia Qin, Shihao Liang, Yining Ye et al. · AI & agents
, using external tools (APIs) to fulfill human instructions. The reason is that current instruction tuning largely focuses on basic language tasks but ignores the tool-use domain.
- 2023-07arXiv 2307.02251RanPAC: Random Projections and Pre-trained Models for Continual Learning
Mark D. McDonnell, Dong Gong, Amin Parveneh et al. · 37th Annual Conference on Neural Information Processing Systems (NeurIPS 2023), Dec 2023, New Orleans, United States · Machine learning
Continual learning (CL) aims to incrementally learn different tasks (such as classification) in a non-stationary data stream without forgetting old ones. Most CL works focus on tackling catastrophic forgetting under a learning-from-scratch paradigm.
- 2023-05arXiv 2305.20050Let's Verify Step by Step
Hunter Lightman, Vineet Kosaraju, Yura Burda et al. · Machine learning
In recent years, large language models have greatly improved in their ability to perform complex multi-step reasoning. However, even state-of-the-art models still regularly produce logical mistakes.
- 2023-05arXiv 2305.18806Prediction Error-based Classification for Class-Incremental Learning
Michał Zając, Tinne Tuytelaars, Gido M. van de Ven · Machine learning
Class-incremental learning (CIL) is a particularly challenging variant of continual learning, where the goal is to learn to discriminate between all classes presented in an incremental fashion.
Cited inStudy: Genesys program prior art - 2023-05arXiv 2305.16291Voyager: An Open-Ended Embodied Agent with Large Language Models
Guanzhi Wang, Yuqi Xie, Yunfan Jiang et al. · AI & agents
We introduce Voyager, the first LLM-powered embodied lifelong learning agent in Minecraft that continuously explores the world, acquires diverse skills, and makes novel discoveries without human intervention.
- 2023-04arXiv 2304.08244API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs
Minghao Li, Yingxiu Zhao, Bowen Yu et al. · Language models & NLP
Recent research has demonstrated that Large Language Models (LLMs) can enhance their capabilities by utilizing external tools. However, three pivotal questions remain unanswered: (1) How effective are current LLMs in utilizing tools?
Cited inStudy: Genesys program prior art - 2023-03arXiv 2303.10725SIESTA: Efficient Online Continual Learning with Sleep
Md Yousuf Harun, Jhair Gallardo, Tyler L. Hayes et al. · Vision
In supervised continual learning, a deep neural network (DNN) is updated with an ever-growing data stream. Unlike the offline setting where data is shuffled, we cannot make any distributional assumptions about the data stream.
Cited inStudy: Genesys program prior art - 2023-03arXiv 2303.07338Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Need
Da-Wei Zhou, Zi-Wen Cai, Han-Jia Ye et al. · Machine learning
Class-incremental learning (CIL) aims to adapt to emerging new classes without forgetting old ones. Traditional CIL models are trained from scratch to continually acquire knowledge as data evolves.
Cited inStudy: Genesys program prior art - 2023-03arXiv 2303.06572Continual Visual Reinforcement Learning with A Life-Long World Model
Minting Pan, Wendong Zhang, Geng Chen et al. · Machine learning
Learning physical dynamics in a series of non-stationary environments is a challenging but essential task for model-based reinforcement learning (MBRL) with visual inputs.
- 2023-01arXiv 2301.04104Mastering Diverse Domains through World Models
Danijar Hafner, Jurgis Pasukonis, Jimmy Ba et al. · AI & agents
Developing a general algorithm that learns to solve tasks across a wide range of applications has been a fundamental challenge in artificial intelligence.
Cited inStudy: Genesys program prior art - 2022-11arXiv 2211.15944The Effectiveness of World Models for Continual Reinforcement Learning
Samuel Kessler, Mateusz Ostaszewski, Michał Bortkiewicz et al. · Machine learning
World models power some of the most efficient reinforcement learning algorithms. In this work, we showcase that they can be harnessed for continual learning - a situation when the agent faces changing environments.
Cited inStudy: Genesys program prior art - 2022-11arXiv 2211.02633A Theoretical Study on Solving Continual Learning
Gyuhak Kim, Changnan Xiao, Tatsuya Konishi et al. · Machine learning
Continual learning (CL) learns a sequence of tasks incrementally. There are two popular CL settings, class incremental learning (CIL) and task incremental learning (TIL).
Cited inThe Capacity AuditInvestors are paying for agents that learn on the job. The hard part is knowing what they forgot.Penalising wrong answers only works if someone else is gradingStudy: Forgetting as selection at scaleStudy: Genesys program prior artInternal notesStudy: Orchestrated specialists and bridge synthesisStudy: Stopping confident wrong expert votes - 2022-10arXiv 2210.10760Scaling Laws for Reward Model Overoptimization
Leo Gao, John Schulman, Jacob Hilton · Machine learning
In reinforcement learning from human feedback, it is common to optimize against a reward model trained to predict human preferences. Because the reward model is an imperfect proxy, optimizing its value too much can hinder ground truth performance, in accordance with Goodhart's law.
Cited inReal judges did not break Best-of-N at N ≤ 32, and measured ε did not predict the budgetThurstone Bounds Best-of-N: A 1927 Identity, Calibrated for Modern VerifiersStudy: Generational competitive continual learningStudy: Genesys program prior artInternal notesStudy: Judge exploitability on real Best-of-NStudy: Logical coherence of LLM world modelsStudy: Stopping confident wrong expert votes - 2022-05arXiv 2205.10787A Dirichlet Process Mixture of Robust Task Models for Scalable Lifelong Reinforcement Learning
Zhi Wang, Chunlin Chen, Daoyi Dong · Machine learning
While reinforcement learning (RL) algorithms are achieving state-of-the-art performance in various challenging tasks, they can easily encounter catastrophic forgetting or interference when faced with lifelong streaming information.
Cited inStudy: Genesys program prior art - 2022-04arXiv 2204.05624Continual Predictive Learning from Videos
Geng Chen, Wendong Zhang, Han Lu et al. · Vision
Predictive learning ideally builds the world model of physical processes in one or more given environments. Typical setups assume that we can collect data from all environments at all times.
- 2022-03arXiv 2203.09450Continual Learning Based on OOD Detection and Task Masking
Gyuhak Kim, Sepideh Esmaeilpour, Changnan Xiao et al. · Vision
Existing continual learning techniques focus on either task incremental learning (TIL) or class incremental learning (CIL) problem, but not both. CIL and TIL differ mainly in that the task-id is provided for each test sample during testing for TIL, but not provided for CIL.
Cited inStudy: Genesys program prior art - 2021-10arXiv 2110.14168Training Verifiers to Solve Math Word Problems
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian et al. · Machine learning
State-of-the-art language models can match human performance on many tasks, but they still struggle to robustly perform multi-step mathematical reasoning.5K high quality linguistically diverse grade school math word problems.
- 2021-05arXiv 2105.10919Continual World: A Robotic Benchmark For Continual Reinforcement Learning
Maciej Wołczyk, Michał Zając, Razvan Pascanu et al. · Machine learning
Continual learning (CL) -- the ability to continuously learn, building on previously acquired knowledge -- is a natural requirement for long-lived autonomous reinforcement learning (RL) agents.
Cited inStudy: Genesys program prior art - 2020-10arXiv 2010.15277Class-incremental learning: survey and performance evaluation on image classification
Marc Masana, Xialei Liu, Bartlomiej Twardowski et al. · Machine learning
For future learning systems, incremental learning is desirable because it allows for: efficient resource usage by eliminating the need to retrain from scratch at the arrival of new data; reduced memory usage by preventing or limiting the amount of data required to be stored -- also important when privacy…
- 2020-06arXiv 2006.10701Deep Reinforcement Learning amidst Lifelong Non-Stationarity
Annie Xie, James Harrison, Chelsea Finn · Machine learning
As humans, our goals and our environment are persistently changing throughout our lifetime based on our experiences, actions, and internal and external drives. In contrast, typical reinforcement learning problem set-ups consider decision processes that are stationary across episodes.
Cited inStudy: Genesys program prior art - 2020-05arXiv 2005.06800Context-aware Dynamics Model for Generalization in Model-Based Reinforcement Learning
Kimin Lee, Younggyo Seo, Seunghyun Lee et al. · Machine learning
Model-based reinforcement learning (RL) enjoys several benefits, such as data-efficiency and planning, by learning a model of the environment's dynamics. However, learning a global model that can generalize across different dynamics is a challenging task.
Cited inStudy: Genesys program prior art - 2019-10arXiv 1910.08348VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learning
Luisa Zintgraf, Kyriacos Shiarlis, Maximilian Igl et al. · Machine learning
Trading off exploration and exploitation in an unknown environment is key to maximising expected return during learning. A Bayes-optimal policy, which does so optimally, conditions its actions not only on the environment state but on the agent's uncertainty about the environment.
Cited inStudy: Genesys program prior art - 2019-04arXiv 1904.07734Three scenarios for continual learning
Gido M. van de Ven, Andreas S. Tolias · Machine learning
Standard artificial neural networks suffer from the well-known issue of catastrophic forgetting, making continual or lifelong learning difficult for machine learning.
- 2019-03arXiv 1903.08254Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables
Kate Rakelly, Aurick Zhou, Deirdre Quillen et al. · Machine learning
Deep reinforcement learning algorithms require large amounts of experience to learn an individual task. While in principle meta-reinforcement learning (meta-RL) algorithms enable agents to learn new skills from small amounts of experience, several major challenges preclude their practicality.
Cited inStudy: Genesys program prior art - 2018-12arXiv 1812.07671Deep Online Learning via Meta-Learning: Continual Adaptation for Model-Based RL
Anusha Nagabandi, Chelsea Finn, Sergey Levine · Machine learning
Humans and animals can learn complex predictive models that allow them to accurately and reliably reason about real-world phenomena, and they can adapt such models extremely quickly in the face of unexpected changes.
Cited inStudy: Genesys program prior art
Built from every arXiv paper cited in our papers, posts, topic hubs, reports and research notes, with titles, authors, dates and abstracts from the arXiv API (updated 2026-11-12). Summaries are the opening sentences of each paper's own abstract, not our assessment; a citation means we used or checked the work, not that we endorse it. Fields follow each paper's primary arXiv category. Papers cited only outside arXiv (journals, proceedings) are not listed yet. Entries are generated from our citations, so a mistyped ID in a note would list the wrong paper; we remove those as we find them. Report a wrong entry.
