NeuroGenesysThe research we build on.
1,514 papers by other researchers that our work cites, from continual learning and verification to language models, agents and hardware. Each entry links to the paper on arXiv and to where we use it. 301 are cited on published pages of this site; the rest in the research notes behind our studies. Our own papers are in the archive.
- 2026-10arXiv 2610.22293Towards Continual Learning: Mitigating Forgetting With Subspace Tuning
Arnav Chavan, Kusha Sareen, Rishabh Tiwari et al. · Machine learning
Large language models (LLMs) acquire new skills through fine-tuning, but it can destroy capabilities they already have: catastrophic forgetting remains a central obstacle to continual learning in LLMs.
- 2026-09arXiv 2609.06986Continual Learning Mechanisms Compose for Long-Horizon Memorization
Zheyuan Zhang, Alvin Zhang, Daniel Khashabi et al. · Machine learning
Language models may need to internalize information that arrives over time and retain it through many subsequent updates.
Cited inEWC protects what its Fisher sees: a literature synthesis on elastic weight consolidation, scale and coherenceTo keep what a model knows, control what it is allowed to moveStudy: Elastic weight consolidation deep diveStudy: Fortifying knowledge in continual learningStudy: Seven continual learning methods tested - 2026-09arXiv 2609.03900Beyond Endpoint Scores: Time- and Capacity-Conditioned Evaluation of Continual Knowledge Updating
Heejin Choi · Machine learning
Continual knowledge-updating methods are often declared superior from one final checkpoint and one conventional adapter rank. We show that this can be insufficient to identify the better operating point.
- 2026-08arXiv 2608.16141Continual-learning rules shape representational drift
Yikai Si, Shanshan Qin · Neuroscience & neural computation
Lifelong learning requires acquiring new knowledge without erasing the old. Yet neural population codes for familiar stimuli and behaviors change over days and weeks.
- 2026-06arXiv 2606.26783Reproducibility Study of "AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models"
Ananth K Suresh, Arya Hariharan · Machine learning
Fang et al.
- 2026-06arXiv 2606.24752Can Scale Save Us From Plasticity Loss in Large Language Models?
J. Fernando Hernandez-Garcia, Tomás Figliolia, Beren Millidge · AI & agents
The loss of plasticity - the ability of a network to learn new information after having already learned older information - is a fundamental challenge in creating artificial neural networks capable of continual learning.
- 2026-06arXiv 2606.09932When RL Fails after SFT: Rejuvenating Model Plasticity for Robust SFT-to-RL Handoff
Runze Liu, Jiashun Liu, Xu Wan et al. · Machine learning
Supervised Fine-Tuning (SFT) followed by Reinforcement Learning (RL) has become a standard pipeline for Large Language Model (LLM) post-training. SFT is expected to provide a useful behavioral prior for RL to further enhance model capabilities.
- 2026-06arXiv 2606.05661Continual Learning Bench: Evaluating Frontier AI Systems in Real-World Stateful Environments
Parth Asawa, Christopher M. Glaze, Gabriel Orlanski et al. · AI & agents
Continual learning, the ability of AI systems to improve through sequential experience, has attracted substantial interest, but no high-quality benchmark exists to evaluate it.
- 2026-06arXiv 2606.03979Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories
Ali Behrouz, Farnoosh Hashemi, Adel Javanmard et al. · Machine learning
The past few decades have witnessed significant advances in the design of machine learning algorithms, from early studies on task-specific shallow models to more general deep Large Language Models (LLMs).
Cited inInvestors are paying for agents that learn on the job. The hard part is knowing what they forgot.To keep what a model knows, control what it is allowed to moveWhat a model lets into long-term memory matters more than how it learnsStudy: Fortifying knowledge in continual learningStudy: Genesys program prior artStudy: Hardware guardian core for continual learningStudy: LLMs modifying open model weightsStudy: Logical coherence of LLM world modelsStudy: Memory world models and source trustStudy: Seven continual learning methods tested - 2026-05arXiv 2605.29495On-Policy Replay for Continual Supervised Fine-Tuning
Yan Chen, Taojie Zhu, Meng Zhang et al. · Machine learning
Continual supervised fine-tuning (SFT) is the de facto recipe for adapting large language models (LLMs) to a stream of downstream tasks, but it suffers from catastrophic forgetting of earlier capabilities.
- 2026-05arXiv 2605.15053TFGN: Task-Free, Replay-Free Continual Pre-Training Without Catastrophic Forgetting at LLM Scale
Anurup Ganguli · Machine learning
Continually pre-training a large language model on heterogeneous text domains, without replay or task labels, has remained an unsolved architectural problem at LLM scale.
- 2026-05arXiv 2605.11685Robust LLM Unlearning Against Relearning Attacks: The Minor Components in Representations Matter
Zeguan Xiao, Xuanzhe Xu, Yong Wang et al. · Language models & NLP
Large language model (LLM) unlearning aims to remove specific data influences from pre-trained model without costly retraining, addressing privacy, copyright, and safety concerns.
- 2026-04arXiv 2604.05248Improving Sparse Memory Finetuning
Satyam Goyal, Anirudh Kanchi, Garv Shah et al. · Machine learning
Large Language Models (LLMs) are typically static after training, yet real-world applications require continual adaptation to new knowledge without degrading existing capabilities.
- 2026-02arXiv 2602.11137Weight Decay Improves Language Model Plasticity
Tessa Han, Sebastian Bordt, Hanlin Zhang et al. · Machine learning
Large language models are typically trained in two broad phases: pretraining to produce a base model, followed by further training to improve downstream performance.
- 2026-02arXiv 2602.08040FIRE: Frobenius-Isometry Reinitialization for Balancing the Stability-Plasticity Tradeoff
Isaac Han, Sangyeon Park, Seungwon Oh et al. · Machine learning
, adapting to new tasks). Standard reinitialization methods, which reinitialize weights toward their original values, are widely used but difficult to tune: conservative reinitializations fail to restore plasticity, while aggressive ones erase useful knowledge.
- 2026-01arXiv 2601.19897Self-Distillation Enables Continual Learning
Idan Shenfeld, Mehul Damani, Jonas Hübotter et al. · Machine learning
Continual learning, enabling models to acquire new skills and knowledge without degrading existing capabilities, remains a fundamental challenge for foundation models.
- 2025-12arXiv 2512.24695Nested Learning: The Illusion of Deep Learning Architectures
Ali Behrouz, Meisam Razaviyayn, Peilin Zhong et al. · Machine learning
Despite the recent progresses, particularly in developing Language Models, there are fundamental challenges and unanswered questions about how such models can continually learn/memorize, self-improve, and find effective solutions.
- 2025-12arXiv 2512.23017Merge before Forget: A Single LoRA Continual Learning via Continual Merging
Fuli Qiao, Mehrdad Mahdavi · Machine learning
Parameter-efficient continual learning has emerged as a promising approach for large language models (LLMs) to mitigate catastrophic forgetting while enabling adaptation to new tasks.
- 2025-12arXiv 2512.04753EtCon: Edit-then-Consolidate for Reliable Knowledge Editing
Ruilin Li, Yibin Wang, Wenhong Zhu et al. · Language models & NLP
Knowledge editing aims to update specific facts in large language models (LLMs) without full retraining. Prior efforts sought to tune the knowledge layers of LLMs, achieving improved performance in controlled, teacher-forced evaluations.
- 2025-12arXiv 2512.20634Real Time Detection and Quantitative Analysis of Spurious Forgetting in Continual Learning
Weiwei Wang · Machine learning
Catastrophic forgetting remains a fundamental challenge in continual learning for large language models. Recent work revealed that performance degradation may stem from spurious forgetting caused by task alignment disruption rather than true knowledge loss.
- 2025-10arXiv 2510.18874Retaining by Doing: The Role of On-Policy Data in Mitigating Forgetting
Howard Chen, Noam Razin, Karthik Narasimhan et al. · Proceedings of the 43rd International Conference on Machine Learning (ICML), 2026 · Machine learning
Adapting language models (LMs) to new tasks via post-training carries the risk of degrading existing capabilities -- a phenomenon classically known as catastrophic forgetting.
- 2025-10arXiv 2510.15103Continual Learning via Sparse Memory Finetuning
Jessy Lin, Luke Zettlemoyer, Gargi Ghosh et al. · Language models & NLP
Modern language models are powerful, but typically static after deployment. A major obstacle to building models that continually learn over time is catastrophic forgetting, where updating on new data erases previously acquired capabilities.
Cited inTo keep what a model knows, control what it is allowed to moveStudy: Fortifying knowledge in continual learningStudy: Genesys program prior artStudy: Hardware guardian core for continual learningStudy: Memory world models and source trustStudy: Seven continual learning methods testedStudy: Weight editing to prevent forgetting - 2025-09arXiv 2509.06100Orthogonal Low-rank Adaptation in Lie Groups for Continual Learning of Large Language Models
Kefan Cao, Shuaicheng Wu · Language models & NLP
Large language models (LLMs) suffer from catastrophic forgetting in sequential multi-task learning., O-LoRA, N-LoRA) mitigate interference via low-rank subspace orthogonality, but additive updates distort the intrinsic geometry of model parameters.
- 2025-09arXiv 2509.04259RL's Razor: Why Online Reinforcement Learning Forgets Less
Idan Shenfeld, Jyothish Pari, Pulkit Agrawal · Machine learning
Comparison of fine-tuning models with reinforcement learning (RL) and supervised fine-tuning (SFT) reveals that, despite similar performance at a new task, RL preserves prior knowledge and capabilities significantly better.
- 2025-08arXiv 2508.16651HiCL: Hippocampal-Inspired Continual Learning
Kushal Kapoor, Wyatt Mackey, Yiannis Aloimonos et al. · Machine learning
We propose HiCL, a novel hippocampal-inspired dual-memory continual learning architecture designed to mitigate catastrophic forgetting by using elements inspired by the hippocampal circuitry.
- 2025-08arXiv 2508.04676GeRe: Towards Efficient Anti-Forgetting in Continual Learning of LLM via General Samples Replay
Yunan Zhang, Shuoran Jiang, Mengchen Zhao et al. · Language models & NLP
The continual learning capability of large language models (LLMs) is crucial for advancing artificial general intelligence.
- 2025-07arXiv 2508.00212Reinitializing weights vs units for maintaining plasticity in neural networks
J. Fernando Hernandez-Garcia, Shibhansh Dohare, Jun Luo et al. · Neuroscience & neural computation
Loss of plasticity is a phenomenon in which a neural network loses its ability to learn when trained for an extended time on non-stationary data. It is a crucial problem to overcome when designing systems that learn continually.
- 2025-07arXiv 2507.11393A Neural Network Model of Complementary Learning Systems: Pattern Separation and Completion for Continual Learning
James P Jun, Vijay Marupudi, Raj Sanjay Shah et al. · Machine learning
Learning new information without forgetting prior knowledge is central to human intelligence. In contrast, neural network models suffer from catastrophic forgetting: a significant degradation in performance on previously learned tasks when acquiring new information.
- 2025-06arXiv 2506.16884The Importance of Being Lazy: Scaling Limits of Continual Learning
Jacopo Graldi, Alessandro Breccia, Giulia Lanzillotta et al. · Machine learning
Despite recent efforts, neural networks still struggle to learn in non-stationary environments, and our understanding of catastrophic forgetting (CF) is far from complete.
- 2025-06arXiv 2506.13045Continual Learning for Generative AI: From LLMs to MLLMs and Beyond
Haiyang Guo, Fanhu Zeng, Fei Zhu et al. · Machine learning
The rapid advancement of generative models has empowered modern AI systems to comprehend and produce highly sophisticated content, even achieving human-level performance in specific domains.
- 2025-06arXiv 2506.10943Self-Adapting Language Models
Adam Zweiger, Jyothish Pari, Han Guo et al. · Machine learning
Large language models (LLMs) are powerful but static; they lack mechanisms to adapt their weights in response to new tasks, knowledge, or examples. We introduce Self-Adapting LLMs (SEAL), a framework that enables LLMs to self-adapt by generating their own finetuning data and update directives.
Cited inCoherence is selected, anchored, then pruned: a literature synthesis on logical coherence in LLM world modelsA model rewrote another model's weights. Read what it took away.Study: Consequence consistent updatesStudy: Continual learning order and companion dataStudy: Fortifying knowledge in continual learningStudy: LLMs modifying open model weightsStudy: Logical coherence of LLM world modelsStudy: Seven continual learning methods testedStudy: Weight editing to prevent forgetting - 2025-06arXiv 2506.06278Distillation Robustifies Unlearning
Bruce W. Lee, Addie Foote, Alex Infanger et al. · Machine learning
Current LLM unlearning methods are not robust. A few steps of finetuning can revert their effects. We begin by showing that this is true even for an idealized form of unlearning: training to imitate a model that was never trained on unwanted information.
- 2025-05arXiv 2506.03189Continual Learning in Vision-Language Models via Aligned Model Merging
Ghada Sokar, Gintare Karolina Dziugaite, Anurag Arnab et al. · Vision
Continual learning is conventionally tackled through sequential fine-tuning, a process that, while enabling adaptation, inherently favors plasticity over the stability needed to retain prior knowledge.
- 2025-05arXiv 2505.15424Gated Integration of Low-Rank Adaptation for Continual Learning of Large Language Models
Yan-Shuo Liang, Jia-Rui Chen, Wu-Jun Li · Language models & NLP
Continual learning (CL), which requires the model to learn multiple tasks sequentially, is crucial for large language models (LLMs). Recently, low-rank adaptation~(LoRA), one of the most representative parameter-efficient fine-tuning (PEFT) methods, has gained increasing attention in CL of LLMs.
- 2025-05arXiv 2505.12512Scalable Strategies for Continual Learning with Replay
Truman Hickok · Machine learning
Future deep learning models will be distinguished by systems that perpetually learn through interaction, imagination, and cooperation, blurring the line between training and inference.
- 2025-05arXiv 2505.11883MINGLE: Mixture of Null-Space Gated Low-Rank Experts for Test-Time Continual Model Merging
Zihuan Qiu, Yi Xu, Chiyuan He et al. · Machine learning
Continual model merging integrates independently fine-tuned models sequentially without access to the original training data, offering a scalable and efficient solution for continual learning.
- 2025-05arXiv 2505.10833MergeBench: A Benchmark for Merging Domain-Specialized LLMs
Yifei He, Siqi Zeng, Yuzheng Hu et al. · Machine learning
Model merging provides a scalable alternative to multi-task training by combining specialized finetuned models through parameter arithmetic, enabling efficient deployment without the need for joint training or access to all task data.
- 2025-05arXiv 2505.05946Full-Parameter Continual Pretraining of Gemma2: Insights into Fluency and Domain Knowledge
Vytenis Šliogeris, Povilas Daniušis, Artūras Nakvosas · Language models & NLP
In this technical report, we empirically investigate the relationship between linguistic fluency and domain knowledge in the context of continual learning with large language models (LLMs).
- 2025-04arXiv 2504.09522How new data permeates LLM knowledge and how to dilute it
Chen Sun, Renat Aksitov, Andrey Zhmoginov et al. · Language models & NLP
Large language models learn and continually learn through the accumulation of gradient-based updates, but how individual pieces of new information affect existing knowledge, leading to both beneficial generalization and problematic hallucination, remains poorly understood.
- 2025-04arXiv 2504.07097Sculpting Subspaces: Constrained Full Fine-Tuning in LLMs for Continual Learning
Nikhil Shivakumar Nayak, Krishnateja Killamsetty, Ligong Han et al. · Machine learning
Continual learning in large language models (LLMs) is prone to catastrophic forgetting, where adapting to new tasks significantly degrades performance on previously learned ones.
Built from every arXiv paper cited in our papers, posts, topic hubs, reports and research notes, with titles, authors, dates and abstracts from the arXiv API (updated 2026-11-12). Summaries are the opening sentences of each paper's own abstract, not our assessment; a citation means we used or checked the work, not that we endorse it. Fields follow each paper's primary arXiv category. Papers cited only outside arXiv (journals, proceedings) are not listed yet. Entries are generated from our citations, so a mistyped ID in a note would list the wrong paper; we remove those as we find them. Report a wrong entry.
