NeuroGenesysThe research we build on.
1,514 papers by other researchers that our work cites, from continual learning and verification to language models, agents and hardware. Each entry links to the paper on arXiv and to where we use it. 301 are cited on published pages of this site; the rest in the research notes behind our studies. Our own papers are in the archive.
- 2026-11arXiv 2611.14326When Should Agents Remember? Falsification-Gated Self-Evolution for LLM Agents
Runxuan Tang, Haoyu Gao, Yuyan Ding et al. · AI & agents
LLM agents increasingly operate in long-horizon, verifier-rich environments, but current self-improvement mechanisms often write episode-derived lessons directly into memory, making local success an unreliable signal for reusable and safe knowledge.
- 2026-10arXiv 2610.25914FisherEdit: Fisher Information Guided Knowledge Editing for Large Language Models
Yizhou Yang, Jiahong Fu, Zhiwu Wang et al. · Language models & NLP
Sequential knowledge editing in Large Language Models (LLMs) often leads to a degradation of general capabilities due to cumulative parameter interference.
- 2026-09arXiv 2609.28395Fine-Tuning LLMs for Translation: General Forgetting Mitigation Does Not Preserve MT-Specific Instruction Following
Niklas Scholz, David Thulke, Abdallah Nasir et al. · Language models & NLP
Fine-tuning large language models on parallel data improves translation quality but can cause catastrophic forgetting. Mitigation methods are generally evaluated by retention on general benchmarks.
- 2026-09arXiv 2609.08067Popular Knowledge Propagates More Errors in LLM Knowledge Updating
Yuji Zhang, Weibing Wang, Cheng Qian et al. · Language models & NLP
Updating a language model's knowledge through fine-tuning is essential for keeping its outputs current, yet can also induce factual forgetting and new hallucinations.
Cited inCoherence is selected, anchored, then pruned: a literature synthesis on logical coherence in LLM world modelsEWC protects what its Fisher sees: a literature synthesis on elastic weight consolidation, scale and coherenceStudy: Elastic weight consolidation deep diveStudy: Logical coherence of LLM world models - 2026-09arXiv 2609.06986Continual Learning Mechanisms Compose for Long-Horizon Memorization
Zheyuan Zhang, Alvin Zhang, Daniel Khashabi et al. · Machine learning
Language models may need to internalize information that arrives over time and retain it through many subsequent updates.
Cited inEWC protects what its Fisher sees: a literature synthesis on elastic weight consolidation, scale and coherenceTo keep what a model knows, control what it is allowed to moveStudy: Elastic weight consolidation deep diveStudy: Fortifying knowledge in continual learningStudy: Seven continual learning methods tested - 2026-07arXiv 2607.04364RL Forgets! Towards Continual Policy Optimization
Mao-Lin Luo, Zhe-Xu Wang, Zi-Hao Zhou et al. · Machine learning
Continual post-training is becoming a central paradigm for adapting vision-language models to evolving tasks. Recent studies report that reinforcement learning is less prone to forgetting than supervised fine-tuning, motivating the view that RL is inherently resistant to forgetting.
- 2026-05arXiv 2605.30148Overcoming Forgetting in LLM Fine-Tuning with Evolution Strategies
Kajetan Schweighofer, Conor F. Hayes, Roberto Dailey et al. · Machine learning
Evolution Strategies (ES) has recently emerged as a competitive alternative to reinforcement learning (RL) for large language model (LLM) fine-tuning, offering advantages through simplicity, scalability, and inference-only training.
- 2026-05arXiv 2605.25525SAE-FD: Sparse Autoencoder Feature Distillation for Continual Learning of Large Language Models
Mingxu Zhang, Yuhan Li, Lujundong Li et al. · Machine learning
Continual learning enables large language models to adapt to evolving tasks without retraining from scratch, yet catastrophic forgetting remains a central obstacle.
- 2026-04arXiv 2604.22407Hidden Failure Modes of Gradient Modification under Adam in Continual Learning, and Adaptive Decoupled Moment Routing as a Repair
Yuelin Hu, Zhenbo Yu, Zhengxue Cheng et al. · Machine learning
, projection, penalty rescaling, replay mixing) while treating Adam as a neutral backend. We show this composition has a hidden failure mode.2).1.8 units.8 units.
- 2026-04arXiv 2604.15574Why Fine-Tuning Encourages Hallucinations and How to Fix It
Guy Kaplan, Zorik Gekhman, Zhen Zhu et al. · Language models & NLP
Large language models are prone to hallucinating factually incorrect statements.~knowledge acquired during pre-training. Since these errors arise as a by-product of knowledge degradation, we explore whether established continual learning tools can mitigate them.
- 2026-03arXiv 2603.09938Model Merging in the Era of Large Language Models: Methods, Applications, and Future Directions
Mingyang Song, Mao Zheng · Language models & NLP
Model merging combines the parameters of multiple neural networks into a single model without additional training.
- 2026-02arXiv 2602.17559Revisiting Weight Regularization for Low-Rank Continual Learning
Yaoyue Zheng, Yin Zhang, Joost van de Weijer et al. · Machine learning
Continual Learning (CL) with large-scale pre-trained models (PTMs) has recently gained wide attention, shifting the focus from training from scratch to continually adapting PTMs.
- 2025-12arXiv 2512.02914Martingale Score: An Unsupervised Metric for Bayesian Rationality in LLM Reasoning
Zhonghao He, Tianyi Qiu, Hirokazu Shirado et al. · AI & agents
Recent advances in reasoning techniques have substantially improved the performance of large language models (LLMs), raising expectations for their ability to provide accurate, truthful, and reliable information.
- 2025-11arXiv 2511.08567The Path Not Taken: RLVR Provably Learns Off the Principals
Hanqing Zhu, Zhenyu Zhang, Hanxian Huang et al. · Machine learning
Reinforcement Learning with Verifiable Rewards (RLVR) reliably improves the reasoning performance of large language models, yet it appears to modify only a small fraction of parameters.
- 2025-10arXiv 2510.27400Balancing Knowledge Updates: Toward Unified Modular Editing in LLMs
Jiahao Liu, Zijian Wang, Kuo Zhao et al. · Language models & NLP
Knowledge editing has emerged as an efficient approach for updating factual knowledge in large language models (LLMs). It typically locates knowledge storage modules and then modifies their parameters.
- 2025-10arXiv 2510.21978Beyond Reasoning Gains: Mitigating General-Capability Forgetting in Large Reasoning Models
Hoang Phan, Xianjun Yang, Yuanshun Yao et al. · Machine learning
Reinforcement learning with verifiable rewards (RLVR) has delivered impressive gains in mathematical and multimodal reasoning and has become a standard post-training paradigm for contemporary language and vision-language models.
- 2025-09arXiv 2509.23593Avoid Catastrophic Forgetting with Rank-1 Fisher from Diffusion Models
Zekun Wang, Anant Gupta, Zihan Dong et al. · ICLR 2026 · Machine learning
Catastrophic forgetting remains a central obstacle for continual learning in neural models.
- 2025-09arXiv 2509.17348AIMMerging: Adaptive Iterative Model Merging Using Training Trajectories for Language Model Continual Learning
Yujie Feng, Jian Li, Xiaoyu Dong et al. · Language models & NLP
Continual learning (CL) is essential for deploying large language models (LLMs) in dynamic real-world environments without the need for costly retraining.
- 2025-05arXiv 2505.05946Full-Parameter Continual Pretraining of Gemma2: Insights into Fluency and Domain Knowledge
Vytenis Šliogeris, Povilas Daniušis, Artūras Nakvosas · Language models & NLP
In this technical report, we empirically investigate the relationship between linguistic fluency and domain knowledge in the context of continual learning with large language models (LLMs).
- 2025-05arXiv 2505.05017Scalable Multi-Stage Influence Function for Large Language Models via Eigenvalue-Corrected Kronecker-Factored Parameterization
Yuntai Bao, Xuhong Zhang, Tianyu Du et al. · Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence Main Track (2025) 8022-8030 · Language models & NLP
Pre-trained large language models (LLMs) are commonly fine-tuned to adapt to downstream tasks. Since the majority of knowledge is acquired during pre-training, attributing the predictions of fine-tuned LLMs to their pre-training data may provide valuable insights.
- 2025-04arXiv 2504.02107TiC-LM: A Web-Scale Benchmark for Time-Continual LLM Pretraining
Jeffrey Li, Mohammadreza Armandpour, Iman Mirzadeh et al. · Machine learning
Large Language Models (LLMs) trained on historical web data inevitably become outdated. We investigate evaluation strategies and update methods for LLMs as new data becomes available.
Cited inEWC protects what its Fisher sees: a literature synthesis on elastic weight consolidation, scale and coherenceStudy: Continual learning order and companion dataStudy: Elastic weight consolidation deep diveStudy: Fortifying knowledge in continual learningStudy: Hardware guardian core for continual learning - 2025-03arXiv 2503.19939Continual Learning With Quasi-Newton Methods
Steven Vander Eeckt, Hugo Van hamme · IEEE Access, vol. 13, pp. 47485-47499, 2025 · Machine learning
Catastrophic forgetting remains a major challenge when neural networks learn tasks sequentially. Elastic Weight Consolidation (EWC) attempts to address this problem by introducing a Bayesian-inspired regularization loss to preserve knowledge of previously learned tasks.
- 2025-03arXiv 2503.05683WikiBigEdit: Understanding the Limits of Lifelong Knowledge Editing in LLMs
Lukas Thede, Karsten Roth, Matthias Bethge et al. · Language models & NLP
Keeping large language models factually up-to-date is crucial for deployment, yet costly retraining remains a challenge. Knowledge editing offers a promising alternative, but methods are only tested on small-scale or synthetic edit benchmarks.
Cited inEWC protects what its Fisher sees: a literature synthesis on elastic weight consolidation, scale and coherenceStudy: Consequence consistent updatesStudy: Continual learning order and companion dataStudy: Elastic weight consolidation deep diveStudy: Fortifying knowledge in continual learningStudy: LLMs modifying open model weightsStudy: Logical coherence of LLM world modelsStudy: Orchestrated specialists and bridge synthesisStudy: Weight editing to prevent forgetting - 2025-02arXiv 2502.15826CoME: An Unlearning-based Approach to Conflict-free Model Editing
Dahyun Jung, Jaehyung Seo, Jaewook Lee et al. · Language models & NLP
Large language models (LLMs) often retain outdated or incorrect information from pre-training, which undermines their reliability.
- 2025-02arXiv 2502.11756On the Computation of the Fisher Information in Continual Learning
Gido M. van de Ven · Machine learning
One of the most popular methods for continual learning with deep neural networks is Elastic Weight Consolidation (EWC), which involves computing the Fisher Information.
- 2025-02arXiv 2502.04390In Praise of Stubbornness: An Empirical Case for Cognitive-Dissonance Aware Continual Update of Knowledge in LLMs
Simone Clemente, Zied Ben Houidi, Alexis Huet et al. · Language models & NLP
Through systematic empirical investigation, we uncover a fundamental and concerning property of Large Language Models: while they can safely learn facts that don't contradict their knowledge, attempting to update facts with contradictory information triggers catastrophic corruption of unrelated knowledge.
- 2024-12arXiv 2412.06117Paraconsistent Belief Revision: A Replacement-Enriched LFI for Epistemic Entrenchment
Marcelo E. Coniglio, Martin Figallo, Rafael R. Testa · Other
We further develop the formal foundations of Paraconsistent Belief Revision (PBR) by introducing Logics of Formal Inconsistency (LFIs) specifically designed to support the development of epistemic entrenchment-based models for belief change.
- 2024-10arXiv 2410.03122RIPPLECOT: Amplifying Ripple Effect of Knowledge Editing in Language Models via Chain-of-Thought In-Context Learning
Zihao Zhao, Yuchen Yang, Yijiang Li et al. · Language models & NLP
The ripple effect poses a significant challenge in knowledge editing for large language models. Namely, when a single fact is edited, the model struggles to accurately update the related facts in a sequence, which is evaluated by multi-hop questions linked to a chain of related facts.
- 2024-07arXiv 2407.12828Why Does New Knowledge Create Messy Ripple Effects in LLMs?
Jiaxin Qin, Zixuan Zhang, Manling Li et al. · Language models & NLP
Extensive previous research has focused on post-training knowledge editing (KE) for language models (LMs) to ensure that knowledge remains accurate and up-to-date.
Cited inA gradient kernel flags contradictions but does not predict forgetting, and steering rehearsal by it backfiresCoherence is selected, anchored, then pruned: a literature synthesis on logical coherence in LLM world modelsEWC protects what its Fisher sees: a literature synthesis on elastic weight consolidation, scale and coherenceStudy: Elastic weight consolidation deep diveStudy: Logical coherence of LLM world models - 2024-05arXiv 2405.14768WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models
Peng Wang, Zexi Li, Ningyu Zhang et al. · Language models & NLP
Large language models (LLMs) need knowledge updates to meet the ever-growing world facts and correct the hallucinated responses, facilitating the methods of lifelong model editing.
Cited inEWC protects what its Fisher sees: a literature synthesis on elastic weight consolidation, scale and coherenceStudy: Consequence consistent updatesStudy: Elastic weight consolidation deep diveStudy: Fortifying knowledge in continual learningStudy: LLMs modifying open model weightsStudy: Orchestrated specialists and bridge synthesis - 2024-03arXiv 2403.08763Simple and Scalable Strategies to Continually Pre-train Large Language Models
Adam Ibrahim, Benjamin Thérien, Kshitij Gupta et al. · Machine learning
Large language models (LLMs) are routinely pre-trained on billions of tokens, only to start the process over again once new data becomes available. A much more efficient solution is to continually pre-train these models, saving significant compute compared to re-training.
Cited inTo keep what a model knows, control what it is allowed to moveStudy: Elastic weight consolidation deep diveStudy: Fortifying knowledge in continual learningStudy: Hardware guardian core for continual learningStudy: Hierarchical and T shaped MoEStudy: Orchestrated specialists and bridge synthesis - 2024-03arXiv 2403.10557Second-Order Information Matters: Revisiting Machine Unlearning for Large Language Models
Kang Gu, Md Rafi Ur Rashid, Najrin Sultana et al. · Machine learning
With the rapid development of Large Language Models (LLMs), we have witnessed intense competition among the major LLM products like ChatGPT, LLaMa, and Gemini. privacy leakage and copyright violation) of the training corpus still remain underexplored.
- 2024-01arXiv 2401.01286A Comprehensive Study of Knowledge Editing for Large Language Models
Ningyu Zhang, Yunzhi Yao, Bozhong Tian et al. · Language models & NLP
Large Language Models (LLMs) have shown extraordinary capabilities in understanding and generating text that closely mirrors human communication. However, a primary limitation lies in the significant computational demands during training, arising from their extensive parameterization.
- 2023-10arXiv 2310.14152Orthogonal Subspace Learning for Language Model Continual Learning
Xiao Wang, Tianze Chen, Qiming Ge et al. · Language models & NLP
Benefiting from massive corpora and advanced hardware, large language models (LLMs) exhibit remarkable capabilities in language understanding and generation. However, their performance degrades in scenarios where multiple tasks are encountered sequentially, also known as catastrophic forgetting.
Cited inEWC protects what its Fisher sees: a literature synthesis on elastic weight consolidation, scale and coherenceStudy: Continual learning order and companion dataStudy: Elastic weight consolidation deep diveStudy: Fortifying knowledge in continual learningStudy: Genesys program prior artStudy: Hardware guardian core for continual learningStudy: Merging and splitting MoE expertsStudy: Orchestrated specialists and bridge synthesisStudy: Seven continual learning methods tested - 2023-09arXiv 2309.06256Mitigating the Alignment Tax of RLHF
Yong Lin, Hangyu Lin, Wei Xiong et al. · Machine learning
LLMs acquire a wide range of abilities during pre-training, but aligning LLMs under Reinforcement Learning with Human Feedback (RLHF) can lead to forgetting pretrained abilities, which is also known as the alignment tax.
- 2023-08arXiv 2308.07707Fast Machine Unlearning Without Retraining Through Selective Synaptic Dampening
Jack Foster, Stefan Schoepf, Alexandra Brintrup · Machine learning
Machine unlearning, the ability for a machine learning model to forget, is becoming increasingly important to comply with data privacy regulations, as well as to remove harmful, manipulated, or outdated information.
- 2023-08arXiv 2308.03296Studying Large Language Model Generalization with Influence Functions
Roger Grosse, Juhan Bae, Cem Anil et al. · Machine learning
When trying to gain better visibility into a machine learning model in order to understand and mitigate the associated risks, a potentially valuable source of evidence is: which training examples most contribute to a given behavior?
Cited inCoherence is selected, anchored, then pruned: a literature synthesis on logical coherence in LLM world modelsEWC protects what its Fisher sees: a literature synthesis on elastic weight consolidation, scale and coherenceStudy: Decision signatures for conflict detectionStudy: Elastic weight consolidation deep diveStudy: Hardware guardian core for continual learningStudy: Logical coherence of LLM world models - 2023-07arXiv 2307.12976Evaluating the Ripple Effects of Knowledge Editing in Language Models
Roi Cohen, Eden Biran, Ori Yoran et al. · Language models & NLP
Modern language models capture a large body of factual knowledge. However, some facts can be incorrectly induced or become obsolete over time, resulting in factually incorrect generations.
Cited inA gradient kernel flags contradictions but does not predict forgetting, and steering rehearsal by it backfiresCoherence is selected, anchored, then pruned: a literature synthesis on logical coherence in LLM world modelsEWC protects what its Fisher sees: a literature synthesis on elastic weight consolidation, scale and coherenceIterated self-closure settles fast, but only evidence that overrides self-belief makes the settled state trueTo stop a model forgetting, rehearse what it knew. New context won't do it.Study: Consequence consistent updatesStudy: Continual learning order and companion dataStudy: Decision signatures for conflict detectionStudy: Elastic weight consolidation deep diveStudy: Graded succession by earned authorityStudy: Hardware guardian core for continual learningStudy: Logical coherence of LLM world modelsStudy: Orchestrated specialists and bridge synthesis - 2023-05arXiv 2305.15555Deep Reinforcement Learning with Plasticity Injection
Evgenii Nikishin, Junhyuk Oh, Georg Ostrovski et al. · Machine learning
A growing body of evidence suggests that neural networks employed in deep reinforcement learning (RL) gradually lose their plasticity, the ability to learn from new data; however, the analysis and mitigation of this phenomenon is hampered by the complex relationship between plasticity, exploration, and performance…
- 2023-05arXiv 2305.11171TrueTeacher: Learning Factual Consistency Evaluation with Large Language Models
Zorik Gekhman, Jonathan Herzig, Roee Aharoni et al. · Language models & NLP
Factual consistency evaluation is often conducted using Natural Language Inference (NLI) models, yet these models exhibit limited success in evaluating summaries.
Built from every arXiv paper cited in our papers, posts, topic hubs, reports and research notes, with titles, authors, dates and abstracts from the arXiv API (updated 2026-11-12). Summaries are the opening sentences of each paper's own abstract, not our assessment; a citation means we used or checked the work, not that we endorse it. Fields follow each paper's primary arXiv category. Papers cited only outside arXiv (journals, proceedings) are not listed yet. Entries are generated from our citations, so a mistyped ID in a note would list the wrong paper; we remove those as we find them. Report a wrong entry.
