NeuroGenesysThe research we build on.
1,514 papers by other researchers that our work cites, from continual learning and verification to language models, agents and hardware. Each entry links to the paper on arXiv and to where we use it. 301 are cited on published pages of this site; the rest in the research notes behind our studies. Our own papers are in the archive.
- 2026-02arXiv 2603.09988Causally Grounded Mechanistic Interpretability for LLMs with Faithful Natural-Language Explanations
Ajay Pravin Mahale · Language models & NLP
Mechanistic interpretability identifies internal circuits responsible for model behaviors, yet translating these findings into human-understandable explanations remains an open problem.
- 2026-02arXiv 2602.09870Steer2Edit: From Activation Steering to Component-Level Editing
Chung-En Sun, Ge Yan, Zimo Wang et al. · Language models & NLP
Steering methods influence Large Language Model behavior by identifying semantic directions in hidden representations, but are typically realized through inference-time activation interventions that apply a fixed, global modification to the model's internal states.
- 2026-02arXiv 2602.08218Sparsity-Aware Evolution for Model Merging
Huan Zhang, Yanjian Zhang, Guillaume Wisniewski et al. · Machine learning
We propose a sparsity-aware evolutionary (SAE) framework for model merging that involves iterative pruning-merging cycles to act as a novel mutation operator.
- 2026-02arXiv 2602.06359Training Data Selection with Gradient Orthogonality for Efficient Domain Adaptation
Xiyang Zhang, Yuanhe Tian, Hongzhi Wang et al. · Machine learning
Fine-tuning large language models (LLMs) for specialized domains often necessitates a trade-off between acquiring domain expertise and retaining general reasoning capabilities, a phenomenon known as catastrophic forgetting.
- 2026-02arXiv 2602.06665Not All Layers Need Tuning: Selective Layer Restoration Recovers Diversity
Bowen Zhang, Meiyi Wang, Harold Soh · Language models & NLP
Post-training improves instruction-following and helpfulness of large language models (LLMs) but often reduces generation diversity, which leads to repetitive outputs in open-ended settings, a phenomenon known as mode collapse.
- 2026-02arXiv 2602.04509Model-Dowser: Data-Free Importance Probing to Mitigate Catastrophic Forgetting in Multimodal Large Language Models
Hyeontaek Hwang, Nguyen Dinh Son, Daeyoung Kim · Language models & NLP
Fine-tuning Multimodal Large Language Models (MLLMs) on task-specific data is an effective way to improve performance on downstream applications. However, such adaptation often leads to a degradation in generalization on pretrained tasks, a phenomenon known as Catastrophic Forgetting.
- 2026-02arXiv 2602.04521-: Circuit-Restricted Weight Arithmetic for Selective Refusal
Aditya Kasliwal, Pratinav Seth, Vinay Kumar Sankarapu · Language models & NLP
Modern deployments require LLMs to enforce safety policies at scale, yet many controls rely on inference-time interventions that add recurring compute cost and serving complexity.
- 2026-01arXiv 2601.18350Adapter Merging Reactivates Latent Reasoning Traces: A Mechanism Analysis
Junyi Zou · Language models & NLP
Large language models fine-tuned via a two-stage pipeline (domain adaptation followed by instruction alignment) can exhibit non-trivial interference after adapter merging, including the re-emergence of explicit reasoning traces under strict decoding.
- 2026-01arXiv 2601.18699Mechanistic Analysis of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning
Gustav Olaf Yunus Laitinen-Fredriksson Lundstrom-Imanov · Machine learning
Sequential fine-tuning of Large Language Models (LLMs) adaptation to target tasks often triggers catastrophic forgetting, where the acquisition of novel target skills degrades ancestral capabilities.
- 2026-01arXiv 2601.13264Unlearning in LLMs: Methods, Evaluation, and Open Challenges
Tyler Lizzo, Larry Heck · Language models & NLP
Large language models (LLMs) have achieved remarkable success across natural language processing tasks, yet their widespread deployment raises pressing concerns around privacy, copyright, security, and bias.
- 2026-01arXiv 2602.00038LSSF: Safety Alignment for Large Language Models through Low-Rank Safety Subspace Fusion
Guanghao Zhou, Panjia Qiu, Cen Chen et al. · Society, software & economics
The safety mechanisms of large language models (LLMs) exhibit notable fragility, as even fine-tuning on datasets without harmful content may still undermine their safety capabilities.
- 2026-01arXiv 2601.10497MERGETUNE: Continued Fine-Tuning of Vision-Language Models
Wenqing Wang, Da Li, Xiatian Zhu et al. · ICLR2026 · Vision
Fine-tuning vision-language models (VLMs) such as CLIP often leads to catastrophic forgetting of pretrained knowledge. Prior work primarily aims to mitigate forgetting during adaptation; however, forgetting often remains inevitable during this process.
- 2026-01arXiv 2601.09694LLMs can Compress LLMs: Adaptive Pruning by Agents
Sai Varun Kodathala, Rakesh Vunnam · Language models & NLP
As Large Language Models (LLMs) continue to scale, post-training pruning has emerged as a promising approach to reduce computational costs while preserving performance.
- 2025-12arXiv 2512.15938SALVE: Sparse Autoencoder-Latent Vector Editing for Mechanistic Control of Neural Networks
Vegard Flovik · Machine learning
Deep neural networks achieve impressive performance but remain difficult to interpret and control. We present SALVE (Sparse Autoencoder-Latent Vector Editing), a unified "discover, validate, and control" framework that bridges mechanistic interpretability and model editing.
- 2025-12arXiv 2512.20634Real Time Detection and Quantitative Analysis of Spurious Forgetting in Continual Learning
Weiwei Wang · Machine learning
Catastrophic forgetting remains a fundamental challenge in continual learning for large language models. Recent work revealed that performance degradation may stem from spurious forgetting caused by task alignment disruption rather than true knowledge loss.
- 2025-11arXiv 2511.15390Automatic Pruning Discovery for Large Language Models
Haidong Kang, Lihong Lin, Enneng Yang et al. · Vision
Large language models (LLMs) have achieved remarkable performance on a wide range of tasks, hindering real-world deployment due to their massive size., Wanda) tailored for LLMs rely heavily on manual design pruning algorithms, thereby leading to huge labor costs and requires expert knowledge.
- 2025-11arXiv 2511.05852Can Fine-Tuning Erase Your Edits? On the Fragile Coexistence of Knowledge Editing and Adaptation
Yinjie Cheng, Paul Youssef, Christin Seifert et al. · Language models & NLP
Knowledge editing (KE) offers a lightweight alternative to retraining for updating large language models (LLMs). Meanwhile, fine-tuning remains the default operation for adapting LLMs to new domains and tasks.
- 2025-10arXiv 2510.15103Continual Learning via Sparse Memory Finetuning
Jessy Lin, Luke Zettlemoyer, Gargi Ghosh et al. · Language models & NLP
Modern language models are powerful, but typically static after deployment. A major obstacle to building models that continually learn over time is catastrophic forgetting, where updating on new data erases previously acquired capabilities.
Cited inTo keep what a model knows, control what it is allowed to moveStudy: Fortifying knowledge in continual learningStudy: Genesys program prior artStudy: Hardware guardian core for continual learningStudy: Memory world models and source trustStudy: Seven continual learning methods testedStudy: Weight editing to prevent forgetting - 2025-10arXiv 2510.08564How to Teach Large Multimodal Models New Skills
Zhen Zhu, Yiming Gong, Yao Xiao et al. · AI & agents
How can we teach large multimodal models (LMMs) new skills without erasing prior abilities? We study sequential fine-tuning on five target skills while monitoring general ability on eight held-out benchmarks across three model families.
- 2025-09arXiv 2509.23893Dynamic Orthogonal Continual Fine-tuning for Mitigating Catastrophic Forgettings
Zhixin Zhang, Zeming Wei, Meng Sun · Machine learning
Catastrophic forgetting remains a critical challenge in continual learning for large language models (LLMs), where models struggle to retain performance on historical tasks when fine-tuning on new sequential data without access to past datasets.
- 2025-09arXiv 2509.22263Erase or Hide? Suppressing Spurious Unlearning Neurons for Robust Unlearning
Nakyeong Yang, Dong-Kyum Kim, Jea Kwon et al. · Machine learning
Large language models trained on web-scale data can memorize private or sensitive knowledge, raising significant privacy risks.
- 2025-09arXiv 2509.17530An Unlearning Framework for Continual Learning
Sayanta Adhikari, Vishnuprasadh Kumaravelu, P. K. Srijith · Machine learning
Growing concerns surrounding AI safety and data privacy have driven the development of Machine Unlearning as a potential solution. However, current machine unlearning algorithms are designed to complement the offline training paradigm.
- 2025-07arXiv 2507.14198Retention analysis of edited knowledge after fine-tuning
Fufang Wen, Shichang Zhang · Language models & NLP
Large language models (LLMs) store vast amounts of knowledge, which often requires updates to correct factual errors, incorporate newly acquired information, or adapt model behavior.
- 2025-07arXiv 2507.08427ChainEdit: Propagating Ripple Effects in LLM Knowledge Editing through Logical Rule-Guided Chains
Zilu Dong, Xiangqing Shen, Zinong Yang et al. · Language models & NLP
Current knowledge editing methods for large language models (LLMs) struggle to maintain logical consistency when propagating ripple effects to associated facts.
- 2025-06arXiv 2506.10943Self-Adapting Language Models
Adam Zweiger, Jyothish Pari, Han Guo et al. · Machine learning
Large language models (LLMs) are powerful but static; they lack mechanisms to adapt their weights in response to new tasks, knowledge, or examples. We introduce Self-Adapting LLMs (SEAL), a framework that enables LLMs to self-adapt by generating their own finetuning data and update directives.
Cited inCoherence is selected, anchored, then pruned: a literature synthesis on logical coherence in LLM world modelsA model rewrote another model's weights. Read what it took away.Study: Consequence consistent updatesStudy: Continual learning order and companion dataStudy: Fortifying knowledge in continual learningStudy: LLMs modifying open model weightsStudy: Logical coherence of LLM world modelsStudy: Seven continual learning methods testedStudy: Weight editing to prevent forgetting - 2025-06arXiv 2506.03490Beyond Memorization: A Rigorous Evaluation Framework for Medical Knowledge Editing
Shigeng Chen, Linhao Luo, Zhangchi Qiu et al. · Language models & NLP
Recently, knowledge editing (KE) has emerged as a promising approach to update specific facts in Large Language Models (LLMs) without the need for full retraining.
- 2025-05arXiv 2505.16831Unlearning Isn't Deletion: Investigating Reversibility of Machine Unlearning in LLMs
Xiaoyu Xu, Xiang Yue, Yang Liu et al. · Language models & NLP
Unlearning in large language models (LLMs) aims to remove specified data, but its efficacy is typically assessed with task-level metrics like accuracy and perplexity.
- 2025-05arXiv 2505.14679UltraEdit: Training-, Subject-, and Memory-Free Lifelong Editing in Language Models
Xiaojie Gu, Ziying Huang, Jia-Chen Gu et al. · Language models & NLP
Lifelong learning enables large language models (LLMs) to adapt to evolving information by continually updating their internal knowledge. An ideal system should support efficient, wide-ranging updates while preserving existing capabilities and ensuring reliable deployment.
- 2025-05arXiv 2505.11816Continuous Subspace Optimization for Continual Learning
Quan Cheng, Yuanyu Wan, Lingyu Wu et al. · Vision
Continual learning aims to learn multiple tasks sequentially while preserving prior knowledge, but faces the challenge of catastrophic forgetting when adapting to new tasks.
- 2025-04arXiv 2504.08192SAEs Can Improve Unlearning: Dynamic Sparse Autoencoder Guardrails for Precision Unlearning in LLMs
Aashiq Muhamed, Jacopo Bonato, Mona Diab et al. · Machine learning
Machine unlearning is a promising approach to improve LLM safety by removing unwanted knowledge from the model.
- 2025-03arXiv 2503.05683WikiBigEdit: Understanding the Limits of Lifelong Knowledge Editing in LLMs
Lukas Thede, Karsten Roth, Matthias Bethge et al. · Language models & NLP
Keeping large language models factually up-to-date is crucial for deployment, yet costly retraining remains a challenge. Knowledge editing offers a promising alternative, but methods are only tested on small-scale or synthetic edit benchmarks.
Cited inEWC protects what its Fisher sees: a literature synthesis on elastic weight consolidation, scale and coherenceStudy: Consequence consistent updatesStudy: Continual learning order and companion dataStudy: Elastic weight consolidation deep diveStudy: Fortifying knowledge in continual learningStudy: LLMs modifying open model weightsStudy: Logical coherence of LLM world modelsStudy: Orchestrated specialists and bridge synthesisStudy: Weight editing to prevent forgetting - 2025-03arXiv 2503.01090Precise Localization of Memories: A Fine-grained Neuron-level Knowledge Editing Technique for LLMs
Haowen Pan, Xiaozhi Wang, Yixin Cao et al. · Language models & NLP
Knowledge editing aims to update outdated information in Large Language Models (LLMs). A representative line of study is locate-then-edit methods, which typically employ causal tracing to identify the modules responsible for recalling factual knowledge about entities.
- 2025-02arXiv 2502.11447Does Editing Provide Evidence for Localization?
Zihao Wang, Victor Veitch · Machine learning
A basic aspiration for interpretability research in large language models is to "localize" semantically meaningful behaviors to particular components within the LLM.
- 2025-02arXiv 2502.05374Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyond
Chongyu Fan, Jinghan Jia, Yihua Zhang et al. · Machine learning
The LLM unlearning technique has recently been introduced to comply with data regulations and address the safety and ethical concerns of LLMs by removing the undesired data-model influence.
- 2025-02arXiv 2502.01636Lifelong Knowledge Editing requires Better Regularization
Akshat Gupta, Phudish Prateepamornkul, Maochuan Lu et al. · Language models & NLP
Knowledge editing is a promising way to improve factuality in large language models, but recent studies have shown significant model degradation during sequential editing.
- 2025-01arXiv 2501.13453Spurious Forgetting in Continual Learning of Language Models
Junhao Zheng, Xidi Cai, Shengjie Qiu et al. · Machine learning
Recent advancements in large language models (LLMs) reveal a perplexing phenomenon in continual learning: despite extensive training, models experience significant performance declines, raising questions about task alignment and underlying knowledge retention.
- 2024-12arXiv 2412.01004Take Only What You Need: Rank Minimization as an Implicit Forgetting Regularizer in Continual Learning
Haodong Lu, Chongyang Zhao, Minhui Xue et al. · Vision
The central tension in continual learning (CL) is the trade-off between plasticity (acquiring new knowledge) and stability (retaining prior knowledge).
- 2024-10arXiv 2410.17146LiNeS: Post-training Layer Scaling Prevents Forgetting and Enhances Model Merging
Ke Wang, Nikolaos Dimitriadis, Alessandro Favero et al. · Machine learning
Fine-tuning pre-trained models has become the standard approach to endow them with specialized knowledge, but it poses fundamental challenges.
- 2024-10arXiv 2410.16251Can Knowledge Editing Really Correct Hallucinations?
Baixiang Huang, Canyu Chen, Xiongxiao Xu et al. · Language models & NLP
Large Language Models (LLMs) suffer from hallucinations, referring to the non-factual information in generated content, despite their superior capacities across tasks.
- 2024-10arXiv 2410.08827Do Unlearning Methods Remove Information from Language Model Weights?
Aghyad Deeb, Fabien Roger · Machine learning
Large Language Models' knowledge of how to perform cyber-security attacks, create bioweapons, and manipulate humans poses risks of misuse. Previous work has proposed methods to unlearn this knowledge.
Built from every arXiv paper cited in our papers, posts, topic hubs, reports and research notes, with titles, authors, dates and abstracts from the arXiv API (updated 2026-11-12). Summaries are the opening sentences of each paper's own abstract, not our assessment; a citation means we used or checked the work, not that we endorse it. Fields follow each paper's primary arXiv category. Papers cited only outside arXiv (journals, proceedings) are not listed yet. Entries are generated from our citations, so a mistyped ID in a note would list the wrong paper; we remove those as we find them. Report a wrong entry.
