NeuroGenesysThe research we build on.
1,514 papers by other researchers that our work cites, from continual learning and verification to language models, agents and hardware. Each entry links to the paper on arXiv and to where we use it. 301 are cited on published pages of this site; the rest in the research notes behind our studies. Our own papers are in the archive.
- 2026-10arXiv 2610.36994CaLLM: Continual Adaptation of LLMs via Non-Parametric Adapter Routing
Bilge Celik, Anna Vettoruzzo, Joaquin Vanschoren · Machine learning
Continual adaptation of large language models (LLMs) to evolving task distributions while mitigating catastrophic forgetting remains an open problem. Existing strategies often struggle to scale or lack a robust mechanism for zero-shot task-to-expert assignment.
- 2026-08arXiv 2608.27046Performance Foundations of Parallel & Distributed Reasoning Language Models
Maciej Besta, Leonard Schmidt, Lara Nonino et al. · Machine learning
Reinforcement Learning with Verifiable Rewards (RLVR) and other RL-style post-training paradigms have been used for aligning large language models (LLMs) with reasoning standards.
- 2026-08arXiv 2608.09819Macaron-V1: Towards Open Continual Learning with Self-Improvement and Mixture-of-LoRA
Mind Lab, :, Vin Bo et al. · Machine learning
Macaron-V1 is an open agent-model family for experiential intelligence: learning from experience in real environments and continuing to learn after deployment. It is organized around two system goals.
- 2026-07arXiv 2607.23837Latent-LoRA: Compact Latent-Space Adapters with Gradient-Free Routing for Continual Learning
Reza Rahimi Azghan, Gautham Krishna Gudur, Giulia Pedrielli et al. · Machine learning
Large language models generalize well to individual tasks but lack an inherent mechanism for learning them sequentially, leading to catastrophic forgetting.
- 2026-07arXiv 2607.01763Denser Better: Limits of On-Policy Self-Distillation for Continual Post-Training
Meng Wang, Haohan Zhao, Wenzhuo Liu et al. · Machine learning
Continual post-training enables foundation models to acquire new knowledge while preserving existing capabilities. Recent work suggests that on-policy learning can mitigate forgetting, with self-distillation as a particularly attractive approach.
- 2026-06arXiv 2606.22030When Does Belief-Based Agent Memory Help? Reliability-Conditional Updating and Provenance-Capped Poisoning Defense
Pranav Singh · AI & agents
We investigate when belief-based memory actually improves large language model (LLM) agents.
- 2026-06arXiv 2606.18024Catastrophic Forgetting is Low-Rank: A Function-Space Theory for Continual Adaptation
Ido Nitzan Hidekel, Dan Raviv · Machine learning
Catastrophic forgetting in continual adaptation is usually studied through parameter drift, replay, or distillation, but these views do not identify which output-space directions are vulnerable.
- 2026-06arXiv 2606.03979Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories
Ali Behrouz, Farnoosh Hashemi, Adel Javanmard et al. · Machine learning
The past few decades have witnessed significant advances in the design of machine learning algorithms, from early studies on task-specific shallow models to more general deep Large Language Models (LLMs).
Cited inInvestors are paying for agents that learn on the job. The hard part is knowing what they forgot.To keep what a model knows, control what it is allowed to moveWhat a model lets into long-term memory matters more than how it learnsStudy: Fortifying knowledge in continual learningStudy: Genesys program prior artStudy: Hardware guardian core for continual learningStudy: LLMs modifying open model weightsStudy: Logical coherence of LLM world modelsStudy: Memory world models and source trustStudy: Seven continual learning methods tested - 2026-05arXiv 2605.12484Learning, Fast and Slow: Towards LLMs That Adapt Continually
Rishabh Tiwari, Kusha Sareen, Lakshya A Agrawal et al. · Machine learning
, via RL). However, updating model parameters forces them to absorb task-specific information, which can result in catastrophic forgetting and loss of plasticity.
- 2026-04arXiv 2604.14375Modular Continual Learning via Zero-Leakage Reconstruction Routing and Autonomous Task Discovery
Noureddine Kermiche · Machine learning
Catastrophic forgetting remains a primary hurdle in sequential task learning for artificial neural networks. We propose a silicon-native modular architecture that achieves structural parameter isolation using Task-Specific Experts and a distributed, outlier-based Gatekeeper.
- 2026-03arXiv 2603.12658Beyond Static Models: An Evolving Framework for Continual Learning in Large Language Models across Training Stages
Hongyang Chen, Zhongwu Sun, Hongfei Ye et al. · Language models & NLP
Continual learning (CL) has emerged as a pivotal paradigm to enable large language models (LLMs) to dynamically adapt to evolving knowledge and sequential tasks while mitigating catastrophic forgetting, a critical limitation of the static pre-training paradigm inherent to modern LLMs.
- 2026-01arXiv 2602.02543Norm Anchors Make Model Edits Last
Mingda Liu, Zhenghan Zhu, Ze'an Miao et al. · Machine learning
Sequential Locate-and-Edit (L&E) model editing can fail abruptly after many edits.
- 2026-01arXiv 2601.21577Collaborative Parameter Learning: Mitigating Forgetting via Parameter-Level Gradient Analysis
Mutian Yang, Zisen Zhan, Yutong Chen et al. · Machine learning
Catastrophic forgetting during knowledge injection impairs the ability of large language models to acquire new knowledge without overwriting previously mastered knowledge.
- 2026-01arXiv 2601.19897Self-Distillation Enables Continual Learning
Idan Shenfeld, Mehul Damani, Jonas Hübotter et al. · Machine learning
Continual learning, enabling models to acquire new skills and knowledge without degrading existing capabilities, remains a fundamental challenge for foundation models.
- 2026-01arXiv 2601.18699Mechanistic Analysis of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning
Gustav Olaf Yunus Laitinen-Fredriksson Lundstrom-Imanov · Machine learning
Sequential fine-tuning of Large Language Models (LLMs) adaptation to target tasks often triggers catastrophic forgetting, where the acquisition of novel target skills degrades ancestral capabilities.
- 2026-01arXiv 2601.15495Tracking the Limits of Knowledge Propagation: How LLMs Fail at Multi-Step Reasoning with Conflicting Knowledge
Yiyang Feng, Zeming Chen, Haotian Wu et al. · AI & agents
A common solution for mitigating outdated or incorrect information in Large Language Models (LLMs) is to provide updated facts in-context or through knowledge editing.
- 2025-12arXiv 2512.23675End-to-End Test-Time Training for Long Context
Arnuv Tandon, Karan Dalal, Xinhao Li et al. · Machine learning
We formulate long-context language modeling as a problem in continual learning rather than architecture design. Under this formulation, we only use a standard architecture -- a Transformer with sliding-window attention.
- 2025-12arXiv 2512.17299M2RU: Memristive Minion Recurrent Unit for On-Chip Continual Learning at the Edge
Abdullah M. Zyarah, Dhireesha Kudithipudi · Machine learning
Continual learning on edge platforms remains challenging because recurrent networks depend on energy-intensive training procedures and frequent data movement that are impractical for embedded deployments.
- 2025-12arXiv 2512.14741Persistent Backdoor Attacks under Continual Fine-Tuning of LLMs
Jing Cui, Yufei Han, Jianbin Jiao et al. · Security
Backdoor attacks embed malicious behaviors into Large Language Models (LLMs), enabling adversaries to trigger harmful outputs or bypass safety controls. However, the persistence of the implanted backdoors under user-driven post-deployment continual fine-tuning has been rarely examined.
- 2025-11arXiv 2511.01553Online Continual Learning on Intel Loihi 2 via a Co-designed Spiking Neural Network
Elvin Hajizada, Danielle Rager, Timothy Shea et al. · Machine learning
Edge AI must adapt to new classes and shifting data within a tight power budget, but online continual learning (learning from each sample before the next arrives) is where conventional processors are least efficient: each small update pays a full dispatch at full power.
- 2025-10arXiv 2510.15103Continual Learning via Sparse Memory Finetuning
Jessy Lin, Luke Zettlemoyer, Gargi Ghosh et al. · Language models & NLP
Modern language models are powerful, but typically static after deployment. A major obstacle to building models that continually learn over time is catastrophic forgetting, where updating on new data erases previously acquired capabilities.
Cited inTo keep what a model knows, control what it is allowed to moveStudy: Fortifying knowledge in continual learningStudy: Genesys program prior artStudy: Hardware guardian core for continual learningStudy: Memory world models and source trustStudy: Seven continual learning methods testedStudy: Weight editing to prevent forgetting - 2025-09arXiv 2509.04259RL's Razor: Why Online Reinforcement Learning Forgets Less
Idan Shenfeld, Jyothish Pari, Pulkit Agrawal · Machine learning
Comparison of fine-tuning models with reinforcement learning (RL) and supervised fine-tuning (SFT) reveals that, despite similar performance at a new task, RL preserves prior knowledge and capabilities significantly better.
- 2025-08arXiv 2508.15253Conflict-Aware Soft Prompting for Retrieval-Augmented Generation
Eunseong Choi, June Park, Hyeri Lee et al. · Language models & NLP
Retrieval-augmented generation (RAG) enhances the capabilities of large language models (LLMs) by incorporating external knowledge into their input prompts.
- 2025-08arXiv 2508.12531Rethinking Safety in LLM Fine-tuning: An Optimization Perspective
Minseon Kim, Jin Myung Kwak, Lama Alssum et al. · Machine learning
, refusing to respond to harmful user requests, even when using harmless datasets, thus requiring additional safety measures.
- 2025-07arXiv 2507.02770Blueprint, Bootstrap, and Bridge: A Security Look at NVIDIA GPU Confidential Computing
Zhongshu Gu, Enriquillo Valdez, Salman Ahmed et al. · Security
NVIDIA GPU Confidential Computing (GPU-CC) aims to provide secure execution for AI workloads. For end users, enabling GPU-CC is seamless and requires no modifications to existing applications.
- 2025-05arXiv 2505.05017Scalable Multi-Stage Influence Function for Large Language Models via Eigenvalue-Corrected Kronecker-Factored Parameterization
Yuntai Bao, Xuhong Zhang, Tianyu Du et al. · Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence Main Track (2025) 8022-8030 · Language models & NLP
Pre-trained large language models (LLMs) are commonly fine-tuned to adapt to downstream tasks. Since the majority of knowledge is acquired during pre-training, attributing the predictions of fine-tuned LLMs to their pre-training data may provide valuable insights.
- 2025-04arXiv 2504.02107TiC-LM: A Web-Scale Benchmark for Time-Continual LLM Pretraining
Jeffrey Li, Mohammadreza Armandpour, Iman Mirzadeh et al. · Machine learning
Large Language Models (LLMs) trained on historical web data inevitably become outdated. We investigate evaluation strategies and update methods for LLMs as new data becomes available.
Cited inEWC protects what its Fisher sees: a literature synthesis on elastic weight consolidation, scale and coherenceStudy: Continual learning order and companion dataStudy: Elastic weight consolidation deep diveStudy: Fortifying knowledge in continual learningStudy: Hardware guardian core for continual learning - 2025-03arXiv 2503.15888Parameters vs. Context: Fine-Grained Control of Knowledge Reliance in Language Models
Baolong Bi, Shenghua Liu, Yiwei Wang et al. · Language models & NLP
Retrieval-Augmented Generation (RAG) mitigates hallucinations in Large Language Models (LLMs) by integrating external knowledge.
- 2025-03arXiv 2503.03704Memory Injection Attacks on LLM Agents via Query-Only Interaction
Shen Dong, Shaochen Xu, Pengfei He et al. · Machine learning
Agents powered by large language models (LLMs) have demonstrated strong capabilities in a wide range of complex, real-world applications. However, LLM agents with a compromised memory bank may easily produce harmful outputs when the past records retrieved for demonstration are malicious.
- 2025-02arXiv 2502.07274Forget Forgetting: Continual Learning in a World of Abundant Memory
Dongkyu Cho, Taesup Moon, Rumi Chunara et al. · Machine learning
Continual learning (CL) has traditionally focused on minimizing exemplar memory, a constraint often misaligned with modern systems where GPU time, not storage, is the primary bottleneck.
- 2025-01arXiv 2501.13453Spurious Forgetting in Continual Learning of Language Models
Junhao Zheng, Xidi Cai, Shengjie Qiu et al. · Machine learning
Recent advancements in large language models (LLMs) reveal a perplexing phenomenon in continual learning: despite extensive training, models experience significant performance declines, raising questions about task alignment and underlying knowledge retention.
- 2024-12arXiv 2412.09764Memory Layers at Scale
Vincent-Pierre Berges, Barlas Oğuz, Daniel Haziza et al. · Language models & NLP
Memory layers use a trainable key-value lookup mechanism to add extra parameters to a model without increasing FLOPs. Conceptually, sparsely activated memory layers complement compute-heavy dense feed-forward layers, providing dedicated capacity to store and retrieve information cheaply.
- 2024-11arXiv 2411.17367Efficient transformer adaptation for analog in-memory computing via low-rank adapters
Chen Li, Elena Ferro, Corey Lammie et al. · Neuromorphic Computing and Engineering 6, 014011 (2026) · Hardware, photonics & physics
Analog In-Memory Computing (AIMC) offers a promising solution to the von Neumann bottleneck. However, deploying transformer models on AIMC remains challenging due to their inherent need for flexibility and adaptability across diverse tasks.
- 2024-10arXiv 2410.18785Should We Really Edit Language Models? On the Evaluation of Edited Language Models
Qi Li, Xiang Liu, Zhenheng Tang et al. · AI & agents
Model editing has become an increasingly popular alternative for efficiently updating knowledge within language models. Current methods mainly focus on reliability, generalization, and locality, with many methods excelling across these criteria.
- 2024-10arXiv 2410.10868Large Continual Instruction Assistant
Jingyang Qiao, Zhizhong Zhang, Xin Tan et al. · Machine learning
Continual Instruction Tuning (CIT) is adopted to continually instruct Large Models to follow human intent data by data. It is observed that existing gradient update would heavily destroy the performance on previous datasets during CIT process.
- 2024-10arXiv 2410.02355AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models
Junfeng Fang, Houcheng Jiang, Kun Wang et al. · 13th International Conference on Learning Representations (ICLR 2025 Oral) · Language models & NLP
Large language models (LLMs) often exhibit hallucinations due to incorrect or outdated knowledge. Hence, model editing methods have emerged to enable targeted knowledge updates.
- 2024-09arXiv 2409.19315Analog In-Memory Computing Attention Mechanism for Fast and Energy-Efficient Large Language Models
Nathan Leroux, Paul-Philipp Manea, Chirag Sudarshan et al. · Neuroscience & neural computation
Transformer networks, driven by self-attention, are central to Large Language Models. In generative Transformers, self-attention uses cache memory to store token projections, avoiding recomputation at each time step.
- 2024-09arXiv 2409.03992Confidential Computing on NVIDIA Hopper GPUs: A Performance Benchmark Study
Jianwei Zhu, Hang Yin, Peng Deng et al. · Other
This report evaluates the performance impact of enabling Trusted Execution Environments (TEE) on NVIDIA Hopper GPUs for large language model (LLM) inference tasks.
- 2024-07arXiv 2407.09435MUSCLE: A Model Update Strategy for Compatible LLM Evolution
Jessica Echterhoff, Fartash Faghri, Raviteja Vemulapalli et al. · AI & agents
Large Language Models (LLMs) are regularly updated to enhance performance, typically through changes in data or architecture. Within the update process, developers often prioritize improving overall performance metrics, paying less attention to maintaining compatibility with earlier model versions.
- 2024-07arXiv 2407.08699Mitigating Catastrophic Forgetting in Language Transfer via Model Merging
Anton Alexandrov, Veselin Raychev, Mark Niklas Müller et al. · Machine learning
As open-weight large language models (LLMs) achieve ever more impressive performances across a wide range of tasks in English, practitioners aim to adapt these models to different languages.
Built from every arXiv paper cited in our papers, posts, topic hubs, reports and research notes, with titles, authors, dates and abstracts from the arXiv API (updated 2026-11-12). Summaries are the opening sentences of each paper's own abstract, not our assessment; a citation means we used or checked the work, not that we endorse it. Fields follow each paper's primary arXiv category. Papers cited only outside arXiv (journals, proceedings) are not listed yet. Entries are generated from our citations, so a mistyped ID in a note would list the wrong paper; we remove those as we find them. Report a wrong entry.
