NeuroGenesysThe research we build on.
1,514 papers by other researchers that our work cites, from continual learning and verification to language models, agents and hardware. Each entry links to the paper on arXiv and to where we use it. 301 are cited on published pages of this site; the rest in the research notes behind our studies. Our own papers are in the archive.
- 2026-05arXiv 2605.29495On-Policy Replay for Continual Supervised Fine-Tuning
Yan Chen, Taojie Zhu, Meng Zhang et al. · Machine learning
Continual supervised fine-tuning (SFT) is the de facto recipe for adapting large language models (LLMs) to a stream of downstream tasks, but it suffers from catastrophic forgetting of earlier capabilities.
- 2026-05arXiv 2605.29800Nine Judges, Two Effective Votes: Correlated Errors Undermine LLM Evaluation Panels
Guneet Kohli · Language models & NLP
LLM-as-a-judge panels aggregate votes from multiple models, with the expectation that diverse models yield more reliable evaluations. We develop a framework to measure the true informational value of such panels and quantify how far their reliability falls short of the independent-voting ideal.
- 2026-05arXiv 2605.30148Overcoming Forgetting in LLM Fine-Tuning with Evolution Strategies
Kajetan Schweighofer, Conor F. Hayes, Roberto Dailey et al. · Machine learning
Evolution Strategies (ES) has recently emerged as a competitive alternative to reinforcement learning (RL) for large language model (LLM) fine-tuning, offering advantages through simplicity, scalability, and inference-only training.
- 2026-05arXiv 2605.28207Pruning and Distilling Mixture-of-Experts into Dense Language Models
Junhyuck Kim, Jihun Yun, Haechan Kim et al. · Language models & NLP
Mixture-of-Experts (MoE) is now the dominant architecture for frontier language models, yet it requires all expert parameters to be loaded in memory, making it less preferable for memory-constrained deployment.
- 2026-05arXiv 2605.28303From Fact Overwriting to Knowledge Evolution: Causal Editing via On-Policy Self-Distillation
Shuaike Li, Kai Zhang, Xianquan Wang et al. · AI & agents
While Knowledge Editing (KE) enables efficient updates, its dominant Static Fact Overwriting paradigm treats LLMs as discrete databases, forcibly injecting isolated facts.
- 2026-05arXiv 2605.28495Janus-LoRA: A Balanced Low-Rank Adaptation for Continual Learning
Cheng Chen, Pengpeng Zeng, Yuyu Guo et al. · Vision
Low-Rank Adaptation (LoRA) has emerged as a promising paradigm for Continual Learning. It independently updates its low-rank factors ( and ), creating a composite update to the full weight matrix through their interaction.
- 2026-05arXiv 2605.26526Open-Weight LLM Fine-Tuning Defenses are Susceptible to Simple Attacks
Kevin Kuo, Virginia Smith, Chhavi Yadav · Machine learning
Recent defenses for safeguarding open-weight large language models (LLMs) are intended to prevent adversarial usage. Underlying these defenses is an assumption that new harmful behavior is learned through fine-tuning rather than elicited by jailbreaking the model.
- 2026-05arXiv 2605.26820Can Vision-Language-Action Models Learn from Real-World Data Continually without Forgetting?
Jiarun Zhu, Yijun Hong, Xiaoquan Sun et al. · Robotics & control
Vision-Language-Action (VLA) models provide a promising foundation for general-purpose robotics, yet their real-world deployment demands the ability to continually acquire new skills without forgetting prior ones.
Cited inStudy: Genesys program prior art - 2026-05arXiv 2605.27276SIA: Self Improving AI with Harness & Weight Updates
Prannay Hebbar, Yogendra Manawat, Samuel Verboomen et al. · AI & agents
Humans are the bottleneck in building and improving AI. Both the models and the agents that wrap them are written, tuned, and corrected by people. The long-horizon goal of an AI that can figure out how to improve itself remains open.
- 2026-05arXiv 2605.25525SAE-FD: Sparse Autoencoder Feature Distillation for Continual Learning of Large Language Models
Mingxu Zhang, Yuhan Li, Lujundong Li et al. · Machine learning
Continual learning enables large language models to adapt to evolving tasks without retraining from scratch, yet catastrophic forgetting remains a central obstacle.
- 2026-05arXiv 2605.25850TIAR: Trajectory-Informed Advantage Reweighting for LLM Abstention Learning
Muyu Pan, Shu Zhao, Nan Zhang et al. · Language models & NLP
This paper investigates large language model (LLM) abstention learning, specifically using ternary reward, which incentivize truthfulness in large language models.
- 2026-05arXiv 2605.26097Forgetting in Language Models: Capacity, Optimization, and Self-Generated Replay
Martin Marek, Dongkyu Cho, Shikai Qiu et al. · Machine learning
Models trained on a new task typically degrade on prior tasks, a phenomenon known as forgetting. Traditionally, mitigating forgetting has required replaying stored exemplars from prior tasks, which is often impractical.
- 2026-05arXiv 2605.26099Do Language Models Need Sleep? Offline Recurrence for Improved Online Inference
Sangyun Lee, Sean McLeish, Tom Goldstein et al. · Language models & NLP
Transformer-based large language models are increasingly used for long-horizon tasks; however, their attention mechanism scales poorly with context length.
- 2026-05arXiv 2605.26242Can LLMs Introspect? A Reality Check
Shashwat Singh, Tal Linzen, Shauli Ravfogel · AI & agents
Can large language models detect and report their own internal states? A number of recent studies have argued that they can. Drawing on lessons from human metacognition research, we argue that this conclusion may be premature.
- 2026-05arXiv 2605.24426SEAL: Synergistic Co-Evolution of Agents and Learning Environments
Yihao Hu, Zhihao Wen, Pan Wang et al. · Language models & NLP
Large language model (LLM) agents increasingly learn through interaction, yet their training process often evolves unevenly: the policy changes with experience while the learning environment continues to provide largely fixed forms of support.
- 2026-05arXiv 2605.24660How Many Tools Should an LLM Agent See? A Chance-Corrected Answer
Vyzantinos Repantis, Ameya Gawde, Harshvardhan Singh et al. · Retrieval & data
Before an LLM agent can use a tool, a retrieval system must decide which candidate tools to show to the agent. How long should that shortlist be? Show too many tools and the model struggles to choose.
Cited inInternal notes - 2026-05arXiv 2605.24050More Skills, Worse Agents? Skill Shadowing Degrades Performance When Expanding Skill Libraries
Hongwen Song, Song Wei · Society, software & economics
Skill libraries allow LLM agents to load task-specific instructions on demand, letting non-expert users solve domain-specific tasks through natural language without knowing which skills exist or how they work.
- 2026-05arXiv 2605.20803Tunable MAGMAX: Preference-Aware Model Merging for Continual Learning
Kei Hiroshima, Kento Uchida, Shinichi Shirakawa · Machine learning
Continual learning (CL) aims to train models sequentially on multiple tasks while mitigating catastrophic forgetting of previously learned knowledge.
- 2026-05arXiv 2605.20926MemConflict: Evaluating Long-Term Memory Systems Under Memory Conflicts
Zhen Tao, Jinxiang Zhao, Peng Liu et al. · Retrieval & data
Long-term memory systems enable conversational agents based on large language models (LLMs) to retain, retrieve, and apply user-specific information across multi-session interactions.
Cited inInternal notes - 2026-05arXiv 2605.19242PhyWorld: Physics-Faithful World Model for Video Generation
Pu Zhao, Juyi Lin, Timothy Rupprecht et al. · Vision
World simulators can provide safe and scalable environments for training Physical AI systems before real-world deployment. Large video generation models are emerging as a promising basis for such simulators because they can generate diverse and realistic visual futures.
- 2026-05arXiv 2605.20296Spectral Unforgetting: Post-Hoc Recovery of Damaged Capabilities Without Retraining
Aarash Abro, Muhammad Tahir · Machine learning
Fine-tuning a language model for a target task routinely degrades capabilities the training data never explicitly threatened.
- 2026-05arXiv 2605.22863Latent Cache Flow: Model-to-Model Communication Without Text
Maximillian Rossi, Prajwal Raghunath, Eugene Wu · Machine learning
LLM agents today communicate via text, which incurs considerable latency and information loss due to the need to autoregressively decode the sharer model's state and encode at the receiver model.
- 2026-05arXiv 2605.18565MINTEval: Evaluating Memory under Multi-Target Interference in Long-Horizon Agent Systems
Hyunji Lee, Justin Chih-Yao Chen, Joykirat Singh et al. · Language models & NLP
Real-world agents operate over long and evolving horizons, where information is repeatedly updated and may interfere across memories, requiring accurate recall and aggregated reasoning over multiple pieces of information.
Cited inInternal notes - 2026-05arXiv 2605.18610CATA: Continual Machine Unlearning via Conflict-Averse Task Arithmetic
Shen Lin, Junhao Dong, Rongjie Chen et al. · Vision
Vision-language models (VLMs) have shown remarkable ability in aligning visual and textual representations, enabling a wide range of multimodal applications.
- 2026-05arXiv 2605.19193Sequential Consensus for Multi-Agent LLM Debates: A Wald-SPRT compute governor with calibration-based failure detection
Andrea Morandi · Machine learning
Multi-agent LLM debate improves factuality and reasoning, but most recipes pick a fixed round count, over-spending on easy items and under-spending on hard ones.
- 2026-05arXiv 2605.20247CP-MoE: Consistency-Preserving Mixture-of-Experts for Continual Learning
Yang Liu, Toan Nguyen, Flora D. Salim · Machine learning
Catastrophic forgetting remains a major obstacle to continual learning in large language models (LLMs) and vision--language models (VLMs).
- 2026-05arXiv 2605.17571Stable Routing for Mixture-of-Experts in Class-Incremental Learning
Zirui Guo, Quan Cheng, Da-Wei Zhou et al. · Vision
Class-incremental learning (CIL) requires models to learn new classes sequentially while preserving prior knowledge.
- 2026-05arXiv 2605.17609Adaptive Generate-Rank-Verify: Inference-Time Search with Costly Verification
Shaddin Dughmi, Mahdi Haghifam, Yusuf Hakan Kalayci · Machine learning
Many inference-time language-model pipelines combine a cheap reward signal with an expensive verifier, such as exact answer checking in mathematical reasoning or hidden-test execution in code generation.
- 2026-05arXiv 2605.16865MixSD: Mixed Contextual Self-Distillation for Knowledge Injection
Jiarui Liu, Lechen Zhang, Yongjin Yang et al. · Language models & NLP
Supervised fine-tuning (SFT) is widely used to inject new knowledge into language models, but it often degrades pretrained capabilities such as reasoning and general-domain performance.
- 2026-05arXiv 2605.16686Scalable Knowledge Editing for Mixture-of-Experts LLMs via Tensor-Structured Updates
Roman Maksimov, Vladimir Aletov, Dmitry Bylinkin et al. · Machine learning
Knowledge editing (KE) provides a lightweight alternative to repeated fine-tuning of LLMs. However, most existing KE methods target dense feed-forward layers, while modern LLMs increasingly adopt Mixture-of-Experts (MoE) architectures for their superior memory footprint and inference efficiency.
- 2026-05arXiv 2605.14438BEAM: Binary Expert Activation Masking for Dynamic Routing in MoE
Juntong Wu, Jialiang Cheng, Qishen Yin et al. · AI & agents
Mixture-of-Experts (MoE) architectures enhance the efficiency of large language models by activating only a subset of experts per token. However, standard MoE employs a fixed Top-K routing strategy, leading to redundant computation and suboptimal inference latency.
- 2026-05arXiv 2605.14477Test-Time Learning with an Evolving Library
Weijia Xu, Alessandro Sordoni, Chandan Singh et al. · Machine learning
We introduce EvoLib, a test-time learning framework that enables large language models to accumulate, reuse, and evolve knowledge across problem instances without parameter updates or external supervision.
Cited inInternal notes - 2026-05arXiv 2605.15053TFGN: Task-Free, Replay-Free Continual Pre-Training Without Catastrophic Forgetting at LLM Scale
Anurup Ganguli · Machine learning
Continually pre-training a large language model on heterogeneous text domains, without replay or task labels, has remained an unsolved architectural problem at LLM scale.
- 2026-05arXiv 2605.13997HodgeCover: Higher-Order Topological Coverage Drives Compression of Sparse Mixture-of-Experts
Tao Zhong, Dongzhe Zheng, Christine Allen-Blanchette · Machine learning
Sparse Mixture-of-Experts (MoE) layers route tokens through a handful of experts, and learning-free compression of these layers reduces inference cost without retraining.
- 2026-05arXiv 2605.11685Robust LLM Unlearning Against Relearning Attacks: The Minor Components in Representations Matter
Zeguan Xiao, Xuanzhe Xu, Yong Wang et al. · Language models & NLP
Large language model (LLM) unlearning aims to remove specific data influences from pre-trained model without costly retraining, addressing privacy, copyright, and safety concerns.
- 2026-05arXiv 2605.11836More Edits, More Stable: Understanding the Lifelong Normalization in Sequential Model Editing
Xin Ma, Wei Chen, Qi Liu et al. · Machine learning
Lifelong Model Editing aims to continuously update evolving facts in Large Language Models while preserving unrelated knowledge and general capabilities, yet it remains plagued by catastrophic forgetting and model collapse.
- 2026-05arXiv 2605.12128Metaphor Is Not All Attention Needs
Olga Sorokoletova, Francesco Giarrusso, Giacomo De Luca et al. · Language models & NLP
Large language models are increasingly deployed in safety-critical applications, where their ability to resist harmful instructions is essential.
- 2026-05arXiv 2605.12419ORBIT: Preserving Foundational Language Capabilities in GenRetrieval via Origin-Regulated Merging
Neha Verma, Nikhil Mehta, Shao-Chuan Wang et al. · Language models & NLP
Despite the rapid advancements in large language model (LLM) development, fine-tuning them for specific tasks often results in the catastrophic forgetting of their general, language-based reasoning abilities.
- 2026-05arXiv 2605.12484Learning, Fast and Slow: Towards LLMs That Adapt Continually
Rishabh Tiwari, Kusha Sareen, Lakshya A Agrawal et al. · Machine learning
, via RL). However, updating model parameters forces them to absorb task-specific information, which can result in catastrophic forgetting and loss of plasticity.
- 2026-05arXiv 2605.12752Low-Rank Adapters Initialization via Gradient Surgery for Continual Learning
Joana Pasquali, Ramiro N. Barros, Arthur S. Bianchessi et al. · Machine learning
LoRA is widely adopted for continual fine-tuning of Large Language Models due to its parameter efficiency, modularity across tasks, and compatibility with replay strategies.
Built from every arXiv paper cited in our papers, posts, topic hubs, reports and research notes, with titles, authors, dates and abstracts from the arXiv API (updated 2026-11-12). Summaries are the opening sentences of each paper's own abstract, not our assessment; a citation means we used or checked the work, not that we endorse it. Fields follow each paper's primary arXiv category. Papers cited only outside arXiv (journals, proceedings) are not listed yet. Entries are generated from our citations, so a mistyped ID in a note would list the wrong paper; we remove those as we find them. Report a wrong entry.
