NeuroGenesysThe research we build on.
1,514 papers by other researchers that our work cites, from continual learning and verification to language models, agents and hardware. Each entry links to the paper on arXiv and to where we use it. 301 are cited on published pages of this site; the rest in the research notes behind our studies. Our own papers are in the archive.
- 2025-04arXiv 2504.14530Causality for Natural Language Processing
Zhijing Jin · Language models & NLP
Causal reasoning is a cornerstone of human intelligence and a critical capability for artificial systems aiming to achieve advanced understanding and decision-making.
- 2025-04arXiv 2504.13146Antidistillation Sampling
Yash Savani, Asher Trockman, Zhili Feng et al. · AI & agents
Frontier models that generate extended reasoning traces inadvertently produce rich token sequences that can facilitate model distillation. Recognizing this vulnerability, model owners may seek sampling strategies that limit the effectiveness of distillation without compromising model performance.
- 2025-04arXiv 2504.13151MIB: A Mechanistic Interpretability Benchmark
Aaron Mueller, Atticus Geiger, Sarah Wiegreffe et al. · Machine learning
How can we know whether new mechanistic interpretability methods achieve real improvements? In pursuit of lasting evaluation standards, we propose MIB, a Mechanistic Interpretability Benchmark, with two tracks spanning four tasks and five models.
- 2025-04arXiv 2504.13171Sleep-time Compute: Beyond Inference Scaling at Test-time
Kevin Lin, Charlie Snell, Yu Wang et al. · AI & agents
Scaling test-time compute has emerged as a key ingredient for enabling large language models (LLMs) to solve difficult problems, but comes with high latency and inference cost.
- 2025-04arXiv 2504.12461On the Need to Rethink Trust in AI Assistants for Software Development: A Critical Review
Sebastian Baltes, Timo Speith, Brenda Chiteri et al. · Society, software & economics
Trust is a fundamental concept in human decision-making and collaboration that has long been studied in philosophy and psychology.
- 2025-04arXiv 2504.09522How new data permeates LLM knowledge and how to dilute it
Chen Sun, Renat Aksitov, Andrey Zhmoginov et al. · Language models & NLP
Large language models learn and continually learn through the accumulation of gradient-based updates, but how individual pieces of new information affect existing knowledge, leading to both beneficial generalization and problematic hallucination, remains poorly understood.
- 2025-04arXiv 2504.08192SAEs Can Improve Unlearning: Dynamic Sparse Autoencoder Guardrails for Precision Unlearning in LLMs
Aashiq Muhamed, Jacopo Bonato, Mona Diab et al. · Machine learning
Machine unlearning is a promising approach to improve LLM safety by removing unwanted knowledge from the model.
- 2025-04arXiv 2504.08066The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search
Yutaro Yamada, Robert Tjarko Lange, Cong Lu et al. · AI & agents
AI is increasingly playing a pivotal role in transforming how scientific discoveries are made. We introduce The AI Scientist-v2, an end-to-end agentic system capable of producing the first entirely AI generated peer-review-accepted workshop paper.
- 2025-04arXiv 2504.06564Thinking Out Loud: Do Reasoning Models Know When They're Right?
Qingcheng Zeng, Weihao Xuan, Leyang Cui et al. · Language models & NLP
Large reasoning models (LRMs) have recently demonstrated impressive capabilities in complex reasoning tasks by leveraging increased test-time computation and exhibiting behaviors reminiscent of human-like self-reflection.
- 2025-04arXiv 2504.07097Sculpting Subspaces: Constrained Full Fine-Tuning in LLMs for Continual Learning
Nikhil Shivakumar Nayak, Krishnateja Killamsetty, Ligong Han et al. · Machine learning
Continual learning in large language models (LLMs) is prone to catastrophic forgetting, where adapting to new tasks significantly degrades performance on previously learned ones.
- 2025-04arXiv 2504.06188SkillFlow: Scalable and Efficient Agent Skill Retrieval System
Fangzhou Li, Pagkratios Tagkopoulos, Ilias Tagkopoulos · AI & agents
AI agents can extend their capabilities at inference time by loading reusable skills into context, yet equipping an agent with too many skills, particularly irrelevant ones, degrades performance.
Cited inStudy: Genesys program prior art - 2025-04arXiv 2504.06426S'MoRE: Structural Mixture of Residual Experts for Parameter-Efficient LLM Fine-tuning
Hanqing Zeng, Yinglong Xia, Zhuokai Zhao et al. · Language models & NLP
Fine-tuning pre-trained large language models (LLMs) presents a dual challenge of balancing parameter efficiency and model capacity.
- 2025-04arXiv 2504.03579Hallucination Detection on a Budget: Efficient Bayesian Estimation of Semantic Entropy
Kamil Ciosek, Nicolò Felicioni, Sina Ghiassian · Machine learning
Detecting whether an LLM hallucinates is an important research challenge., 2024) of the distribution of generated sequences. We propose a new algorithm for doing that, with two main advantages.
- 2025-04arXiv 2504.01848PaperBench: Evaluating AI's Ability to Replicate AI Research
Giulio Starace, Oliver Jaffe, Dane Sherburn et al. · AI & agents
We introduce PaperBench, a benchmark evaluating the ability of AI agents to replicate state-of-the-art AI research. Agents must replicate 20 ICML 2024 Spotlight and Oral papers from scratch, including understanding paper contributions, developing a codebase, and successfully executing experiments.
Cited inStudy: Superintelligence - 2025-04arXiv 2504.02107TiC-LM: A Web-Scale Benchmark for Time-Continual LLM Pretraining
Jeffrey Li, Mohammadreza Armandpour, Iman Mirzadeh et al. · Machine learning
Large Language Models (LLMs) trained on historical web data inevitably become outdated. We investigate evaluation strategies and update methods for LLMs as new data becomes available.
Cited inEWC protects what its Fisher sees: a literature synthesis on elastic weight consolidation, scale and coherenceStudy: Continual learning order and companion dataStudy: Elastic weight consolidation deep diveStudy: Fortifying knowledge in continual learningStudy: Hardware guardian core for continual learning - 2025-03arXiv 2503.24354ORAL: Prompting Your Large-Scale LoRAs via Conditional Recurrent Diffusion
Rana Muhammad Shahroz Khan, Dongwen Tang, Pingzhi Li et al. · Machine learning
Parameter generation has emerged as a novel paradigm for neural network development, offering an alternative to traditional neural network training by synthesizing high-quality model weights directly.
- 2025-03arXiv 2504.00180Contradiction Detection in RAG Systems: Evaluating LLMs as Context Validators for Improved Information Consistency
Vignesh Gokul, Srikanth Tenneti, Alwarappan Nakkiran · Language models & NLP
Retrieval Augmented Generation (RAG) systems have emerged as a powerful method for enhancing large language models (LLMs) with up-to-date information.
- 2025-03arXiv 2503.21227LLaVA-CMoE: Towards Continual Mixture of Experts for Large Vision-Language Models
Hengyuan Zhao, Ziqin Wang, Qixin Sun et al. · Language models & NLP
Mixture of Experts (MoE) architectures have recently advanced the scalability and adaptability of large language models (LLMs) for continual multimodal learning.
- 2025-03arXiv 2503.21878Is Best-of-N the Best of Them? Coverage, Scaling, and Optimality in Inference-Time Alignment
Audrey Huang, Adam Block, Qinghua Liu et al. · AI & agents
Inference-time computation offers a powerful axis for scaling the performance of language models. However, naively increasing computation in techniques like Best-of-N sampling can lead to performance degradation due to reward hacking.
- 2025-03arXiv 2503.20523GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Driving
Lloyd Russell, Anthony Hu, Lorenzo Bertoni et al. · Vision
Generative models offer a scalable and flexible paradigm for simulating complex environments, yet current approaches fall short in addressing the domain-specific requirements of autonomous driving - such as multi-agent interactions, fine-grained control, and multi-camera consistency.
- 2025-03arXiv 2503.19939Continual Learning With Quasi-Newton Methods
Steven Vander Eeckt, Hugo Van hamme · IEEE Access, vol. 13, pp. 47485-47499, 2025 · Machine learning
Catastrophic forgetting remains a major challenge when neural networks learn tasks sequentially. Elastic Weight Consolidation (EWC) attempts to address this problem by introducing a Bayesian-inspired regularization loss to preserve knowledge of previously learned tasks.
- 2025-03arXiv 2503.20018Experience Replay Addresses Loss of Plasticity in Continual Learning
Jiuqi Wang, Rohan Chandra, Shangtong Zhang · Machine learning
Loss of plasticity is one of the main challenges in continual learning with deep neural networks, where neural networks trained via backpropagation gradually lose their ability to adapt to new tasks and perform significantly worse than their freshly initialized counterparts.
- 2025-03arXiv 2503.18866Reasoning to Learn from Latent Thoughts
Yangjun Ruan, Neil Band, Chris J. Maddison et al. · Machine learning
Compute scaling for language model (LM) pretraining has outpaced the growth of human-written texts, leading to concerns that data will become the bottleneck to LM scaling.
- 2025-03arXiv 2503.19206Overtrained Language Models Are Harder to Fine-Tune
Jacob Mitchell Springer, Sachin Goyal, Kaiyue Wen et al. · Language models & NLP
Large language models are pre-trained on ever-growing token budgets under the assumption that better pre-training performance translates to improved downstream models.
- 2025-03arXiv 2503.16779Chain-of-Tools: Utilizing Massive Unseen Tools in the CoT Reasoning of Frozen Language Models
Mengsong Wu, Tong Zhu, Han Han et al. · Language models & NLP
Tool learning can further broaden the usage scenarios of large language models (LLMs). However most of the existing methods either need to finetune that the model can only use tools seen in the training data, or add tool demonstrations into the prompt with lower efficiency.
Cited inInternal notes - 2025-03arXiv 2503.15888Parameters vs. Context: Fine-Grained Control of Knowledge Reliance in Language Models
Baolong Bi, Shenghua Liu, Yiwei Wang et al. · Language models & NLP
Retrieval-Augmented Generation (RAG) mitigates hallucinations in Large Language Models (LLMs) by integrating external knowledge.
- 2025-03arXiv 2503.15299Inside-Out: Hidden Factual Knowledge in LLMs
Zorik Gekhman, Eyal Ben David, Hadas Orgad et al. · Language models & NLP
This work presents a framework for assessing whether large language models (LLMs) encode more factual knowledge in their parameters than what they express in their outputs.
- 2025-03arXiv 2503.15427Direct comparison of stochastic driven nonlinear dynamical systems for combinatorial optimization
Junpeng Hou, Amin Barzegar, Helmut G. Katzgraber · Phys. Rev. E 112, 035301 (2025) · Hardware, photonics & physics
Combinatorial optimization problems are ubiquitous in industrial applications. However, finding optimal or close-to-optimal solutions can often be extremely hard.
- 2025-03arXiv 2503.14499Measuring AI Ability to Complete Long Software Tasks
Thomas Kwa, Ben West, Joel Becker et al. · NeurIPS 2025 · AI & agents
Despite rapid progress on AI benchmarks, the real-world meaning of benchmark performance remains unclear. To quantify the capabilities of AI systems in terms of human capabilities, we propose a new metric: 50%-task-completion time horizon.
- 2025-03arXiv 2503.13575Analytic Subspace Routing: How Recursive Least Squares Works in Continual Learning of Large Language Model
Kai Tong, Kang Pan, Xiao Zhang et al. · Machine learning
Large Language Models (LLMs) possess encompassing capabilities that can process diverse language-related tasks. However, finetuning on LLMs will diminish this general skills and continual finetuning will further cause severe degradation on accumulated knowledge.
- 2025-03arXiv 2503.13657Why Do Multi-Agent LLM Systems Fail?
Mert Cemri, Melissa Z. Pan, Shuyi Yang et al. · AI & agents
Despite enthusiasm for Multi-Agent LLM Systems (MAS), their performance gains on popular benchmarks are often minimal. This gap highlights a critical need for a principled understanding of why MAS fail.
- 2025-03arXiv 2503.11127Don't Forget It! Conditional Sparse Autoencoder Clamping Works for Unlearning
Matthew Khoriaty, Andrii Shportko, Gustavo Mercier et al. · Machine learning
Recent developments in Large Language Model (LLM) capabilities have brought great potential but also posed new risks. For example, LLMs with knowledge of bioweapons, advanced chemistry, or cyberattacks could cause violence if placed in the wrong hands or during malfunctions.
- 2025-03arXiv 2503.11926Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation
Bowen Baker, Joost Huizinga, Leo Gao et al. · AI & agents
Mitigating reward hacking--where AI systems misbehave due to flaws or misspecifications in their learning objectives--remains a key challenge in constructing capable and aligned models.
- 2025-03arXiv 2503.07137A Comprehensive Survey of Mixture-of-Experts: Algorithms, Theory, and Applications
Siyuan Mu, Sen Lin · Machine learning
Artificial intelligence (AI) has achieved astonishing successes in many domains, especially with the recent breakthroughs in the development of foundational large models.
- 2025-03arXiv 2503.10657RouterEval: A Comprehensive Benchmark for Routing LLMs to Explore Model-level Scaling Up in LLMs
Zhongzhan Huang, Guoming Ling, Yupei Lin et al. · Language models & NLP
Routing large language models (LLMs) is a new paradigm that uses a router to recommend the best LLM from a pool of candidates for a given input., a capable router can significantly enhance the performance of this paradigm as the number of candidates increases.
- 2025-03arXiv 2503.05683WikiBigEdit: Understanding the Limits of Lifelong Knowledge Editing in LLMs
Lukas Thede, Karsten Roth, Matthias Bethge et al. · Language models & NLP
Keeping large language models factually up-to-date is crucial for deployment, yet costly retraining remains a challenge. Knowledge editing offers a promising alternative, but methods are only tested on small-scale or synthetic edit benchmarks.
Cited inEWC protects what its Fisher sees: a literature synthesis on elastic weight consolidation, scale and coherenceStudy: Consequence consistent updatesStudy: Continual learning order and companion dataStudy: Elastic weight consolidation deep diveStudy: Fortifying knowledge in continual learningStudy: LLMs modifying open model weightsStudy: Logical coherence of LLM world modelsStudy: Orchestrated specialists and bridge synthesisStudy: Weight editing to prevent forgetting - 2025-03arXiv 2503.05029Continual Pre-training of MoEs: How robust is your router?
Benjamin Thérien, Charles-Étienne Joseph, Zain Sarwar et al. · Machine learning
Sparsely-activated Mixture of Experts (MoE) transformers are promising architectures for foundation models.
- 2025-03arXiv 2503.03704Memory Injection Attacks on LLM Agents via Query-Only Interaction
Shen Dong, Shaochen Xu, Pengfei He et al. · Machine learning
Agents powered by large language models (LLMs) have demonstrated strong capabilities in a wide range of complex, real-world applications. However, LLM agents with a compromised memory bank may easily produce harmful outputs when the past records retrieved for demonstration are malicious.
- 2025-03arXiv 2503.07639Mixture of Experts Made Intrinsically Interpretable
Xingyi Yang, Constantin Venhoff, Ashkan Khakzar et al. · Machine learning
Neurons in large language models often exhibit \emph{polysemanticity}, simultaneously encoding multiple unrelated concepts and obscuring interpretability.
- 2025-03arXiv 2503.02701MindBridge: Scalable and Cross-Model Knowledge Editing via Memory-Augmented Modality
Shuaike Li, Kai Zhang, Qi Liu et al. · AI & agents
Knowledge editing is a technique for efficiently and accurately updating the knowledge of large language models (LLMs) to alleviate obsolescence and correct errors.
Built from every arXiv paper cited in our papers, posts, topic hubs, reports and research notes, with titles, authors, dates and abstracts from the arXiv API (updated 2026-11-12). Summaries are the opening sentences of each paper's own abstract, not our assessment; a citation means we used or checked the work, not that we endorse it. Fields follow each paper's primary arXiv category. Papers cited only outside arXiv (journals, proceedings) are not listed yet. Entries are generated from our citations, so a mistyped ID in a note would list the wrong paper; we remove those as we find them. Report a wrong entry.
