NeuroGenesysThe research we build on.
1,514 papers by other researchers that our work cites, from continual learning and verification to language models, agents and hardware. Each entry links to the paper on arXiv and to where we use it. 301 are cited on published pages of this site; the rest in the research notes behind our studies. Our own papers are in the archive.
- 2025-03arXiv 2503.01090Precise Localization of Memories: A Fine-grained Neuron-level Knowledge Editing Technique for LLMs
Haowen Pan, Xiaozhi Wang, Yixin Cao et al. · Language models & NLP
Knowledge editing aims to update outdated information in Large Language Models (LLMs). A representative line of study is locate-then-edit methods, which typically employ causal tracing to identify the modules responsible for recalling factual knowledge about entities.
- 2025-03arXiv 2503.01763Retrieval Models Aren't Tool-Savvy: Benchmarking Tool Retrieval for Large Language Models
Zhengliang Shi, Yuhan Wang, Lingyong Yan et al. · Language models & NLP
Tool learning aims to augment large language models (LLMs) with diverse tools, enabling them to act as agents for solving practical tasks.
- 2025-02arXiv 2502.20379Multi-Agent Verification: Scaling Test-Time Compute with Multiple Verifiers
Shalev Lifshitz, Sheila A. McIlraith, Yilun Du · AI & agents
By utilizing more computational resources at test-time, large language models (LLMs) can improve without additional training. One common strategy uses verifiers to evaluate candidate outputs.
- 2025-02arXiv 2502.18864Accelerating scientific discovery with Co-Scientist
Juraj Gottweis, Wei-Hung Weng, Alexander Daryin et al. · AI & agents
Scientific discovery is driven by scientists generating novel hypotheses for complex problems that undergo rigorous experimental validation. To augment this process, we introduce Co-Scientist, a multi-agent AI system built on Gemini for structured scientific thinking and hypothesis generation.
- 2025-02arXiv 2502.18940MathTutorBench: A Benchmark for Measuring Open-ended Pedagogical Capabilities of LLM Tutors
Jakub Macina, Nico Daheim, Ido Hakimi et al. · Language models & NLP
Evaluating the pedagogical capabilities of AI-based tutoring models is critical for making guided progress in the field. Yet, we lack a reliable, easy-to-use, and simple-to-run evaluation that reflects the pedagogical abilities of models.
- 2025-02arXiv 2502.19261Drop-Upcycling: Training Sparse Mixture of Experts with Partial Re-initialization
Taishi Nakamura, Takuya Akiba, Kazuki Fujii et al. · Language models & NLP
The Mixture of Experts (MoE) architecture reduces the training and inference cost significantly compared to a dense model of equivalent capacity. Upcycling is an approach that initializes and trains an MoE model using a pre-trained dense model.
- 2025-02arXiv 2502.18581Scalable Best-of-N Selection for Large Language Models via Self-Certainty
Zhewei Kang, Xuandong Zhao, Dawn Song · Language models & NLP
Best-of-N selection is a key technique for improving the reasoning performance of Large Language Models (LLMs) through increased test-time computation. Current state-of-the-art methods often employ computationally intensive reward models for response evaluation and selection.
- 2025-02arXiv 2502.17416Reasoning with Latent Thoughts: On the Power of Looped Transformers
Nikunj Saunshi, Nishanth Dikkala, Zhiyuan Li et al. · Language models & NLP
Large language models have shown remarkable reasoning abilities and scaling laws suggest that large parameter count, especially along the depth axis, is the primary driver.
- 2025-02arXiv 2502.17420The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independence
Tom Wollschläger, Jannes Elstner, Simon Geisler et al. · Machine learning
The safety alignment of large language models (LLMs) can be circumvented through adversarially crafted inputs, yet the mechanisms by which these attacks bypass safety barriers remain poorly understood.
- 2025-02arXiv 2502.17424Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs
Jan Betley, Daniel Tan, Niels Warncke et al. · Language models & NLP
We present a surprising result regarding LLMs and alignment. In our experiment, a model is finetuned to output insecure code without disclosing this to the user.
- 2025-02arXiv 2502.16681Are Sparse Autoencoders Useful? A Case Study in Sparse Probing
Subhash Kantamneni, Joshua Engels, Senthooran Rajamanoharan et al. · Machine learning
Sparse autoencoders (SAEs) are a popular method for interpreting concepts represented in large language model (LLM) activations.
- 2025-02arXiv 2502.15652Empowering LLMs with Logical Reasoning: A Comprehensive Survey
Fengxiang Cheng, Haoxuan Li, Fenrong Liu et al. · AI & agents
Large language models (LLMs) have achieved remarkable successes on various tasks.
- 2025-02arXiv 2502.15924Improving Consistency in Large Language Models through Chain of Guidance
Harsh Raj, Vipul Gupta, Domenic Rosati et al. · Language models & NLP
Consistency is a fundamental dimension of trustworthiness in Large Language Models (LLMs). For humans to be able to trust LLM-based applications, their outputs should be consistent when prompted with inputs that carry the same meaning or intent.
- 2025-02arXiv 2502.14427Token-Level Density-Based Uncertainty Quantification Methods for Eliciting Truthfulness of Large Language Models
Artem Vazhentsev, Lyudmila Rvanova, Ivan Lazichny et al. · Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics · Language models & NLP
Uncertainty quantification (UQ) is a prominent approach for eliciting truthful answers from large language models (LLMs). To date, information-based and consistency-based UQ have been the dominant UQ methods for text generation via LLMs.
- 2025-02arXiv 2502.14802From RAG to Memory: Non-Parametric Continual Learning for Large Language Models
Bernal Jiménez Gutiérrez, Yiheng Shu, Weijian Qi et al. · Language models & NLP
Our ability to continuously acquire, organize, and leverage knowledge is a key feature of human intelligence that AI systems must approximate to unlock their full potential.
Cited inStudy: Graphyte spec - 2025-02arXiv 2502.14855Prompt-to-Leaderboard
Evan Frick, Connor Chen, Joseph Tennyson et al. · Machine learning
Large language model (LLM) evaluations typically rely on aggregated metrics like accuracy or human preference, averaging across users and prompts. This averaging obscures user- and prompt-specific variations in model performance.
- 2025-02arXiv 2502.15826CoME: An Unlearning-based Approach to Conflict-free Model Editing
Dahyun Jung, Jaehyung Seo, Jaewook Lee et al. · Language models & NLP
Large language models (LLMs) often retain outdated or incorrect information from pre-training, which undermines their reliability.
- 2025-02arXiv 2502.14143Multi-Agent Risks from Advanced AI
Lewis Hammond, Alan Chan, Jesse Clifton et al. · AI & agents
The rapid development of advanced AI agents and the imminent deployment of many instances of these agents will give rise to multi-agent systems of unprecedented complexity.
- 2025-02arXiv 2502.17480Brain-to-Text Decoding: A Non-invasive Approach via Typing
Jarod Lévy, Mingfang Zhang, Svetlana Pinet et al. · Hardware, photonics & physics
Modern neuroprostheses can now restore communication in patients who have lost the ability to speak or move. However, these invasive devices entail risks inherent to neurosurgery.
- 2025-02arXiv 2502.11447Does Editing Provide Evidence for Localization?
Zihao Wang, Victor Veitch · Machine learning
A basic aspiration for interpretability research in large language models is to "localize" semantically meaningful behaviors to particular components within the LLM.
- 2025-02arXiv 2502.11756On the Computation of the Fisher Information in Continual Learning
Gido M. van de Ven · Machine learning
One of the most popular methods for continual learning with deep neural networks is Elastic Weight Consolidation (EWC), which involves computing the Fisher Information.
- 2025-02arXiv 2502.11177The Mirage of Model Editing: Revisiting Evaluation in the Wild
Wanli Yang, Fei Sun, Jiajun Tan et al. · Language models & NLP
Despite near-perfect results reported in the literature, the effectiveness of model editing in real-world applications remains unclear.
- 2025-02arXiv 2502.10928Probing Semantic Routing in Large Mixture-of-Expert Models
Matthew Lyle Olson, Neale Ratzlaff, Musashi Hinck et al. · Machine learning
In the past year, large (>100B parameter) mixture-of-expert (MoE) models have become increasingly common in the open domain. While their advantages are often framed in terms of efficiency, prior work has also explored functional differentiation through routing behavior.
- 2025-02arXiv 2502.09056Adapting Language-Specific LLMs to a Reasoning Model in One Day via Model Merging -- An Open Recipe
Kunat Pipatanakul, Pittawat Taveekitworachai, Potsawee Manakul et al. · Language models & NLP
This paper investigates data selection and model merging methodologies aimed at incorporating advanced reasoning capabilities such as those of DeepSeek R1 into language-specific large language models (LLMs), with a particular focus on the Thai LLM.
- 2025-02arXiv 2502.08177SycEval: Evaluating LLM Sycophancy
Aaron Fanous, Jacob Goldberg, Ank A. Agarwal et al. · AI & agents
Large language models (LLMs) are increasingly applied in educational, clinical, and professional settings, but their tendency for sycophancy -- prioritizing user agreement over independent reasoning -- poses risks to reliability.
- 2025-02arXiv 2502.08691AgentSociety: Large-Scale Simulation of LLM-Driven Generative Agents Advances Understanding of Human Behaviors and Society
Jinghua Piao, Yuwei Yan, Jun Zhang et al. · Society, software & economics
Understanding human behavior and society is a central focus in social sciences, with the rise of generative social science marking a significant paradigmatic shift.
- 2025-02arXiv 2502.08773Universal Model Routing for Efficient LLM Inference
Wittawat Jitkrittum, Harikrishna Narasimhan, Ankit Singh Rawat et al. · Language models & NLP
Model routing is a simple technique for reducing the inference cost of large language models (LLMs), wherein one maintains a pool of candidate LLMs, and learns to route each prompt to the smallest feasible LLM.
- 2025-02arXiv 2502.08788Stop Overvaluing Multi-Agent Debate -- We Must Rethink Evaluation and Embrace Model Heterogeneity
Hangfan Zhang, Zhiyao Cui, Jianhao Chen et al. · Language models & NLP
Multi-agent debate (MAD) has gained significant attention as a promising line of research to improve the factual accuracy and reasoning capabilities of large language models (LLMs).
- 2025-02arXiv 2502.07274Forget Forgetting: Continual Learning in a World of Abundant Memory
Dongkyu Cho, Taesup Moon, Rumi Chunara et al. · Machine learning
Continual learning (CL) has traditionally focused on minimizing exemplar memory, a constraint often misaligned with modern systems where GPU time, not storage, is the primary bottleneck.
- 2025-02arXiv 2502.06210Achieving Deep Continual Learning via Evolution
Aojun Lu, Junchao Ke, Chunhui Ding et al. · Machine learning
Deep neural networks, despite their remarkable success, remain fundamentally limited in their ability to perform Continual Learning (CL).
- 2025-02arXiv 2502.05793On Reference (In-)Determinacy in Natural Language Inference
Sihao Chen, Chaitanya Malaviya, Alex Fabrikant et al. · Language models & NLP
, the premise and hypothesis are assumed to refer to the same context when human raters annotate a label.
- 2025-02arXiv 2502.18482MixLLM: Dynamic Routing in Mixed Large Language Models
Xinyuan Wang, Yanchi Liu, Wei Cheng et al. · Language models & NLP
Large Language Models (LLMs) exhibit potential artificial generic intelligence recently, however, their usage is costly with high response latency.
- 2025-02arXiv 2502.06884Learning Conformal Abstention Policies for Adaptive Risk Management in Large Language and Vision-Language Models
Sina Tayebati, Divake Kumar, Nastaran Darabi et al. · Machine learning
Large Language and Vision-Language Models (LLMs/VLMs) are increasingly used in safety-critical applications, yet their opaque decision-making complicates risk assessment and reliability.
- 2025-02arXiv 2502.05171Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach
Jonas Geiping, Sean McLeish, Neel Jain et al. · Machine learning
We study a novel language model architecture that is capable of scaling test-time computation by implicitly reasoning in latent space. Our model works by iterating a recurrent block, thereby unrolling to arbitrary depth at test-time.
- 2025-02arXiv 2502.05172Joint MoE Scaling Laws: Mixture of Experts Can Be Memory Efficient
Jan Ludziejewski, Maciej Pióro, Jakub Krajewski et al. · Machine learning
Mixture of Experts (MoE) architectures have significantly increased computational efficiency in both research and real-world applications of large-scale machine learning models.
- 2025-02arXiv 2502.05374Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyond
Chongyu Fan, Jinghan Jia, Yihua Zhang et al. · Machine learning
The LLM unlearning technique has recently been introduced to comply with data regulations and address the safety and ethical concerns of LLMs by removing the undesired data-model influence.
- 2025-02arXiv 2502.04294Prediction-Powered E-Values
Daniel Csillag, Claudio José Struchiner, Guilherme Tegoni Goedert · Machine learning
Quality statistical inference requires a sufficient amount of data, which can be missing or hard to obtain. To this end, prediction-powered inference has risen as a promising methodology, but existing approaches are largely limited to Z-estimation problems such as inference of means and quantiles.
Cited inInternal notes - 2025-02arXiv 2502.04313Great Models Think Alike and this Undermines AI Oversight
Shashwat Goel, Joschka Struber, Ilze Amanda Auzina et al. · Machine learning
As Language Model (LM) capabilities advance, evaluating and supervising them at scale is getting harder for humans. There is hope that other language models can automate both these tasks, which we refer to as ''AI Oversight''.
- 2025-02arXiv 2502.03009Scaling Laws for Upcycling Mixture-of-Experts Language Models
Seng Pei Liew, Takuya Kato, Sho Takase · Machine learning
Pretraining large language models (LLMs) is resource-intensive, often requiring months of training time even with high-end GPU clusters.
- 2025-02arXiv 2502.03350Optimal Task Order for Continual Learning of Multiple Tasks
Ziyan Li, Naoki Hiratani · Machine learning
Continual learning of multiple tasks remains a major challenge for neural networks. Here, we investigate how task order influences continual learning and propose a strategy for optimizing it.
Built from every arXiv paper cited in our papers, posts, topic hubs, reports and research notes, with titles, authors, dates and abstracts from the arXiv API (updated 2026-11-12). Summaries are the opening sentences of each paper's own abstract, not our assessment; a citation means we used or checked the work, not that we endorse it. Fields follow each paper's primary arXiv category. Papers cited only outside arXiv (journals, proceedings) are not listed yet. Entries are generated from our citations, so a mistyped ID in a note would list the wrong paper; we remove those as we find them. Report a wrong entry.
