NeuroGenesysThe research we build on.
1,514 papers by other researchers that our work cites, from continual learning and verification to language models, agents and hardware. Each entry links to the paper on arXiv and to where we use it. 301 are cited on published pages of this site; the rest in the research notes behind our studies. Our own papers are in the archive.
- 2026-08arXiv 2609.20846Rewarding Efficient Reasoning Improves Abstention on Underspecified Tasks in Reasoning Models
Polina Tsvilodub, Max Höth, Michael Franke et al. · Language models & NLP
While modern large reasoning models (LRMs) excel at providing correct answers in many tasks, we provide additional evidence that they often struggle with a critical capability: knowing when to abstain from answering.
- 2026-08arXiv 2608.02528Uncertainty Is Not Enough: Value-of-Information Routing for Mixtures of LoRA Experts
Tom Saliencro, Rohan Desai, Priya Nair et al. · Machine learning
Mixtures of low-rank adaptation experts increase parameter-efficient capacity by routing each input through a subset of adapters. Recent dynamic routers activate more experts when the router or prediction is uncertain.
- 2026-07arXiv 2608.00301Abstention as an Action Can Kill Both the Reward Gradient and the KL Anchor: Collapse Law and Repair for Error-Penalized Reinforcement Learning
Xujun Che, Yuchen Yuan, Weida Zhao et al. · Machine learning
Error-penalized scoring rules ( for a correct answer, for a wrong one, $0$ for abstaining) are increasingly prescribed against hallucination: a rational agent facing such a rule answers exactly when its correctness probability exceeds Chow's threshold .
- 2026-07arXiv 2607.26052Spend Experts Where You Are Unsure: Confidence-Adaptive Routing for Mixture-of-Experts LoRA
Tom Saliencro, Rohan Desai, Priya Nair et al. · Machine learning
Mixture-of-Experts (MoE) variants of Low-Rank Adaptation (LoRA) route every token to a fixed number of experts . Tokens differ in how uncertain the model is about them, so a single k over-spends on easy tokens and under-serves hard ones.
- 2026-07arXiv 2607.10738To Answer or to Abstain: Mitigating Search-Agent Hallucinations via Abstention-Aware Reinforcement Learning
Fengji Zhang, Tianyu Fan, Yuxiang Zheng et al. · Machine learning
Recent advances in equipping Large Language Models (LLMs) with search tools and outcome-reward reinforcement learning (RL) have achieved new state-of-the-art results on open-domain QA tasks.
- 2026-06arXiv 2606.31404Wisdom Of The (AI) Crowd: Investigating Artificial Swarm Intelligence In Large Language Models
Justin Brenne, Christian Meske · AI & agents
Human swarm intelligence demonstrates remarkable collective accuracy but faces scalability constraints in cost, coordination, and time.
- 2026-05arXiv 2605.25850TIAR: Trajectory-Informed Advantage Reweighting for LLM Abstention Learning
Muyu Pan, Shu Zhao, Nan Zhang et al. · Language models & NLP
This paper investigates large language model (LLM) abstention learning, specifically using ternary reward, which incentivize truthfulness in large language models.
- 2026-05arXiv 2605.09702Calibrate, Don't Curate: Label-Efficient Estimation from Noisy LLM Judges
Yanran Li · Statistics & theory
Multi-judge evaluation is increasingly used to assess LLMs and reward models, and the prevailing heuristic is to curate: keep the most accurate judges and discard weaker ones.
- 2026-04arXiv 2604.03904I-CALM: Incentivizing Confidence-Aware Abstention for LLM Selective Answering
Haotian Zong, Binze Li, Yufei Long et al. · Language models & NLP
Large language models (LLMs) often produce confident but incorrect answers, in part because standard evaluation incentives reward guessing over expressing uncertainty.
- 2026-04arXiv 2604.22779KARL: Mitigating Hallucinations in LLMs via Knowledge-Boundary-Aware Reinforcement Learning
Cheng Gao, Cheng Huang, Kangyang Luo et al. · Machine learning
Enabling large language models (LLMs) to appropriately abstain from answering questions beyond their knowledge is crucial for mitigating hallucinations.
- 2026-03arXiv 2603.14623Proactive Routing to Interpretable Surrogates with Distribution-Free Safety Guarantees
Iqtedar Uddin, Mazin Khider, André Bauer · Machine learning
Model routing determines whether to use an accurate black-box model or a simpler surrogate that approximates it at lower cost or greater interpretability.
- 2026-02arXiv 2602.20273The Truthfulness Spectrum Hypothesis
Zhuofan Josh Ying, Shauli Ravfogel, Nikolaus Kriegeskorte et al. · Machine learning
Large language models (LLMs) have been reported to linearly encode truthfulness, yet recent work questions this finding's generality.
- 2026-02arXiv 2603.06612Consensus is Not Verification: Why Crowd Wisdom Strategies Fail for LLM Truthfulness
Yegor Denisov-Blanch, Joshua Kazdan, Jessica Chudnovsky et al. · Machine learning
Pass@k and other methods of scaling inference compute can improve language model performance in domains with external verifiers, including mathematics and code, where incorrect candidates can be filtered reliably.
- 2026-01arXiv 2601.14210DRIFT: Detecting Representational Inconsistencies for Factual Truthfulness
Rohan Bhatnagar, Youran Sun, Chi Andrew Zhang et al. · Language models & NLP
LLMs often produce fluent but incorrect answers, yet detecting such hallucinations typically requires multiple sampling passes or post-hoc verification, adding significant latency and cost.
- 2025-10arXiv 2510.10193SAFER: Risk-Constrained Sample-then-Filter in Large Language Models
Qingni Wang, Yue Fan, Xin Eric Wang · AI & agents
As large language models (LLMs) are increasingly deployed in risk-sensitive applications such as real-world open-ended question answering (QA), ensuring the trustworthiness of their outputs has become critical.
- 2025-09arXiv 2509.04664Why Language Models Hallucinate
Adam Tauman Kalai, Ofir Nachum, Santosh S. Vempala et al. · Language models & NLP
Like students facing hard exam questions, large language models sometimes guess when uncertain, producing plausible yet incorrect statements instead of admitting uncertainty.
- 2025-06arXiv 2507.00029LoRA-Mixer: Coordinate Modular LoRA Experts Through Serial Attention Routing
Wenbing Li, Zikai Song, Hang Zhou et al. · Machine learning
Recent attempts to combine low-rank adaptation (LoRA) with mixture-of-experts (MoE) for multi-task adaptation of Large Language Models (LLMs) often replace whole attention/FFN layers with switch experts or append parallel expert branches, undermining parameter efficiency and limiting task specialization.
- 2025-06arXiv 2506.08572The Geometries of Truth Are Orthogonal Across Tasks
Waiss Azizian, Michael Kirchhof, Eugene Ndiaye et al. · Machine learning
Large Language Models (LLMs) have demonstrated impressive generalization capabilities across various tasks, but their claim to practical relevance is still mired by concerns on their reliability.
- 2025-06arXiv 2506.09038AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions
Polina Kirichenko, Mark Ibrahim, Kamalika Chaudhuri et al. · AI & agents
For Large Language Models (LLMs) to be reliably deployed in both everyday and high-stakes domains, knowing when not to answer is equally critical as answering correctly.
- 2025-06arXiv 2506.07962Correlated Errors in Large Language Models
Elliot Kim, Avi Garg, Kenny Peng et al. · Language models & NLP
Diversity in training data, architecture, and providers is assumed to mitigate homogeneity in LLMs. However, we lack empirical evidence on whether different LLMs differ meaningfully.
- 2025-06arXiv 2506.00823Probing the Geometry of Truth: Consistency and Generalization of Truth Directions in LLMs Across Logical Transformations and Question Answering Tasks
Yuntai Bao, Xuhong Zhang, Tianyu Du et al. · Language models & NLP
Large language models (LLMs) are trained on extensive datasets that encapsulate substantial world knowledge. However, their outputs often include confidently stated inaccuracies.
- 2025-02arXiv 2502.06884Learning Conformal Abstention Policies for Adaptive Risk Management in Large Language and Vision-Language Models
Sina Tayebati, Divake Kumar, Nastaran Darabi et al. · Machine learning
Large Language and Vision-Language Models (LLMs/VLMs) are increasingly used in safety-critical applications, yet their opaque decision-making complicates risk assessment and reliability.
- 2025-01arXiv 2501.01818Rerouting LLM Routers
Avital Shafran, Roei Schuster, Thomas Ristenpart et al. · Security
LLM routers aim to balance quality and cost of generation by classifying queries and routing them to a cheaper or more expensive LLM depending on their complexity.
- 2024-12arXiv 2412.19792InfAlign: Inference-aware language model alignment
Ananth Balashankar, Ziteng Sun, Jonathan Berant et al. · Machine learning
Language model alignment is a critical step in training modern generative language models. Alignment targets to improve win rate of a sample from the aligned model against the base model.
- 2024-12arXiv 2412.15571Continual Learning Using a Kernel-Based Method Over Foundation Models
Saleh Momeni, Sahisnu Mazumder, Bing Liu · Machine learning
Continual learning (CL) learns a sequence of tasks incrementally. This paper studies the challenging CL setting of class-incremental learning (CIL). CIL has two key challenges: catastrophic forgetting (CF) and inter-task class separation (ICS).
- 2024-11arXiv 2411.17501The Limits of Inference Scaling Through Resampling
Benedikt Stroebl, Sayash Kapoor, Arvind Narayanan · Machine learning
Recent research has generated hope that inference scaling, such as resampling solutions until they pass verifiers like unit tests, could allow weaker models to match stronger ones.
- 2024-11arXiv 2411.14257Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Models
Javier Ferrando, Oscar Obeso, Senthooran Rajamanoharan et al. · Language models & NLP
Hallucinations in large language models are a widespread problem, yet the mechanisms behind whether models will hallucinate are poorly understood, limiting our ability to solve this problem.
- 2024-11arXiv 2411.13619Non-Linear Outlier Synthesis for Out-of-Distribution Detection
Lars Doorenbos, Raphael Sznitman, Pablo Márquez-Neila · Vision
The reliability of supervised classifiers is severely hampered by their limitations in dealing with unexpected inputs, leading to great interest in out-of-distribution (OOD) detection.
- 2024-11arXiv 2411.00430Class Incremental Learning with Task-Specific Batch Normalization and Out-of-Distribution Detection
Zhiping Zhou, Xuchen Xie, Yiqiao Qiu et al. · Machine learning
This study focuses on incremental learning for image classification, exploring how to reduce catastrophic forgetting of all learned knowledge when access to old data is restricted.
- 2024-10arXiv 2410.02707LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations
Hadas Orgad, Michael Toker, Zorik Gekhman et al. · Language models & NLP
Large language models (LLMs) often produce errors, including factual inaccuracies, biases, and reasoning failures, collectively referred to as "hallucinations".
- 2024-06arXiv 2406.18665RouteLLM: Learning to Route LLMs with Preference Data
Isaac Ong, Amjad Almahairi, Vincent Wu et al. · Machine learning
Large language models (LLMs) exhibit impressive capabilities across a wide range of tasks, yet the choice of which model to use often involves a trade-off between performance and cost.
Cited inRoute on shared statistics, never expert confidence: the router architecture for frozen expertsStudy: Hierarchical and T shaped MoEInternal notesStudy: Model self knowledge and confident errorsStudy: Optimal router architectureStudy: Orchestrated specialists and bridge synthesisStudy: Stopping confident wrong expert votes - 2024-06arXiv 2406.16989Retrieval-Augmented Mixture of LoRA Experts for Uploadable Machine Learning
Ziyu Zhao, Leilei Gan, Guoyin Wang et al. · Machine learning
Low-Rank Adaptation (LoRA) offers an efficient way to fine-tune large language models (LLMs). Its modular and plug-and-play nature allows the integration of various domain-specific LoRAs, enhancing LLM capabilities.
- 2024-06arXiv 2406.15927Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMs
Jannik Kossen, Jiatong Han, Muhammed Razzak et al. · Language models & NLP
We propose semantic entropy probes (SEPs), a cheap and reliable method for uncertainty quantification in Large Language Models (LLMs). Hallucinations, which are plausible-sounding but factually incorrect and arbitrary model generations, present a major challenge to the practical adoption of LLMs.
Cited inStable wrong beliefs evade every self-check: a literature synthesis on model self-knowledge and confident errorsStudy: Decision signatures for conflict detectionStudy: Logical coherence of LLM world modelsStudy: Model self knowledge and confident errorsStudy: Stopping confident wrong expert votes - 2024-06arXiv 2406.15479Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging
Zhenyi Lu, Chenghao Fan, Wei Wei et al. · Language models & NLP
In the era of large language models, model merging is a promising way to combine multiple task-specific models into a single multitask model without extra training.
- 2024-05arXiv 2405.11157Towards Modular LLMs by Building and Reusing a Library of LoRAs
Oleksiy Ostapenko, Zhan Su, Edoardo Maria Ponti et al. · Machine learning
The growing number of parameter-efficient adaptations of a base large language model (LLM) calls for studying whether we can reuse such trained adapters to improve performance for new tasks.
- 2024-05arXiv 2405.05904Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?
Zorik Gekhman, Gal Yona, Roee Aharoni et al. · Language models & NLP
When large language models are aligned via supervised fine-tuning, they may encounter new factual information that was not acquired through pre-training.
Cited inStable wrong beliefs evade every self-check: a literature synthesis on model self-knowledge and confident errorsTo stop a model forgetting, rehearse what it knew. New context won't do it.Study: Consequence consistent updatesStudy: Continual learning order and companion dataStudy: Fortifying knowledge in continual learningStudy: Generational competitive continual learningStudy: Graded succession by earned authorityStudy: Logical coherence of LLM world modelsStudy: Model self knowledge and confident errorsStudy: Orchestrated specialists and bridge synthesisStudy: Stopping confident wrong expert votes - 2024-04arXiv 2404.09523More, better or different? Trade-offs between group size and competence development in jury theorems
Gustaf Arrhenius, Klas Markström · Society, software & economics
In many circumstances there is a trade off between the number of voters and the time they can be given before having to make a decision since both aspects are costly.
- 2024-04arXiv 2405.01563Mitigating LLM Hallucinations via Conformal Abstention
Yasin Abbasi Yadkori, Ilja Kuzborskij, David Stutz et al. · Machine learning
, by saying "I don't know") in a general domain, instead of resorting to possibly "hallucinating" a non-sensical or incorrect answer.
- 2024-03arXiv 2403.05612Unfamiliar Finetuning Examples Control How Language Models Hallucinate
Katie Kang, Eric Wallace, Claire Tomlin et al. · Machine learning
Large language models are known to hallucinate when faced with unfamiliar queries, but the underlying mechanism that govern how models hallucinate are not yet fully understood.
- 2024-03arXiv 2403.02419Are More LLM Calls All You Need? Towards Scaling Laws of Compound Inference Systems
Lingjiao Chen, Jared Quincy Davis, Boris Hanin et al. · Machine learning
Many recent state-of-the-art results in language tasks were achieved using compound systems that perform multiple Language Model (LM) calls and aggregate their responses.
Built from every arXiv paper cited in our papers, posts, topic hubs, reports and research notes, with titles, authors, dates and abstracts from the arXiv API (updated 2026-11-12). Summaries are the opening sentences of each paper's own abstract, not our assessment; a citation means we used or checked the work, not that we endorse it. Fields follow each paper's primary arXiv category. Papers cited only outside arXiv (journals, proceedings) are not listed yet. Entries are generated from our citations, so a mistyped ID in a note would list the wrong paper; we remove those as we find them. Report a wrong entry.
