NeuroGenesysThe research we build on.
1,514 papers by other researchers that our work cites, from continual learning and verification to language models, agents and hardware. Each entry links to the paper on arXiv and to where we use it. 301 are cited on published pages of this site; the rest in the research notes behind our studies. Our own papers are in the archive.
- 2025-03arXiv 2503.13657Why Do Multi-Agent LLM Systems Fail?
Mert Cemri, Melissa Z. Pan, Shuyi Yang et al. · AI & agents
Despite enthusiasm for Multi-Agent LLM Systems (MAS), their performance gains on popular benchmarks are often minimal. This gap highlights a critical need for a principled understanding of why MAS fail.
- 2025-02arXiv 2502.14427Token-Level Density-Based Uncertainty Quantification Methods for Eliciting Truthfulness of Large Language Models
Artem Vazhentsev, Lyudmila Rvanova, Ivan Lazichny et al. · Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics · Language models & NLP
Uncertainty quantification (UQ) is a prominent approach for eliciting truthful answers from large language models (LLMs). To date, information-based and consistency-based UQ have been the dominant UQ methods for text generation via LLMs.
- 2025-02arXiv 2502.08177SycEval: Evaluating LLM Sycophancy
Aaron Fanous, Jacob Goldberg, Ank A. Agarwal et al. · AI & agents
Large language models (LLMs) are increasingly applied in educational, clinical, and professional settings, but their tendency for sycophancy -- prioritizing user agreement over independent reasoning -- poses risks to reliability.
- 2025-01arXiv 2501.12835Adaptive Retrieval Without Self-Knowledge? Bringing Uncertainty Back Home
Viktor Moskvoretskii, Maria Lysyuk, Mikhail Salnikov et al. · Language models & NLP
Retrieval Augmented Generation (RAG) improves correctness of Question Answering (QA) and addresses hallucinations in Large Language Models (LLMs), yet greatly increase computational costs.
- 2025-01arXiv 2501.04899SUGAR: Leveraging Contextual Confidence for Smarter Retrieval
Hanna Zubkova, Ji-Hoon Park, Seong-Whan Lee · Language models & NLP
Bearing in mind the limited parametric knowledge of Large Language Models (LLMs), retrieval-augmented generation (RAG) which supplies them with the relevant external knowledge has served as an approach to mitigate the issue of hallucinations to a certain extent.
- 2024-12arXiv 2412.12472Knowledge Boundary of Large Language Models: A Survey
Moxin Li, Yong Zhao, Wenxuan Zhang et al. · Language models & NLP
Although large language models (LLMs) store vast amount of knowledge in their parameters, they still have limitations in the memorization and utilization of certain knowledge, leading to undesired behaviors such as generating untruthful and inaccurate responses.
- 2024-11arXiv 2411.14257Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Models
Javier Ferrando, Oscar Obeso, Senthooran Rajamanoharan et al. · Language models & NLP
Hallucinations in large language models are a widespread problem, yet the mechanisms behind whether models will hallucinate are poorly understood, limiting our ability to solve this problem.
- 2024-11arXiv 2411.04368Measuring short-form factuality in large language models
Jason Wei, Nguyen Karina, Hyung Won Chung et al. · Language models & NLP
We present SimpleQA, a benchmark that evaluates the ability of language models to answer short, fact-seeking questions. We prioritized two properties in designing this eval.
- 2024-10arXiv 2410.20783Graph-based Uncertainty Metrics for Long-form Language Model Outputs
Mingjian Jiang, Yangjun Ruan, Prasanna Sattigeri et al. · Language models & NLP
Recent advancements in Large Language Models (LLMs) have significantly improved text generation capabilities, but these systems are still known to hallucinate, and granular uncertainty estimation for long-form LLM generations remains challenging.
- 2024-10arXiv 2410.13787Looking Inward: Language Models Can Learn About Themselves by Introspection
Felix J Binder, James Chua, Tomek Korbak et al. · Language models & NLP
Humans acquire knowledge by observing the external world, but also by introspection., thoughts and feelings) that is not accessible to external observers. Can LLMs introspect?
- 2024-10arXiv 2410.09724Taming Overconfidence in LLMs: Reward Calibration in RLHF
Jixuan Leng, Chengsong Huang, Banghua Zhu et al. · Language models & NLP
Language model calibration refers to the alignment between the confidence of the model and the actual performance of its responses.
- 2024-10arXiv 2410.02707LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations
Hadas Orgad, Michael Toker, Zorik Gekhman et al. · Language models & NLP
Large language models (LLMs) often produce errors, including factual inaccuracies, biases, and reasoning failures, collectively referred to as "hallucinations".
- 2024-07arXiv 2407.18418Know Your Limits: A Survey of Abstention in Large Language Models
Bingbing Wen, Jihan Yao, Shangbin Feng et al. · Language models & NLP
Abstention, the refusal of large language models (LLMs) to provide an answer, is increasingly recognized for its potential to mitigate hallucinations and enhance safety in LLM systems.
- 2024-07arXiv 2407.21057Multi-group Uncertainty Quantification for Long-form Text Generation
Terrance Liu, Zhiwei Steven Wu · Language models & NLP
While past works have shown how uncertainty quantification can be applied to large language model (LLM) outputs, the question of whether resulting uncertainty guarantees still hold within sub-groupings of data remains open.
- 2024-07arXiv 2407.12831Truth is Universal: Robust Detection of Lies in LLMs
Lennart Bürger, Fred A. Hamprecht, Boaz Nadler · Language models & NLP
Large Language Models (LLMs) have revolutionised natural language processing, exhibiting impressive human-like capabilities. In particular, LLMs are capable of "lying", knowingly outputting false statements.
Cited inCoherence is selected, anchored, then pruned: a literature synthesis on logical coherence in LLM world modelsStable wrong beliefs evade every self-check: a literature synthesis on model self-knowledge and confident errorsStudy: Logical coherence of LLM world modelsStudy: Model self knowledge and confident errors - 2024-06arXiv 2406.19276VERISCORE: Evaluating the factuality of verifiable claims in long-form text generation
Yixiao Song, Yekyung Kim, Mohit Iyyer · Language models & NLP
, 2024), decompose an input text into "atomic claims" and verify each against a knowledge base like Wikipedia., can plausibly be proven true or false). We address this issue with VERISCORE, a metric for diverse long-form generation tasks that contain both verifiable and unverifiable content.
- 2024-06arXiv 2406.18665RouteLLM: Learning to Route LLMs with Preference Data
Isaac Ong, Amjad Almahairi, Vincent Wu et al. · Machine learning
Large language models (LLMs) exhibit impressive capabilities across a wide range of tasks, yet the choice of which model to use often involves a trade-off between performance and cost.
Cited inRoute on shared statistics, never expert confidence: the router architecture for frozen expertsStudy: Hierarchical and T shaped MoEInternal notesStudy: Model self knowledge and confident errorsStudy: Optimal router architectureStudy: Orchestrated specialists and bridge synthesisStudy: Stopping confident wrong expert votes - 2024-06arXiv 2406.15927Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMs
Jannik Kossen, Jiatong Han, Muhammed Razzak et al. · Language models & NLP
We propose semantic entropy probes (SEPs), a cheap and reliable method for uncertainty quantification in Large Language Models (LLMs). Hallucinations, which are plausible-sounding but factually incorrect and arbitrary model generations, present a major challenge to the practical adoption of LLMs.
Cited inStable wrong beliefs evade every self-check: a literature synthesis on model self-knowledge and confident errorsStudy: Decision signatures for conflict detectionStudy: Logical coherence of LLM world modelsStudy: Model self knowledge and confident errorsStudy: Stopping confident wrong expert votes - 2024-06arXiv 2406.15627Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph
Roman Vashurin, Ekaterina Fadeeva, Artem Vazhentsev et al. · Transactions of the Association for Computational Linguistics, 13 (2025) 220-248 · Language models & NLP
The rapid proliferation of large language models (LLMs) has stimulated researchers to seek effective and efficient approaches to deal with LLM hallucinations and low-quality outputs.
- 2024-06arXiv 2406.12673Estimating Knowledge in Large Language Models Without Generating a Single Token
Daniela Gottesman, Mor Geva · Language models & NLP
To evaluate knowledge in large language models (LLMs), current methods query the model and then evaluate its generated responses. In this work, we ask whether evaluation can be done before the model has generated any text.
- 2024-06arXiv 2406.12045-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains
Shunyu Yao, Noah Shinn, Pedram Razavi et al. · AI & agents
Existing benchmarks do not test language agents on their interaction with human users or ability to follow domain-specific rules, both of which are vital for deploying them in real world applications.
Cited inStudy: Genesys program prior artInternal notesStudy: Model self knowledge and confident errors - 2024-06arXiv 2406.09714Large language model validity via enhanced conformal prediction methods
John J. Cherian, Isaac Gibbs, Emmanuel J. Candès · Machine learning
We develop new conformal inference methods for obtaining validity guarantees on the output of large language models (LLMs). Prior work in conformal language modeling identifies a subset of the text that satisfies a high-probability guarantee of correctness.
- 2024-06arXiv 2406.10099Know the Unknown: An Uncertainty-Sensitive Method for LLM Instruction Tuning
Jiaqi Li, Yixuan Tang, Yi Yang · Language models & NLP
Large language models (LLMs) demonstrate remarkable capabilities but face challenges from hallucinations, which typically arise from insufficient knowledge or context.
- 2024-06arXiv 2406.08391Large Language Models Must Be Taught to Know What They Don't Know
Sanyam Kapoor, Nate Gruver, Manley Roberts et al. · Machine learning
When using large language models (LLMs) in high-stakes applications, we need to know when we can trust their predictions.
- 2024-06arXiv 2406.02543To Believe or Not to Believe Your LLM
Yasin Abbasi Yadkori, Ilja Kuzborskij, András György et al. · Machine learning
We explore uncertainty quantification in large language models (LLMs), with the goal to identify when uncertainty in responses given a query is large.
- 2024-05arXiv 2405.20003Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similarities
Alexander Nikitin, Jannik Kossen, Yarin Gal et al. · Machine learning
Uncertainty quantification in Large Language Models (LLMs) is crucial for applications where safety and reliability are important. In particular, uncertainty can be used to improve the trustworthiness of LLMs by detecting factually incorrect model responses, commonly called hallucinations.
- 2024-05arXiv 2405.05904Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?
Zorik Gekhman, Gal Yona, Roee Aharoni et al. · Language models & NLP
When large language models are aligned via supervised fine-tuning, they may encounter new factual information that was not acquired through pre-training.
Cited inStable wrong beliefs evade every self-check: a literature synthesis on model self-knowledge and confident errorsTo stop a model forgetting, rehearse what it knew. New context won't do it.Study: Consequence consistent updatesStudy: Continual learning order and companion dataStudy: Fortifying knowledge in continual learningStudy: Generational competitive continual learningStudy: Graded succession by earned authorityStudy: Logical coherence of LLM world modelsStudy: Model self knowledge and confident errorsStudy: Orchestrated specialists and bridge synthesisStudy: Stopping confident wrong expert votes - 2024-04arXiv 2404.14618Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing
Dujian Ding, Ankur Mallick, Chi Wang et al. · Machine learning
, edge) devices, tend to lag behind in terms of response quality. Therefore in this work we propose a hybrid inference approach which combines their respective strengths to save cost and maintain quality.
- 2024-04arXiv 2404.10198ClashEval: Quantifying the tug-of-war between an LLM's internal prior and external evidence
Kevin Wu, Eric Wu, James Zou · Language models & NLP
Retrieval augmented generation (RAG) is frequently used to mitigate hallucinations and provide up-to-date knowledge for large language models (LLMs).
Cited inStable wrong beliefs evade every self-check: a literature synthesis on model self-knowledge and confident errorsTo keep what a model knows, control what it is allowed to moveWhat a model lets into long-term memory matters more than how it learnsStudy: Fortifying knowledge in continual learningStudy: Hardware guardian core for continual learningStudy: Memory world models and source trustStudy: Model self knowledge and confident errors - 2024-04arXiv 2404.10136Language Model Cascades: Token-level uncertainty and beyond
Neha Gupta, Harikrishna Narasimhan, Wittawat Jitkrittum et al. · Language models & NLP
Recent advances in language models (LMs) have led to significant improvements in quality on complex NLP tasks, but at the expense of increased inference costs.
- 2024-04arXiv 2405.01563Mitigating LLM Hallucinations via Conformal Abstention
Yasin Abbasi Yadkori, Ilja Kuzborskij, David Stutz et al. · Machine learning
, by saying "I don't know") in a general domain, instead of resorting to possibly "hallucinating" a non-sensical or incorrect answer.
- 2024-03arXiv 2403.20279LUQ: Long-text Uncertainty Quantification for LLMs
Caiqi Zhang, Fangyu Liu, Marco Basaldella et al. · Language models & NLP
Large Language Models (LLMs) have demonstrated remarkable capability in a variety of NLP tasks. However, LLMs are also prone to generate nonfactual content.
- 2024-03arXiv 2403.18802Long-form factuality in large language models
Jerry Wei, Chengrun Yang, Xinying Song et al. · Language models & NLP
Large language models (LLMs) often generate content that contains factual errors when responding to fact-seeking prompts on open-ended topics.
- 2024-03arXiv 2403.05612Unfamiliar Finetuning Examples Control How Language Models Hallucinate
Katie Kang, Eric Wallace, Claire Tomlin et al. · Machine learning
Large language models are known to hallucinate when faced with unfamiliar queries, but the underlying mechanism that govern how models hallucinate are not yet fully understood.
- 2024-02arXiv 2402.10978Language Models with Conformal Factuality Guarantees
Christopher Mohri, Tatsunori Hashimoto · Machine learning
Guaranteeing the correctness and factuality of language model (LM) outputs is a major open problem. In this work, we propose conformal factuality, a framework that can ensure high probability correctness guarantees for LMs by connecting language modeling and conformal prediction.
- 2024-02arXiv 2402.03744INSIDE: LLMs' Internal States Retain the Power of Hallucination Detection
Chao Chen, Kai Liu, Ze Chen et al. · Language models & NLP
Knowledge hallucination have raised widespread concerns for the security and reliability of deployed LLMs.
- 2024-02arXiv 2402.00367Don't Hallucinate, Abstain: Identifying LLM Knowledge Gaps via Multi-LLM Collaboration
Shangbin Feng, Weijia Shi, Yike Wang et al. · Language models & NLP
Despite efforts to expand the knowledge of large language models (LLMs), knowledge gaps -- missing or outdated information in LLMs -- might always persist given the evolving nature of knowledge.
- 2024-01arXiv 2401.06730Relying on the Unreliable: The Impact of Language Models' Reluctance to Express Uncertainty
Kaitlyn Zhou, Jena D. Hwang, Xiang Ren et al. · Language models & NLP
As natural language becomes the default interface for human-AI interaction, there is a need for LMs to appropriately communicate uncertainties in downstream applications.
- 2023-12arXiv 2312.07000Alignment for Honesty
Yuqing Yang, Ethan Chern, Xipeng Qiu et al. · Language models & NLP
Recent research has made significant strides in aligning large language models (LLMs) with helpfulness and harmlessness.
- 2023-11arXiv 2311.09677R-Tuning: Instructing Large Language Models to Say `I Don't Know'
Hanning Zhang, Shizhe Diao, Yong Lin et al. · Language models & NLP
Large language models (LLMs) have revolutionized numerous domains with their impressive performance but still face their challenges. A predominant issue is the propensity for these models to generate non-existent facts, a concern termed hallucination.
Built from every arXiv paper cited in our papers, posts, topic hubs, reports and research notes, with titles, authors, dates and abstracts from the arXiv API (updated 2026-11-12). Summaries are the opening sentences of each paper's own abstract, not our assessment; a citation means we used or checked the work, not that we endorse it. Fields follow each paper's primary arXiv category. Papers cited only outside arXiv (journals, proceedings) are not listed yet. Entries are generated from our citations, so a mistyped ID in a note would list the wrong paper; we remove those as we find them. Report a wrong entry.
