NeuroGenesysThe research we build on.
1,514 papers by other researchers that our work cites, from continual learning and verification to language models, agents and hardware. Each entry links to the paper on arXiv and to where we use it. 301 are cited on published pages of this site; the rest in the research notes behind our studies. Our own papers are in the archive.
- 2024-07arXiv 2407.01032Overcoming Common Flaws in the Evaluation of Selective Classification Systems
Jeremias Traub, Till J. Bungert, Carsten T. Lüth et al. · Machine learning
Selective Classification, wherein models can reject low-confidence predictions, promises reliable translation of machine-learning based classification systems to real-world scenarios such as clinical diagnostics.
- 2024-07arXiv 2407.01376Badllama 3: removing safety finetuning from Llama 3 in minutes
Dmitrii Volkov · Machine learning
We show that extensive LLM safety fine-tuning is easily subverted when an attacker has access to model weights.
- 2024-06arXiv 2406.19764Belief Revision: The Adaptability of Large Language Models Reasoning
Bryan Wilie, Samuel Cahyawijaya, Etsuko Ishii et al. · Language models & NLP
The capability to reason from text is crucial for real-world NLP applications. Real-world scenarios often involve incomplete or evolving data. In response, individuals update their beliefs and understandings accordingly.
- 2024-06arXiv 2406.19276VERISCORE: Evaluating the factuality of verifiable claims in long-form text generation
Yixiao Song, Yekyung Kim, Mohit Iyyer · Language models & NLP
, 2024), decompose an input text into "atomic claims" and verify each against a knowledge base like Wikipedia., can plausibly be proven true or false). We address this issue with VERISCORE, a metric for diverse long-form generation tasks that contain both verifiable and unverifiable content.
- 2024-06arXiv 2406.19354Fundamental Problems With Model Editing: How Should Rational Belief Revision Work in LLMs?
Peter Hase, Thomas Hofweber, Xiang Zhou et al. · Language models & NLP
The model editing problem concerns how language models should learn new facts about the world over time.
- 2024-06arXiv 2406.18219A Closer Look into Mixture-of-Experts in Large Language Models
Ka Man Lo, Zeyu Huang, Zihan Qiu et al. · Language models & NLP
Mixture-of-experts (MoE) is gaining increasing attention due to its unique properties and remarkable performance, especially for language tasks.
- 2024-06arXiv 2406.18665RouteLLM: Learning to Route LLMs with Preference Data
Isaac Ong, Amjad Almahairi, Vincent Wu et al. · Machine learning
Large language models (LLMs) exhibit impressive capabilities across a wide range of tasks, yet the choice of which model to use often involves a trade-off between performance and cost.
Cited inRoute on shared statistics, never expert confidence: the router architecture for frozen expertsStudy: Hierarchical and T shaped MoEInternal notesStudy: Model self knowledge and confident errorsStudy: Optimal router architectureStudy: Orchestrated specialists and bridge synthesisStudy: Stopping confident wrong expert votes - 2024-06arXiv 2406.17557The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale
Guilherme Penedo, Hynek Kydlíček, Loubna Ben allal et al. · Language models & NLP
The performance of a large language model (LLM) depends heavily on the quality and size of its pretraining dataset. However, the pretraining datasets for state-of-the-art open LLMs like Llama 3 and Mixtral are not publicly available and very little is known about how they were created.
- 2024-06arXiv 2406.16437Theory on Mixture-of-Experts in Continual Learning
Hongbo Li, Sen Lin, Lingjie Duan et al. · Machine learning
Continual learning (CL) has garnered significant attention because of its ability to adapt to new tasks that arrive over time. Catastrophic forgetting (of old tasks) has been identified as a major issue in CL, as the model adapts to new tasks.
- 2024-06arXiv 2406.16554LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training
Tong Zhu, Xiaoye Qu, Daize Dong et al. · Language models & NLP
Mixture-of-Experts (MoE) has gained increasing popularity as a promising framework for scaling up large language models (LLMs). However, training MoE from scratch in a large-scale setting still suffers from data-hungry and instability problems.
- 2024-06arXiv 2406.16797Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs
Ashwinee Panda, Berivan Isik, Xiangyu Qi et al. · Language models & NLP
Existing methods for adapting large language models (LLMs) to new tasks are not suited to multi-task adaptation because they modify all the model weights -- causing destructive interference between tasks.
- 2024-06arXiv 2406.16989Retrieval-Augmented Mixture of LoRA Experts for Uploadable Machine Learning
Ziyu Zhao, Leilei Gan, Guoyin Wang et al. · Machine learning
Low-Rank Adaptation (LoRA) offers an efficient way to fine-tune large language models (LLMs). Its modular and plug-and-play nature allows the integration of various domain-specific LoRAs, enhancing LLM capabilities.
- 2024-06arXiv 2406.15927Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMs
Jannik Kossen, Jiatong Han, Muhammed Razzak et al. · Language models & NLP
We propose semantic entropy probes (SEPs), a cheap and reliable method for uncertainty quantification in Large Language Models (LLMs). Hallucinations, which are plausible-sounding but factually incorrect and arbitrary model generations, present a major challenge to the practical adoption of LLMs.
Cited inStable wrong beliefs evade every self-check: a literature synthesis on model self-knowledge and confident errorsStudy: Decision signatures for conflict detectionStudy: Logical coherence of LLM world modelsStudy: Model self knowledge and confident errorsStudy: Stopping confident wrong expert votes - 2024-06arXiv 2406.15627Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph
Roman Vashurin, Ekaterina Fadeeva, Artem Vazhentsev et al. · Transactions of the Association for Computational Linguistics, 13 (2025) 220-248 · Language models & NLP
The rapid proliferation of large language models (LLMs) has stimulated researchers to seek effective and efficient approaches to deal with LLM hallucinations and low-quality outputs.
- 2024-06arXiv 2406.14026Demystifying Language Model Forgetting with Low-rank Example Associations
Xisen Jin, Xiang Ren · Machine learning
Large language models (LLMs) suffer from forgetting of upstream knowledge when fine-tuned. Despite efforts on mitigating forgetting, few have investigated how forgotten upstream examples are dependent on newly learned tasks.
- 2024-06arXiv 2406.13233AdaMoE: Token-Adaptive Routing with Null Experts for Mixture-of-Experts Language Models
Zihao Zeng, Yibo Miao, Hongcheng Gao et al. · AI & agents
Mixture of experts (MoE) has become the standard for constructing production-level large language models (LLMs) due to its promise to boost model capacity without causing significant overheads.
- 2024-06arXiv 2406.13356Unlearning or Obfuscating? Jogging the Memory of Unlearned LLMs via Benign Relearning
Shengyuan Hu, Yiwei Fu, Zhiwei Steven Wu et al. · Machine learning
Machine unlearning is a promising approach to mitigate undesirable memorization of training data in ML models. However, in this work we show that existing approaches for unlearning in LLMs are surprisingly susceptible to a simple set of .
- 2024-06arXiv 2406.12673Estimating Knowledge in Large Language Models Without Generating a Single Token
Daniela Gottesman, Mor Geva · Language models & NLP
To evaluate knowledge in large language models (LLMs), current methods query the model and then evaluate its generated responses. In this work, we ask whether evaluation can be done before the model has generated any text.
- 2024-06arXiv 2406.11717Refusal in Language Models Is Mediated by a Single Direction
Andy Arditi, Oscar Obeso, Aaquib Syed et al. · Machine learning
Conversational large language models are fine-tuned for both instruction-following and safety, resulting in models that obey benign requests but refuse harmful ones.
- 2024-06arXiv 2406.11813How Do Large Language Models Acquire Factual Knowledge During Pretraining?
Hoyeon Chang, Jinho Park, Seonghyeon Ye et al. · Language models & NLP
Despite the recent observation that large language models (LLMs) can store substantial factual knowledge, there is a limited understanding of the mechanisms of how they acquire factual knowledge through pretraining.
- 2024-06arXiv 2406.12045-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains
Shunyu Yao, Noah Shinn, Pedram Razavi et al. · AI & agents
Existing benchmarks do not test language agents on their interaction with human users or ability to follow domain-specific rules, both of which are vital for deploying them in real world applications.
Cited inStudy: Genesys program prior artInternal notesStudy: Model self knowledge and confident errors - 2024-06arXiv 2406.15479Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging
Zhenyi Lu, Chenghao Fan, Wei Wei et al. · Language models & NLP
In the era of large language models, model merging is a promising way to combine multiple task-specific models into a single multitask model without extra training.
- 2024-06arXiv 2406.09714Large language model validity via enhanced conformal prediction methods
John J. Cherian, Isaac Gibbs, Emmanuel J. Candès · Machine learning
We develop new conformal inference methods for obtaining validity guarantees on the output of large language models (LLMs). Prior work in conformal language modeling identifies a subset of the text that satisfies a high-probability guarantee of correctness.
- 2024-06arXiv 2406.10099Know the Unknown: An Uncertainty-Sensitive Method for LLM Instruction Tuning
Jiaqi Li, Yixuan Tang, Yi Yang · Language models & NLP
Large language models (LLMs) demonstrate remarkable capabilities but face challenges from hallucinations, which typically arise from insufficient knowledge or context.
- 2024-06arXiv 2406.10162Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Models
Carson Denison, Monte MacDiarmid, Fazl Barez et al. · AI & agents
In reinforcement learning, specification gaming occurs when AI systems learn undesired behaviors that are highly rewarded due to misspecified training goals.
- 2024-06arXiv 2406.07814Collective Constitutional AI: Aligning a Language Model with Public Input
Saffron Huang, Divya Siddarth, Liane Lovitt et al. · Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency. 1395-1417 · AI & agents
There is growing consensus that language model (LM) developers should not be the sole deciders of LM behavior, creating a need for methods that enable the broader public to collectively shape the behavior of LM systems that affect them.
- 2024-06arXiv 2406.08391Large Language Models Must Be Taught to Know What They Don't Know
Sanyam Kapoor, Nate Gruver, Manley Roberts et al. · Machine learning
When using large language models (LLMs) in high-stakes applications, we need to know when we can trust their predictions.
- 2024-06arXiv 2406.09455Pandora: Towards General World Model with Natural Language Actions and Video States
Jiannan Xiang, Guangyi Liu, Yi Gu et al. · Vision
World models simulate future states of the world in response to different actions. They facilitate interactive content creation and provides a foundation for grounded, long-horizon reasoning.
- 2024-06arXiv 2406.06858FLUX: Fast Software-based Communication Overlap On GPUs Through Kernel Fusion
Li-Wen Chang, Wenlei Bao, Qi Hou et al. · Machine learning
Large deep learning models have demonstrated strong ability to solve many tasks across a wide range of applications. Those large models typically require training and inference to be distributed.
Cited inThe Wrong Phase - 2024-06arXiv 2406.07155Scaling Large Language Model-based Multi-Agent Collaboration
Chen Qian, Zihao Xie, YiFei Wang et al. · AI & agents
Recent breakthroughs in large language model-driven autonomous agents have revealed that multi-agent collaboration often surpasses each individual through collective reasoning.
- 2024-06arXiv 2406.07515Beyond Model Collapse: Scaling Up with Synthesized Data Requires Verification
Yunzhen Feng, Elvis Dohmatob, Pu Yang et al. · Machine learning
Large Language Models (LLM) are increasingly trained on data generated by other LLM, either because generated text and images become part of the pre-training corpus, or because synthetized data is used as a replacement for expensive human-annotation.
- 2024-06arXiv 2406.06461Reasoning in Token Economies: Budget-Aware Evaluation of LLM Reasoning Strategies
Junlin Wang, Siddhartha Jain, Dejiao Zhang et al. · Language models & NLP
A diverse array of reasoning strategies has been proposed to elicit the capabilities of large language models. However, in this paper, we point out that traditional evaluations which focus solely on performance metrics miss a key factor: the increased effectiveness due to additional compute.
- 2024-06arXiv 2406.06519UMBRELA: UMbrela is the (Open-Source Reproduction of the) Bing RELevance Assessor
Shivani Upadhyay, Ronak Pradeep, Nandan Thakur et al. · Retrieval & data
Copious amounts of relevance judgments are necessary for the effective training and accurate evaluation of retrieval systems. Conventionally, these judgments are made by human assessors, rendering this process expensive and laborious.
Cited inInternal notes - 2024-06arXiv 2406.04692Mixture-of-Agents Enhances Large Language Model Capabilities
Junlin Wang, Jue Wang, Ben Athiwaratkun et al. · Language models & NLP
Recent advances in large language models (LLMs) demonstrate substantial capabilities in natural language understanding and generation tasks. With the growing number of LLMs, how to harness the collective expertise of multiple LLMs is an exciting open direction.
- 2024-06arXiv 2406.03689Evaluating the World Model Implicit in a Generative Model
Keyon Vafa, Justin Y. Chen, Ashesh Rambachan et al. · Language models & NLP
Recent work suggests that large language models may implicitly learn world models. How should we assess this possibility? We formalize this question for the case where the underlying reality is governed by a deterministic finite automaton.
- 2024-06arXiv 2406.04127Are We Done with MMLU?
Aryo Pradipta Gema, Joshua Ong Jun Leang, Giwon Hong et al. · Language models & NLP
Maybe not. We identify and analyse errors in the popular Massive Multitask Language Understanding (MMLU) benchmark. Even though MMLU is widely adopted, our analysis demonstrates numerous ground truth errors that obscure the true capabilities of LLMs.
- 2024-06arXiv 2406.04268Open-Endedness is Essential for Artificial Superhuman Intelligence
Edward Hughes, Michael Dennis, Jack Parker-Holder et al. · Machine learning
In recent years there has been a tremendous surge in the general capabilities of AI systems, mainly fuelled by training foundation models on internetscale data. Nevertheless, the creation of openended, ever self-improving AI remains elusive.
- 2024-06arXiv 2406.04496Time Sensitive Knowledge Editing through Efficient Finetuning
Xiou Ge, Ali Mousavi, Edouard Grave et al. · Language models & NLP
Large Language Models (LLMs) have demonstrated impressive capability in different tasks and are bringing transformative changes to many domains. However, keeping the knowledge in LLMs up-to-date remains a challenge once pretraining is complete.
- 2024-06arXiv 2406.06608The Prompt Report: A Systematic Survey of Prompt Engineering Techniques
Sander Schulhoff, Michael Ilie, Nishant Balepur et al. · Language models & NLP
Generative Artificial Intelligence (GenAI) systems are increasingly being deployed across diverse industries and research domains. Developers and end-users interact with these systems through the use of prompting and prompt engineering.
- 2024-06arXiv 2406.02543To Believe or Not to Believe Your LLM
Yasin Abbasi Yadkori, Ilja Kuzborskij, András György et al. · Machine learning
We explore uncertainty quantification in large language models (LLMs), with the goal to identify when uncertainty in responses given a query is large.
Built from every arXiv paper cited in our papers, posts, topic hubs, reports and research notes, with titles, authors, dates and abstracts from the arXiv API (updated 2026-11-12). Summaries are the opening sentences of each paper's own abstract, not our assessment; a citation means we used or checked the work, not that we endorse it. Fields follow each paper's primary arXiv category. Papers cited only outside arXiv (journals, proceedings) are not listed yet. Entries are generated from our citations, so a mistyped ID in a note would list the wrong paper; we remove those as we find them. Report a wrong entry.
