NeuroGenesysThe research we build on.
1,514 papers by other researchers that our work cites, from continual learning and verification to language models, agents and hardware. Each entry links to the paper on arXiv and to where we use it. 301 are cited on published pages of this site; the rest in the research notes behind our studies. Our own papers are in the archive.
- 2023-06arXiv 2306.17165An Efficient General-Purpose Modular Vision Model via Multi-Task Heterogeneous Training
Zitian Chen, Mingyu Ding, Yikang Shen et al. · Vision
We present a model that can perform multiple vision tasks and can be adapted to other downstream tasks efficiently. Despite considerable progress in multi-task learning, most efforts focus on learning from multi-label data: a single image set with multiple task labels.
- 2023-06arXiv 2306.13812Maintaining Plasticity in Deep Continual Learning
Shibhansh Dohare, J. Fernando Hernandez-Garcia, Parash Rahman et al. · Machine learning
Modern deep-learning systems are specialized to problem settings in which training occurs once and then never again, as opposed to continual-learning settings in which training occurs continually.
- 2023-06arXiv 2306.12646Learnability and Algorithm for Continual Learning
Gyuhak Kim, Changnan Xiao, Tatsuya Konishi et al. · Machine learning
This paper studies the challenging continual learning (CL) setting of Class Incremental Learning (CIL). CIL learns a sequence of tasks consisting of disjoint sets of concepts or classes.
- 2023-06arXiv 2306.13063Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs
Miao Xiong, Zhiyuan Hu, Xinyang Lu et al. · Language models & NLP
Empowering large language models to accurately express confidence in their answers is essential for trustworthy decision-making.
- 2023-06arXiv 2306.12142Synaptic metaplasticity with multi-level memristive devices
Simone D'Agostino, Filippo Moro, Tifenn Hirtzlin et al. · Neuroscience & neural computation
Deep learning has made remarkable progress in various tasks, surpassing human performance in some cases. However, one drawback of neural networks is catastrophic forgetting, where a network trained on one task forgets the solution when learning a new one.
- 2023-06arXiv 2306.11644Textbooks Are All You Need
Suriya Gunasekar, Yi Zhang, Jyoti Aneja et al. · Language models & NLP
5 (1B tokens).5% on MBPP.
- 2023-06arXiv 2306.09983Evaluating Superhuman Models with Consistency Checks
Lukas Fluri, Daniel Paleka, Florian Tramèr · Machine learning
If machine learning models were to achieve superhuman abilities at various reasoning or decision-making tasks, how would we go about evaluating such models, given that humans would necessarily be poor proxies for ground truth?
- 2023-06arXiv 2306.10193Conformal Language Modeling
Victor Quach, Adam Fisch, Tal Schuster et al. · Language models & NLP
We propose a novel approach to conformal prediction for generative language models (LMs). Standard conformal prediction produces prediction sets -- in place of single predictions -- that have rigorous, statistical performance guarantees.
- 2023-06arXiv 2306.09300Exploring Magnetism of Lead-free Halide Double Perovskites: A High-Throughput First-Principles Study
Utkarsh Singh, Johan Klarbring, Igor A. Abrikosov et al. · Hardware, photonics & physics
We have performed a comprehensive, first-principles high-throughput study of the magnetic properties of halide double perovskites, , with magnetic ions occupying one or both B and B$^\prime$ sites.
- 2023-06arXiv 2306.09301OpenOOD v1.5: Enhanced Benchmark for Out-of-Distribution Detection
Jingyang Zhang, Jingkang Yang, Pengyun Wang et al. · Machine learning
Out-of-Distribution (OOD) detection is critical for the reliable operation of open-world intelligent systems. Despite the emergence of an increasing number of OOD detection methods, the evaluation inconsistencies present challenges for tracking the progress in this field.
- 2023-06arXiv 2306.09306Propagating Knowledge Updates to LMs Through Distillation
Shankar Padmanabhan, Yasumasa Onoe, Michael J. Q. Zhang et al. · Language models & NLP
Modern language models have the capacity to store and use immense amounts of knowledge about real-world entities, but it remains unclear how to update such knowledge stored in model parameters.
- 2023-06arXiv 2306.05685Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng et al. · Language models & NLP
Evaluating large language model (LLM) based chat assistants is challenging due to their broad capabilities and the inadequacy of existing benchmarks in measuring human preferences.
Cited inInternal notes - 2023-06arXiv 2306.04181Benchmarking Foundation Models with Language-Model-as-an-Examiner
Yushi Bai, Jiahao Ying, Yixin Cao et al. · Language models & NLP
Numerous benchmarks have been established to assess the performance of foundation models on open-ended question answering, which serves as a comprehensive test of a model's ability to understand and generate language in a manner similar to humans.
- 2023-06arXiv 2306.04529Git-Theta: A Git Extension for Collaborative Development of Machine Learning Models
Nikhil Kandpal, Brian Lester, Mohammed Muqeeth et al. · Machine learning
Currently, most machine learning models are trained by centralized teams and are rarely updated. In contrast, open-source software development involves the iterative development of a shared artifact through distributed collaboration using a version control system.
- 2023-06arXiv 2306.03341Inference-Time Intervention: Eliciting Truthful Answers from a Language Model
Kenneth Li, Oam Patel, Fernanda Viégas et al. · Machine learning
We introduce Inference-Time Intervention (ITI), a technique designed to enhance the "truthfulness" of large language models (LLMs). ITI operates by shifting model activations during inference, following a set of directions across a limited number of attention heads.
- 2023-06arXiv 2306.03745Soft Merging of Experts with Adaptive Routing
Mohammed Muqeeth, Haokun Liu, Colin Raffel · Machine learning
Sparsely activated neural networks with conditional computation learn to route their inputs through different "expert" subnetworks, providing a form of modularity that densely activated models lack.
- 2023-06arXiv 2306.02561LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion
Dongfu Jiang, Xiang Ren, Bill Yuchen Lin · Language models & NLP
We present LLM-Blender, an ensembling framework designed to attain consistently superior performance by leveraging the diverse strengths of multiple open-source large language models (LLMs).
- 2023-06arXiv 2306.03288Deep Learning From Crowdsourced Labels: Coupled Cross-entropy Minimization, Identifiability, and Regularization
Shahana Ibrahim, Tri Nguyen, Xiao Fu · Machine learning
Using noisy crowdsourced labels from multiple annotators, a deep learning-based end-to-end (E2E) system aims to learn the label correction mechanism and the neural classifier simultaneously.
- 2023-06arXiv 2306.01708TIES-Merging: Resolving Interference When Merging Models
Prateek Yadav, Derek Tam, Leshem Choshen et al. · Machine learning
, further fine-tuning a pre-trained model on a downstream task - can confer significant advantages, including improved downstream performance, faster convergence, and better sample efficiency.
Cited inStudy: Decision signatures for conflict detectionStudy: Fortifying knowledge in continual learningStudy: Generational competitive continual learningStudy: Hierarchical and T shaped MoEStudy: LLMs modifying open model weightsStudy: Merging and splitting MoE expertsStudy: Weight editing to prevent forgetting - 2023-06arXiv 2306.01711OMNI: Open-endedness via Models of human Notions of Interestingness
Jenny Zhang, Joel Lehman, Kenneth Stanley et al. · AI & agents
Open-ended algorithms aim to learn new, interesting behaviors forever. That requires a vast environment search space, but there are thus infinitely many possible tasks.
- 2023-06arXiv 2306.00802Birth of a Transformer: A Memory Viewpoint
Alberto Bietti, Vivien Cabannes, Diane Bouchacourt et al. · Machine learning
Large language models based on transformers have achieved great empirical successes. However, as they are deployed more widely, there is a growing need to better understand their internal mechanisms in order to make them more reliable.
- 2023-05arXiv 2305.20050Let's Verify Step by Step
Hunter Lightman, Vineet Kosaraju, Yura Burda et al. · Machine learning
In recent years, large language models have greatly improved in their ability to perform complex multi-step reasoning. However, even state-of-the-art models still regularly produce logical mistakes.
- 2023-05arXiv 2305.18806Prediction Error-based Classification for Class-Incremental Learning
Michał Zając, Tinne Tuytelaars, Gido M. van de Ven · Machine learning
Class-incremental learning (CIL) is a particularly challenging variant of continual learning, where the goal is to learn to discriminate between all classes presented in an incremental fashion.
Cited inStudy: Genesys program prior art - 2023-05arXiv 2305.19118Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate
Tian Liang, Zhiwei He, Wenxiang Jiao et al. · Language models & NLP
Modern large language models (LLMs) like ChatGPT have shown remarkable performance on general language tasks but still struggle on complex reasoning tasks, which drives the research on cognitive behaviors of LLMs to explore human-like problem-solving strategies.
- 2023-05arXiv 2305.18153Do Large Language Models Know What They Don't Know?
Zhangyue Yin, Qiushi Sun, Qipeng Guo et al. · Language models & NLP
Large language models (LLMs) have a wealth of knowledge that allows them to excel in various Natural Language Processing (NLP) tasks. Current research focuses on enhancing their performance within their existing knowledge.
- 2023-05arXiv 2305.17493The Curse of Recursion: Training on Generated Data Makes Models Forget
Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao et al. · Machine learning
Stable Diffusion revolutionised image creation from descriptive text.5) and GPT-4 demonstrated astonishing performance across a variety of language tasks. ChatGPT introduced such language models to the general public.
- 2023-05arXiv 2305.15852Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigation
Niels Mündler, Jingxuan He, Slobodan Jenko et al. · Language models & NLP
Large language models (large LMs) are susceptible to producing text that contains hallucinated content. An important instance of this problem is self-contradiction, where the LM generates two contradictory sentences within the same context.
- 2023-05arXiv 2305.16291Voyager: An Open-Ended Embodied Agent with Large Language Models
Guanzhi Wang, Yuqi Xie, Yunfan Jiang et al. · AI & agents
We introduce Voyager, the first LLM-powered embodied lifelong learning agent in Minecraft that continuously explores the world, acquires diverse skills, and makes novel discoveries without human intervention.
- 2023-05arXiv 2305.14795MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions
Zexuan Zhong, Zhengxuan Wu, Christopher D. Manning et al. · Language models & NLP
The information stored in large language models (LLMs) falls out of date quickly, and retraining from scratch is often not an option. This has recently given rise to a range of techniques for injecting new facts through updating model weights.
Cited inTrust the premises, not the confidence: which belief to drop when an LLM contradicts itself, and why the solver is not the bottleneckStudy: Consequence consistent updatesStudy: Continual learning order and companion dataStudy: Graded succession by earned authorityStudy: LLMs modifying open model weightsStudy: Logical coherence of LLM world modelsStudy: Orchestrated specialists and bridge synthesisStudy: Weight editing to prevent forgetting - 2023-05arXiv 2305.14975Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback
Katherine Tian, Eric Mitchell, Allan Zhou et al. · Language models & NLP
A trustworthy real-world prediction system should produce well-calibrated confidence scores; that is, its confidence in an answer should be indicative of the likelihood that the answer is correct, enabling deferral to an expert in cases of low-confidence predictions.
- 2023-05arXiv 2305.15555Deep Reinforcement Learning with Plasticity Injection
Evgenii Nikishin, Junhyuk Oh, Georg Ostrovski et al. · Machine learning
A growing body of evidence suggests that neural networks employed in deep reinforcement learning (RL) gradually lose their plasticity, the ability to learn from new data; however, the analysis and mitigation of this phenomenon is hampered by the complex relationship between plasticity, exploration, and performance…
- 2023-05arXiv 2305.13712Knowledge of Knowledge: Exploring Known-Unknowns Uncertainty with Large Language Models
Alfonso Amayuelas, Kyle Wong, Liangming Pan et al. · Language models & NLP
This paper investigates the capabilities of Large Language Models (LLMs) in the context of understanding their knowledge and uncertainty over questions. Specifically, we focus on addressing known-unknown questions, characterized by high uncertainty due to the absence of definitive answers.
- 2023-05arXiv 2305.14250Language Models with Rationality
Nora Kassner, Oyvind Tafjord, Ashish Sabharwal et al. · Language models & NLP
While large language models (LLMs) are proficient at question-answering (QA), it is not always clear how (or even if) an answer follows from their latent "beliefs".
- 2023-05arXiv 2305.14251FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation
Sewon Min, Kalpesh Krishna, Xinxi Lyu et al. · Language models & NLP
Evaluating the factuality of long-form text generated by large language models (LMs) is non-trivial because (1) generations often contain a mixture of supported and unsupported pieces of information, making binary judgments of quality inadequate, and (2) human evaluation is time-consuming and costly.
- 2023-05arXiv 2305.14325Improving Factuality and Reasoning in Language Models through Multiagent Debate
Yilun Du, Shuang Li, Antonio Torralba et al. · Language models & NLP
Large language models (LLMs) have demonstrated remarkable capabilities in language generation, understanding, and few-shot learning in recent years.
- 2023-05arXiv 2305.18323ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models
Binfeng Xu, Zhiyuan Peng, Bowen Lei et al. · Language models & NLP
Augmented Language Models (ALMs) blend the reasoning capabilities of Large Language Models (LLMs) with tools that allow for knowledge retrieval and action execution.
- 2023-05arXiv 2305.12616Conformal Prediction With Conditional Guarantees
Isaac Gibbs, John J. Cherian, Emmanuel J. Candès · Statistics & theory
We consider the problem of constructing distribution-free prediction sets with finite-sample conditional guarantees. Prior work has shown that it is impossible to provide exact conditional coverage universally in finite samples.
- 2023-05arXiv 2305.12766Understanding Emergent In-Context Learning from a Kernel Regression Perspective
Chi Han, Ziqi Wang, Han Zhao et al. · Language models & NLP
Large language models (LLMs) have initiated a paradigm shift in transfer learning.
- 2023-05arXiv 2305.12827Task Arithmetic in the Tangent Space: Improved Editing of Pre-Trained Models
Guillermo Ortiz-Jimenez, Alessandro Favero, Pascal Frossard · Advances in Neural Information Processing Systems 36 (NeurIPS 2023) · Machine learning
Task arithmetic has recently emerged as a cost-effective and scalable approach to edit pre-trained models directly in weight space: By adding the fine-tuned weights of different tasks, the model's performance can be improved on these tasks, while negating them leads to task forgetting.
- 2023-05arXiv 2305.13281LM vs LM: Detecting Factual Errors via Cross Examination
Roi Cohen, May Hamri, Mor Geva et al. · Language models & NLP
A prominent weakness of modern language models (LMs) is their tendency to generate factually incorrect text, which hinders their usability. A natural question is whether such factual errors can be detected automatically.
Built from every arXiv paper cited in our papers, posts, topic hubs, reports and research notes, with titles, authors, dates and abstracts from the arXiv API (updated 2026-11-12). Summaries are the opening sentences of each paper's own abstract, not our assessment; a citation means we used or checked the work, not that we endorse it. Fields follow each paper's primary arXiv category. Papers cited only outside arXiv (journals, proceedings) are not listed yet. Entries are generated from our citations, so a mistyped ID in a note would list the wrong paper; we remove those as we find them. Report a wrong entry.
