NeuroGenesysThe research we build on.
1,514 papers by other researchers that our work cites, from continual learning and verification to language models, agents and hardware. Each entry links to the paper on arXiv and to where we use it. 301 are cited on published pages of this site; the rest in the research notes behind our studies. Our own papers are in the archive.
- 2023-11arXiv 2311.17371Should we be going MAD? A Look at Multi-Agent Debate Strategies for LLMs
Andries Smit, Paul Duckworth, Nathan Grinsztajn et al. · Language models & NLP
Recent advancements in large language models (LLMs) underscore their potential for responding to inquiries in various domains. However, ensuring that generative agents provide accurate and reliable answers remains an ongoing challenge.
- 2023-11arXiv 2311.08702Debate Helps Supervise Unreliable Experts
Julian Michael, Salsabila Mahdi, David Rein et al. · AI & agents
As AI systems are used to answer more difficult questions and potentially help create new knowledge, judging the truthfulness of their outputs becomes more difficult and more important.
- 2023-11arXiv 2311.08516LLMs cannot find reasoning errors, but can correct them given the error location
Gladys Tyen, Hassan Mansoor, Victor Cărbune et al. · AI & agents
g., 2023). In this paper, we show that poor self-correction performance stems from LLMs' inability to find logical mistakes, rather than their ability to correct a known mistake.
- 2023-10arXiv 2310.14152Orthogonal Subspace Learning for Language Model Continual Learning
Xiao Wang, Tianze Chen, Qiming Ge et al. · Language models & NLP
Benefiting from massive corpora and advanced hardware, large language models (LLMs) exhibit remarkable capabilities in language understanding and generation. However, their performance degrades in scenarios where multiple tasks are encountered sequentially, also known as catastrophic forgetting.
Cited inEWC protects what its Fisher sees: a literature synthesis on elastic weight consolidation, scale and coherenceStudy: Continual learning order and companion dataStudy: Elastic weight consolidation deep diveStudy: Fortifying knowledge in continual learningStudy: Genesys program prior artStudy: Hardware guardian core for continual learningStudy: Merging and splitting MoE expertsStudy: Orchestrated specialists and bridge synthesisStudy: Seven continual learning methods tested - 2023-10arXiv 2310.06272Let Models Speak Ciphers: Multiagent Debate through Embeddings
Chau Pham, Boyi Liu, Yingxiang Yang et al. · Language models & NLP
Discussion and debate among Large Language Models (LLMs) have gained considerable attention due to their potential to enhance the reasoning ability of LLMs.
- 2023-10arXiv 2310.01798Large Language Models Cannot Self-Correct Reasoning Yet
Jie Huang, Xinyun Chen, Swaroop Mishra et al. · Language models & NLP
Large Language Models (LLMs) have emerged as a groundbreaking technology with their unparalleled text generation capabilities across various applications. Nevertheless, concerns persist regarding the accuracy and appropriateness of their generated content.
Cited inCoherence is selected, anchored, then pruned: a literature synthesis on logical coherence in LLM world modelsStable wrong beliefs evade every self-check: a literature synthesis on model self-knowledge and confident errorsStudy: Civilization as a superintelligence modelStudy: Cross generational teaching and examinationStudy: Logical coherence of LLM world modelsStudy: Model self knowledge and confident errorsStudy: Orchestrated specialists and bridge synthesis - 2023-09arXiv 2309.13007ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs
Justin Chih-Yao Chen, Swarnadeep Saha, Mohit Bansal · Language models & NLP
Large Language Models (LLMs) still struggle with natural language reasoning tasks. Motivated by the society of minds (Minsky, 1988), we propose ReConcile, a multi-model multi-agent framework designed as a round table conference among diverse LLM agents.
- 2023-09arXiv 2309.11495Chain-of-Verification Reduces Hallucination in Large Language Models
Shehzaad Dhuliawala, Mojtaba Komeili, Jing Xu et al. · Language models & NLP
Generation of plausible yet incorrect factual information, termed hallucination, is an unsolved issue in large language models. We study the ability of language models to deliberate on the responses they give in order to correct their mistakes.
- 2023-08arXiv 2308.08155AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation
Qingyun Wu, Gagan Bansal, Jieyu Zhang et al. · AI & agents
AutoGen is an open-source framework that allows developers to build LLM applications via multiple agents that can converse with each other to accomplish tasks. AutoGen agents are customizable, conversable, and can operate in various modes that employ combinations of LLMs, human inputs, and tools.
- 2023-08arXiv 2308.06948MixBCT: Towards Self-Adapting Backward-Compatible Training
Yu Liang, Yufeng Zhang, Shiliang Zhang et al. · Vision
Backward-compatible training circumvents the need for expensive updates to the old gallery database when deploying an advanced new model in the retrieval system.
- 2023-08arXiv 2308.00352MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework
Sirui Hong, Mingchen Zhuge, Jiaqi Chen et al. · AI & agents
Remarkable progress has been made on automated problem solving through societies of agents based on large language models (LLMs). Existing LLM-based multi-agent systems can already solve simple dialogue tasks.
- 2023-07arXiv 2307.13269LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition
Chengsong Huang, Qian Liu, Bill Yuchen Lin et al. · Language models & NLP
Low-rank adaptations (LoRA) are often employed to fine-tune large language models (LLMs) for new tasks.
- 2023-07arXiv 2307.12976Evaluating the Ripple Effects of Knowledge Editing in Language Models
Roi Cohen, Eden Biran, Ori Yoran et al. · Language models & NLP
Modern language models capture a large body of factual knowledge. However, some facts can be incorrectly induced or become obsolete over time, resulting in factually incorrect generations.
Cited inA gradient kernel flags contradictions but does not predict forgetting, and steering rehearsal by it backfiresCoherence is selected, anchored, then pruned: a literature synthesis on logical coherence in LLM world modelsEWC protects what its Fisher sees: a literature synthesis on elastic weight consolidation, scale and coherenceIterated self-closure settles fast, but only evidence that overrides self-belief makes the settled state trueTo stop a model forgetting, rehearse what it knew. New context won't do it.Study: Consequence consistent updatesStudy: Continual learning order and companion dataStudy: Decision signatures for conflict detectionStudy: Elastic weight consolidation deep diveStudy: Graded succession by earned authorityStudy: Hardware guardian core for continual learningStudy: Logical coherence of LLM world modelsStudy: Orchestrated specialists and bridge synthesis - 2023-07arXiv 2307.07924ChatDev: Communicative Agents for Software Development
Chen Qian, Wei Liu, Hongzhang Liu et al. · Society, software & economics
Software development is a complex task that necessitates cooperation among multiple members with diverse skills. Numerous studies used deep learning to improve specific phases in a waterfall model, such as design, coding, and testing.
- 2023-06arXiv 2306.09300Exploring Magnetism of Lead-free Halide Double Perovskites: A High-Throughput First-Principles Study
Utkarsh Singh, Johan Klarbring, Igor A. Abrikosov et al. · Hardware, photonics & physics
We have performed a comprehensive, first-principles high-throughput study of the magnetic properties of halide double perovskites, , with magnetic ions occupying one or both B and B$^\prime$ sites.
- 2023-06arXiv 2306.02561LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion
Dongfu Jiang, Xiang Ren, Bill Yuchen Lin · Language models & NLP
We present LLM-Blender, an ensembling framework designed to attain consistently superior performance by leveraging the diverse strengths of multiple open-source large language models (LLMs).
- 2023-05arXiv 2305.20050Let's Verify Step by Step
Hunter Lightman, Vineet Kosaraju, Yura Burda et al. · Machine learning
In recent years, large language models have greatly improved in their ability to perform complex multi-step reasoning. However, even state-of-the-art models still regularly produce logical mistakes.
- 2023-05arXiv 2305.19118Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate
Tian Liang, Zhiwei He, Wenxiang Jiao et al. · Language models & NLP
Modern large language models (LLMs) like ChatGPT have shown remarkable performance on general language tasks but still struggle on complex reasoning tasks, which drives the research on cognitive behaviors of LLMs to explore human-like problem-solving strategies.
- 2023-05arXiv 2305.16291Voyager: An Open-Ended Embodied Agent with Large Language Models
Guanzhi Wang, Yuqi Xie, Yunfan Jiang et al. · AI & agents
We introduce Voyager, the first LLM-powered embodied lifelong learning agent in Minecraft that continuously explores the world, acquires diverse skills, and makes novel discoveries without human intervention.
- 2023-05arXiv 2305.14795MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions
Zexuan Zhong, Zhengxuan Wu, Christopher D. Manning et al. · Language models & NLP
The information stored in large language models (LLMs) falls out of date quickly, and retraining from scratch is often not an option. This has recently given rise to a range of techniques for injecting new facts through updating model weights.
Cited inTrust the premises, not the confidence: which belief to drop when an LLM contradicts itself, and why the solver is not the bottleneckStudy: Consequence consistent updatesStudy: Continual learning order and companion dataStudy: Graded succession by earned authorityStudy: LLMs modifying open model weightsStudy: Logical coherence of LLM world modelsStudy: Orchestrated specialists and bridge synthesisStudy: Weight editing to prevent forgetting - 2023-05arXiv 2305.14251FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation
Sewon Min, Kalpesh Krishna, Xinxi Lyu et al. · Language models & NLP
Evaluating the factuality of long-form text generated by large language models (LMs) is non-trivial because (1) generations often contain a mixture of supported and unsupported pieces of information, making binary judgments of quality inadequate, and (2) human evaluation is time-consuming and costly.
- 2023-05arXiv 2305.14325Improving Factuality and Reasoning in Language Models through Multiagent Debate
Yilun Du, Shuang Li, Antonio Torralba et al. · Language models & NLP
Large language models (LLMs) have demonstrated remarkable capabilities in language generation, understanding, and few-shot learning in recent years.
- 2023-05arXiv 2305.18323ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models
Binfeng Xu, Zhiyuan Peng, Bowen Lei et al. · Language models & NLP
Augmented Language Models (ALMs) blend the reasoning capabilities of Large Language Models (LLMs) with tools that allow for knowledge retrieval and action execution.
- 2023-05arXiv 2305.13281LM vs LM: Detecting Factual Errors via Cross Examination
Roi Cohen, May Hamri, Mor Geva et al. · Language models & NLP
A prominent weakness of modern language models (LMs) is their tendency to generate factually incorrect text, which hinders their usability. A natural question is whether such factual errors can be detected automatically.
- 2023-05arXiv 2305.05176FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance
Lingjiao Chen, Matei Zaharia, James Zou · Machine learning
There is a rapidly growing number of large language models (LLMs) that users can query for a fee.g. GPT-4, ChatGPT, J1-Jumbo, and find that these models have heterogeneous pricing structures, with fees that can differ by two orders of magnitude.
- 2023-03arXiv 2303.17580HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face
Yongliang Shen, Kaitao Song, Xu Tan et al. · Language models & NLP
Solving complicated AI tasks with different domains and modalities is a key step toward artificial general intelligence. While there are numerous AI models available for various domains and modalities, they cannot handle complicated AI tasks autonomously.
- 2023-03arXiv 2303.14177Scaling Expert Language Models with Unsupervised Domain Discovery
Suchin Gururangan, Margaret Li, Mike Lewis et al. · Language models & NLP
Large language models are typically trained densely: all parameters are updated with respect to all inputs. This requires synchronization of billions of parameters across thousands of GPUs.
- 2023-03arXiv 2303.08774GPT-4 Technical Report
OpenAI, Josh Achiam, Steven Adler et al. · Language models & NLP
We report the development of GPT-4, a large-scale, multimodal model which can accept image and text inputs and produce text outputs.
- 2022-11arXiv 2211.13218CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learning
James Seale Smith, Leonid Karlinsky, Vyshnavi Gutta et al. · Vision
Computer vision models suffer from a phenomenon known as catastrophic forgetting when learning novel concepts from continuously shifting training data.
- 2022-11arXiv 2211.11875Enhancing Self-Consistency and Performance of Pre-Trained Language Models through Natural Language Inference
Eric Mitchell, Joseph J. Noh, Siyan Li et al. · Language models & NLP
While large pre-trained language models are powerful, their predictions often lack logical consistency across test inputs.'.
Cited inCoherence is selected, anchored, then pruned: a literature synthesis on logical coherence in LLM world modelsTrust the premises, not the confidence: which belief to drop when an LLM contradicts itself, and why the solver is not the bottleneckStudy: Logical coherence of LLM world modelsStudy: Orchestrated specialists and bridge synthesis - 2022-11arXiv 2211.11031Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors
Thomas Hartvigsen, Swami Sankaranarayanan, Hamid Palangi et al. · Machine learning
Deployed language models decay over time due to shifting inputs, changing user needs, or emergent world-knowledge gaps. When such problems are identified, we want to make targeted edits while avoiding expensive retraining.
- 2022-11arXiv 2211.02633A Theoretical Study on Solving Continual Learning
Gyuhak Kim, Changnan Xiao, Tatsuya Konishi et al. · Machine learning
Continual learning (CL) learns a sequence of tasks incrementally. There are two popular CL settings, class incremental learning (CIL) and task incremental learning (TIL).
Cited inThe Capacity AuditInvestors are paying for agents that learn on the job. The hard part is knowing what they forgot.Penalising wrong answers only works if someone else is gradingStudy: Forgetting as selection at scaleStudy: Genesys program prior artInternal notesStudy: Orchestrated specialists and bridge synthesisStudy: Stopping confident wrong expert votes - 2022-10arXiv 2210.02406Decomposed Prompting: A Modular Approach for Solving Complex Tasks
Tushar Khot, Harsh Trivedi, Matthew Finlayson et al. · Language models & NLP
Few-shot prompting is a surprisingly powerful way to use Large Language Models (LLMs) to solve various tasks.
- 2022-08arXiv 2208.03306Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Models
Margaret Li, Suchin Gururangan, Tim Dettmers et al. · Language models & NLP
We present Branch-Train-Merge (BTM), a communication-efficient algorithm for embarrassingly parallel training of large language models (LLMs).
- 2022-07arXiv 2207.05221Language Models (Mostly) Know What They Know
Saurav Kadavath, Tom Conerly, Amanda Askell et al. · Language models & NLP
We study whether language models can evaluate the validity of their own claims and predict which questions they will be able to answer correctly. We first show that larger models are well-calibrated on diverse multiple choice and true/false questions when they are provided in the right format.
- 2022-06arXiv 2206.06520Memory-Based Model Editing at Scale
Eric Mitchell, Charles Lin, Antoine Bosselut et al. · AI & agents
Even the largest neural networks make errors, and once-correct predictions can become invalid as the world changes. Model editors make local updates to the behavior of base (pre-trained) models to inject updated knowledge or correct undesirable behaviors.
- 2022-05arXiv 2205.10625Least-to-Most Prompting Enables Complex Reasoning in Large Language Models
Denny Zhou, Nathanael Schärli, Le Hou et al. · AI & agents
Chain-of-thought prompting has demonstrated remarkable performance on various natural language reasoning tasks. However, it tends to perform poorly on tasks which requires solving problems harder than the exemplars shown in the prompts.
- 2022-04arXiv 2204.04799DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning
Zifeng Wang, Zizhao Zhang, Sayna Ebrahimi et al. · Machine learning
Continual learning aims to enable a single model to learn a sequence of tasks without catastrophic forgetting.
- 2022-03arXiv 2203.11171Self-Consistency Improves Chain of Thought Reasoning in Language Models
Xuezhi Wang, Jason Wei, Dale Schuurmans et al. · Language models & NLP
Chain-of-thought prompting combined with pre-trained large language models has achieved encouraging results on complex reasoning tasks. In this paper, we propose a new decoding strategy, self-consistency, to replace the naive greedy decoding used in chain-of-thought prompting.
- 2021-12arXiv 2112.08654Learning to Prompt for Continual Learning
Zifeng Wang, Zizhao Zhang, Chen-Yu Lee et al. · Machine learning
The mainstream paradigm behind continual learning has been to adapt the model parameters to non-stationary data distributions, where catastrophic forgetting is the central challenge.
Built from every arXiv paper cited in our papers, posts, topic hubs, reports and research notes, with titles, authors, dates and abstracts from the arXiv API (updated 2026-11-12). Summaries are the opening sentences of each paper's own abstract, not our assessment; a citation means we used or checked the work, not that we endorse it. Fields follow each paper's primary arXiv category. Papers cited only outside arXiv (journals, proceedings) are not listed yet. Entries are generated from our citations, so a mistyped ID in a note would list the wrong paper; we remove those as we find them. Report a wrong entry.
