NeuroGenesysThe research we build on.
1,514 papers by other researchers that our work cites, from continual learning and verification to language models, agents and hardware. Each entry links to the paper on arXiv and to where we use it. 301 are cited on published pages of this site; the rest in the research notes behind our studies. Our own papers are in the archive.
- 2025-05arXiv 2505.23735ATLAS: Learning to Optimally Memorize the Context at Test Time
Ali Behrouz, Zeman Li, Praneeth Kacham et al. · Language models & NLP
Transformers have been established as the most popular backbones in sequence modeling, mainly due to their effectiveness in in-context retrieval tasks and the ability to learn at scale.
- 2025-05arXiv 2505.17762Resolving Conflicting Evidence in Automated Fact-Checking: A Study on Retrieval-Augmented LLMs
Ziyu Ge, Yuhao Wu, Daniel Wai Kit Chin et al. · Language models & NLP
Large Language Models (LLMs) augmented with retrieval mechanisms have demonstrated significant potential in fact-checking tasks by integrating external knowledge.
- 2025-05arXiv 2505.00843OET: Optimization-based prompt injection Evaluation Toolkit
Jinsheng Pan, Xiaogeng Liu, Chaowei Xiao · Security
Large Language Models (LLMs) have demonstrated remarkable capabilities in natural language understanding and generation, enabling their widespread adoption across various domains.
- 2025-04arXiv 2504.19622From Evidence to Belief: A Bayesian Epistemology Approach to Language Models
Minsu Kim, Sangryul Kim, James Thorne · AI & agents
This paper investigates the knowledge of language models from the perspective of Bayesian epistemology. We explore how language models adjust their confidence and responses when presented with evidence with varying levels of informativeness and reliability.
- 2025-03arXiv 2503.20523GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Driving
Lloyd Russell, Anthony Hu, Lorenzo Bertoni et al. · Vision
Generative models offer a scalable and flexible paradigm for simulating complex environments, yet current approaches fall short in addressing the domain-specific requirements of autonomous driving - such as multi-agent interactions, fine-grained control, and multi-camera consistency.
- 2025-03arXiv 2503.03704Memory Injection Attacks on LLM Agents via Query-Only Interaction
Shen Dong, Shaochen Xu, Pengfei He et al. · Machine learning
Agents powered by large language models (LLMs) have demonstrated strong capabilities in a wide range of complex, real-world applications. However, LLM agents with a compromised memory bank may easily produce harmful outputs when the past records retrieved for demonstration are malicious.
- 2025-02arXiv 2502.02737SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model
Loubna Ben Allal, Anton Lozhkov, Elie Bakouch et al. · Language models & NLP
While large language models have facilitated breakthroughs in many applications of artificial intelligence, their inherent largeness makes them computationally expensive and challenging to deploy in resource-constrained settings.
- 2025-02arXiv 2502.00592M+: Extending MemoryLLM with Scalable Long-Term Memory
Yu Wang, Dmitry Krotov, Yuanzhe Hu et al. · Language models & NLP
Equipping large language models (LLMs) with latent-space memory has attracted increasing attention as they can extend the context window of existing language models.
- 2025-01arXiv 2501.09038Do generative video models understand physical principles?
Saman Motamed, Laura Culp, Kevin Swersky et al. · Vision
AI video generation is undergoing a revolution, with quality and realism advancing rapidly.
- 2025-01arXiv 2501.03575Cosmos World Foundation Model Platform for Physical AI
NVIDIA, :, Niket Agarwal et al. · Vision
Physical AI needs to be trained digitally first. It needs a digital twin of itself, the policy model, and a digital twin of the world, the world model. In this paper, we present the Cosmos World Foundation Model Platform to help developers build customized world models for their Physical AI setups.
- 2025-01arXiv 2501.01956Metadata Conditioning Accelerates Language Model Pre-training
Tianyu Gao, Alexander Wettig, Luxi He et al. · Language models & NLP
The vast diversity of styles, domains, and quality levels present in language model pre-training corpora is essential in developing general model capabilities, but efficiently learning and deploying the correct behaviors exemplified in each of these heterogeneous data sources is challenging.
- 2025-01arXiv 2501.00879TrustRAG: Enhancing Robustness and Trustworthiness in Retrieval-Augmented Generation
Huichi Zhou, Kin-Hei Lee, Zhonghao Zhan et al. · Language models & NLP
Retrieval-Augmented Generation (RAG) enhances large language models (LLMs) by integrating external knowledge sources, enabling more accurate and contextually relevant responses tailored to user queries.
- 2024-12arXiv 2412.09764Memory Layers at Scale
Vincent-Pierre Berges, Barlas Oğuz, Daniel Haziza et al. · Language models & NLP
Memory layers use a trainable key-value lookup mechanism to add extra parameters to a model without increasing FLOPs. Conceptually, sparsely activated memory layers complement compute-heavy dense feed-forward layers, providing dedicated capacity to store and retrieve information cheaply.
- 2024-11arXiv 2411.02385How Far is Video Generation from World Model: A Physical Law Perspective
Bingyi Kang, Yang Yue, Rui Lu et al. · Vision
OpenAI's Sora highlights the potential of video generation for developing world models that adhere to fundamental physical laws. However, the ability of video generation models to discover such laws purely from visual data without human priors can be questioned.
- 2024-10arXiv 2410.05451SecAlign: Defending Against Prompt Injection with Preference Optimization
Sizhe Chen, Arman Zharmagambetov, Saeed Mahloujifar et al. · Security
Large language models (LLMs) are becoming increasingly prevalent in modern software systems, interfacing between the user and the Internet to assist with tasks that require advanced language understanding.
- 2024-07arXiv 2407.12784AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases
Zhaorun Chen, Zhen Xiang, Chaowei Xiao et al. · Machine learning
LLM agents have demonstrated remarkable performance across various applications, primarily due to their advanced capabilities in reasoning, utilizing external knowledge and tools, calling APIs, and executing actions to interact with environments.
- 2024-06arXiv 2406.17557The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale
Guilherme Penedo, Hynek Kydlíček, Loubna Ben allal et al. · Language models & NLP
The performance of a large language model (LLM) depends heavily on the quality and size of its pretraining dataset. However, the pretraining datasets for state-of-the-art open LLMs like Llama 3 and Mixtral are not publicly available and very little is known about how they were created.
- 2024-06arXiv 2406.09455Pandora: Towards General World Model with Natural Language Actions and Video States
Jiannan Xiang, Guangyi Liu, Yi Gu et al. · Vision
World models simulate future states of the world in response to different actions. They facilitate interactive content creation and provides a foundation for grounded, long-horizon reasoning.
- 2024-06arXiv 2406.03689Evaluating the World Model Implicit in a Generative Model
Keyon Vafa, Justin Y. Chen, Ashesh Rambachan et al. · Language models & NLP
Recent work suggests that large language models may implicitly learn world models. How should we assess this possibility? We formalize this question for the case where the underlying reality is governed by a deterministic finite automaton.
- 2024-06arXiv 2406.06608The Prompt Report: A Systematic Survey of Prompt Engineering Techniques
Sander Schulhoff, Michael Ilie, Nishant Balepur et al. · Language models & NLP
Generative Artificial Intelligence (GenAI) systems are increasingly being deployed across diverse industries and research domains. Developers and end-users interact with these systems through the use of prompting and prompt engineering.
- 2024-04arXiv 2404.13208The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions
Eric Wallace, Kai Xiao, Reimar Leike et al. · Security
Today's LLMs are susceptible to prompt injections, jailbreaks, and other attacks that allow adversaries to overwrite a model's original instructions with their own malicious prompts.
- 2024-04arXiv 2404.10198ClashEval: Quantifying the tug-of-war between an LLM's internal prior and external evidence
Kevin Wu, Eric Wu, James Zou · Language models & NLP
Retrieval augmented generation (RAG) is frequently used to mitigate hallucinations and provide up-to-date knowledge for large language models (LLMs).
Cited inStable wrong beliefs evade every self-check: a literature synthesis on model self-knowledge and confident errorsTo keep what a model knows, control what it is allowed to moveWhat a model lets into long-term memory matters more than how it learnsStudy: Fortifying knowledge in continual learningStudy: Hardware guardian core for continual learningStudy: Memory world models and source trustStudy: Model self knowledge and confident errors - 2024-04arXiv 2404.06809Not All Contexts Are Equal: Teaching LLMs Credibility-aware Generation
Ruotong Pan, Boxi Cao, Hongyu Lin et al. · Language models & NLP
The rapid development of large language models has led to the widespread adoption of Retrieval-Augmented Generation (RAG), which integrates external knowledge to alleviate knowledge bottlenecks and mitigate hallucinations.
- 2024-04arXiv 2404.01019Source-Aware Training Enables Knowledge Attribution in Language Models
Muhammad Khalifa, David Wadden, Emma Strubell et al. · Language models & NLP
Large language models (LLMs) learn a vast amount of knowledge during pretraining, but they are often oblivious to the source(s) of such knowledge. We investigate the problem of intrinsic source citation, where LLMs are required to cite the pretraining source supporting a generated response.
- 2024-03arXiv 2403.11901Larimar: Large Language Models with Episodic Memory Control
Payel Das, Subhajit Chaudhury, Elliot Nelson et al. · Machine learning
Efficient and accurate updating of knowledge stored in Large Language Models (LLMs) is one of the most pressing research challenges today. This paper presents Larimar - a novel, brain-inspired architecture for enhancing LLMs with a distributed episodic memory.
- 2024-02arXiv 2402.07867PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models
Wei Zou, Runpeng Geng, Binghui Wang et al. · Security
Large language models (LLMs) have achieved remarkable success due to their exceptional generative capabilities. Despite their success, they also have inherent limitations such as a lack of up-to-date knowledge and hallucination.
- 2023-06arXiv 2306.03288Deep Learning From Crowdsourced Labels: Coupled Cross-entropy Minimization, Identifiability, and Regularization
Shahana Ibrahim, Tri Nguyen, Xiao Fu · Machine learning
Using noisy crowdsourced labels from multiple annotators, a deep learning-based end-to-end (E2E) system aims to learn the label correction mechanism and the neural classifier simultaneously.
- 2023-05arXiv 2305.13300Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts
Jian Xie, Kai Zhang, Jiangjie Chen et al. · Language models & NLP
By providing external information to large language models (LLMs), tool augmentation (including retrieval augmentation) has emerged as a promising solution for addressing the limitations of LLMs' static parametric memory.
Cited inCoherence is selected, anchored, then pruned: a literature synthesis on logical coherence in LLM world modelsSuccessors should earn authority, not win contests: graded succession in continual learningStudy: Decision signatures for conflict detectionStudy: Graded succession by earned authorityStudy: Logical coherence of LLM world modelsStudy: Memory world models and source trustStudy: Model self knowledge and confident errors - 2023-05arXiv 2305.10429DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining
Sang Michael Xie, Hieu Pham, Xuanyi Dong et al. · Language models & NLP
, Wikipedia, books, web text) greatly affect language model (LM) performance.
- 2023-03arXiv 2303.06572Continual Visual Reinforcement Learning with A Life-Long World Model
Minting Pan, Wendong Zhang, Geng Chen et al. · Machine learning
Learning physical dynamics in a series of non-stationary environments is a challenging but essential task for model-based reinforcement learning (MBRL) with visual inputs.
- 2022-10arXiv 2210.13382Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task
Kenneth Li, Aspen K. Hopkins, David Bau et al. · Machine learning
Language models show a surprising range of capabilities, but the source of their apparent competence is unclear. Do these networks just memorize a collection of surface statistics, or do they rely on internal representations of the process that generates the sequences they see?
- 2022-04arXiv 2204.05624Continual Predictive Learning from Videos
Geng Chen, Wendong Zhang, Han Lu et al. · Vision
Predictive learning ideally builds the world model of physical processes in one or more given environments. Typical setups assume that we can collect data from all environments at all times.
- 2021-10arXiv 2110.00175DualNet: Continual Learning, Fast and Slow
Quang Pham, Chenghao Liu, Steven Hoi · Machine learning
According to Complementary Learning Systems (CLS) theory~\citep{mcclelland1995there} in neuroscience, humans do effective \emph{continual learning} through two complementary systems: a fast learning system centered on the hippocampus for rapid learning of the specifics and individual experiences, and a slow…
- 2019-08arXiv 1908.02240Biologically inspired sleep algorithm for artificial neural networks
Giri P Krishnan, Timothy Tadros, Ramyaa Ramyaa et al. · Neuroscience & neural computation
Sleep plays an important role in incremental learning and consolidation of memories in biological systems.
- 2017-12arXiv 1712.04577Learning From Noisy Singly-labeled Data
Ashish Khetan, Zachary C. Lipton, Anima Anandkumar · Machine learning
Supervised learning depends on annotated examples, which are taken to be the \emph{ground truth}. But these labels often come from noisy crowdsourcing platforms, like Amazon Mechanical Turk.
- 2017-05arXiv 1705.08690Continual Learning with Deep Generative Replay
Hanul Shin, Jung Kwon Lee, Jaehong Kim et al. · AI & agents
Attempts to train a comprehensive artificial intelligence capable of solving multiple tasks have been impeded by a chronic problem called catastrophic forgetting.
Built from every arXiv paper cited in our papers, posts, topic hubs, reports and research notes, with titles, authors, dates and abstracts from the arXiv API (updated 2026-11-12). Summaries are the opening sentences of each paper's own abstract, not our assessment; a citation means we used or checked the work, not that we endorse it. Fields follow each paper's primary arXiv category. Papers cited only outside arXiv (journals, proceedings) are not listed yet. Entries are generated from our citations, so a mistyped ID in a note would list the wrong paper; we remove those as we find them. Report a wrong entry.
