NeuroGenesysThe research we build on.
1,514 papers by other researchers that our work cites, from continual learning and verification to language models, agents and hardware. Each entry links to the paper on arXiv and to where we use it. 301 are cited on published pages of this site; the rest in the research notes behind our studies. Our own papers are in the archive.
- 2023-09arXiv 2309.14316Physics of Language Models: Part 3.1, Knowledge Storage and Extraction
Zeyuan Allen-Zhu, Yuanzhi Li · Language models & NLP
")., cheating), or by genuinely learning to extract knowledge from sources like Wikipedia? In this paper, we investigate this issue using a controlled biography dataset.
- 2023-09arXiv 2309.12288The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"
Lukas Berglund, Meg Tong, Max Kaufmann et al. · Language models & NLP
We expose a surprising failure of generalization in auto-regressive large language models (LLMs). If a model is trained on a sentence of the form "A is B", it will not automatically generalize to the reverse direction "B is A".
- 2023-07arXiv 2307.12976Evaluating the Ripple Effects of Knowledge Editing in Language Models
Roi Cohen, Eden Biran, Ori Yoran et al. · Language models & NLP
Modern language models capture a large body of factual knowledge. However, some facts can be incorrectly induced or become obsolete over time, resulting in factually incorrect generations.
Cited inA gradient kernel flags contradictions but does not predict forgetting, and steering rehearsal by it backfiresCoherence is selected, anchored, then pruned: a literature synthesis on logical coherence in LLM world modelsEWC protects what its Fisher sees: a literature synthesis on elastic weight consolidation, scale and coherenceIterated self-closure settles fast, but only evidence that overrides self-belief makes the settled state trueTo stop a model forgetting, rehearse what it knew. New context won't do it.Study: Consequence consistent updatesStudy: Continual learning order and companion dataStudy: Decision signatures for conflict detectionStudy: Elastic weight consolidation deep diveStudy: Graded succession by earned authorityStudy: Hardware guardian core for continual learningStudy: Logical coherence of LLM world modelsStudy: Orchestrated specialists and bridge synthesis - 2023-05arXiv 2305.14795MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions
Zexuan Zhong, Zhengxuan Wu, Christopher D. Manning et al. · Language models & NLP
The information stored in large language models (LLMs) falls out of date quickly, and retraining from scratch is often not an option. This has recently given rise to a range of techniques for injecting new facts through updating model weights.
Cited inTrust the premises, not the confidence: which belief to drop when an LLM contradicts itself, and why the solver is not the bottleneckStudy: Consequence consistent updatesStudy: Continual learning order and companion dataStudy: Graded succession by earned authorityStudy: LLMs modifying open model weightsStudy: Logical coherence of LLM world modelsStudy: Orchestrated specialists and bridge synthesisStudy: Weight editing to prevent forgetting - 2022-06arXiv 2207.00099Measuring Forgetting of Memorized Training Examples
Matthew Jagielski, Om Thakkar, Florian Tramèr et al. · Machine learning
Machine learning models exhibit two seemingly contradictory phenomena: training data memorization, and various forms of forgetting. In memorization, models overfit specific training examples and become susceptible to privacy attacks.
- 2022-05arXiv 2205.13323The Effect of Task Ordering in Continual Learning
Samuel J. Bell, Neil D. Lawrence · Machine learning
We investigate the effect of task ordering on continual learning performance. We conduct an extensive series of empirical experiments on synthetic and naturalistic datasets and show that reordering tasks significantly affects the amount of catastrophic forgetting.
- 2022-05arXiv 2205.09588How catastrophic can catastrophic forgetting be in linear regression?
Itay Evron, Edward Moroshko, Rachel Ward et al. · 35th Annual Conference on Learning Theory (2022) · Machine learning
To better understand catastrophic forgetting, we study fitting an overparameterized linear model to a sequence of tasks with different input distributions.
- 2022-04arXiv 2204.14211TemporalWiki: A Lifelong Benchmark for Training and Evaluating Ever-Evolving Language Models
Joel Jang, Seonghyeon Ye, Changho Lee et al. · Language models & NLP
Language Models (LMs) become outdated as the world changes; they often fail to perform tasks requiring recent factual information which was absent or different during training, a phenomenon called temporal misalignment.
- 2022-02arXiv 2202.05262Locating and Editing Factual Associations in GPT
Kevin Meng, David Bau, Alex Andonian et al. · Language models & NLP
We analyze the storage and recall of factual associations in autoregressive transformer language models, finding evidence that these associations correspond to localized, directly-editable computations.
- 2020-11arXiv 2011.09161Positive-Congruent Training: Towards Regression-Free Model Updates
Sijie Yan, Yuanjun Xiong, Kaustav Kundu et al. · Vision
Reducing inconsistencies in the behavior of different versions of an AI system can be as important in practice as reducing its overall error.
Built from every arXiv paper cited in our papers, posts, topic hubs, reports and research notes, with titles, authors, dates and abstracts from the arXiv API (updated 2026-11-12). Summaries are the opening sentences of each paper's own abstract, not our assessment; a citation means we used or checked the work, not that we endorse it. Fields follow each paper's primary arXiv category. Papers cited only outside arXiv (journals, proceedings) are not listed yet. Entries are generated from our citations, so a mistyped ID in a note would list the wrong paper; we remove those as we find them. Report a wrong entry.
