NeuroGenesysThe research we build on.
1,514 papers by other researchers that our work cites, from continual learning and verification to language models, agents and hardware. Each entry links to the paper on arXiv and to where we use it. 301 are cited on published pages of this site; the rest in the research notes behind our studies. Our own papers are in the archive.
- 2024-04arXiv 2404.03592ReFT: Representation Finetuning for Language Models
Zhengxuan Wu, Aryaman Arora, Zheng Wang et al. · Language models & NLP
Parameter-efficient finetuning (PEFT) methods seek to adapt large neural models via updates to a small number of weights.
- 2024-03arXiv 2403.14236A Unified Framework for Model Editing
Akshat Gupta, Dev Sajnani, Gopala Anumanchipalli · Machine learning
ROME and MEMIT are largely believed to be two different model editing algorithms, with the major difference between them being the ability to perform batched edits.
- 2024-03arXiv 2403.14472Detoxifying Large Language Models via Knowledge Editing
Mengru Wang, Ningyu Zhang, Ziwen Xu et al. · Language models & NLP
This paper investigates using knowledge editing techniques to detoxify Large Language Models (LLMs). We construct a benchmark, SafeEdit, which covers nine unsafe categories with various powerful attack prompts and equips comprehensive metrics for systematic evaluation.
- 2024-03arXiv 2403.13257Arcee's MergeKit: A Toolkit for Merging Large Language Models
Charles Goddard, Shamane Siriwardhana, Malikeh Ehghaghi et al. · Language models & NLP
The rapid expansion of the open-source language model landscape presents an opportunity to merge the competencies of these model checkpoints by combining their parameters.
- 2024-03arXiv 2403.13355BadEdit: Backdooring large language models by model editing
Yanzhou Li, Tianlin Li, Kangjie Chen et al. · Security
Mainstream backdoor attack methods typically demand substantial tuning data for poisoning, limiting their practicality and potentially degrading the overall performance when applied to Large Language Models (LLMs).
- 2024-03arXiv 2403.13187Evolutionary Optimization of Model Merging Recipes
Takuya Akiba, Makoto Shing, Yujin Tang et al. · Nat Mach Intell (2025) · Neuroscience & neural computation
Large language models (LLMs) have become increasingly capable, but their development often requires substantial computational resources.
- 2024-03arXiv 2403.07175Rebuilding ROME : Resolving Model Collapse during Sequential Model Editing
Akshat Gupta, Sidharth Baskaran, Gopala Anumanchipalli · Language models & NLP
Recent work using Rank-One Model Editing (ROME), a popular model editing method, has shown that there are certain facts that the algorithm is unable to edit without breaking the model.
- 2024-03arXiv 2403.03218The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning
Nathaniel Li, Alexander Pan, Anjali Gopal et al. · Machine learning
The White House Executive Order on Artificial Intelligence highlights the risks of large language models (LLMs) empowering malicious actors in developing biological, cyber, and chemical weapons.
- 2024-02arXiv 2402.16817Investigating the Effectiveness of HyperTuning via Gisting
Jason Phang · Language models & NLP
, 2023) is a simple method for training models to compress information into fewer token representations using a modified attention mask, and can serve as an economical approach to training Transformer-based hypernetworks.
- 2023-12arXiv 2312.15166SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scaling
Dahyun Kim, Chanjun Park, Sanghoon Kim et al. · Language models & NLP
7 billion parameters, demonstrating superior performance in various natural language processing (NLP) tasks. Inspired by recent efforts to efficiently up-scale LLMs, we present a method for scaling LLMs called depth up-scaling (DUS), which encompasses depthwise scaling and continued pretraining.
- 2023-11arXiv 2311.03099Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch
Le Yu, Bowen Yu, Haiyang Yu et al. · Language models & NLP
In this paper, we unveil that Language Models (LMs) can acquire new capabilities by assimilating parameters from homologous models without retraining or GPUs.
- 2023-10arXiv 2310.20624LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70B
Simon Lermen, Charlie Rogers-Smith, Jeffrey Ladish · Machine learning
AI developers often apply safety alignment procedures to prevent the misuse of their AI systems.
- 2023-10arXiv 2310.03693Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!
Xiangyu Qi, Yi Zeng, Tinghao Xie et al. · Language models & NLP
Optimizing large language models (LLMs) for downstream use cases often involves the customization of pre-trained LLMs through further fine-tuning.5 Turbo on custom datasets also encourage this practice.
- 2023-06arXiv 2306.01708TIES-Merging: Resolving Interference When Merging Models
Prateek Yadav, Derek Tam, Leshem Choshen et al. · Machine learning
, further fine-tuning a pre-trained model on a downstream task - can confer significant advantages, including improved downstream performance, faster convergence, and better sample efficiency.
Cited inStudy: Decision signatures for conflict detectionStudy: Fortifying knowledge in continual learningStudy: Generational competitive continual learningStudy: Hierarchical and T shaped MoEStudy: LLMs modifying open model weightsStudy: Merging and splitting MoE expertsStudy: Weight editing to prevent forgetting - 2023-05arXiv 2305.14795MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions
Zexuan Zhong, Zhengxuan Wu, Christopher D. Manning et al. · Language models & NLP
The information stored in large language models (LLMs) falls out of date quickly, and retraining from scratch is often not an option. This has recently given rise to a range of techniques for injecting new facts through updating model weights.
Cited inTrust the premises, not the confidence: which belief to drop when an LLM contradicts itself, and why the solver is not the bottleneckStudy: Consequence consistent updatesStudy: Continual learning order and companion dataStudy: Graded succession by earned authorityStudy: LLMs modifying open model weightsStudy: Logical coherence of LLM world modelsStudy: Orchestrated specialists and bridge synthesisStudy: Weight editing to prevent forgetting - 2022-12arXiv 2212.04089Editing Models with Task Arithmetic
Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman et al. · Machine learning
, improving their performance on a downstream task or mitigating biases learned during pre-training -- is a common practice when developing machine learning systems.
- 2022-11arXiv 2211.12485HyperTuning: Toward Adapting Large Language Models without Back-propagation
Jason Phang, Yi Mao, Pengcheng He et al. · Language models & NLP
Fine-tuning large language models for different tasks can be costly and inefficient, and even methods that reduce the number of tuned parameters still require full gradient-based optimization.
- 2022-10arXiv 2210.07229Mass-Editing Memory in a Transformer
Kevin Meng, Arnab Sen Sharma, Alex Andonian et al. · Language models & NLP
Recent work has shown exciting promise in updating large language models with new memories, so as to replace obsolete information or add specialized knowledge. However, this line of work is predominantly limited to updating single associations.
Cited inEWC protects what its Fisher sees: a literature synthesis on elastic weight consolidation, scale and coherenceStudy: Consequence consistent updatesStudy: Decision signatures for conflict detectionStudy: Elastic weight consolidation deep diveStudy: Fortifying knowledge in continual learningStudy: LLMs modifying open model weights - 2022-03arXiv 2203.05482Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time
Mitchell Wortsman, Gabriel Ilharco, Samir Yitzhak Gadre et al. · Machine learning
The conventional recipe for maximizing model accuracy is to (1) train multiple models with various hyperparameters and (2) pick the individual model which performs best on a held-out validation set, discarding the remainder.
Built from every arXiv paper cited in our papers, posts, topic hubs, reports and research notes, with titles, authors, dates and abstracts from the arXiv API (updated 2026-11-12). Summaries are the opening sentences of each paper's own abstract, not our assessment; a citation means we used or checked the work, not that we endorse it. Fields follow each paper's primary arXiv category. Papers cited only outside arXiv (journals, proceedings) are not listed yet. Entries are generated from our citations, so a mistyped ID in a note would list the wrong paper; we remove those as we find them. Report a wrong entry.
