NeuroGenesysThe research we build on.
1,514 papers by other researchers that our work cites, from continual learning and verification to language models, agents and hardware. Each entry links to the paper on arXiv and to where we use it. 301 are cited on published pages of this site; the rest in the research notes behind our studies. Our own papers are in the archive.
- 2025-08arXiv 2508.15805ALAS: Autonomous Learning Agent for Self-Updating Language Models
Dhruv Atreja · Language models & NLP
Large language models (LLMs) often have a fixed knowledge cutoff, limiting their accuracy on emerging information. We present ALAS (Autonomous Learning Agent System), a modular pipeline that continuously updates an LLM's knowledge with minimal human intervention.
- 2025-08arXiv 2508.06601Deep Ignorance: Filtering Pretraining Data Builds Tamper-Resistant Safeguards into Open-Weight LLMs
Kyle O'Brien, Stephen Casper, Quentin Anthony et al. · Machine learning
Open-weight AI systems offer unique benefits, including enhanced transparency, open research, and decentralized access. However, they are vulnerable to tampering attacks which can efficiently elicit harmful behaviors by modifying weights or activations.
- 2025-08arXiv 2508.03153Estimating Worst-Case Frontier Risks of Open-Weight LLMs
Eric Wallace, Olivia Watkins, Miles Wang et al. · Machine learning
In this paper, we study the worst-case frontier risks of releasing gpt-oss. We introduce malicious fine-tuning (MFT), where we attempt to elicit maximum capabilities by fine-tuning gpt-oss to be as capable as possible in two domains: biology and cybersecurity.
- 2025-07arXiv 2507.21509Persona Vectors: Monitoring and Controlling Character Traits in Language Models
Runjin Chen, Andy Arditi, Henry Sleight et al. · Language models & NLP
Large language models interact with users through a simulated 'Assistant' persona. While the Assistant is typically trained to be helpful, harmless, and honest, it sometimes deviates from these ideals.
- 2025-07arXiv 2508.00914Knowledge Editing for Multi-Hop Question Answering Using Semantic Analysis
Dominic Simon, Rickard Ewetz · AI & agents
Large Language Models (LLMs) require lightweight avenues of updating stored information that has fallen out of date.
- 2025-07arXiv 2507.03662Re-Emergent Misalignment: How Narrow Fine-Tuning Erodes Safety Alignment in LLMs
Jeremiah Giordani · Machine learning
Recent work has shown that fine-tuning large language models (LLMs) on code with security vulnerabilities can result in misaligned and unsafe behaviors across broad domains.
- 2025-06arXiv 2506.16406Drag-and-Drop LLMs: Zero-Shot Prompt-to-Weights
Zhiyuan Liang, Dongwen Tang, Yuhao Zhou et al. · Machine learning
Modern Parameter-Efficient Fine-Tuning (PEFT) methods such as low-rank adaptation (LoRA) reduce the cost of customizing large language models (LLMs), yet still require a separate optimization run for every downstream dataset.
- 2025-06arXiv 2506.11613Model Organisms for Emergent Misalignment
Edward Turner, Anna Soligo, Mia Taylor et al. · Machine learning
Recent work discovered Emergent Misalignment (EM): fine-tuning large language models on narrowly harmful datasets can lead them to become broadly misaligned.
- 2025-06arXiv 2506.10943Self-Adapting Language Models
Adam Zweiger, Jyothish Pari, Han Guo et al. · Machine learning
Large language models (LLMs) are powerful but static; they lack mechanisms to adapt their weights in response to new tasks, knowledge, or examples. We introduce Self-Adapting LLMs (SEAL), a framework that enables LLMs to self-adapt by generating their own finetuning data and update directives.
Cited inCoherence is selected, anchored, then pruned: a literature synthesis on logical coherence in LLM world modelsA model rewrote another model's weights. Read what it took away.Study: Consequence consistent updatesStudy: Continual learning order and companion dataStudy: Fortifying knowledge in continual learningStudy: LLMs modifying open model weightsStudy: Logical coherence of LLM world modelsStudy: Seven continual learning methods testedStudy: Weight editing to prevent forgetting - 2025-06arXiv 2506.06105Text-to-LoRA: Instant Transformer Adaption
Rujikorn Charakorn, Edoardo Cetin, Yujin Tang et al. · Machine learning
While Foundation Models provide a general tool for rapid content creation, they regularly require task-specific adaptation. Traditionally, this exercise involves careful curation of datasets and repeated fine-tuning of the underlying model.
- 2025-05arXiv 2505.24428Model Unlearning via Sparse Autoencoder Subspace Guided Projections
Xu Wang, Zihao Li, Benyou Wang et al. · Language models & NLP
Large language models (LLMs) store vast amounts of information, making them powerful yet raising privacy and safety concerns when selective knowledge removal is required.
- 2025-05arXiv 2505.22954Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agents
Jenny Zhang, Shengran Hu, Cong Lu et al. · AI & agents
Today's AI systems have human-designed, fixed architectures and cannot autonomously and continuously improve themselves. The advance of AI could itself be automated.
- 2025-05arXiv 2505.20809Improved Representation Steering for Language Models
Zhengxuan Wu, Qinan Yu, Aryaman Arora et al. · Language models & NLP
Steering methods for language models (LMs) seek to provide fine-grained and interpretable control over model generations by variously changing model inputs, weights, or representations to adjust behavior.
- 2025-05arXiv 2505.14679UltraEdit: Training-, Subject-, and Memory-Free Lifelong Editing in Language Models
Xiaojie Gu, Ziying Huang, Jia-Chen Gu et al. · Language models & NLP
Lifelong learning enables large language models (LLMs) to adapt to evolving information by continually updating their internal knowledge. An ideal system should support efficient, wide-ranging updates while preserving existing capabilities and ensuring reliable deployment.
- 2025-05arXiv 2505.10833MergeBench: A Benchmark for Merging Domain-Specialized LLMs
Yifei He, Siqi Zeng, Yuzheng Hu et al. · Machine learning
Model merging provides a scalable alternative to multi-task training by combining specialized finetuned models through parameter arithmetic, enabling efficient deployment without the need for joint training or access to all task data.
- 2025-04arXiv 2504.13146Antidistillation Sampling
Yash Savani, Asher Trockman, Zhili Feng et al. · AI & agents
Frontier models that generate extended reasoning traces inadvertently produce rich token sequences that can facilitate model distillation. Recognizing this vulnerability, model owners may seek sampling strategies that limit the effectiveness of distillation without compromising model performance.
- 2025-03arXiv 2503.24354ORAL: Prompting Your Large-Scale LoRAs via Conditional Recurrent Diffusion
Rana Muhammad Shahroz Khan, Dongwen Tang, Pingzhi Li et al. · Machine learning
Parameter generation has emerged as a novel paradigm for neural network development, offering an alternative to traditional neural network training by synthesizing high-quality model weights directly.
- 2025-03arXiv 2503.14499Measuring AI Ability to Complete Long Software Tasks
Thomas Kwa, Ben West, Joel Becker et al. · NeurIPS 2025 · AI & agents
Despite rapid progress on AI benchmarks, the real-world meaning of benchmark performance remains unclear. To quantify the capabilities of AI systems in terms of human capabilities, we propose a new metric: 50%-task-completion time horizon.
- 2025-03arXiv 2503.11127Don't Forget It! Conditional Sparse Autoencoder Clamping Works for Unlearning
Matthew Khoriaty, Andrii Shportko, Gustavo Mercier et al. · Machine learning
Recent developments in Large Language Model (LLM) capabilities have brought great potential but also posed new risks. For example, LLMs with knowledge of bioweapons, advanced chemistry, or cyberattacks could cause violence if placed in the wrong hands or during malfunctions.
- 2025-03arXiv 2503.05683WikiBigEdit: Understanding the Limits of Lifelong Knowledge Editing in LLMs
Lukas Thede, Karsten Roth, Matthias Bethge et al. · Language models & NLP
Keeping large language models factually up-to-date is crucial for deployment, yet costly retraining remains a challenge. Knowledge editing offers a promising alternative, but methods are only tested on small-scale or synthetic edit benchmarks.
Cited inEWC protects what its Fisher sees: a literature synthesis on elastic weight consolidation, scale and coherenceStudy: Consequence consistent updatesStudy: Continual learning order and companion dataStudy: Elastic weight consolidation deep diveStudy: Fortifying knowledge in continual learningStudy: LLMs modifying open model weightsStudy: Logical coherence of LLM world modelsStudy: Orchestrated specialists and bridge synthesisStudy: Weight editing to prevent forgetting - 2025-03arXiv 2503.02701MindBridge: Scalable and Cross-Model Knowledge Editing via Memory-Augmented Modality
Shuaike Li, Kai Zhang, Qi Liu et al. · AI & agents
Knowledge editing is a technique for efficiently and accurately updating the knowledge of large language models (LLMs) to alleviate obsolescence and correct errors.
- 2025-02arXiv 2502.17420The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independence
Tom Wollschläger, Jannes Elstner, Simon Geisler et al. · Machine learning
The safety alignment of large language models (LLMs) can be circumvented through adversarially crafted inputs, yet the mechanisms by which these attacks bypass safety barriers remain poorly understood.
- 2025-02arXiv 2502.11177The Mirage of Model Editing: Revisiting Evaluation in the Wild
Wanli Yang, Fei Sun, Jiajun Tan et al. · Language models & NLP
Despite near-perfect results reported in the literature, the effectiveness of model editing in real-world applications remains unclear.
- 2025-02arXiv 2502.09056Adapting Language-Specific LLMs to a Reasoning Model in One Day via Model Merging -- An Open Recipe
Kunat Pipatanakul, Pittawat Taveekitworachai, Potsawee Manakul et al. · Language models & NLP
This paper investigates data selection and model merging methodologies aimed at incorporating advanced reasoning capabilities such as those of DeepSeek R1 into language-specific large language models (LLMs), with a particular focus on the Thai LLM.
- 2025-02arXiv 2502.05209Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilities
Zora Che, Stephen Casper, Robert Kirk et al. · Security
Evaluations of large language model (LLM) risks and capabilities are increasingly being incorporated into AI risk management and governance frameworks. Currently, most risk evaluations are conducted by designing inputs that elicit harmful behaviors from the system.
- 2025-01arXiv 2501.11587Recurrent Diffusion for Large-Scale Parameter Generation
Kai Wang, Dongwen Tang, Wangbo Zhao et al. · Machine learning
Parameter generation has long struggled to match the scale of today large vision and language models, curbing its broader utility.
- 2024-12arXiv 2412.05346BadGPT-4o: stripping safety finetuning from GPT models
Ekaterina Krupkina, Dmitrii Volkov · Security
2023's simple fine-tuning poisoning technique strips GPT-4o's safety guardrails without degrading the model. The BadGPT attack matches best white-box jailbreaks on HarmBench and StrongREJECT.
- 2024-10arXiv 2410.20526Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoders
Zhengfu He, Wentao Shu, Xuyang Ge et al. · Machine learning
Sparse Autoencoders (SAEs) have emerged as a powerful unsupervised method for extracting sparse representations from language models, yet scalable training remains a significant challenge.
- 2024-10arXiv 2410.19278Applying sparse autoencoders to unlearn knowledge in language models
Eoin Farrell, Yeu-Tong Lau, Arthur Conmy · Machine learning
We investigate whether sparse autoencoders (SAEs) can be used to remove knowledge from language models. We use the biology subset of the Weapons of Mass Destruction Proxy dataset and test on the gemma-2b-it and gemma-2-2b-it language models.
- 2024-10arXiv 2410.15639Can Large Language Models Invent Algorithms to Improve Themselves?: Algorithm Discovery for Recursive Self-Improvement through Reinforcement Learning
Yoichi Ishibashi, Taro Yano, Masafumi Oyamada · Language models & NLP
Large Language Models (LLMs) have achieved remarkable capabilities, yet their improvement methods remain fundamentally constrained by human design. We present Self-Developing, a framework that enables LLMs to autonomously discover, implement, and refine their own improvement algorithms.
- 2024-10arXiv 2410.14735Agent Skill Acquisition for Large Language Models via CycleQD
So Kuroki, Taishi Nakamura, Takuya Akiba et al. · Language models & NLP
Training large language models to acquire specific skills remains a challenging endeavor. Conventional training approaches often struggle with data distribution imbalances and inadequacies in objective functions that do not align well with task-specific performance.
- 2024-10arXiv 2410.04207Learning on LoRAs: GL-Equivariant Processing of Low-Rank Weight Spaces for Large Finetuned Models
Theo Putterman, Derek Lim, Yoav Gelberg et al. · Machine learning
Low-rank adaptations (LoRAs) have revolutionized the finetuning of large foundation models, enabling efficient adaptation even with limited computational resources.
- 2024-10arXiv 2410.02355AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models
Junfeng Fang, Houcheng Jiang, Kun Wang et al. · 13th International Conference on Learning Representations (ICLR 2025 Oral) · Language models & NLP
Large language models (LLMs) often exhibit hallucinations due to incorrect or outdated knowledge. Hence, model editing methods have emerged to enable targeted knowledge updates.
- 2024-09arXiv 2409.18025An Adversarial Perspective on Machine Unlearning for AI Safety
Jakub Łucki, Boyi Wei, Yangsibo Huang et al. · Machine learning
Large language models are finetuned to refuse questions about hazardous knowledge, but these protections can often be bypassed. Unlearning methods aim at completely removing hazardous capabilities from models and make them inaccessible to adversaries.
- 2024-08arXiv 2408.00761Tamper-Resistant Safeguards for Open-Weight LLMs
Rishub Tamirisa, Bhrugu Bharathi, Long Phan et al. · Machine learning
Rapid advances in the capabilities of large language models (LLMs) have raised widespread concerns regarding their potential for malicious use. Open-weight LLMs present unique challenges, as existing safeguards lack robustness to tampering attacks that modify model weights.
- 2024-07arXiv 2407.01376Badllama 3: removing safety finetuning from Llama 3 in minutes
Dmitrii Volkov · Machine learning
We show that extensive LLM safety fine-tuning is easily subverted when an attacker has access to model weights.
- 2024-06arXiv 2406.11717Refusal in Language Models Is Mediated by a Single Direction
Andy Arditi, Oscar Obeso, Aaquib Syed et al. · Machine learning
Conversational large language models are fine-tuned for both instruction-following and safety, resulting in models that obey benign requests but refuse harmful ones.
- 2024-05arXiv 2405.14577Representation Noising: A Defence Mechanism Against Harmful Finetuning
Domenic Rosati, Jan Wehner, Kai Williams et al. · Language models & NLP
Releasing open-source large language models (LLMs) presents a dual-use risk since bad actors can easily fine-tune these models for harmful purposes. Even without the open release of weights, weight stealing and fine-tuning APIs make closed models vulnerable to harmful fine-tuning attacks (HFAs).
- 2024-05arXiv 2405.14768WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models
Peng Wang, Zexi Li, Ningyu Zhang et al. · Language models & NLP
Large language models (LLMs) need knowledge updates to meet the ever-growing world facts and correct the hallucinated responses, facilitating the methods of lifelong model editing.
Cited inEWC protects what its Fisher sees: a literature synthesis on elastic weight consolidation, scale and coherenceStudy: Consequence consistent updatesStudy: Elastic weight consolidation deep diveStudy: Fortifying knowledge in continual learningStudy: LLMs modifying open model weightsStudy: Orchestrated specialists and bridge synthesis - 2024-04arXiv 2404.14394A Multimodal Automated Interpretability Agent
Tamar Rott Shaham, Sarah Schwettmann, Franklin Wang et al. · AI & agents
This paper describes MAIA, a Multimodal Automated Interpretability Agent. MAIA is a system that uses neural models to automate neural model understanding tasks like feature interpretation and failure mode discovery.
Built from every arXiv paper cited in our papers, posts, topic hubs, reports and research notes, with titles, authors, dates and abstracts from the arXiv API (updated 2026-11-12). Summaries are the opening sentences of each paper's own abstract, not our assessment; a citation means we used or checked the work, not that we endorse it. Fields follow each paper's primary arXiv category. Papers cited only outside arXiv (journals, proceedings) are not listed yet. Entries are generated from our citations, so a mistyped ID in a note would list the wrong paper; we remove those as we find them. Report a wrong entry.
