NeuroGenesysThe research we build on.
1,514 papers by other researchers that our work cites, from continual learning and verification to language models, agents and hardware. Each entry links to the paper on arXiv and to where we use it. 301 are cited on published pages of this site; the rest in the research notes behind our studies. Our own papers are in the archive.
- 2024-10arXiv 2410.16251Can Knowledge Editing Really Correct Hallucinations?
Baixiang Huang, Canyu Chen, Xiongxiao Xu et al. · Language models & NLP
Large Language Models (LLMs) suffer from hallucinations, referring to the non-factual information in generated content, despite their superior capacities across tasks.
- 2024-10arXiv 2410.14670Decomposing The Dark Matter of Sparse Autoencoders
Joshua Engels, Logan Riggs, Max Tegmark · Machine learning
Sparse autoencoders (SAEs) are a promising technique for decomposing language model activations into interpretable linear features. However, current SAEs fall short of completely explaining model performance, resulting in "dark matter": unexplained variance in activations.
- 2024-10arXiv 2410.13284Learning to Route LLMs with Confidence Tokens
Yu-Neng Chuang, Prathusha Kameswara Sarma, Parikshit Gopalan et al. · Language models & NLP
Large language models (LLMs) have demonstrated impressive performance on several tasks and are increasingly deployed in real-world applications. However, especially in high-stakes settings, it becomes vital to know when the output of an LLM may be unreliable.
- 2024-10arXiv 2410.13787Looking Inward: Language Models Can Learn About Themselves by Introspection
Felix J Binder, James Chua, Tomek Korbak et al. · Language models & NLP
Humans acquire knowledge by observing the external world, but also by introspection., thoughts and feelings) that is not accessible to external observers. Can LLMs introspect?
- 2024-10arXiv 2410.14735Agent Skill Acquisition for Large Language Models via CycleQD
So Kuroki, Taishi Nakamura, Takuya Akiba et al. · Language models & NLP
Training large language models to acquire specific skills remains a challenging endeavor. Conventional training approaches often struggle with data distribution imbalances and inadequacies in objective functions that do not align well with task-specific performance.
- 2024-10arXiv 2410.11782G-Designer: Architecting Multi-agent Communication Topologies via Graph Neural Networks
Guibin Zhang, Yanwei Yue, Xiangguo Sun et al. · AI & agents
Recent advancements in large language model (LLM)-based agents have demonstrated that collective intelligence can significantly surpass the capabilities of individual agents, primarily due to well-crafted inter-agent communication topologies.
- 2024-10arXiv 2410.11805NesTools: A Dataset for Evaluating Nested Tool Learning Abilities of Large Language Models
Han Han, Tong Zhu, Xiang Zhang et al. · Language models & NLP
Large language models (LLMs) combined with tool learning have gained impressive results in real-world applications. During tool learning, LLMs may call multiple tools in nested orders, where the latter tool call may take the former response as its input parameters.
Cited inStudy: Genesys program prior art - 2024-10arXiv 2410.10179Is Parameter Collision Hindering Continual Learning in LLMs?
Shuo Yang, Kun-Peng Ning, Yu-Yang Liu et al. · Machine learning
Large Language Models (LLMs) often suffer from catastrophic forgetting when learning multiple tasks sequentially, making continual learning (CL) essential for their dynamic deployment.
- 2024-10arXiv 2410.10813LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory
Di Wu, Hongwei Wang, Wenhao Yu et al. · Language models & NLP
Recent large language model (LLM)-driven chat assistant systems have integrated memory components to track user-assistant chat histories, enabling more accurate and personalized responses.
- 2024-10arXiv 2410.09724Taming Overconfidence in LLMs: Reward Calibration in RLHF
Jixuan Leng, Chengsong Huang, Banghua Zhu et al. · Language models & NLP
Language model calibration refers to the alignment between the confidence of the model and the actual performance of its responses.
- 2024-10arXiv 2410.09870ChroKnowledge: Unveiling Chronological Knowledge of Language Models in Multiple Domains
Yein Park, Chanwoong Yoon, Jungwoo Park et al. · Language models & NLP
Large language models (LLMs) have brought significant changes to many aspects of our lives. However, assessing and ensuring their chronological knowledge remains challenging.
- 2024-10arXiv 2410.08589Retraining-Free Merging of Sparse MoE via Hierarchical Clustering
I-Chun Chen, Hsu-Shen Liu, Wei-Fang Sun et al. · Machine learning
Sparse Mixture-of-Experts (SMoE) models represent a significant advancement in large language model (LLM) development through their efficient parameter utilization.
- 2024-10arXiv 2410.08827Do Unlearning Methods Remove Information from Language Model Weights?
Aghyad Deeb, Fabien Roger · Machine learning
Large Language Models' knowledge of how to perform cyber-security attacks, create bioweapons, and manipulate humans poses risks of misuse. Previous work has proposed methods to unlearn this knowledge.
- 2024-10arXiv 2410.07524Upcycling Large Language Models into Mixture of Experts
Ethan He, Abhinav Khattar, Ryan Prenger et al. · Language models & NLP
Upcycling pre-trained dense language models into sparse mixture-of-experts (MoE) models is an efficient approach to increase the model capacity of already trained models.
- 2024-10arXiv 2410.08047Divide and Translate: Compositional First-Order Logic Translation and Verification for Complex Logical Reasoning
Hyun Ryu, Gyeongman Kim, Hyemin S. Lee et al. · The Thirteenth International Conference on Learning Representations (ICLR 2025) · Language models & NLP
Complex logical reasoning tasks require a long sequence of reasoning, which a large language model (LLM) with chain-of-thought prompting still falls short. To alleviate this issue, neurosymbolic approaches incorporate a symbolic solver.
- 2024-10arXiv 2410.07095MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering
Jun Shern Chan, Neil Chowdhury, Oliver Jaffe et al. · Language models & NLP
We introduce MLE-bench, a benchmark for measuring how well AI agents perform at machine learning engineering.
Cited inStudy: Superintelligence - 2024-10arXiv 2410.07137Cheating Automatic LLM Benchmarks: Null Models Achieve High Win Rates
Xiaosen Zheng, Tianyu Pang, Chao Du et al. · Language models & NLP
0, Arena-Hard-Auto, and MT-Bench, have become popular for evaluating language models due to their cost-effectiveness and scalability compared to human evaluation.
Cited inStudy: Genesys program prior art - 2024-10arXiv 2410.07163Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning
Chongyu Fan, Jiancheng Liu, Licong Lin et al. · Language models & NLP
, copyrighted or harmful content) while preserving model utility. Despite the increasing demand for unlearning, a technically-grounded optimization framework is lacking.
- 2024-10arXiv 2410.07348MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation Experts
Peng Jin, Bo Zhu, Li Yuan et al. · Machine learning
In this work, we aim to simultaneously enhance the effectiveness and efficiency of Mixture-of-Experts (MoE) methods. To achieve this, we propose MoE++, a general and heterogeneous MoE framework that integrates both Feed-Forward Network~(FFN) and zero-computation experts.
- 2024-10arXiv 2410.05779LightRAG: Simple and Fast Retrieval-Augmented Generation
Zirui Guo, Lianghao Xia, Yanhua Yu et al. · Retrieval & data
Retrieval-Augmented Generation (RAG) systems enhance large language models (LLMs) by integrating external knowledge sources, enabling more accurate and contextually relevant responses tailored to user needs.
Cited inStudy: Graphyte spec - 2024-10arXiv 2410.05817Probing Language Models on Their Knowledge Source
Zineddine Tighidet, Andrea Mogini, Jiali Mei et al. · Language models & NLP
Large Language Models (LLMs) often encounter conflicts between their learned, internal (parametric knowledge, PK) and external knowledge provided during inference (contextual knowledge, CK).
- 2024-10arXiv 2410.10868Large Continual Instruction Assistant
Jingyang Qiao, Zhizhong Zhang, Xin Tan et al. · Machine learning
Continual Instruction Tuning (CIT) is adopted to continually instruct Large Models to follow human intent data by data. It is observed that existing gradient update would heavily destroy the performance on previous datasets during CIT process.
- 2024-10arXiv 2410.05229GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models
Iman Mirzadeh, Keivan Alizadeh, Hooman Shahrokhi et al. · Machine learning
Recent advancements in Large Language Models (LLMs) have sparked interest in their formal reasoning capabilities, particularly in mathematics. The GSM8K benchmark is widely used to assess the mathematical reasoning of models on grade-school-level questions.
- 2024-10arXiv 2410.05451SecAlign: Defending Against Prompt Injection with Preference Optimization
Sizhe Chen, Arman Zharmagambetov, Saeed Mahloujifar et al. · Security
Large language models (LLMs) are becoming increasingly prevalent in modern software systems, interfacing between the user and the Internet to assist with tasks that require advanced language understanding.
- 2024-10arXiv 2410.04207Learning on LoRAs: GL-Equivariant Processing of Low-Rank Weight Spaces for Large Finetuned Models
Theo Putterman, Derek Lim, Yoav Gelberg et al. · Machine learning
Low-rank adaptations (LoRAs) have revolutionized the finetuning of large foundation models, enabling efficient adaptation even with limited computational resources.
- 2024-10arXiv 2410.03122RIPPLECOT: Amplifying Ripple Effect of Knowledge Editing in Language Models via Chain-of-Thought In-Context Learning
Zihao Zhao, Yuchen Yang, Yijiang Li et al. · Language models & NLP
The ripple effect poses a significant challenge in knowledge editing for large language models. Namely, when a single fact is edited, the model struggles to accurately update the related facts in a sequence, which is evaluated by multi-hop questions linked to a chain of related facts.
- 2024-10arXiv 2410.03617What Matters for Model Merging at Scale?
Prateek Yadav, Tu Vu, Jonathan Lai et al. · Machine learning
Model merging aims to combine multiple expert models into a more capable single model, offering benefits such as reduced storage and serving costs, improved generalization, and support for decentralized model development.
- 2024-10arXiv 2410.03834GraphRouter: A Graph-based Router for LLM Selections
Tao Feng, Yanzhen Shen, Jiaxuan You · AI & agents
The rapidly growing number and variety of Large Language Models (LLMs) present significant challenges in efficiently selecting the appropriate LLM for a given query, especially considering the trade-offs between performance and computational cost.
- 2024-10arXiv 2410.02173Efficiently Deploying LLMs with Controlled Risk
Michael J. Zellinger, Matt Thomson · Machine learning
Deploying large language models in production requires simultaneous attention to efficiency and risk control. Prior work has shown the possibility to cut costs while maintaining similar accuracy, but has neglected to focus on risk control.
- 2024-10arXiv 2410.02197Beyond Bradley-Terry Models: A General Preference Model for Language Model Alignment
Yifan Zhang, Ge Zhang, Yue Wu et al. · AI & agents
Modeling human preferences is crucial for aligning foundation models with human values. Traditional reward modeling methods, such as the Bradley-Terry (BT) reward model, fall short in expressiveness, particularly in addressing intransitive preferences.
- 2024-10arXiv 2410.02223EmbedLLM: Learning Compact Representations of Large Language Models
Richard Zhuang, Tianhao Wu, Zhaojin Wen et al. · Language models & NLP
With hundreds of thousands of language models available on Huggingface today, efficiently evaluating and utilizing these models across various downstream, tasks has become increasingly critical.
- 2024-10arXiv 2410.02330Llama SLayer 8B: Shallow Layers Hold the Key to Knowledge Injection
Tianxiang Chen, Zhentao Tan, Tao Gong et al. · Language models & NLP
As a manner to augment pre-trained large language models (LLM), knowledge injection is critical to develop vertical domain large models and has been widely studied.
- 2024-10arXiv 2410.02355AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models
Junfeng Fang, Houcheng Jiang, Kun Wang et al. · 13th International Conference on Learning Representations (ICLR 2025 Oral) · Language models & NLP
Large language models (LLMs) often exhibit hallucinations due to incorrect or outdated knowledge. Hence, model editing methods have emerged to enable targeted knowledge updates.
- 2024-10arXiv 2410.02707LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations
Hadas Orgad, Michael Toker, Zorik Gekhman et al. · Language models & NLP
Large language models (LLMs) often produce errors, including factual inaccuracies, biases, and reasoning failures, collectively referred to as "hallucinations".
- 2024-10arXiv 2410.02879Position: LLM Unlearning Benchmarks are Weak Measures of Progress
Pratiksha Thaker, Shengyuan Hu, Neil Kale et al. · Language models & NLP
Unlearning methods have the potential to improve the privacy and safety of large language models (LLMs) by removing sensitive or harmful information post hoc. The LLM unlearning research community has increasingly turned toward empirical benchmarks to assess the effectiveness of such methods.
- 2024-10arXiv 2410.03782DaWin: Training-free Dynamic Weight Interpolation for Robust Adaptation
Changdae Oh, Yixuan Li, Kyungwoo Song et al. · Machine learning
Adapting a pre-trained foundation model on downstream tasks should ensure robustness against distribution shifts without the need to retrain the whole model.
- 2024-10arXiv 2410.01380Knowledge Entropy Decay during Language Model Pretraining Hinders New Knowledge Acquisition
Jiyeon Kim, Hyunji Lee, Hyowon Cho et al. · Language models & NLP
In this work, we investigate how a model's tendency to broadly integrate its parametric knowledge evolves throughout pretraining, and how this behavior affects overall performance, particularly in terms of knowledge acquisition and forgetting.
- 2024-09arXiv 2409.19878HDMoLE: Mixture of LoRA Experts with Hierarchical Routing and Dynamic Thresholds for Fine-Tuning LLM-based ASR Models
Bingshen Mu, Kun Wei, Qijie Shao et al. · Other
Recent advancements in integrating Large Language Models (LLM) with automatic speech recognition (ASR) have performed remarkably in general domains.
- 2024-09arXiv 2409.19886RouterDC: Query-Based Router by Dual Contrastive Learning for Assembling Large Language Models
Shuhao Chen, Weisen Jiang, Baijiong Lin et al. · Machine learning
Recent works show that assembling multiple off-the-shelf large language models (LLMs) can harness their complementary abilities. To achieve this, routing is a promising method, which learns a router to select the most suitable LLM for each query.
- 2024-09arXiv 2409.19315Analog In-Memory Computing Attention Mechanism for Fast and Energy-Efficient Large Language Models
Nathan Leroux, Paul-Philipp Manea, Chirag Sudarshan et al. · Neuroscience & neural computation
Transformer networks, driven by self-attention, are central to Large Language Models. In generative Transformers, self-attention uses cache memory to store token projections, avoiding recomputation at each time step.
Built from every arXiv paper cited in our papers, posts, topic hubs, reports and research notes, with titles, authors, dates and abstracts from the arXiv API (updated 2026-11-12). Summaries are the opening sentences of each paper's own abstract, not our assessment; a citation means we used or checked the work, not that we endorse it. Fields follow each paper's primary arXiv category. Papers cited only outside arXiv (journals, proceedings) are not listed yet. Entries are generated from our citations, so a mistyped ID in a note would list the wrong paper; we remove those as we find them. Report a wrong entry.
