NeuroGenesysThe research we build on.
1,514 papers by other researchers that our work cites, from continual learning and verification to language models, agents and hardware. Each entry links to the paper on arXiv and to where we use it. 301 are cited on published pages of this site; the rest in the research notes behind our studies. Our own papers are in the archive.
- 2020-10arXiv 2010.11465Beta Embeddings for Multi-Hop Logical Reasoning in Knowledge Graphs
Hongyu Ren, Jure Leskovec · AI & agents
One of the fundamental problems in Artificial Intelligence is to perform complex multi-hop logical reasoning over the facts captured by a knowledge graph (KG). This problem is challenging, because KGs can be massive and incomplete.
- 2020-10arXiv 2010.04003A Theoretical Analysis of Catastrophic Forgetting through the NTK Overlap Matrix
Thang Doan, Mehdi Bennani, Bogdan Mazoure et al. · Proceedings of the 24th International Conference on Artificial Intelligence and Statistics (AISTATS 2021) · Machine learning
Continual learning (CL) is a setting in which an agent has to learn from an incoming stream of data during its entire lifetime. Although major advances have been made in the field, one recurring problem which remains unsolved is that of Catastrophic Forgetting (CF).
- 2020-09arXiv 2009.14794Rethinking Attention with Performers
Krzysztof Choromanski, Valerii Likhosherstov, David Dohan et al. · Machine learning
We introduce Performers, Transformer architectures which can estimate regular (softmax) full-rank-attention Transformers with provable accuracy, but using only linear (as opposed to quadratic) space and time complexity, without relying on any priors such as sparsity or low-rankness.
- 2020-07arXiv 2008.02217Hopfield Networks is All You Need
Hubert Ramsauer, Bernhard Schäfl, Johannes Lehner et al. · Neuroscience & neural computation
We introduce a modern Hopfield network with continuous states and a corresponding update rule. The new Hopfield network can store exponentially (with the dimension of the associative space) many patterns, retrieves the pattern with one update, and has exponentially small retrieval errors.
- 2020-06arXiv 2006.16236Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention
Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas et al. · Machine learning
Transformers achieve remarkable performance in several tasks but due to their quadratic complexity, with respect to the input's length, they are prohibitively slow for very long sequences.
- 2020-06arXiv 2006.08591Monotone operator equilibrium networks
Ezra Winston, J. Zico Kolter · Machine learning
Implicit-depth models such as Deep Equilibrium Networks have recently been shown to match or exceed the performance of traditional deep networks while being much more memory efficient.
- 2020-02arXiv 2002.08484Estimating Training Data Influence by Tracing Gradient Descent
Garima Pruthi, Frederick Liu, Mukund Sundararajan et al. · Machine learning
We introduce a method called TracIn that computes the influence of a training example on a prediction made by the model. The idea is to trace how the loss on the test point changes during the training process whenever the training example of interest was utilized.
- 2020-02arXiv 2002.05969Query2box: Reasoning over Knowledge Graphs in Vector Space using Box Embeddings
Hongyu Ren, Weihua Hu, Jure Leskovec · Machine learning
Answering complex logical queries on large-scale incomplete knowledge graphs (KGs) is a fundamental yet challenging task.
- 2019-09arXiv 1909.01377Deep Equilibrium Models
Shaojie Bai, J. Zico Kolter, Vladlen Koltun · Machine learning
We present a new approach to modeling sequential data: the deep equilibrium model (DEQ).
- 2019-08arXiv 1908.11775Transformer Dissection: A Unified Understanding of Transformer's Attention via the Lens of Kernel
Yao-Hung Hubert Tsai, Shaojie Bai, Makoto Yamada et al. · Machine learning
Transformer is a powerful architecture that achieves superior performance on various sequence learning tasks, including neural machine translation, language understanding, and sequence prediction.
- 2019-02arXiv 1902.10197RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space
Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie et al. · Machine learning
We study the problem of learning representations of entities and relations in knowledge graphs for predicting missing links. The success of such a task heavily relies on the ability of modeling and inferring the patterns of (or between) the relations.
- 2018-11arXiv 1811.02524Solving SAT and MaxSAT with a Quantum Annealer: Foundations, Encodings, and Preliminary Results
Zhengbing Bian, Fabian Chudak, William Macready et al. · Hardware, photonics & physics
Quantum annealers (QAs) are specialized quantum computers that minimize objective functions over discrete variables by physically exploiting quantum effects. Current QA platforms allow for the optimization of quadratic objectives defined over binary variables (qubits), also known as Ising problems.
- 2018-06arXiv 1806.07572Neural Tangent Kernel: Convergence and Generalization in Neural Networks
Arthur Jacot, Franck Gabriel, Clément Hongler · In Advances in neural information processing systems (pp. 8571-8580) 2018 · Machine learning
At initialization, artificial neural networks (ANNs) are equivalent to Gaussian processes in the infinite-width limit, thus connecting them to kernel methods. a new kernel: the Neural Tangent Kernel (NTK).
- 2018-03arXiv 1803.10122World Models
David Ha, Jürgen Schmidhuber · Machine learning
We explore building generative neural network models of popular reinforcement learning environments. Our world model can be trained quickly in an unsupervised manner to learn a compressed spatial and temporal representation of the environment.
- 2017-11arXiv 1711.11157A Semantic Loss Function for Deep Learning with Symbolic Knowledge
Jingyi Xu, Zilu Zhang, Tal Friedman et al. · AI & agents
This paper develops a novel methodology for using symbolic knowledge in deep learning. From first principles, we derive a semantic loss function that bridges between neural output vectors and logical constraints.
- 2017-07arXiv 1707.00355Evaluating Ising Processing Units with Integer Programming
Carleton Coffrin, Harsha Nagarajan, Russell Bent · Statistics & theory
The recent emergence of novel computational devices, such as adiabatic quantum computers, CMOS annealers, and optical parametric oscillators, present new opportunities for hybrid-optimization algorithms that are hardware accelerated by these devices.
- 2017-05arXiv 1705.08690Continual Learning with Deep Generative Replay
Hanul Shin, Jung Kwon Lee, Jaehong Kim et al. · AI & agents
Attempts to train a comprehensive artificial intelligence capable of solving multiple tasks have been impeded by a chronic problem called catastrophic forgetting.
- 2016-09arXiv 1609.03543Logical Induction
Scott Garrabrant, Tsvi Benson-Tilsen, Andrew Critch et al. · AI & agents
We present a computable algorithm that assigns probabilities to every logical statement in a given formal language, and refines those probabilities over time.
- 2016-06arXiv 1606.01164Dense Associative Memory for Pattern Recognition
Dmitry Krotov, John J Hopfield · Advances in Neural Information Processing Systems 29 (2016), 1172-1180 · Neuroscience & neural computation
A model of associative memory is studied, which stores and reliably retrieves many more patterns than the number of neurons in the network. We propose a simple duality between this dense associative memory and neural networks commonly used in deep learning.
- 2016-05arXiv 1605.09522Kernel Mean Embedding of Distributions: A Review and Beyond
Krikamol Muandet, Kenji Fukumizu, Bharath Sriperumbudur et al. · Foundations and Trends in Machine Learning: Vol. 10: No. 1-2, pp 1-141 (2017) · Machine learning
A Hilbert space embedding of a distribution---in short, a kernel mean embedding---has recently emerged as a powerful tool for machine learning and inference.
- 2015-11arXiv 1511.06361Order-Embeddings of Images and Language
Ivan Vendrov, Ryan Kiros, Sanja Fidler et al. · Machine learning
Hypernymy, textual entailment, and image captioning can be seen as special cases of a single visual-semantic hierarchy over words, sentences, and images. In this paper we advocate for explicitly modeling the partial order structure of this hierarchy.
- 2013-01arXiv 1301.6725Loopy Belief Propagation for Approximate Inference: An Empirical Study
Kevin Murphy, Yair Weiss, Michael I. Jordan · AI & agents
The most dramatic instance OF this IS the near Shannon - limit performance OF Turbo Codes codes whose decoding algorithm IS equivalent TO loopy belief propagation IN a chain - structured Bayesian network.
- 2010-02arXiv 1002.3307Graph Zeta Function in the Bethe Free Energy and Loopy Belief Propagation
Yusuke Watanabe, Kenji Fukumizu · Advances in Neural Information Processing Systems 22, pages 2017-2025 · AI & agents
We propose a new approach to the analysis of Loopy Belief Propagation (LBP) by establishing a formula that connects the Hessian of the Bethe free energy with the edge zeta function.
Built from every arXiv paper cited in our papers, posts, topic hubs, reports and research notes, with titles, authors, dates and abstracts from the arXiv API (updated 2026-11-12). Summaries are the opening sentences of each paper's own abstract, not our assessment; a citation means we used or checked the work, not that we endorse it. Fields follow each paper's primary arXiv category. Papers cited only outside arXiv (journals, proceedings) are not listed yet. Entries are generated from our citations, so a mistyped ID in a note would list the wrong paper; we remove those as we find them. Report a wrong entry.
