NeuroGenesysWhat is analog in-memory computing?
Last updated • By Will Schulz
Analog in-memory computing (AIMC) performs matrix–vector multiplication inside the memory array that stores a neural network's weights. It uses device physics, Ohm's law for multiplication and Kirchhoff's current law for summation, instead of moving weights to a separate processor, which targets the energy cost of data movement in AI inference.
How does analog in-memory computing work?
A review by Sebastian et al. (Nature Nanotechnology, 2020) describes in-memory computing as a non-von Neumann approach in which "certain computational tasks are performed in place in the memory itself by exploiting the physical attributes of the memory devices." Both charge-based and resistance-based devices are explored, including phase-change memory, RRAM, flash and SRAM.
IBM demonstrated two milestones in 2023: a 14-nm chip with 64 AIMC cores of 256×256 phase-change memory (Le Gallo et al., Nature Electronics), whose authors stress that AIMC "must be combined with on-chip digital operations and communication" for end-to-end gains, and an analog-AI chip for speech recognition (Ambrogio et al., Nature).
What are the limits of analog in-memory computing?
Limited precision, device noise and drift, and costly analog-to-digital conversion at the periphery; nonlinear activations are hard in analog. Our own cost-model work adds a transformer-specific limit: AIMC accelerates static-weight matrix multiplies but cannot touch attention's growing KV cache. In the AIMC window paper, the decode advantage on a transformer dilutes from about 3.9× at 2k tokens to about 1.05× at 131k, while a sliding attention window keeps it roughly constant. The last hardware flank finds analog prefill survives only as an energy-bound, short-prompt niche, and The wrong phase notes that a widely relayed ~150 TOPS/W analog figure is an 8-bit test-chip claim, against about 24 TOPS/W for a shipping card.
Where can I read more on NeuroGenesys?
Sources
- Memory devices and applications for in-memory computing — Nature Nanotechnology (Sebastian et al.), 2020
- A 64-core mixed-signal in-memory compute chip based on phase-change memory for deep neural network inference — Nature Electronics (Le Gallo et al.), 2023
- An analog-AI chip for energy-efficient speech recognition and transcription — Nature (Ambrogio et al.), 2023
- Which sequence architecture keeps analog in-memory compute's advantage? — NeuroGenesys, July 2026
