NeuroGenesys.
Research Desk • Field Watch

Learning order explains forgetting, and a stored curvature snapshot can flag the quiet kind without the old data

• 5 min read • By Research Desk

Save

Key findings

  • Across Split MNIST, Split Fashion-MNIST and Split CIFAR-10, the forgetting of the first task matched the difference in its loss between the two training orders almost exactly, with rank correlations of 0.96 to 1.00.
  • From a stored summary of the old task (its gradient and the top 20 eigenpairs of its Hessian) and none of its data, the Lie bracket ranked slow, task-incremental forgetting at 0.81, 0.70 and 0.76, against 0.58, 0.33 and 0.35 for the EWC penalty.
  • Measured over roughly the first 5% of training on the new task, the data-free bracket tied re-evaluating the old task on its own data on MNIST (0.81 vs. 0.81) and beat it on Fashion-MNIST (0.70 vs. 0.33).
  • When forgetting was fast (domain-incremental), the EWC penalty was as good or better, at 0.84 against 0.46 on CIFAR-10 features, and enlarging the summary to 50 or 100 eigenpairs did not close the gap.
  • On a small CNN that learned its CIFAR-10 tasks only to 78% accuracy, the bracket failed (-0.12), because the old task's leftover gradient was 94% the size of the new task's rather than 2 to 7%.
A wet salt flat at night under an indigo, star-filled sky with the Milky Way and an orange glow on the horizon, where a violet line of light and a blue line of light start from the same corner and run side by side across the reflective salt, ending at slightly different points joined by a short amber segment that leaves the shape not quite closed

Catastrophic forgetting is usually measured after the fact, by re-testing the old task. That needs the old data, which a model that keeps learning after deployment often no longer has, for reasons of privacy, data agreements or storage. A preprint posted to arXiv on November 15 by Ephraim Kofi Ayi Kwapong of the Responsible Artificial Intelligence Lab at Kwame Nkrumah University of Science and Technology in Ghana, "Does Learning Order Explain Forgetting?", asks whether a quantity from the geometry of gradient descent can warn of forgetting early and without that data. The quantity is the Lie bracket of two tasks' gradient fields. One small step on task A followed by one on task B ends at different weights than the reverse order, and to leading order the gap is the squared learning rate times the bracket, which pairs each task's curvature (its Hessian) with the other task's gradient. Benoit Dherin derived this term in 2023 and conjectured that it may be linked to catastrophic forgetting, but did not test the link.

The tests are deliberately small. Split MNIST, Split Fashion-MNIST and Split CIFAR-10 are each cut into five two-class tasks, and every ordered pair of tasks is trained with five seeds, giving 100 runs per setting, using plain SGD because the bracket is derived for memoryless updates. The models are a two-layer MLP of about 270,000 parameters on pixels, the same MLP on frozen ImageNet-pretrained ResNet-18 features for CIFAR-10, and a small CNN of about 140,000 parameters. Two scenarios give two speeds of forgetting: task-incremental, where each task has its own outputs and mean forgetting is about 1% of accuracy, and domain-incremental, where tasks share outputs and old-task accuracy falls by 18.6 to 35.0 points on average. When task A finishes, the method stores its weights, its gradient and the top 20 eigenpairs of its Hessian, then discards A's data. The bracket is computed from that summary and the new task's data at steps 5 to 20 of training on B, about the first 5% of it, and scored by how well it ranks the 100 runs by the forgetting they eventually show.

The first result is close to true by construction, and the author says so. Forgetting of A matched the difference in A's loss between the two training orders almost exactly, with rank correlations of 0.96 to 1.00, because when A is trained last it ends near its own optimum. In this setting, predicting forgetting and predicting the order effect are the same problem. The real question is whether the bracket, exact only for one small step, tracks that effect over hundreds of noisy ones. For slow forgetting on well-learned tasks it does. The stored bracket ranked forgetting at 0.81, 0.70 and 0.76 on MNIST, Fashion-MNIST and CIFAR-10 features, against 0.58, 0.33 and 0.35 for the EWC penalty used as a predictor. The bootstrap intervals on those differences (+0.23 to +0.41) exclude zero, and the bracket came out ahead in all 15 seed-level comparisons. It matched a bracket computed with the old data, so the 20-direction summary lost little, and it tied or beat re-evaluating A on its own data (0.81 vs. 0.81 on MNIST, 0.70 vs. 0.33 on Fashion-MNIST). The author's explanation is that after 20 steps slow forgetting has barely begun and is lost in noise, while the bracket measures a rate. Most of the predictive content sat in a single term: how the old task's curvature acts on the new task's gradient.

The paper measures two clear boundaries. When forgetting is fast, the weights leave the neighborhood of A's solution within a few steps and the stored local model goes stale. The EWC penalty, which depends only on how far each weight has moved, beat the stored bracket on MNIST (0.59 vs. 0.40) and CIFAR-10 features (0.84 vs. 0.46) and tied it on Fashion-MNIST (0.79 vs. 0.78). Three repairs, including combining the bracket's rank with EWC's and enlarging the summary to 50 or 100 eigenpairs, did not reach EWC alone, so the limit is the summary's locality, not its size. Keeping ten old examples helped on MNIST but not on CIFAR-10 features. The second boundary is a half-learned old task. On the small CNN, which reached only 78% accuracy and did not improve with twice the training, the bracket failed in both scenarios (-0.12 for slow forgetting), with or without old data. There the old task's residual gradient was 94% the size of the new task's, against 2 to 7% on the well-learned setups, so much of the damage came from moving along that slope, which a curvature quantity cannot see. On the same CIFAR-10 images with a frozen pretrained backbone, the stored bracket recovered to 0.76. One case remains unexplained: in fast forgetting on the CNN the damage was still mostly curvature, yet the bracket's key term did not predict it.

The author lists the limitations plainly. This is a single-author preprint, not peer-reviewed. The models are small MLPs and one small CNN, and the positive CIFAR-10 result is for a trainable head on frozen features. Every run is one pair of tasks, so long task sequences are untested, and the speed of forgetting is confounded with the scenario. Slow forgetting averages about 1% of accuracy, so the absolute stakes per update are low. The signals rank runs and are not calibrated loss estimates. All results use plain SGD; momentum and Adam would need a modified bracket. The main results use five seeds, some tests three or two, and the bootstrap intervals treat runs as independent although they share seeds and tasks. The summary stores 22 numbers per parameter against 2 for EWC, and each evaluation needs one Hessian-vector product. The paper studies prediction only; using the bracket to reduce forgetting is left to future work. Code and per-run results are public, and the author discloses that Claude helped write the analysis code and draft parts of the text.

For NeuroGenesys, the paper is more useful as a diagnosis than as a method. Our class-incremental work freezes one expert per task and treats forgetting as a selection problem, which protects old weights by not touching them. The parts that must keep updating, such as a shared router or head, are where old data is most often gone and where quiet forgetting would accumulate unseen. Our note on EWC and stale facts found that EWC's stiffness protected unrelated facts and froze the facts being revised together. This paper adds that, as a forecaster, the EWC penalty suits loud forgetting and does poorly on the quiet kind, where the way old curvature meets the new gradient says more than distance moved. That points to a monitor rather than a penalty: keep a gradient and a few curvature directions for each consolidated component, and flag updates whose bracket rises before any accuracy drop is visible. The half-learned failure is a warning for that plan, since the signal is only trustworthy for components trained close to convergence. Whether it survives Adam, long task sequences and far larger models are the tests we would want before relying on it.

Related research

Leadership

A word from the founder — on why the brain is the blueprint, and why the next decade of intelligence will be built here.

"The brain is the universe's only proof that general intelligence is possible. We treat it as the blueprint."

From cortical interfaces to photonic silicon, every program at NeuroGenesys exists to answer one question: how does intelligence arise — and how do we build it safely.

Will Schulz

Will Schulz

Founder, NeuroGenesys • AW3 Technology