Grokking: The Phase Transition After Memorization

Power et al. watched a transformer memorize division modulo 97 while performing near chance on unseen equations. After far more training, with no new data, test accuracy climbed toward 100%. The network had stored the answers first and discovered the rule later.

How a network changes its mind

The 2022 experiment used half of the possible input pairs for training. Once training accuracy approached 100%, ordinary early stopping would have ended the run. Continuing optimization exposed a second timescale: the memorizing solution slowly gave way to a rule that covered the entire operation table.

For modular addition, that rule can be written as a rotation:

θ(a + b) = θ(a) + θ(b) mod 2π

Nanda et al. (2023) reverse-engineered a one-layer transformer solving addition modulo 113. Its embeddings mapped integers onto circles, while neurons combined sines and cosines at five key frequencies. The visible jump in test accuracy concealed three quieter stages.

Weight decay often helps because a lookup table costs more parameter norm than a compact arithmetic circuit. The optimizer can therefore reach zero training error quickly, then spend thousands of steps replacing the expensive answer sheet with a cheaper algorithm.

Where it shows up

Study	Task	Delayed result
Power et al., 2022	Division modulo 97	Chance-level validation rose toward 100%
Nanda et al., 2023	Addition modulo 113	Five Fourier frequencies carried the learned rule
Liu et al., 2022	MNIST, IMDb, QM9	Grokking induced through data size and initialization choices
Mohamadi et al., 2024	Modular addition	Theory tied the delay to leaving kernel-like training

The sharp arithmetic examples matter because every possible input can be enumerated. They turn concept transformer architecture from a black box into a laboratory specimen.

What's contested

“Phase transition” describes the accuracy curve better than it describes the internal computation. Nanda et al. found continuous progress in circuit formation before the abrupt test jump. Other accounts emphasize shrinking weight norms, a change from kernel-like to feature-learning dynamics, or altered geometry around decision boundaries.

The scope is also unsettled. Grokking can be induced on images, language, and molecular data, but those demonstrations often require small datasets or unusual initialization. It does not yet explain every sudden capability discussed in concept emergence in llms.

Why this has to do with other realms

A physics phase transition looks discontinuous at the scale being measured even when microscopic variables have been moving for some time. Grokking carries the same warning: an abrupt observable need not have an abrupt cause. That places it beside concept emergence, while the replacement of a lookup table by a short rule connects it to compression in concept information theory.

It also complicates concept scaling laws. More training can change not only how well a model performs, but which internal solution performs the task.

An open question

Could concept mechanistic interpretability detect a generalizing circuit early enough to predict grokking while validation accuracy is still flat?

Key Sources

Power et al. (2022), “Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets” - the original modular-arithmetic experiments.
Nanda et al. (2023), “Progress Measures for Grokking via Mechanistic Interpretability” - the Fourier-circuit reconstruction.
Liu, Michaud and Tegmark (2022), “Omnigrok” - evidence beyond algorithmic datasets and the loss-landscape account.
Mohamadi et al. (2024), “Why Do You Grok?” - a theoretical account of modular addition.

Abhishek's take

The useful lesson for me is not “train longer.” It is that a flat validation curve may hide a circuit assembling below the measurement threshold. I want progress measures that reveal that circuit before another 10,000 training steps merely burn compute.

Tags: #grokking #generalization #phase-transitions #mechanistic-interpretability #neural-networks

Abhishek