Grokking: The Late Generalization of a Neural Network

A small neural network, perfect during training but mediocre in testing after 1,000 steps, ultimately managed to solve new problems almost perfectly by around 20,000 steps, without any additional data or tuning. Internal analyses revealed a circular representation of numbers, discovered through gradient descent, and a late shift from a memorized solution to a general method. This phenomenon, dubbed "grokking," raises questions about the use of early stopping and the interpretation of flat metrics.

Misleading Metrics and Questioning Early Stopping

Observed solely through training and testing accuracy, grokking resembles a long flat phase followed by a sudden shift. A standard early stopping rule would have nearly interrupted this transition, as these rules aim to halt training when no improvement is seen. This situation raises questions about its frequency in larger, more significant models, which may be stopped before a general solution emerges; there is still no complete answer. It remains unclear whether a similar mechanism occurs invisibly in current large models. From the outside, memorization and understanding may appear identical until the point of divergence, and seemingly "stationary" training may precede a decisive change.

A Late Shift Quantified on a Simple Task

On modular addition, the network achieved a perfect score on the training data in less than 1,000 steps while remaining around 10% on unseen problems. The researchers continued training, adding thousands and then tens of thousands of steps without altering the procedure or introducing new data. At some point, performance on unseen questions surged to nearly 100%, and by around 20,000 steps, accuracy was close to 100% in both training and testing. This marked discrepancy between immediate success on training and late generalization has been termed "grokking."

What the Network Built Internally, Piece by Piece

An internal examination of the model, continued through a follow-up study in 2023, revealed the emergence of a circular representation of numbers and the use of rotations to combine them, rediscovering trigonometry without explicit supervision. Gradient descent developed this method, where each number occupies a point on the circle and addition corresponds to two successive rotations before reading the final position. The construction of this structure took time, coexisting with a memorized solution limited to familiar examples. The general method took over when it was sufficiently complete to surpass memorization. From the outside, the construction phase appeared null; internally, it was the heart of the process.

Framework of the Experiment and Initial Failure of Generalization

In 2022, a small team trained a tiny neural network on modular addition, illustrated by 8 plus 7 equaling 3 on a 12-hour clock. The network quickly mastered the training answers but initially failed to generalize, a classic pattern of memorization rather than learning the rule. The experimental setup did not involve large models or specific architectures at this stage.

A Science Fiction Term to Name the Shift

The term "grokking" comes from Robert Heinlein's novel, Stranger in a Strange Land, where it denotes an understanding so deep that it becomes intuitive. In the experiment, the model did not gradually progress by simulating the correct answer: at a hidden moment, it stopped simulating and began to "know." The advanced lesson does not specifically pertain to transformers or modular arithmetic.

A School Analogy to Contextualize the Phenomenon

The proposed analogy is that of a student who, after cramming and memorizing answers, continues to review the same content. Weeks later, a breakthrough occurs: they truly understand the subject and succeed on new problems. This breakthrough happens long after the student seems to have completed their revisions.