In 2022, Alethea Power and her collaborators were working on training small transformers to do maths. As training progressed something interesting happened: as a model learned modular arithmetic, memorising the training data, seemed to plateau in its improvement, and then, later in training suddenly jumped to being able to generalise. Task accuracy would go from near zero to near perfect in a few thousands 'steps' of training. What they observed is called 'grokking' and has given its name to different AI products and models.
The problem they trained the model on was (a + b) mod 113. Modular arithmetic is like doing maths on a clock face. On a 12-hour clock, 10 + 4 is not 14; it's 2 o'clock. Mod 113 is the same idea, but imagine it on a much larger clockface: add two numbers, and if the result goes past 113, wrap back to the start.
A year later Neel Nanda and his team did some investigations into what was going on in the model. Inside, they found what looked like an algorithm: a Fourier circuit, computing modular addition through trigonometric identities. The model seemed to have learned, or constructed, in a precise and inspectable way, what modular arithmetic was.
This became one of the most cited results in mechanistic interpretability: a simple system, opened up, revealing an algorithm of the sort a mathematician would write.
With DST we can look at these results again; and, as with Othello, not refute any of the research done, but try to offer another perspective.
A note on setup before we begin. When the first version of this battery returned results this strong, the first question was whether we were testing Nanda's architecture or our own. We weren't, quite: three small things had drifted. No causal attention mask, ReLU instead of GELU, biases in the MLP layers. So we re-ran the whole battery on a faithful rebuild: causal mask, GELU, no MLP bias. Every finding below replicates across both versions. The numbers quoted are from the second faithful rerun.
The structural reading of models trained on this problem says: the model has discovered modular addition, expressed as a Fourier circuit. The DST reading says: the model has learned a (mod 113, addition) shaped surface that happens to express itself in trigonometric features, because trigonometric features are an efficient encoding of cyclic data. If the goal of compressing the data into the ideal map involves finding features that corroborate what we think of as the most efficient, this seems, in principle, like a good thing for the model to do.
If the structural reading is correct, the model's representations perhaps could generalise across moduli, across operations, across seeds. If the distributional reading is correct, each circuit should be specific: to the operation, to the modulus, to the seed.
So we trained six models (I know, always training models). Three on (a + b) mod 113, differing only in their random seed. Two on (a + b) mod 127. One on (a × b) mod 113.
We start, as with Othello, with the probes. A probe is a small classifier trained to read a model's internal activations and predict something we care about; here, the inputs a and b. If a probe succeeds, the information is geometrically present in the model's representations.
Self-probes work almost perfectly. Every model can read its own state with 99.9% accuracy: the geometry is there.
But cross-probes, a probe trained on Seed A and then applied to Seed B, perform at 0.90%. Chance is 0.88%.
Control-verified: an untrained network of the same architecture decodes (a+b) mod 113 at 0.0% against a 0.9% chance floor, so the privacy here is a fact about learned circuits, not about the input.
To check this with a second instrument we use CKA, or Centred Kernel Alignment. CKA is a measure of how similarly two neural networks structure their internal representations; it returns a number between 0 and 1, where 1 means the models arrange information in essentially the same geometry and 0 means there is no shared structure to find. Across our three seeds, CKA is 0.0008. A third measure, Procrustes distance, returns 1.394: essentially √2, which is the theoretical maximum for representations of this kind, the point at which two geometries are as misaligned as they can possibly be. The representations are not rotated versions of each other; they look as separate as they can get.
When we look directly at the Fourier circuits each model has built we can see this even more clearly. Seed 0 uses dominant frequencies {110, 3, 48, 65, 75}. Seed 1 uses {112, 1, 24, 89, 59}. Seed 2 uses {81, 32, 20, 93, 111}. The overlap across any pair of seeds is zero.
Three models, trained on identical data with identical hyperparameters, all reaching 100% accuracy through Fourier circuits, share not a single dominant frequency. The circuit is absolutely real, but the specific frequencies it uses are arbitrary.
There is a reasonable objection here: perhaps the 'different' frequencies are secretly the same information in a different form. Fourier features come in conjugate pairs (frequency k and frequency p - k together carry the same cyclic content), so in principle two seeds could be using the same underlying features under different labels. But even after pairing the conjugates, the overlap across seeds is still zero. The encodings seem private.
There is a further objection: three seeds is a very small sample. With only three sets of five frequencies drawn from 113 candidates, zero pairwise overlap has about a 47% probability under pure random selection. So the n=3 result is suggestive, not decisive. To tighten the question, we trained thirty more seeds and compared the observed overlap against a Monte Carlo null in which each seed picks five frequencies uniformly at random.
| Statistic | Observed | Null mean | 95% CI | P-value |
|---|---|---|---|---|
| Mean pairwise overlap | 0.22 | 0.22 | [0.18, 0.27] | 0.49 P(obs ≤ null) |
| Fraction of pairs with zero overlap | 86.2% | 79.2% | [75.2%, 82.8%] | < 0.0001 P(obs ≥ null) |
| Max times any single frequency appears across seeds | 5 | 4.9 | [4, 7] | 0.69 P(obs ≥ null) |
With thirty seeds the story is more precise than with three. No frequency is preferred; the most any single frequency appears across seeds is five. Pairs with zero shared frequencies occur at 86.2%, modestly above the random-null's 79.2% (p < 0.0001). The circuit is real, and the specific frequencies each seed settles on are essentially arbitrary.
Three things show up in the larger sample. The mean pairwise overlap is 0.22 shared frequencies per pair, which matches the random-null mean (0.22) exactly. No frequency is preferred across seeds: the most any single frequency appears is five out of thirty seeds, which is what the random null predicts. The fraction of pairs with zero overlap is 86.2%, modestly above the null's 79.2% (p < 0.0001) - a real but small dispersion effect.
Taken together: the circuit is real, and the specific frequencies each seed settles on are essentially arbitrary. Most pairs share nothing not because the model is avoiding shared representations, but because random selection of five things from 113 naturally produces mostly-disjoint small subsets. The strong form of the claim - that seeds build "entirely different" encodings - was overstated at n=3. The weaker form - that no particular frequencies are structurally preferred, and the specific Fourier basis each model lands on is arbitrary - holds cleanly at n=30.
(Just for fun, here is the graph at n=300)
| Statistic | Observed | Null mean | 95% CI | P-value |
|---|---|---|---|---|
| Mean pairwise overlap | 0.224 | 0.223 | [0.219, 0.228] | 0.59 P(obs ≤ null) |
| Fraction of pairs with zero overlap | 85.4% | 79.2% | [78.8%, 79.6%] | < 0.001 P(obs ≥ null) |
| Max times any single frequency appears | 23 | 23.4 | [21, 27] | 0.65 P(obs ≥ null) |
With three hundred seeds the picture is the same as with thirty. The zero-overlap fraction is still modestly elevated above the random null (85.4% vs 79.2%), but the mean pairwise overlap and the maximum single-frequency count both sit at what uniform random selection would produce. The Fourier circuits are real and they are seed-specific; the specific frequencies each model settles on are essentially arbitrary.
One further check protects everything above. Across the programme we ran the same control on every cross-probe result: take an untrained network of the identical architecture and probe it identically. In the board game domains that control was deflating, a network that had learned nothing read the board nearly as well as the trained ones, because the board comes with the input. Here it is the opposite. An untrained network decodes (a+b) mod 113 at exactly 0.0%, below even the 0.9% chance floor. The decodability of modular addition is created entirely by training, so the privacy these probes report is real and control-verified: the seeds built genuinely different learned circuits, not one input signal viewed from different angles.
Even if seeds diverge, perhaps a model that has learned modular addition of some kind should be able to minimally transfer to a different modulus. The rule is the same, but the wrap point differs.
We tested the mod 113 model on (a + b) mod 127. The raw accuracy was 40.4%. This looked, at first at least, like a real transfer.
But since your solo researcher is neither an AI researcher nor a mathematician, there was a confound. For any pair where a + b is less than 113, the answer is identical under both moduli: the wrap point is irrelevant because the sum doesn't reach it. The number of in-vocabulary pairs where this happens is 6,441 out of 16,129, or 39.9%. That is not roughly the observed accuracy; it is the observed accuracy. The 'transfer' had precisely the magnitude you would expect if it were arithmetic overlap and nothing else.
So in a second analysis we restrict the test to the 6,328 pairs where the two moduli give different answers.
The mod 113 model's accuracy on the mod 127 task: 0.00%. Not a single pair. The model produced (a + b) mod 113 with 99.56% accuracy and the mod 127 answer never. This seems to be the signature of having specifically learned (a + b) mod 113.
We tightened the test once more. Of the 6,328 pairs, we kept only the 4,851 where the mod 127 answer sits inside the model's output range, so the model could in principle produce it. Same result: 0.00%. Not one pair.
The same test from addition to multiplication: 0.88%. Exact chance. Multiplication to addition: 0.86%.
Maybe the circuit itself is real even if it doesn't transfer. We can try to test this directly. Activation patching is a technique for probing internal computation: we run the model on an input, reach into its internal state, and swap out the part that represents one of the numbers. If we swap b for b + 1, and what the model is doing really is addition, then its output should shift by exactly one.
That's what happens. The output shifts by one, cleanly, with 100% reliability. We also looked at the probability the model assigns to a near-miss, an answer off by one in either direction. It is zero, to six decimal places.
Gaussian noise applied to the input embeddings, as a fraction of the embedding's typical norm.
A genuine arithmetic algorithm should be insensitive to a small distortion in its inputs; the rule 'add the two numbers, mod 113' doesn't care if a is represented slightly imprecisely. But it seems that perhaps the Fourier circuit isn't an arithmetic algorithm. Instead, it looks more like a precisely tuned distributional pattern, depending on exact relationships between exact frequencies.
Distort the inputs and the pattern degrades. It seems the cleanness of the circuit and its fragility are not in tension; they are the same property seen from two angles.
The Fourier circuits are real in the sense that they are discoverable and compute. In their own way they are a beautiful demonstration of the extraordinary power of statistical compression to find the most efficient encoding and decoding of a problem. Another demonstration of why we should not take transformer based models lightly. But they don't show the characteristic qualities of an algorithm, a learned operation that can be transferred.
The grokking experiments are, for me, beautiful maps drawn in trigonometry.