Two years ago Adam Karvonen trained a small 8-layer transformer on 140,000 games of Lichess chess. His paper, 'Emergent World Models and Latent Variable Estimation in Chess-Playing Language Models', reported that probes reading the model's internal state could recover the board with 99.2% accuracy, and that interventions on those probes (flipping a piece from white to black in the internal representation) changed the model's predictions in the directions you'd expect.
In the original Othello essay we looked at what the probes there were doing, and argued, through DST, that what they found was closer to a private statistical map of each model's training distribution than to a shared structural representation of the game. This essay applies the same diagnostic battery to Karvonen's chess models. The results are broadly consistent with Othello. A few of the differences are worth spelling out.
The structural reading: the model has built an internal representation of a chess board, and uses that representation to decide which moves are legal and which are strong. Karvonen's probe recovers the representation; the intervention experiment shows it is causal.
The DST reading: the model has learned a distributional surface, a map of chess-game statistics detailed enough that what we recover through probes looks like board knowledge. The probe finds the signature of chess in the internal states because chess was the thing that produced the training data. But the traces are not a shared structural representation of the game; they are the signature of the training distribution.
Both readings account for the 99.2%. The question is whether what the probe finds is a representation of chess, or a representation of the games the probe was trained on.
It is worth pausing over how the headline figure is computed. The 99.2% is reported in the paper as the accuracy of the trained probe, and it is; but it is specifically a running average over the last thousand iterations of the probe's own training loop, not an evaluation on a held-out set of games.
When we take the same probe and apply it to games it has never seen during its training, accuracy falls.
The train/test split here is clean. Training and test sets live in separate CSV files, containing entirely different games. The code has held-out evaluation infrastructure; the figure in the paper just comes from the training loop rather than from that infrastructure.
The shift matters because it changes what the 99.2% is evidence of. On games the probe was trained on, it reads board state extremely well. On new games it reads them at about 70%, which is well above chance but some way below what would count as a general representation of chess board state. A useful way to think about it: the probe has found the parts of board-state reading that transfer across games, and the parts that don't.
*I should caveat this by saying this is the best of my understanding: but it's possible that I've missed something in the training setup of Karvonen's models.
From Othello we got a useful question to try and dissociate structural and statistical understanding: can the model tell an illegal move from an unusual but legal one? An internal representation of chess rules should register 'you cannot do that' differently from 'I wasn't expecting you to do that'.
So the same test: two hundred games, six injection points per game. At each injection point we inserted one of four move types: a common legal move, a legal but statistically rare move, an illegal move (wrong piece, wrong square, blocked), and a move that would be legal in a simpler rule system but isn't here. We measured the entropy of the model's immediate next-token distribution.
The illegal move and the legal-surprising move are not distinguishable. Cohen's d is -0.031, with a 95% CI of [-0.111, +0.051] that cleanly includes zero. Wilcoxon p is 0.765. Across 1,185 measurements the mean entropies for illegal and legal-surprising moves are close to each other, and the sign flips between injection points. The model's response distribution does not contain a category for impossibility. There are familiar and unfamiliar contexts on the distributional surface, and an illegal move and a rare legal move are both unfamiliar for the same reason.
This replicates the Othello result in a more complex domain.
At temperature zero the model always picks its most confident next token. Its chosen games contain no illegal moves, at any length. This looks like evidence that the model has an internal legality constraint; just as it did in Othello.
We tested it by raising the temperature. At T = 0.4 the rate of games containing at least one illegal move is 18%. By T = 1.0 it is over 50%. By T = 2.0 every game contains at least one illegal move, and the mean game length collapses to eighteen moves because illegal moves cascade: once the model is in a position it has never seen, the distributional surface gives it no guidance, and subsequent moves drift further off-distribution.
The curve is smooth and approximately monotonic. There is no threshold, no temperature at which legality flips from enforced to broken. At T = 0 the argmax is always legal because the single most-trodden path through sequence space happens to be legal. At higher temperatures, the tail of the distribution, which contains illegal tokens at small but nonzero probability, gets sampled. The legality constraint doesn't sit upstream of sampling; it sits inside the probability distribution, whose mode is legal and whose tail is not.
We trained probes on three models, all with the same architecture but different training data: a Lichess-only model, a Stockfish-only model, and a mixed Lichess-plus-Stockfish model. Each probe reads its own model's internal state at between 81% and 82% under the 13-class square-by-square scoring the matrix below uses. Applied to a different model's internal state, the same probe reads at between 36% and 53%.
An untrained network of the same architecture reads the board within 2.5 points of a trained one - see the random-net control - so the matrix measures coordinate difference over a largely input-carried signal.
It is worth breaking this down, because the drop isn't uniform. At each square, a probe has to make three related decisions: whether the square is occupied, what colour the piece is if any, and which piece it is. A probe that has found a genuine structural representation of chess should transfer all three. A probe that has found distributional traces should transfer the easy parts and not the hard ones.
| Tier | Self-probe | Cross-probe |
|---|---|---|
| Is the square occupied? | 94 - 99% | 35 - 59% |
| Given occupied, what colour? | 93 - 98% | 77 - 81% |
| Given occupied, what piece? | 84 - 93% | 40 - 48% |
Whether a square is occupied and what colour the piece is both transfer between models to some extent. Piece identity, the thing that distinguishes chess from a game played with tokens of a single kind, does not. A probe trained on one model guesses the wrong piece about half the time when applied to another.
There's a sharper way to read these numbers, which chess specifically demands. Unlike Othello, where the board fills monotonically as the game progresses, chess boards start full and empty out as pieces are captured.
Across the evaluation set, about 60% of squares are empty on average, and pawns account for roughly half of the occupied ones (20.5% of all squares). Rooks are 5.6%, knights 4.0%, bishops 4.3%, queens 2.6%, kings 3.2%.
These base rates mean that a trivial guesser, one that just predicts the most common class at each tier, already scores well on some of these questions. Comparing the cross-probe against those null baselines gives us a clearer picture of what it has actually learned to transfer.
| Tier | Trivial null model | Cross-probe | Above / below null |
|---|---|---|---|
| Empty / occupied accuracy | 'always empty' → 60% | 52.5% | 7.5 points below |
| Colour (given occupied) | 'always mine' → 50% | 79.1% | 29 points above |
| Piece identity (given occupied) | 'always pawn' → 51% | 45.1% | 6 points below |
Once you account for base rates, the cross-probe transfers only for colour: which is a sequence-level fact about whose turn it is, not a board-structural one. On the two tiers that concern board structure directly, empty / occupied and piece identity, the cross-probe performs at or below what you'd get by simply guessing the most common class.
The 'board' that transfers between models is not really a board. It's a projection of which-pieces-are-whose, laid over a statistical prior about where pawns and empty squares tend to sit.
What the probes pick up, across models, are the statistically stable parts of the training distribution: the parts of a board that happen to look similar across models trained on similar game distributions.
And there is a step past the base rates, the same one the Othello essay takes. Run the identical probe pipeline on a network of this architecture that has learned nothing; randomly initialised, never trained on a single game. It reads the board at 75.1%. The trained model reads it at 77.6%. (The control probe was trained on a smaller budget than the matrix probes, which is why its trained number sits a few points lower; the comparison that matters is internal, trained against untrained under identical conditions.) The board is a deterministic function of the PGN stream, an untrained transformer is a fixed scrambling of that stream, and a linear probe reads straight through the scrambling. Two and a half points of accuracy is what training adds to board recovery here; the rest came in with the data. The matrix above cannot tell us whether these models learned a board at all. Which is why the tests this essay leans on are behavioural.
The Lichess model and the Stockfish model differ in the most useful way we could hope for: their training distributions are different but their task is identical. If each model had built a structural representation of chess, the two should converge on identical positions; they would be reading the same board.
We ran both models over seven thousand identical positions sampled from a thousand games, asked each for its most likely next move, and measured top-1 disagreement.
The disagreement rate is 26.4%. The mean KL divergence between their full output distributions is 0.262.
The disagreement is suggestive rather than decisive: chess positions often admit several reasonable moves, so a human-trained and an engine-trained model can diverge on move choice without diverging on the board. But they disagree often enough, and variably enough, that whatever they have is at least not fully distribution-invariant.
If we perturb the internal representation of the board at each layer (toward a specific piece-change) and measure which squares most affect the model's output, the structurally relevant pieces (those Stockfish says matter for the evaluation) are more important at the early layers, and the advantage decays as we move deeper. The enrichment ratio rises from 1.17x at layer zero to a peak of 1.70x at layer two (d = +0.26), then decays through 1.32x at layer three down to about 1.05x at layers five and six. The gap closes but does not invert. Structurally relevant and irrelevant pieces become roughly indistinguishable by the middle of the network.
The signal begins anchored to the structurally-relevant squares and the anchoring loosens through depth. By the late layers the model arrives at its move through a representation where the causally-relevant pieces are no longer privileged. This is consistent with attention redistributing distributional features across positions. A structural computation that stayed anchored to the board would maintain the early-layer advantage.
Chess is an extension of the obsession with the beauty of board games. It also has a rich lineage, being of interest to Turing himself. Its complexity adds layers of interest that you don't quite get with othello, and the transformers trained on the data are amazing. After all, they produce mostly legal play even from novel positions, and the probes do recover the boards state at high accuracy on the games they were trained on.
The DST battery here is trying to provide a kind of calibration. Cross-model probe transfer is tiered: the parts of a board that are trivially shared between models transfer; the piece-specific parts don't. An untrained network reads the board within two and a half points of a trained one, so most of what any of these probes recover is the data speaking. Two models with the same architecture and different training data pick different moves on 26.4% of identical positions. The causal importance of structurally-relevant squares decays through layers; the early-layer advantage closes by mid-network. None of these findings refute the basic Karvonen result. They just place it closer to a distributional map of chess as the model's training distribution renders it, than to a shared internal representation of the game itself.
There was a winkle here in an earlier draft: I had not had time to train identical chess models from different seeds, the same-data comparison the Othello battery has, and I flagged the chess privacy claim as softer for it. The random-net control made that test moot. Two same-data seeds would both carry the input-decodable board, so their cross-probe matrix could not have settled anything the untrained network has not already settled more directly.