Triptych

I've been working on this personal side-project, a triptych of ideas about AI, on and off for a little over a year. It explores, and then tries to offer, a different way of thinking about what AI systems do and what they are. At its heart the project is about two things: how language itself shapes how we describe and experience machine cognition; and the role that time itself plays in how humans and machines think.

The first part introduces the concept of 'decompression', the way in which we ascribe intelligence and experience to the outputs of AI models. The second tries to examine the roots of this in a historical context, tracing a line from Helmholtz through to Hinton. It asks how this context might have influenced our investigation of neural networks, and suggests a different way of thinking about them: distributional surface theory (DST). The third and final part, Beethoven and the Bitter Lesson, tries to unpack all of this in the context of time and cyclicality.

This triptych started out with a hobby experiment: trying to train a transformer model to write better metaphors.

The extension and blending of language is one of the purest expressions of human creativity. Metaphor makes vivid the connections between words, compounding the effect of reading or hearing them as they are superimposed, compared and contrasted.

It seemed like fun to see if I could get a neural network to do something similar.

The experiment went something like this. I took the 2,500 most common words in the English language and for each of them generated a list of 'features'.

Each feature for a word was a word, or phase, generated against a list of 52 domains. These domains tried to capture the underlying categories we associate with language; the conceptual spaces we connect to its individual pieces.

To give an example, here are three words from the final dataset, 'snow', 'light' and 'blood', and a small subset of their domains:

Despite being so different individually, you can see they share features; both in terms of their close similarities and their diametric contrasts.

All three include the features 'innocence' and 'purity'. Light and snow share 'clarity', the conceptual opposite to blood's 'rage'. Both snow and light are also evocative of the idea of 'blinding' and 'whiteness'.

The way a poet blends these domains together is what gives their combination a specific effect. You can see this, using the precise words above, in Shakespeare's Othello, at a high point in the play's tragedy:

'Yet I'll not shed her blood, / Nor scar that whiter skin of hers than snow, / And smooth as monumental alabaster. / Yet she must die... Put out the light, and then put out the light.'

It is the collapsing of the space between these words, the tying up of the conceptual, rather than literal, understanding of what they mean, that gives the writing a bodily force. The power of metaphor is in this blending.

So the idea was to replicate this 'physics of language', the gravitational conceptual pull of words, not their statistical co-occurrence.

I knew that trying to embed conceptual knowledge was one of Richard Sutton's bitter lessons: "building in how we think we think does not work in the long run"; but since it was a personal side project, it seemed worth a try. Predictably, it failed completely.

To understand some of the why, I began reading about the 'internal world models' of transformer based AI systems. The research suggested that AI systems were developing forms of internal representation. If it was possible to understand how they were creating these, it might reveal what was missing from the metaphor experiment.

I started downloading and playing with the publicly available models used in academic research. A few times I trained my own versions of these, puzzling my way through the process. In parallel I began reading some of the history and sociology around computer cognition, trying to make sense of its historical roots.

'Distributional Surface' was what I began calling, for myself, the characteristic quality I experienced investigating these systems. In my metaphor work, any conceptual instantiation was washed out by the statistical surface created in the training process. Once the probabilistic relationships were mapped the models seemed to always follow a probabilistic path across that surface, always. Yet what they were doing had the flavour of understanding until you looked deeper. When you saw the outputs it seemed they had built a conceptual framework. This experience, the flavour of understanding, was what produced the work on the first section.

The 'theory' bit of DST, the second section, came out of trying to cohere more fully what I was finding. I was looking for a way of characterising what these models were doing, but also to connect what research had previously found to beliefs that were already latent in the field; sometimes going back to its inception. DST doesn't try to challenge the evidence of what researchers working on world models found, but perhaps to characterise it in a different way.

In the end, this characterisation brought me back, again and again, to time and temporality. The contrast between how humans hold, and then resolve, multiple ideas simultaneously and how AI systems build their outputs through sequential processing. The third section looks at this in the context of music, Gödel and quantum cognition.

The essays are designed to be readable for a non-technical audience (written mostly for fun, sometimes for friends and family). Alongside them are a set of technical 'experiments'.

I use the word experiments lightly, since I'm neither technical enough to do them on my own, nor in any way qualified to be doing mechanistic interpretability research. The irony will not be lost on anyone that almost all experiments were produced and analysed in collaboration with Claude Code. All the model checkpoints, scripts and files are available here.

I'm putting this out not because I think it has much chance of surviving technical scrutiny, but perhaps because it will be of interest to people trying to think about what AI systems are. If any of the work holds up as directionally, I hope it could be stimulating for more capable researchers to take a different approach to stress testing these systems. If AI ends up being the most powerful technology we see in our lifetimes the questions it raises are worth trying to think, talk and write about; even the setting of a hobby project like this.

My personal belief is that the fear we have about the dangers these new forms of 'intelligence' and 'sentience' pose to humanity is probably misplaced. That they will be transformative, shocking for our economies and potentially very dangerous is not necessarily a function of either of those things.

If we understand AI as a technology perhaps it can help us build a path to understanding our own cognition and consciousness better. For the first time we have systems that can produce language, images and music, domains previously exclusive to humans. In how AI models produce these, and the ways they differ to how we do, we may begin to unlock negative definitions, ways of understanding the mind by subtracting what it is not.

I hope you enjoy the essays.