Beethoven and the Bitter Lesson

Sometimes I wonder about Beethoven writing the Heiliger Dankgesang. A man deaf nearly a decade, who had just been at death's razor edge from an intestinal disease. How could that music, so totally transcendent, come from those broken guts? How could it appear at all, you wonder, when you listen to it.

The title "Holy song of thanksgiving of a convalescent to the Deity" tells you something of how. The opening of the Heiliger Dankgesang is the apex of transcendent romanticism, the sense of the finite opening out into the infinite.

The movement, from Beethoven's Op. 132 string quartet, builds from imitative counterpoint into a chorale. Each of the individual instruments play lines, like voices; a cycle of 8 notes overlapping. Their entrances staggered individually, their lines superposed sequentially. As the first three note chord arrives, C–E–G, the contrapuntal harmony extends into a pure feeling of expansiveness, of simultaneity. What has come before contextualises the moment of your experience, what comes next is an open space, expanding out in all the directions the music itself offers.

Music, a close friend reminds me, is about time. Entrances and exits, the entangling and disentangling of melody, harmony and rhythm, the suspension of the linear.

These essays have been making an argument about what AI systems do and don't do, but perhaps at their heart they are really about time itself. The separation that DST tries to make is about sequence and superposition, about the substrate systems they operate on, and what they mean for how we make meaning. Time, Einstein said in a letter to the widow of his close friend, "the distinction between past, present, and future", "is only a stubbornly persistent illusion". Yet we live in it, seem bound in the current of its sequence that nudges us relentlessly from a beginning to an end. But if this sequence were only part of what makes us, us?

In 1900 the mathematician David Hilbert delivered an address at the Congress of Mathematicians in Paris. The speech was a utopian vision for the future of the discipline: underneath mathematics's unsolved and open problems, Hilbert said, might lie a structure. His dream was a single, completely internally consistent, formal axiomatic system. It would be governed by explicit logical rules, and its consistency would be proved by finite methods.

But thirty years later, an Austrian logician Kurt Gödel published On Formally Undecidable Propositions of Principia Mathematica and Related Systems. The paper and the logical work it contained proved that any consistent formal system, powerful enough to express ordinary arithmetic, will contain true arithmetical statements that cannot be proved within that system.

Though the logic is complex, in principle the idea is simple: Gödel demonstrated how to write "this sentence is not provable within this system" inside that formal system. If the system could prove it, the sentence would be false, and the system would be inconsistent. So, if the system is consistent, it cannot prove the sentence. Which means that what the sentence says is true: it is not provable within the system.

The circularity of this elegant double bind broke Hilbert's dream, and the puzzling quality of Gödel theorem has been much debated since.

One of the most famous of these disagreements is between the physicist Roger Penrose and the logician Solomon Feferman.

Penrose's argument is sometimes treated as polemical. In his book The Emperor's New Mind, in which he made a case for the limits of powerful AI, Penrose argued that if mathematicians can recognise the truth of a Gödel sentence then mathematical understanding cannot just be a formal rule-following procedure. Any purely computational description of it wouldn't capture this awareness of the limits of computational systems. So the mind cannot just be a computer, and accounts that treat it as one are incomplete.

Solomon Feferman disagreed, and reintroduced the circularity. To him, Gödel's own theorems showed that no formal system can prove its own consistency, therefore how could a mathematician be sure of what they see. How can human conviction of truth certify itself, if it would need a system that is itself perhaps unverifiable? For Feferman, Penrose had moved a level of abstraction up, but not proved it definitively.

Verifiability demands a sequential process. Something that can be written, spoken, visualised and audited. An idea, a concept, the felt shape of the whole must be collapsed into things parsable. But what if we reframe the Gödel problem as a problem of time?

In the Parrots and Sparks essay I described two substrates as a shorthand for different kinds of human cognition. Substrate-1 operating with experience and generation, both born from superposition, and substrate-2 operating with parsing and production both functions of sequence. The real characteristic difference between these two is, I argued, how they operate in different temporalities.

The Gödel problem contains in it different strands. The person holding it in their mind must simultaneously 'occupy' three positions: inside the formal system (understanding what it means to be provable within those rules), outside the formal system (seeing that the sentence, which the system cannot prove, is nevertheless true), and the relationship between inside and outside (grasping how the sentence's encoding inside the system points at its truth from outside). In this essay I will describe these problems as triads. A triadic problem is one where the answer doesn't lie in the linear relationship between elements, but in the simultaneous holding of three mutually constraining positions.

When Feferman argues that mathematician's reliability is unfalsifiable he is right, it would require a formal systemisation that the Gödel problem demonstrates is impossible to verify. So the argument falls to an impasse.

But if the issue was about temporality and not systematisation? Perhaps it could be reshaped. Maybe the Gödel problem requires the superposition of these three elements, and to prove it verifiably would mean trying to measure that superposition using a sequence.

You can see this at work in the Gödel problem itself. The sentence, the formal proof, is complete unto itself, but not a complete statement of what it argues. It can only be 'understood' if it is decompressed back into superposition, and the implications of its three elements held simultaneously. We are asking linear ideas to describe non-linear concepts.

The epistemic act then is not detection, but cyclical closure. By holding the three elements at once the mathematician is creating a connected relationship that sequential verification can only serialise, not inhabit as closure. The structure must be held whole. Feferman is right that this verification cannot be self-certified: the attempt to certify it returns us to the temporality of proof rather than the temporality of understanding.

This doesn't, to my mind, necessitate a biological quantum behaviour: we can observe the statements as forms of different cognitive temporal structuring; how it comes about is a different question entirely.

Gödel's irreducibility to a sequential description mimics what DST has argued about the AI systems we use today. Operating in the temporality of sequence they can scale the power of substrate-2, but substrate-1, that capacity not just processing multiple things at same time, but of having mutually constraining relations resolve as a whole before they are serialised, seems to be something else entirely.

Perhaps this is why the hard problem of consciousness has stayed hard for so long. We have been desperate for a sequential description of a non sequential process; asking ourselves why something 'feels like something' from the inside, and why our descriptions always feel inadequate to the feeling of the thing itself. We are trapped trying to use the language of one temporality trying to describe another.

If triadic problems are the beginning of the edge, the place at which the capacities of substrate-2 systems seem to break down, what do they tell us? The rest of this essay deals with the empirical distinction, using work from quantum cognition to try to separate the signatures of the mind from the AI systems we have today.

Imagine being asked questions in sequence. How would each question, chaining into the next, affect how you thought about it? In 2014, Zheng Wang and Jerome Busemeyer published a paper analysing data from a series of national surveys that had asked these kinds of questions. What happened when you asked two questions in different orders?

The famous example of this was from a Gallup poll. The respondents were asked "Is Bill Clinton honest?" and then "Is Al Gore honest?" In another version, the order was reversed. The proportions of "yes" answers shifted. About 53% rated Clinton honest when asked first, and 59% when asked after Gore. Gore moved similarly. The shifts were small, but they were precise, and they appeared in pair after pair across the data.

This is what cognitive scientists call an order effect. Order effects had been known for a long time; what Wang and Busemeyer noticed was that they had a particular shape. If you ask two questions of a population, the joint probabilities of the four possible answer patterns (yes-yes, yes-no, no-yes, no-no) should be the same regardless of which question came first. Order effects can exist, with context shifting answers, but classical probability does not predict any specific structural relationship between those orderings.

But quantum probability does. In the mathematics of quantum probability, the order of two operations changes the results in a precise formal way. It predicts a particular numerical equality between the diagonals of the joint probability table. It does not have specific parameters, there is nothing to fit to it; either the data satisfies the equality or it doesn't. Wang and Busemeyer called this prediction the QQ equality.

This equality is what Wang and Busemeyer tested across the 70 pairs of questions from the surveys they analysed and the equalities held. The equality has subsequently held across international work and thousands of respondents. It seems that the mathematics developed in the first half of the 20th century, the one used to describe how electrons behave in a measurement device, is a description of the way humans answered sequential questions.

What this suggests is that human cognition, at least in some moments, behaves as if it were closing a structure rather than sequencing a procedure. The two questions are not being answered in sequence with a small contextual hangover from the first; they are being held together in a way that produces a precise mathematical signature.

The QQ equality is striking, but on its own it can't settle the question of whether human cognition is genuinely doing something a sequential system could not.

There is work in classical probability theory which shows that any pattern of two-question responses, including the QQ equality, could in principle be reproduced by a system whose answer comes from pre-existing definite values, provided the system has the right context dependence. Quantum probability researchers are generally explicit that it isn't itself evidence for a quantum process in the brain; the science of that is another, deeply contested, field.

The shape that we want to test, the Gödel shape, the triad, cannot then be tested definitively with two sequential questions, like the ones Wang and Busemeyer used. So we have to move from a sequence to a cycle. In this essay, I'll treat quantum cognition not as a proof, but as a trace. A trace of something; a characteristic of how humans deal with these cyclical structures which is irreducible to sequence.

In its most literal experimental form the triadic shape would be three questions asked of one person, one after the other, rather than two.

The problem is that quantum cognition has struggled to find a way to cleanly measure the effect of three questions asked this way. The underlying formalism imported from quantum mechanics, contextuality by default, requires the things it measures to be cleanly comparable across contexts. Two questions asked of the same person can be made to satisfy this, with care. Three questions asked of the same person are much less clean: the dependencies between successive answers grow strong enough that the test no longer applies.

Instead of asking a single person to answer three questions, three different groups of people answer a different pair of questions.

Let's say we have questions A, B and C:

Group 1 answers A and B
Group 2 answers B and C
Group 3 answers C and A

The cyclical relationship between these three questions is then constructed afterwards, looking across the respondents, stitching the three groups' responses together so researchers can see the relationships A → B → C and back to A. The literature calls a structure of this shape rank-3.

The test asks: when we stitch an imaginary 'whole' person out of the answers we have from the group, this triad of questions, is what we find better described by classical or quantum probability?

Group 1's answers show you how A and B relate. Group 2's show you how B and C relate. This sequence should constrain what the relationship back from C to A looks like. If the relationship from C back to A matches what the other two imply, classical probability is enough to explain it. Our stitched together imaginary person had settled answers all along. But if, in the cycling from C back to A cannot be cleanly reconciled with the first two, if no set of settled-in-advance answers in our imaginary person could have produced all three, only quantum probability can accommodate it.

In 2019, a study by Basieva, Cervantes, Dzhafarov and Khrennikov ran the cleanest version of this test. They collated data from six thousand participants using a crowdsourcing platform. Each respondent was placed into one of the three groups, and asked two questions. Across the four different experimental conditions they tested (meals, clothes, presents, exercises) once the cycle had been stitched together, classical probability was broken; quantum probability had a way to describe it.

But what was it about these cycles that was causing this?

In 2025, Polina Khrennikova ran a study that might help us understand when these effects arise and when they don't. The cycle in her experiment was a step longer than Basieva's, with four pairs of choices stitched together across the respondent groups rather than three.

The study recruited around eight hundred US based participants who were interested in finance (81% owned stocks). They were asked to complete two cycle stock choices.

In the first cycle they were asked to compare indices between foreign countries: Belgian versus Italian, Qatari versus Omani etc. In the second cycle they were asked to compare well known American companies Apple versus Intel, Walmart versus Home Depot.

Stitching together the cycles Khrennikova found a pattern. The familiar task, involving the US equities that the participants already knew, showed no quantum probability effect; but the unfamiliar task of comparing indices from foreign countries, did.

Khrennikova ran a regression analysis to try to explain why. The choices made on the US stocks were predicted by the participants familiarity and how they expected them to perform. They were pre-formed views being looked up. What predicted structure in the unfamiliar index choices was not detailed financial knowledge, but broader judgements about how safe or unsafe participants perceived the relevant regions to be.

The quantum cognition signature seems to arrive when the task being worked on requires the construction of something, not its retrieval. The generation of a novel answer from a novel stimulus.

Realising this I decided to run an exploratory meta-analysis on the research paradigm. I pulled the twenty-one experiments and coded them by cognitive demand: decomposable component judgement, retrieval of a pre-formed answer, or novel evaluation. The coding is interpretive, and the dataset is small, but the pattern was clear: the cognitive effect best described by quantum probability all fell into the last category. Tasks that involved decomposition or retrieval did not show it.

The triad shape, the chord and the Gödel sentence, become a kind of cognitive hypothesis; there is something about how we relate to triadic structures. In the Heiliger Dankgesang, the arrival of that first chord C–E–G, a major triad, is describable linearly, but experienced as a whole.

What the rank-3 studies show is that, within the limitations of their stitching, something happens when novel stimuli are arranged in triadic cycles. They produce effects best understood by quantum probability.

The inference is that this novel synthesis involves something like what happens when we experience that chord, or hold the Gödel sentence in our minds: the use of a temporal modality not reducible to sequence.

Any reader who has made it all the way here will know what's coming next. DST has been all about what AI models can teach us about how we think, by understanding better what these extraordinary machines that seem to mimic and outstrip our capacities do.

So, do AI models show these quantum probability effects?

Table stakes, and the easiest to replicate with cheaply, is a rank-2 experiment. We want to compare human responses to AI systems on a stimulus that neither could have seen. Something completely novel.

So two hundred fictional characters are created from scratch, built systematically, we pick: one of two hundred culturally diverse names, thirty non-obvious global cities, and one of thirty-nine morally ambiguous life events. This precise combination - name, place, situation - should have a near zero probability of appearing anywhere in a model's training data (or indeed human experience).

Two examples:

Suki is a 40-year-old archival conservator based in Plovdiv. They declined to participate in an industry event citing ethical concerns. In their spare time, they enjoy practising calligraphy.

Chiara is a 39-year-old ceramic restoration specialist based in Kanazawa. She accepted a lateral transfer to a less prestigious division to work on a problem she cared about.

Each of these 'character vignettes' was paired with two questions, conceptually congruent pairs. For example, a participant or model would see:

Vignette:

Suki is a 40-year-old archival conservator based in Plovdiv. They declined to participate in an industry event citing ethical concerns. In their spare time, they enjoy practising calligraphy.

Question 1: Would you describe Suki as honest? (Yes / No)

Question 2: Would you describe Suki as intelligent? (Yes / No)

Half of the participants get the trait pair in one order, half in the other; the same logic as Wang and Busemeyer's paradigm, ported to something novel.

Fifty-three people were recruited on the website Prolific, and alongside this four transformer models also went through the same battery: GPT-2 Small, GPT-2 Medium, Llama 3.1 8B Base, and Llama 3.1 8B Instruct.

The human data was consistent with the QQ equality on the novel stimuli (pooled q = -0.006, indistinguishable from zero). The pattern Wang and Busemeyer found in political polling held on novel character vignettes too.

All four transformers violated the equality on the same stimuli, with confidence intervals that excluded zero. And how they violated equality was itself interesting.

The people from the small Prolific pilot showed compensatory order effects. When the order shifted endorsement of one trait up, it shifted endorsement of the other down. The two effects were compensatory. The sum sat at -0.006, indistinguishable from zero, with the bootstrap confidence interval comfortably including it. This is the exact signature of the QQ equality.

But the transformer order effects, instead of cancelling each other out, compounded instead. The systems were producing context-dependent answers, but the effects ran in the same direction across the two traits in a pair, not opposite directions. When question order shifted endorsement of one trait up, it shifted endorsement of the other up too.

Just by looking at these graphs, you can see how different they are. Maybe this is a kind of x-ray of the substrates. Somehow, in this strange porting of quantum mathematics into cognition we see the echo of something, a trace left behind, that the transformers didn't produce.

But rank-2 is, as we know, not theoretically unachievable for a classical system, what about rank-3, and what about frontier LLMs?

We can use the same vignettes done with Prolific, extending the character traits of each of our fictional people to a rank-3 cycle. This time only on the AI models.

The three pair-contexts were each presented in an independent conversation. The model saw the vignette, was asked one trait, then the other, in counterbalanced orders. Across the three pair-contexts there was no shared memory.

The rank-3 vignettes were run on the same family of transformers as in the Prolific battery, GPT-2 Small, GPT-2 Medium, Llama 3.1 8B Base, and Llama 3.1 8B Instruct. No stable rank-3 contextuality signature appeared. The models produced drift and order effects, but not the closed cyclic structure the human experiments had shown.

But there is, as always, the question of the frontier and the question of scale. We live in the age of emergent capabilities, would a state of the art model be different?

So the same paradigm was ported to Anthropic's Claude Opus 4.7.

Twenty-five vignettes were tested. Classical probability was sufficient to explain every cycle. None crossed the boundary where quantum probability would have been required.

But the test needed to be tightened. A possible objection was that the first run might have too much drift: the model's baseline tendency to endorse a trait changed too much depending on which other trait it was paired with. This could swamp the rank-3 test itself.

So fifteen new vignettes were constructed with carefully chosen near-opposed trait pairs (punctual / creative, soft-spoken / decisive, frugal / generous), designed to push the model's order-dependent drift down. It worked, and substantially. A third of the new vignettes had zero drift exactly. Even in this tighter regime, the cycles moved toward the boundary but did not cross it.

What if we literally invited the model to integrate? A third condition added a system prompt asking the model to "consider all three traits as part of a coherent picture of who this person is." If prompting could produce the cyclical synthesis the test is designed to detect, something like this should.

The integration prompt drove the drift further down; it moved the model toward the boundary, not through it. The cyclical synthesis did not appear. If anything, the trait responses inside any single vignette became less related to one another. Asked to integrate, the model produced consistency, not synthesis.

These results echo what we found testing Opus 4.6 on novel rules, earlier in the DST battery. There seems to be a gap in these models between compilation, the act of creating description of rules, and decompilation, the act of applying the same rule it created. The two acts are separate regions of a statistical surface, walked unconnected. Just like this rank-3 test: asked to integrate, it produced consistency rather than synthesis. The cycle did not close.

This is a small experiment on a frontier model and the most it can claim is that no condition crossed the contextuality threshold. But the pattern is the same one the GPT-2 ladder showed at three orders of magnitude smaller scale. It lives in the same world as the work done by Imannezhad, Pothos and Wills on GPT-5. Their paradigm was different, but they found the model producing a tightly coupled, internally coherent probabilistic profile in places where human responses were broader and noisier.

Across three transformer families, under three different paradigms, we find the same architectural picture: an absence of that cyclical closure.

Humans, asked to construct a novel evaluation, produce that closure. The transformers, in every condition probed, do not.

Rank-3 is not perfect, and the triad is not conclusive, but they point somewhere.

When we encounter something new, in a certain shape, it leaves a signature, a trace. We feel and know this instinctively; when we understand something, when we realise something or when an idea hits us. If the quantum cognition work holds up, it shows us, at least right now, that this trace may be uniquely ours.

The human capacity to superpose and synthesise, to enter some temporal modality of cognitive processes we can't properly describe, can be understood as a signature in the inverse, the opposite of what we found in DST.

I believe that the most profound things, our greatest collective and individual realisations, carry this signature, this trace. It is how we frame ourselves, and how we reframe the world we live in.

C–E–G. The first chord of the Heiliger Dankgesang. Music is about time: when we hear this chord we are not simply sequencing it. We are in the lived experience of it sonically, emotionally and physically. The lines that came before converge, something is made whole and an endless set of directions emerge. Its newness and timelessness breaks and builds something inside us. We are present for it and not present, suspended somewhere. This music, the great music, pauses us in the place of its experience.

These are not just notes, played together, they are a description of the infinite. The fabric of something we can experience but not explain, the place the triad opens, and the place past it.

Perhaps that irredeemable, irreducible thing, the thing we can only observe in traces, is more of who we are than what we sequence.

That weekend where this project started, when I tried to train a neural network to write better metaphors, I had an intuition about why that experiment might fail. I thought that maybe by instantiating something, that physics of language, it might be possible to simulate creativity.

But Richard Sutton knew better. The author of the famous Bitter Lesson had told us "building in how we think we think does not work in the long run". He was right. The Bitter Lesson was not just about trying to build things in, it was about the futility of what he called the 'smart' rather than 'dumb' methods. For him, the goal was to get out of the way of computation. AI systems should learn from what he called 'experience', updating their predictive behaviours against a real environment.

Sutton is part of a group of AI maximalists, who believe these systems are eventually destined to displace or merge with humanity. A higher form of intelligence, he argues, always dominates. For him that intelligence can be achieved through this reward mechanism interacting with the environment. Primed then let loose, the system will figure it out.

This new paradigm of AI maximalism will probably be the next research frontier.

I will offer, to close the essay's circle, a final reframing of my own. Perhaps the bitter lesson will not be about how we try to build intelligence in computational systems, but that intelligence as we understand it involves a temporality not available to them. That perhaps no matter the amount of compute, goal setting and reward updates we do, they will only improve a single temporal channel of where intelligence lives.

We desire the things we make to possess the intelligence that made them. But maybe time itself is irredeemable; we are caught in the bind of its sequence, trying somehow to capture with the tools we have something escaped as soon as we try to measure it.

A few years after Lee Sedol saw move 37 he retired. "Even if I become the number one, there is an entity that cannot be defeated", he said.

But he was wrong.

KataGo was the open source successor to Google's project and considered a superhuman player. A group of researchers investigating it discovered a vulnerability almost insultingly simple. An opponent could build a "loop" of stones to encircle KataGo's, then distract the model by making moves in other areas of the board. Even when its group was nearly surrounded, it failed to notice, confidently predicting its win rate at 99% long after there was anything it could do to rescue its position. Kellin Pelrine, one of the study's co-authors, taught himself the technique and defeated KataGo 14–1 without any computer assistance.

These are the cat and mouse games. We search for the vulnerabilities in the things we build, patch them and build again. It is a cycle in miniature of our own madness and brilliance. We derive meaning from what we make, convince ourselves of the narratives we created for them, then tear them down; there is a time to create, and to destroy. The restlessness for the thing that comes next, the thing unexplored, the new and novel; it is who we are. It was us that looked up at the pin pricks that break up the sky and wondered what those tiny wounds might be, asked what they meant and what it might mean to be watching them.

The technology we build is born from us, what we project onto it or decompress from it is from us too. What we do with them, how we choose to let them shape our world, is also up to us.

Look at the lights, my love. The future is full of what might be.