The Undiscovered Country

On a podcast with Lex Fridman, Demis Hassabis, the co-founder of Google DeepMind posited a thought experiment: an interesting test for AGI, he mused, might be to drop an AI model into 1900 and see if it could independently come up with the theory of special relativity.[1] He refined the idea later at IISc Bangalore, deciding that 1911 would be a better date, and that general relativity, that even more extraordinary accomplishment, would be the goal to aim for: "that's the kind of true test of whether we have a full AGI system".

At the heart of this is an equivalence that has been explored in the other DST essays: that the brain is a complex computer. Hassabis described his position in a Twitter back and forth with the AI researcher Yann LeCun, using the language of the field's godfather: "the human brain (and AI foundation models) are approximate Turing Machines."

The logic runs something like this: the output of the brain is a form of computation. Therefore any sufficiently powerful machine, operating on a computational basis, should be able to reproduce or exceed the computation of the brain itself. The theory of general relativity, then, should be tractable for a machine.

I'd like to suggest an extension to this test. If this AGI is a machine capable of synthesising novel ideas through a computational process equivalent to the mind, then this ability should extend to all domains where human knowledge has been produced.

For example, this AGI system should, dropped in Elizabethan England in 1600, be able to produce Hamlet.

The AGI system, transported back, would presumably have access to all the available knowledge of the time. Perhaps it would have done its Latin and Greek at Stratford Grammar school, filled out a commonplace book and read all of Livy. It would have studied Montaigne's essays, whose ideas appear in patches throughout Hamlet.

But as we try to place the AI system in this time, to visualise it producing the play, there's an imaginative gap. Shakespeare's language is so embodied in experience. Hamlet is a creative synthesis of the intellectual and emotional questions that arrive out of conscious experience itself; all done with language that stretches and reshapes past its common use. It's also a thing written so that people would perform it in front of other people; consciousness observing consciousness, in front of consciousness. It is deeply entwined with the human experience itself.

And yet, in the formulation of approximate Turing Machines, the play was generated through computational processes. So, if the framework holds, this imaginative gap is an illusion, and a sufficiently powerful AI system, dropped into 1600, should be able to produce Hamlet.

There are obvious objections to the shape of this thought experiment, and I'll try to work through them one by one, attempting to show why this extension to the test is valid.

The first objection would likely be that of equivalence: general relativity and Hamlet are not the same thing. One is testable scientific theory and the other is a piece of literature which doesn't make testable predictions about the world. The equivalence is malformed. Let me try and state the case:

In this formulation general relativity is a kind of compression. Einstein was resolving the equivalence principle - the puzzling identity between inertial and gravitational mass - into a geometric framework in which gravity is the curvature of spacetime itself. Mercury's perihelion shift, which Newtonian gravity could not explain, fell out as a prediction; so did the bending of light around massive objects, a phenomenon nobody had thought to demand an explanation for until the theory predicted it. General relativity is a solution to a problem presented by the universe.

Hamlet isn't anything like that, it adds something to the world not demanded by the universe itself, not 'constrained' by empirical reality. It does not 'resolve' any underlying tension in the experience of reality, either. The play then, is a fundamentally different thing to the theory: general relativity finds the simplest description that accounts for known phenomena, demanded by those phenomena; and Hamlet is a novel exploration of an experience.

If the test is whether a sufficiently powerful computational system can discover novel ideas in a constrained search space, then Hamlet isn't an equivalent.

The problem is that this treats the domain of the output, physics and literature, as evidence of a different underlying process. I would suggest that the universe did not 'generate' either problem. It is the puzzling quality of what it is like to be in the universe which created both the questions and the answers. Physics and literature are each areas of human endeavor that attempt to create and answer questions about experience. So we can see both general relativity and Hamlet as their own acts of compression: the synthesis of a new way of describing, interpreting and understanding the problems that we generate for ourselves by being in the universe, and how we can understand them.

Einstein synthesises the known physics of the time into a novel framework; this generates the theory of general relativity. Because this is 'science' we presume that it lives in a totally different category to literature; but why should it? Why can't physics and literature share resemblances?

General relativity itself is not a complete theory. It is, like all scientific theories, the best current supposition we have, with the language that we have created to represent the universe, of a series of phenomena we experience. Indeed the fact that general relativity and quantum mechanics have not been reconciled is evidence of the inevitable incompleteness.

Similarly Shakespeare is synthesising experience, generated by being inside the universe, into a novel description of that experience. When Shakespeare allows the consciousness of his character to reflect on the question of his own existence, he is compressing Montaigne, Livy, all the questions that have plagued humanity of what life and death mean (as well as what might come after), and synthesising these into a statement: "to be, or not to be, that is the question".

Hamlet himself, through his words, makes suppositions and statements about the nature of the problems that plague, and will plague, humanity. These statements have been stress tested against the human consciousness for more than 400 years, and we agree they are still important.

Both general relativity and Hamlet are understood as foundational but not complete, critical engagement and the birth of new works that deal directly with some of their themes and questions.

So the two share structural features though they are from different domains. And given these similar structural features, and the core supposition that humans are approximate Turing machines, the two must be the novel result of powerful computation over contained search space. Therefore, Hamlet is a valid equivalent to extend the test.

Even if a skeptic grants that the two problems do share an equivalent shape, it would be reasonable to make a second argument: they cannot be evaluated in the same way.

General relativity is stress tested against the universe itself, whether it 'generated' the problem or not. The 1919 eclipse expedition measures the bending of starlight the theory predicted, and it either matches the prediction or it doesn't. The compression can in principle be checked against the thing it compresses.

This means, in theory, that an AI system dropped in 1911 could generate thousands of candidate theories - notably different to how Einstein arrived at his theory of general relativity - and then stress test them against the available, empirical evidence. The search space has a verification method, however imperfect and human-mediated: does the theory more elegantly describe the phenomena being observed.

Hamlet, in contrast, has no external verification method. There is not an objective set of criteria which can be laid out, against which it can be measured as a perfect 'compression' of the human condition, or the condition of Elizabethan England, or as an expression of dramatic structure. The verification happens when a sufficiently complex entity encounters the piece of art itself, and recognises something in the experience of encountering it - which we loosely term as 'valuable' or 'important'.

So even if we grant that the two problems are solved in an equivalent way - through the synthesis and recombination of complex ideas into a single new set of ideas or statements - they do not share the same verification criteria. In the AGI general relativity question it is at least possible to see the brute-force search over the whole space of knowledge as a potential way of arriving at the general relativity outcome; how would this be at all possible with something like Hamlet?

But I think this might be another category error. Though there are distinct approaches for how a scientific theory and play are 'verified' they again share more features than might be immediately visible. Principally that they are 'verified' over time.

General relativity was not agreed as fact the moment Einstein published it: it was established empirically in stages, and then integrated into the frame of our understanding. It is still not 'perfectly verified'. As mentioned before, the complication of quantum mechanics isn't fully resolved; the unreconciled tension between these two still exists today. But yes, the theory is nonetheless testable under the following criteria: it describes a solution to the experience of the universe which is granted to be more faithful than the alternatives.

Now, let's take the same idea - a solution to the experience of the universe which is granted to be more faithful than the alternatives - and see if we can apply it to Hamlet.

Over the past four centuries the play has become the canonical example of what it means, in literature, to grapple with the experience of consciousness; this being a problem produced by being and experiencing the universe.

Its success is, to some degree, that it describes the observed experience more faithfully than its other literary alternatives.

So the verifiability question has a problem: it tries to privilege a certain kind of verification. It does not acknowledge that verification is a complex entanglement of the specific criteria each discipline uses and the sociological process of the societies that produce them.

In a sense then, the impossibility of 'verifying' Hamlet cannot be a limitation of the thought experiment. First (and again), if the brain is an approximate Turing complete machine, writing Hamlet can only be a different kind of problem to general relativity. If it's a different kind of problem inside the same search space, it must be admissible as a test. Second, if we dropped the AGI into 1600 and were able to observe its writing, we should be able to use the subjective verification criteria that elevated Hamlet on the outputs of that machine.

Whether we agree on verifiability doesn't resolve the problem of the space where the synthesis of novel ideas is being made. Do the search spaces of science and literature have any equivalence?

If we were arguing about special relativity, we could say the ideas were already in the field by 1905: Lorentz's mature transformations, Poincaré's mathematical formulation, Minkowski's geometric reformulation. The physicists of the early 20th century were converging towards something like special relativity - Einstein's being alive is not a necessary condition to get us to special relativity, or an equivalent theory. Therefore the AGI system would have a fair chance of synthesising what was pre-existing.

But general relativity is one of those singular things. It doesn't seem to have a framework partially built sitting underneath it. Some of the theory's predictions, including the bending of light around massive objects, were not even known to be problems demanding an explanation before the theory predicted them.

The same can be said for Hamlet. There isn't an obvious convergence towards 'to be, or not to be'. Yes there were playwrights like Marlowe and Kyd, but they were not converging towards writing Hamlet. If Shakespeare had died in 1599 we wouldn't have Hamlet.

Both Hamlet and general relativity sit in a special category, seemingly deeply embedded in, and wholly outside of, the sequence of knowledge in their disciplines. Both seem sui generis.

In any case, let's make a concession to the shape of the experiment, removing its specificity. Let's take a version of the test that Hassabis himself has posited. The AGI system in 1911 and the AGI system in 1600 do not need to produce the exact theory of general relativity or the exact text of Hamlet. Instead they are both tasked with producing something equivalent or better.

And here the problem crystallises. Again, if we take the supposition at face value, then Shakespeare and Einstein's brains were Turing machines. So this AGI system must be able to produce a work of Shakespearian or Einsteinian equivalence. But what is it searching for?

In the special-relativity case it would seem there is a value function that exists prior to the theory: the phenomena are there, the contradictions are there, and a candidate theory can be tested against them. The value function precedes the solution.

For general relativity this is less obvious. Some of the constraints existed; but the criterion the theory satisfies, "a geometric reformulation of gravity that recovers Newton in the weak-field limit and predicts new phenomena", was not a target before Einstein constructed it.

In the Hamlet case, the value function certainly does not exist before the work does. The criteria by which Hamlet is judged great, that a play could dramatise consciousness examining itself in real time, that dramatic language could carry philosophical weight of that density, that inaction could be the subject of a drama, did not exist before Shakespeare created them. The combined value function was created in the act of writing itself. The work reshaped the evaluative landscape. You would need to produce Hamlet in order to know what you were looking for, which means you can't have been looking for it.

And this circularity extends backwards. The convergence that preceded the work only looks like convergence because the work arrived. This is seductive: it makes us feel that discovery is not part of a complex socio-cultural process, but part of an inevitable advancement. But before each formulation, the space of possibility was open in every direction. The value function isn't just created by the work, it's projected onto the history that preceded it, making the convergence look like it was always leading here.

This is not a 'search space' problem, it is something else entirely.

So I would suggest the value function problem just described is not unique to Hamlet or to literature; and is exactly the same in the most complex problems of physics.

The initial value function is internal, created in the act of generation. In the case of general relativity, there was no Mercury-perihelion-crisis demanding a geometric reformulation; the reformulation is what made Mercury's perihelion legible as evidence in the first place.

And so we have a circular problem.

The AGI system dropped into either 1911 or 1600 needs to produce something that reorganises the search space it was produced in. It would need to generate an evaluative criteria while synthesising the novel answer. This is an act of simultaneity: stepping outside the boundary of what came before in order to reorganise it; in this frame production and evaluation are the same co-occurrent.

The line between what can be synthesised from the existing searchable space, and what can only be synthesised in the act of reorganising it is not always clear. There will be cases where sequential processing can blur the boundary.

But the AGI test applied to 1600 makes the problem vivid: the act of generating something genuinely novel and evaluating whether it is novel are not two separate processes. They are the same process.

So, generating novel spaces, that do not exist until someone creates them, is the ultimate test of AGI. Hassabis was absolutely right. Applying the thought experiment to the whole scope of human knowledge - making sure to include literature, music, philosophy - reminds us of the extraordinary capacity of the human mind, and how far the frame of the 'approximate Turing machine' must stretch to contain it.

In the final essay, Beethoven and the Bitter Lesson, I try to characterise why the imaginative stretch from today's systems to AGI is so difficult. Perhaps the gap between today's systems and one that could rewrite Hamlet, or rediscover general relativity, is not computational power, but the temporal modality that gives rise to simultaneity itself. Something that may not be available to sequential systems at all.