Reasoning Beyond Words: The Post-LLM Roadmap

A professor at UCL, a former key member of AlphaGo, argues that language models lack true reasoning. He advocates for an architecture inspired by the game of Go, centered around an explicit epistemic state and independent evidence control, for sensitive fields such as medicine or research.

Traceable Decisions Required in High-Risk Uses

In medicine, engineering, and scientific research, it is not enough to know the conclusion of a system: one must be able to trace the path that led there. When a diagnosis or treatment fails, it should be possible to identify whether the reasoning, the evidence used, or the assumptions are to blame. The proposed approach requires an independent entity that evaluates each step based on its actual contribution to reducing uncertainty and only allows belief updates that are supported by evidence. The desired systems must produce conclusions derived from verifiable sequences of evidence, inferences, and belief revisions, a condition deemed necessary for sectors like drug discovery, materials, climate, or diagnostics. Conversely, merely increasing the intuitive part of a system would not be sufficient to ensure the reliability of machine intelligence; scaling would refine intuition without making it more deliberative.

A Reasoning Architecture Centered on an Epistemic State

The proposed device relies on maintaining an epistemic state that catalogs what is established, what is in doubt, what has been dismissed, and the questions still open. Reasoning is viewed as a series of moves that modify this state to advance knowledge and reduce uncertainty. Recent advances in neural models, including LLMs, are considered tools for suggesting approaches based on available knowledge and resources, interacting with tools via APIs or code, and contributing to the evaluation of assertions in light of available evidence. With strict validation and update rules, the system could accumulate certified knowledge and improve its reasoning policy based on its experiences, in the spirit of an amplified scientific method aimed at knowledge resistant to scrutiny.

What AlphaGo Did Differently: An Explicit Game Tree

AlphaGo maintained an explicit record of what it knew through a game tree. Each move and position were annotated with judgments from its neural networks, and the tree was continuously updated throughout the reasoning process before synthesizing this information to choose the move to play. Thousands of future branches were explored, complementing the intuitions provided by the networks with structured deliberation. This articulation between intuitions and explicit exploration constitutes the core of what is presented as a genuine capacity for reasoning.

The Limitations of LLMs and Thought Chains

Language models operate by iteratively selecting the next token, functioning in a manner akin to a rapid and associative mode. Following the rise of ChatGPT, the introduction of thought chains brought improvements, particularly in mathematics and programming, but without establishing a separate deliberation mechanism: the intermediate steps remain generated by the same prolonged prediction. Three shortcomings are noted. First, the absence of an explicit, persistent, and inspectable epistemic state listing hypotheses, degrees of confidence, evidence, and pending questions, revised over time with new information. Second, the lack of separation between knowledge and manipulation procedures, which remain intertwined in the network weights rather than represented in a set of beliefs. Finally, studies have shown that thought chains can be produced retrospectively, sometimes in the form of narratives that do not reflect the actual calculation path.

Historical Matches to Frame the Debate

Deep Blue defeated Garry Kasparov in 1997 by evaluating 200 million positions per second while looking six to eight moves ahead for each player, according to rules coded by humans. Go presents a different complexity, as the value of a stone depends on group dynamics and territory over dozens of moves, to the extent that calculating even a fraction of the possibilities would require billions of years of supercomputer time. AlphaGo combined a policy network, which estimated about a 1 in 10,000 chance that a human would play move 37, with a search exploring a tree with thousands of branches. It retained this move even though it seemed implausible by mere intuition. This articulation recalls the distinction popularized by Daniel Kahneman between intuitive thinking and deliberative thinking. The series concluded with a 4-1 victory for AlphaGo, after a second match marked by a 37th move on the fifth line that led some commentators to believe there was a bug, when Lee Sedol later described the machine as creative.

The Author and His Journey

Thore Graepel, holder of the machine learning chair at University College London and a former key member of the AlphaGo team at DeepMind, indicates that he has recently left Google DeepMind. He claims to work towards ensuring that AI serves human flourishing. His position leads him to advocate for a new approach to machine reasoning, aimed at sensitive applications such as drug discovery, materials, climate, or diagnostics.