Richard Sutton: Generative AI Fails in Science

Richard Sutton, a prominent Turing Award winner, has recently expressed his reservations about the ability of generative AI systems to make genuine scientific discoveries. According to him, these systems lack a crucial competency: the evaluation and development of their own results.

Large language models and other image generators, while effective at mimicking existing examples, often fail to innovate significantly. Sutton emphasizes that, although these models can produce impressive results, these are often due to the quality of the source material. In contrast, truly innovative results are rare and often unreliable, a situation he illustrates with a researcher’s joke: "This work is both innovative and good. Unfortunately, the parts that are good are not innovative, and the parts that are innovative are not good."

Nevertheless, Sutton acknowledges the usefulness of generative AI in areas such as summarization or entertainment, where innovation is not necessarily the primary goal. However, he insists that for science, imitation is not enough. Scientific discovery relies on a three-step process: variation, evaluation, and selective retention. Systems must generate options, test them, and retain those that work.

Systems like AlphaGo, AlphaFold, and Claude Code illustrate this ability to go beyond mere generation. These models incorporate evaluation loops that enable true creativity and discovery. For instance, a move in Go can be evaluated by its ability to increase the chances of winning, while a program can be tested for its ability to execute correctly.

Sutton also critiques the current direction of the AI industry, which focuses too much on increasing the size of language models without allowing for continuous and autonomous learning. He advocates for AI agents that interact with their environment and learn continuously, building internal models of the world and developing new strategies.

In his Oak architecture, Sutton proposes an approach where agents start without specialized knowledge, interact with their environment, and form abstract concepts over time. This approach requires reliable continuous learning, a challenge that current neural networks still struggle to meet.