OpenAI: Karpathy Uses Claude Opus 5 to Create a 3D Scene

OpenAI: Karpathy Uses Claude Opus 5 to Create a 3D Scene

Andrej Karpathy used the AI model Claude Opus 5 to transform a single paragraph from "The Lord of the Rings" into a 3D scene in a browser, generating 5,500 lines of code in about two hours.

The entire process cost around ten dollars. The model autonomously placed and animated objects, but made mistakes as it could only verify its own video output through screenshots.

For Karpathy, who works at Anthropic, this test demonstrates the potential for custom, low-cost game worlds created on demand. He views the result more as an ambiance test than a true benchmark.

One paragraph from "The Lord of the Rings" as input, 5,500 lines of code as output. Andrej Karpathy asked Claude Opus 5 to transform Tolkien's opening into a 3D scene in a browser.

The model worked for about two hours, with a budget of one million tokens (approximately 10 dollars), and rendered everything in Three.js, a graphics library for 3D in the browser. Karpathy published the source code online so that others could replay and modify the scene.

Karpathy argues that the famous pelican test is nearly saturated. Simon Willison's fun benchmark asks a model to draw an SVG of a pelican riding a bicycle, a prompt chosen to be absurd enough that the subject does not already exist in the training data. The idea goes even further back. In 2023, Microsoft asked GPT-4 to draw a unicorn in TikZ as part of its Sparks of AGI paper and presented it as evidence of spatial reasoning. At that time, GPT-4 was only trained on textual data, unlike the multimodal models that now power every major chatbot.

Karpathy describes the result as rough but fun. Eleven Labs provided the audio, while the model autonomously placed and animated objects. No one would build worlds like that by hand, Karpathy says, but with a model, it's almost free. He sees potential for on-demand game worlds where users could be integrated as side characters or protagonists.

One problem remains. The model cannot view its own video output and therefore had to rely on screenshots, which led to errors. Other users have asked Opus 5 to write entire browser games, ranging from first-person shooters to Minecraft clones. None of this produces true benchmarks. These are ambiance tests.

Karpathy is a co-founder of OpenAI and was responsible for AI at Tesla. He then led the educational startup Eureka Labs and has been working on pre-training research at Anthropic since May 2026.