Same LLM, Different Results: The Role of Workflow

Practitioners accessing similar models achieve divergent results. The difference lies not only in the model itself but also in the organization of work around it, from the initial framing to responses to unsatisfactory outputs.

The Work Framework Weighs More Than the Model's Capability Alone

The mere capability of the model is insufficient to explain the differences in results. It is the way the task is framed, the preserved context, the actions assigned to the AI, and the manner of responding to unsatisfactory outputs that play a decisive role.

Two Illustrative Approaches: Explanatory Fable and Toolchain

Amanda Askell uses a prompt that asks the AI to explain a complex idea in the form of a fable before naming it. Andrej Karpathy integrates language models into a broader workflow, combining web research, file analysis, Python, artifact production, and coding. These methods lead to very different ways of working around the same type of model.

A Common Foundation, A Persistent Question

Although users have access to similar models, one question persists: why do we observe such discrepancies in results? This inquiry accompanies two years of practice with AI agents and the observation of profiles exploring various uses of these tools.