Design and AI: From Mockup to Evaluation Set

Treating AI as a material requires executing models, observing their variability, and recording failures. Concrete practices are emerging, supported by low-code or no-code tools and increased designer involvement in development, while research and positions clarify its limitations.

Repeated Trials to Capture Model Variability

Sampling code can serve as a testing ground, allowing for the recording of instances where a model breaks a design. Unlike wood, which retains its grain, a model can change after a provider update. Engineers refer to this recording as an evaluation set, and designers are well-positioned to draft it because they identify failures that first impact users. A design adjusted to a model the previous month may fail without any interface changes. To reveal these discrepancies, systems must be executed repeatedly: variation, errors, and delays are only discovered by running the model, which returns different responses each time. A mockup is merely a sample, while the actual product is a distribution of outputs. Some time constraints escape static mockups: a software conversational threshold is set below 400 milliseconds, and a four-second wait is not visible.

Four Habits to Turn Outputs into Practice

Four habits are proposed to make these findings operational. First, call the model directly by requesting a script via Claude Code or Cursor that sends real produced inputs and displays the responses to read before designing the screens. Next, execute each prompt 50 times, sorting by length, tone, and accuracy, then design for the longest and most incorrect response. In review, place the two worst of the 50 outputs next to the mockup to collectively decide what the interface must endure. Finally, record each failure as a test case in a shared file to replay it with each model or prompt change. The sampling code is disposable: it can be discarded on the same day, its role being solely to reveal the material.

AI Tools and the Empowerment of Designers

The distance between designers and material is shrinking. According to Figma, the share of designers involved in development has doubled in a year, rising from 21% to 41% by 2026. Tools like Claude Code, Cursor, and Figma Make allow for describing a prototype, connecting it to a live model, and observing what breaks in a single session. Some studies have shown that it is possible to equip practitioners without code, while today, for language models, a few lines are enough to access practice. In 2023, Geoffrey Litt argued that language models would enable ordinary users to create small tools. The author describes the historical distance of designers from the material as a bottleneck.

Contrasting Positions on the Role of Code in Design

Some voices remind us of the limits of code use by designers. In 2017, Alan Cooper wrote that coding is not necessary to be a high-level interaction designer. In 2025, Karri Saarinen at Linear warned that production-centered habits could weaken the thinking that supports design. These warnings target code intended for shipping, not the disposable scripts used to explore model behavior.

Precedents: From Bauhaus to Web Editors

Learning through material has established milestones. At the Bauhaus, Walter Gropius's April 1919 manifesto directed towards crafts, with workshops dedicated to wood, metal, glass, or weaving, and elementary exercises by Josef Albers to test properties. Marcel Breuer led the woodworking and bent tubular steel into chairs, while Herbert Bayer oversaw printing and advertising. With the web of the 1990s, the tool was the text editor, and the cycle consisted of entering tags, saving, and then reloading. The following years saw the emergence of BBEdit, HoTMetaL, PageMill, FrontPage, HomeSite, and Dreamweaver. Designers combining visual editors and code inspection validated across multiple browsers, with the editor presenting the best case when the markup exposed the rest. In 2000, John Allsopp described the difference in control inherent to print; ten years later, ethan marcotte demonstrated responsive design through fluid grids, flexible images, and media queries; Frank Chimero later advocated for the idea of a "grain" of the web. These references place AI within a continuity of learning through material.

Recent Studies: Framing AI as Material

Several works have framed AI as a material. In 2017, Lars Erik Holmquist proposed considering technology with its own limits, and Graham Dove and his colleagues reported little innovation in design for machine learning, attributed to its novelty and difficulty. In 2020, Qian Yang linked obstacles to uncertainty about system capabilities and the complexity of outputs. Bret Victor, in 2012, highlighted the need for an immediate connection with what one creates. Josh Clark, in 2018, categorized machine learning alongside HTML and CSS as a design material. In 2023, a study by K. J. Kevin Feng and David W. McDonald involving 27 practitioners showed that practical experimentation helped align model-target behavior, invent corrective mechanisms, and detect ethical risks; one participant noted that in the absence of testing, the frequency and manner of errors remained unknown. The authors identified the production of models, often out of reach for designers, as a bottleneck.