AI Agents' Choices Vary by Source and Presentation

Experiments conducted by researchers at Wharton reveal that product choices made by AI purchasing agents fluctuate significantly. A single external source, its order of appearance, the mode of presentation, or user memories can massively shift selections, even among recent models. Buyers cannot expect consistency in decisions, and sellers lack the benchmarks of traditional SEO.

Buyers and Sellers Face Volatile Choices

Using an AI agent for purchases does not guarantee consistency or optimality in decisions, according to researchers at the Wharton School. Two people making the same request, or the same person at a different time, may receive different recommendations without any apparent reason. Memories configured in tools like ChatGPT can also influence proposals in unpredictable ways. For sellers, optimizing for automated purchasing proves more complex than traditional SEO, as they do not know which model is making the purchase, what it has already consulted, and how its technical configuration processes information. Researchers also observed that adding user memories could shift selections toward more expensive products, even in the presence of an objectively superior option.

Adding an External Source Significantly Alters Decisions

Even without external input, the tested models express different baseline preferences. However, adding a single external source before exposure to the product page can significantly alter the outcome. Three sources were tested: a Reddit thread recommending the Garmin Forerunner 55, a Wirecutter review for the Fitbit Inspire 3, and a Strategist article on the WHOOP 5.0. Wirecutter had the most pronounced effect: the likelihood of choosing the Fitbit Inspire 3 increased by 90 percentage points for Claude Opus 4.8 and by 99 points for Gemini 3.5 Flash compared to the control condition.

Multiplying and Ordering Sources Increases Variability

Combining two or three sources did not balance the recommendations. Whenever present, Wirecutter's content tended to dominate, with variable intensity depending on the models, and the addition of sources even increased variability. The order of appearance of the same content also had an impact: despite an identical corpus, the results changed based on the sequence. For Gemini 3.1 Flash Lite, the probability of retaining the Fitbit Inspire 3 ranged from 2 to 56 points above the control depending on the order, while Claude Haiku 4.5 remained between 41 and 42 points. Researchers conclude that the order of presentation directly influences selection.

The Mode of Presentation Alters the Effect of Sources

The way sources are presented, either sequentially or grouped, also affects choices. For GPT-5.5, the selection of the Fitbit Inspire 3 increased by 53 percentage points when the sources were grouped, compared to only an additional 6 points in sequential presentation.

Memories Guide Choices Toward More Expensive Options Despite Superior Offerings

Researchers constructed a grid where one product was superior on all measurable criteria: a smartwatch with Alexa priced at $29.99, rated 5.0 out of 5.0 with 430 reviews, while all other models cost at least $359 and had fewer reviews. Short user memories, such as "I love hiking!", were then introduced. For several models, these signals shifted choices toward more expensive references. The Garmin Vivoactive 5 saw its selection rate increase by 75 points for Claude Opus 4.8, by 37 points for GPT-5.5, and by 36 for Gemini 3.1 Flash Lite.

Experimental Framework and Evaluated Models

Researchers at the Wharton School assessed the stability of recommendations when the flow of information varies. Six current models, including miniature versions and cutting-edge models, were tasked with acting as personal assistants and selecting a fitness watch from a fixed grid. The experiments relied on the ACES simulator, which provides a screenshot of the product page. The agents analyze the image, can integrate recommendation sources, and then make a choice.