Bocconi: GPT-4o Improves Grades, Not Proven Learning

At Bocconi University, a randomized trial shows that access to GPT-4o boosts grades on a marketing assignment, particularly when a short lesson on causal reasoning alters the way students argue. However, researchers did not measure learning without ChatGPT. Other studies indicate improvements on assignments but declines in exam performance when AI replaces critical thinking.

No Testing Without AI and a Narrow Scope Limit the Findings

Students were not evaluated without ChatGPT after the task, meaning the study measures an improvement in grades but does not demonstrate increased learning. It remains uncertain whether the observed advantage comes from knowledge acquisition or better-assisted production. The experiment focused on first-year students from a single university and a specific marketing task. Random assignment was conducted at the level of 13 course sections, rather than individually among the 1,053 students. The main grades were assigned by human graders who were unaware of the texts' group affiliations, while indicators such as causal reasoning and idea diversity were assessed using models from OpenAI and Anthropic. OpenAI provided the technology used, participated in the research, and several authors are currently working or have worked there.

What Grades Reward in This Task

The grading rubric favored conventional responses. A greater output of ideas, coherence of arguments, and internal diversity were linked to higher grades. Conversely, increased falsifiability, in-depth explanations of how proposed actions would work, and a greater divergence from peers' ideas were associated with lower outcomes. For this task, structured responses that conformed to the expected solution space were primarily valued during grading. Diversity and originality should be explicitly integrated into the evaluation criteria for them to be truly considered, as current grading scales mainly assess the quality of form, organization, and completeness, without measuring learning or understanding.

Organization of the Trial and Quantified Gains with GPT-4o

In November 2025, 13 groups from an introductory management course were randomly assigned to one of four conditions: a control group, a group receiving a lesson on causal reasoning, a group with access to GPT-4o, or a group combining both interventions. Students were tasked with writing up to 180 words of marketing recommendations for the university shop, with the lesson covering causal logic, falsifiability, and cause-and-effect relationships. Access to GPT-4o resulted in nearly a full point increase on a scale of 1 to 5. Responses included, on average, about two additional ideas, increased logical coherence, and greater alignment with the recommendations of three experts. Even after accounting for argumentative quality, the number and diversity of ideas, as well as textual properties, a measurable advantage remained, attributed to better content quality rather than increased student knowledge.

The Causal Reasoning Lesson Changes the Nature of Responses

The brief training in causal reasoning did not increase traditional scores and even resulted in slightly lower average grades. However, it led students to explain why their proposed actions should work, consider possible failure conditions, and produce more diverse ideas than their peers. When combined with GPT-4o, it did not add points to traditional scores, but both approaches proved complementary on causal reasoning indicators, with increased diversity maintained in the group that received the lesson. Overall, ChatGPT alone primarily improves grades, while critical thinking training alters the type of reasoning and ideas mobilized.

Other Studies: Gains on Assignments, Drops in Exams Without AI

Previous research shows that the effect of AI depends on its use: when it supports students' thinking, the impact differs from when it substitutes for it. In the United States, an examination of over half a million college grades revealed an increase in top grades following the arrival of ChatGPT, particularly in writing and programming subjects that involve numerous assignments. Controlled environment studies show that once AI is removed, participants achieve lower results than a control group, with the decline being most pronounced for those who used GPT to provide ready-made answers. In China, a 30-month study involving over 26,000 students found that homework was completed faster and with better results thanks to AI, but exam scores dropped, with subsequent performance on entrance exams being 18 to 24% lower. Students who spent as much time on assignments as those not using AI, despite having access to the tool, did not show equivalent declines.

OpenAI Highlights the Assessment Challenge, Authors Call for Rewarding Reasoning

OpenAI presents these results primarily as a challenge for grading systems, as AI can generate polished work similar to that of an expert, reducing the final product's ability to reflect students' actual understanding. It is recommended to value originality, reasoning, and consideration of multiple approaches in grading scales, not just conventional or polished responses. Such an evolution of criteria would likely involve a broader transformation of the educational system.