OpenAI and Tartu Innovate in AI Educational Assessment

⚡

Key Takeaways

1OpenAI and the University of Tartu have developed the Learning Outcomes Measurement Suite to assess the impact of AI on learning.

2This framework aims to address the shortcomings of current methods that focus on narrow performance signals such as exam scores.

3Randomized controlled trials are underway to validate this framework, with participation from partners such as Stanford and MIT.

💡Why it matters — This project could transform the way educational systems integrate AI, providing a more comprehensive understanding of its impact on learning.

AI and Education: A Revolution in Progress

Education is one of the sectors where artificial intelligence (AI) promises to radically transform learning methods. Tools like ChatGPT offer personalized support, accessible to any student, regardless of location or time. However, the education sector is still in its early stages of understanding the impact of AI on learning outcomes.

Last year, a team of researchers examined the use of tools like the study mode. The results showed promising improvements in student performance. However, a crucial question remains: how can we assess the influence of AI on learners' progress over time, and not just during a final exam?

A Challenge for the Educational Ecosystem

Currently, most research methods focus on narrow performance signals, such as exam scores. These methods do not allow for an evaluation of how students actually learn with AI in real-world contexts, nor how this usage shapes learning outcomes over time.

To address this gap, OpenAI, in collaboration with the University of Tartu in Estonia and the SCALE Initiative at the Stanford Accelerator for Learning, has developed the Learning Outcomes Measurement Suite. This framework aims to support the longitudinal measurement of learning outcomes across different educational contexts.

Validation and Future Research

A thorough validation of this framework is underway through a randomized controlled trial. Additionally, further research is planned with the founding organizations of the Learning Lab, OpenAI's research ecosystem on learning. This includes researchers from Arizona State University, the UCL Knowledge Lab, and the MIT Media Lab, building on previous collaborative studies.

Today, OpenAI shares an insight into how the Learning Outcomes Measurement Suite works and its significance. Ultimately, the goal is to publish more research and make this framework available as a public resource for schools, universities, and educational systems worldwide.

A Look to the Future

The use of AI tools by students can take various forms, ranging from seeking quick answers to problem-solving with tutor-like support. To encourage deeper interaction with ChatGPT, OpenAI introduced the study mode last year.

This mode is powered by customized system instructions, designed in collaboration with teachers, scientists, and pedagogy experts. These instructions reflect fundamental behaviors that support genuine learning, utilizing learning structures, comprehension checks, and guided practices.

To test the effectiveness of this pedagogical interaction style with AI, a randomized study was conducted with over 300 students preparing for exams in neuroscience and microeconomics. Although the analysis is still ongoing, initial results are encouraging. They suggest that this style of interaction, fostered by features like study mode, can enhance learning outcomes.

Thoughtful Integration of AI

According to Susanna Loeb, Professor of Education and Director of the SCALE Initiative at Stanford University, this research is essential for understanding how AI can be thoughtfully integrated into schools. The goal is to support rigorous academic learning while cultivating critical thinking, creativity, curiosity, and students' self-confidence as learners.

By making the Learning Outcomes Measurement Suite publicly accessible, OpenAI hopes to provide schools and universities around the world with a valuable tool to assess the impact of AI on learning.