StudentSim: Simulated Students to Train AI Tutors

Microsoft and the University of Illinois present StudentSim, a method for creating replicas of students from minimal data and making them respond to a tutor's guidance. Tests report better performance than GPT-5.4 in chess, English, and mathematics, with a proof of concept evaluated by professionals. This is a demonstrator, and the code is published.

A chess proof of concept achieves top scores

Professional chess players evaluated three versions of a tutor: one without additional training, one trained with GPT-5.4 as a student, and one based on StudentSim. Among these versions, the one trained with StudentSim stood out in three aspects: it displayed the lowest number of serious factual errors, and received the highest ratings for the quality of explanations and the ability to adapt to each learner. During this study, the learner showed a preference for questions that guided towards the solution rather than direct instructions. In contrast, the tutor trained with GPT-5.4 exhibited lower factual accuracy compared to the version without additional training. This is an experimental demonstration and not a claim of having designed the optimal tutor. The choice of chess as a testing ground is explained by the ability for a engine to objectively evaluate the relevance of a move in a given situation, while grading written essays or certain math responses remains difficult due to the lack of reliable tools for open-ended answers. The system's code is available on GitHub.

With minimal data, StudentSim learns in two stages

The lack of data is the main obstacle. In an English writing corpus, the median student has only written three essays, and more than two-thirds have five or fewer. Training a replica directly on so few examples leads to overfitting, according to researchers. StudentSim operates in two steps: first, a base model learns from all students in a given subject, capturing common errors and how responses are revised after a tutor's hint; then, this model is adapted to each student with their scarce recordings. In all tested subjects, Alibaba's Qwen3-4B-Instruct serves as the foundation.

Against GPT-5.4, the personalized approach performs better

The method was evaluated on 60 students in chess, English as a foreign language, and mathematics, using public datasets from real learners. StudentSim outperforms GPT-5.4 in these three disciplines when the latter is configured to simulate a student's behavior. For chess, it correctly anticipates a player's next move about twice as often and almost systematically applies the proposed corrections, while GPT-5.4 and chess-specialized models achieve lower performance. Researchers note that each category of method has its own limitations: GPT-5.4 considers hints but does not reproduce the typical mistakes of a learner, while chess-specialized models mimic a player's game but do not take verbal cues into account. In a similar situation, three human players opted for three different moves; StudentSim was able to replicate each of these choices, while a chess model consistently proposed the most probable move and GPT-5.4 made mistakes in all three situations.

In the ecosystem, pilots and limits of personalization

Microsoft is also conducting AI tutor experiments with students in real conditions. In Nigeria, a pilot project allowed students to use Copilot twice a week over a six-week period, with progress on tests equivalent to nearly two additional years of learning. Meanwhile, OpenAI and Google offer Study Mode and Guided Learning, which rely on system instructions and models tailored for teaching, without considering the individual learner's profile. In the absence of this adaptation, AI assistance can hinder performance: studies indicate that users perform worse after brief AI help than those who work alone. Other research also shows the limitations of replicas: around 1,000 real people were replicated in 2024 through two-hour interviews, and a separate study reports errors; nine open models imitating users on X, Bluesky, and Reddit became less accurate as they appeared more human.

Goal: replicas that make mistakes, learn, and react

The system designed by Microsoft and the University of Illinois generates realistic student replicas from limited data to provide quick feedback where collecting from real students would be too costly and slow. AI tutors benefit from adaptation to individual strengths and weaknesses, but determining the right guidance requires time and resources; training a tutor with a wide range of students is deemed prohibitive, which hinders progress compared to AI models. Researchers therefore propose using distinct digital replicas for each student. They argue that current approaches only address one out of two competencies: faithfully reproducing behavior without understanding the tutor's explanations, or following advice without matching the abilities of the imitated student. StudentSim formalizes these two capabilities into measurable objectives, evaluating the correspondence to typical responses and errors as well as revision after assistance. Training tutors requires both a realistic starting point and a replica that reacts to instructions. The team plans to extend its work to model the acquisition, retention, and forgetting of knowledge over multiple sessions.