What's Happening?
A recent study conducted by researchers from the University of Toronto, following students in 18 Tennessee middle schools from 2024 to 2026, indicates that most students did not want to use Khanmigo, Khan Academy's AI tutor. The study, circulated by the National
Bureau of Economic Research, found that while almost all students initially tried Khanmigo, many stopped using it when the AI refused to provide direct answers and instead offered hints or asked guiding questions. This behavior was observed despite the AI being designed to act more like a human tutor, withholding answers and guiding students through problems. The researchers, Philip Oreopoulos and Nina Low, noted that students largely declined to seek help for their confusion from the AI tutor. The study focused on low-achieving students who were at least one grade level behind their peers and received extra remedial math periods. Some were assigned to use Khan Academy with Khanmigo available, while others received traditional remedial math work.
Why It's Important?
This study highlights a significant challenge in the integration of AI educational tools into the U.S. education system: student engagement. While AI enthusiasts have promoted AI tutors as a solution to improve learning outcomes, particularly for struggling students, the findings suggest that the mere availability of such tools does not guarantee their effective utilization. The modest gains in math performance observed in students using Khan Academy with Khanmigo were no greater than those found in previous studies without an AI assistant, indicating that Khanmigo did not add significant value in its initial iteration. This raises questions about the design and implementation strategies for AI in education, emphasizing the need for tools that not only provide academic support but also actively encourage student interaction and sustained use. The findings could influence how educational institutions and technology developers approach future AI-driven learning solutions, potentially shifting focus towards more engaging and user-centric designs.
What's Next?
Following the study's findings, Sal Khan, founder and CEO of Khan Academy, acknowledged the low engagement with Khanmigo and stated that the company has already made changes to the AI assistant. The updated version of Khanmigo is now integrated directly into Khan Academy, rather than being in a separate tab, and its behavior is adjusted based on student interaction. If a student asks for help before attempting a problem, Khanmigo offers minimal hints. However, if a student gets a problem wrong, the AI automatically intervenes to offer assistance and work through the problem collaboratively. Khan Academy is also exploring further incentives, such as giving students 'credit' for using Khanmigo after making a mistake, allowing bot-assisted redos to count towards mastering a skill. Future research will likely focus on whether these modifications increase student engagement and, more importantly, whether consistent use of the AI tutor leads to improved learning outcomes.
Beyond the Headlines
The study's implications extend beyond the immediate effectiveness of Khanmigo, touching upon broader ethical and pedagogical considerations in AI education. The initial design of Khanmigo, which aimed to prevent cheating by not directly providing answers, inadvertently led to student disengagement. This highlights a tension between preventing misuse and fostering genuine learning through AI. The challenge lies in creating AI tools that are both effective and appealing to students, encouraging them to grapple with difficult concepts rather than seeking shortcuts. Furthermore, the study underscores the importance of rigorous, independent research in evaluating educational technologies before widespread adoption. The findings also prompt a re-evaluation of how 'help' is perceived and utilized by students, suggesting that the intrinsic motivation to overcome confusion might be a more significant factor in learning than the availability of advanced tools. The long-term success of AI in education will depend on its ability to adapt to human learning behaviors and motivations, rather than simply offering technological solutions.
