What's Happening?
A study conducted by researchers at Harvard University has revealed that ChatGPT, a language model developed by OpenAI, significantly underperformed compared to doctoral students in a series of cognitive
tasks. The study involved students from Harvard's Principles of Molecular Biology course and aimed to assess the AI's ability to perform tasks at various cognitive levels. While the researchers hypothesized that ChatGPT would perform similarly to students on lower cognitive levels, the AI struggled particularly with 'apply' level tasks, which involve identifying and rationalizing experimental controls. The study found that students outperformed ChatGPT in 'remember', 'understand', 'apply', and 'analyze' questions, with the AI earning a 66 percent average compared to the students' 87 percent. The research highlighted the AI's difficulty in multi-step, compositional thinking, despite improvements in newer models.
Why It's Important?
The findings of this study underscore the limitations of current AI models in educational settings, particularly in tasks requiring critical thinking and application of knowledge. This has significant implications for the integration of AI in educational environments, where reliance on such technology could lead to gaps in learning outcomes. The study suggests that while AI can assist in certain areas, it cannot yet replace the nuanced understanding and problem-solving abilities of human students. This highlights the need for educators to design assessments that are robust to AI interference and to continue emphasizing traditional teaching methods that foster deep learning and comprehension.
What's Next?
The study's lead author, John Peters, suggests that educational institutions should focus on creating assessments that minimize the motivation to use AI, such as lowering the stakes of homework assignments. Additionally, Peters advocates for the use of oral exams to ensure that students' learning is authentic and not influenced by AI. As AI technology continues to evolve, it will be crucial for educators and policymakers to monitor its impact on learning and to adapt teaching strategies accordingly. OpenAI has been contacted for comment on the study's findings, which may prompt further discussions on the role of AI in education.
