What's Happening?
The National Council on Measurement in Education (NCME) is hosting an event titled 'Teaching and Learning Measurement in the Age of AI: Metacognition, Evaluation, and Exercises.' Scheduled for September 30, from 1:00 pm to 2:00 pm EDT, this session is presented
by the Training and Professional Development Committee. The event will explore how large language models (LLMs) are changing the landscape of educational measurement. It will focus on methods for teaching and learning with AI, emphasizing metacognition and executive function, and addressing popular misconceptions that AI models might reproduce. Participants will be introduced to example exercises that utilize AI to build understanding rather than bypass it, such as collaborative coding with a model and tracing foundational study ideas.
Why It's Important?
This event is crucial for educators and professionals in the field of educational measurement as it directly addresses the impact of artificial intelligence on learning and assessment. The rise of LLMs presents both opportunities and challenges, making it essential to adapt teaching methodologies to ensure students develop critical thinking and problem-solving skills that AI cannot replicate. By focusing on metacognition and evaluating AI's reasoning, the session aims to equip learners with the ability to critically engage with AI-generated content. This shift in educational focus is vital for maintaining the integrity and depth of learning in an AI-augmented world, ensuring that future professionals can effectively leverage AI tools while retaining human expertise and judgment.
What's Next?
Participants will learn to identify the inferences AI models make, anticipate common misconceptions reproduced by AI, and adapt classroom exercises to integrate AI constructively. A key takeaway will be the ability to draft reusable 'AI skills'—short, specific instructions for LLMs that define the model's goal, standard, and procedure, thereby allowing the learner to maintain executive control over the task. The session will be led by Joseph H. Grochowalski, PhD, a Senior Psychometrician at the College Board and co-chair of the NCME Training and Professional Development Committee. This event signals an ongoing effort within educational measurement to develop best practices for integrating AI into curricula and assessment, with future workshops and discussions likely to follow.
Beyond the Headlines
The discussion around AI's role in educational measurement extends beyond practical classroom applications to deeper philosophical questions about the nature of knowledge and learning. If AI can solve complex problems, what then is the purpose of human education? The emphasis on metacognition and critical evaluation suggests a shift from rote memorization to higher-order thinking skills, where understanding 'how' AI arrives at an answer becomes as important as the answer itself. This could lead to a redefinition of educational goals, prioritizing skills like ethical reasoning, creativity, and the ability to formulate insightful questions that AI cannot yet generate. The event also touches upon the challenge of AI models uncritically reproducing 'popular misconceptions,' highlighting the need for human expertise to curate and validate information in an AI-driven information landscape.
