AI in Mental Healthcare Requires Human Supervision to Ensure Safety and Effectiveness
A commentary in the Journal of Medical Systems argues that while human oversight is widely touted as a safeguard for artificial intelligence (AI) in medicine, it often falls short in practice. The authors, Jonas Ver Berne and Reinhilde Jacobs of KU Leuven and Karolinska Institutet, distinguish between 'cognitive oversight' and 'normative oversight.' Cognitive oversight refers to a clinician's ability to genuinely understand the AI's reasoning, data, and uncertainties. Normative oversight is the clinician's capacity to take responsibility for the AI-assisted decision. The commentary asserts that as AI systems become more complex and 'agentic' (planning multi-step actions and integrating diverse data), clinicians' cognitive oversight erodes, while their legal and moral responsibility (normative oversight) remains intact. This creates a dangerous asymmetry where accountability persists without comprehension. 
