AMA Calls for Stronger Safeguards on AI Chatbots in Mental Health
The American Medical Association (AMA) has urged federal lawmakers to implement stronger safeguards for artificial intelligence (AI) chatbots used in mental health care. The AMA's call comes amid increasing use of AI-enabled tools for mental health support, which, while innovative, pose risks such as emotional dependency, misinformation, and inadequate crisis response. The AMA has recommended several measures, including enforcing transparency standards, creating a risk-based oversight framework, mandating ongoing safety monitoring, and requiring strict data protection standards. A recent survey by Rock Health found that 32% of respondents use AI chatbots for health information, with 28% using them for mental health management. However, research from Mass General Brigham indicates that while AI models can achieve correct diagnoses, they often fail in differential diagnosis, highlighting the need for AI to augment rather than replace physician reasoning.