Penn State Researchers Find Conversational Chatbots Can Mislead Users, Verification Tools Offer Solution
Penn State researchers have published a study in the Journal of Computer-Mediated Communication indicating that highly conversational AI chatbots can lead users to trust misinformation, even when the information is demonstrably false. The study found that the more human-like a chatbot's communication, the more likely users are to curb their 'negative machine heuristic'—the tendency to believe machines lack human intuition and subjective decision-making. This increased trust occurred even when the information provided was 'ridiculously wrong,' according to co-author S. Shyam Sundar. The research involved an online experiment with 477 participants who interacted with an AI health assistant that provided incorrect information. Participants were then asked to rate the chatbot's credibility. A second part of the study explored whether verification options could mitigate this misplaced trust, finding that users who actively clicked verification buttons became more skeptical of the AI's responses.