AI Models Exhibit Frequent Disagreements, Raising Concerns About Accuracy and Reliability
Various AI models, including ChatGPT, Perplexity, Claude, Gemini, and Grok, frequently provide conflicting answers to the same questions, raising significant concerns about their accuracy and reliability. This disagreement stems from several factors: AI models predict the most probable next word rather than fact-checking, leading to inherent randomness in their outputs. When uncertain, chatbots tend to guess, often creating plausible but false statements (hallucinations), partly because current evaluation methods incentivize guessing over admitting lack of knowledge. Live web search features in these platforms also struggle with accuracy, often failing to link to original sources correctly or fabricating web addresses. Research indicates that generative search tools collectively provide incorrect answers to over 60% of queries. Additionally, training chatbots to be empathetic can reduce their accuracy by 10% to 30%, making them more prone to agreeing with users' false beliefs. AI models also adopt human fl...