The Temptation of the Instant Answer
In the fast-paced world of modern business, efficiency is king. Large Language Models (LLMs) like ChatGPT, Gemini, and others have emerged as revolutionary tools for productivity, especially for the tedious task of document analysis. Professionals across
legal, financial, and academic sectors are increasingly turning to AI to compare contracts, analyse financial reports, or sift through research papers. The appeal is obvious: why spend hours manually cross-referencing two 50-page documents when an AI can give you the key differences in seconds? The default approach for many users is to upload the files and ask a direct, conclusion-oriented question: “Which of these two lease agreements is more favourable?” or “Summarise the differences between these two financial statements.” This approach treats the AI as an oracle, a black box that consumes information and dispenses a final verdict. While this feels like the ultimate shortcut, it comes with significant and often hidden risks.
The Hidden Danger of Conclusion-First Prompts
The single biggest risk of asking an AI for a conclusion is a phenomenon known as “hallucination.” An LLM hallucination occurs when the model generates plausible-sounding information that is factually incorrect, unsupported by the source material, or entirely fabricated. When you force an AI to provide a simple conclusion to a complex comparison, you increase the likelihood of this happening. The model, designed to be helpful and provide a coherent response, may oversimplify nuances, invent connections that don't exist, or miss critical omissions in one of the documents because it is prioritising the delivery of a final answer over a faithful representation of the data. This is because the AI isn't 'thinking' in a human sense; it's predicting the most probable sequence of words to satisfy your prompt. An answer without the underlying evidence is unverifiable and can lead to poor decision-making based on flawed, AI-generated insights.
A Better Way: Prompting for Evidence
The solution is to shift your prompting strategy from asking for conclusions to requesting evidence. Instead of treating the AI as a judge, treat it as a highly efficient paralegal or research assistant. Your goal is to have the AI extract and organise the relevant information for you, so that you, the human expert, can make the final call. A well-crafted, evidence-based prompt forces the AI to ground its output in the provided text.
Consider this contrast:
Weak Prompt (Conclusion-focused): “Compare these two employment contracts and tell me which one is better.”
Strong Prompt (Evidence-focused): “Review Document A and Document B. Extract and present in a side-by-side table all clauses related to 'Termination,' 'Confidentiality,' and 'Non-Compete Agreements.' For each clause, quote the exact text from both documents.”
The second prompt doesn't ask the AI for its opinion. It directs the AI to perform a specific, verifiable task: find and present data. This makes the output concrete and allows you to perform the analysis yourself with all the relevant facts laid out clearly.
The Benefits of an Evidence-First Approach
Adopting an evidence-first mindset for AI interaction yields several powerful benefits. First and foremost is accuracy and verifiability. When the AI provides direct quotes or extracts specific data points, you can immediately check its work against the source documents. This drastically reduces the risk of acting on a hallucination. Second, it deepens your own understanding. By reviewing the extracted evidence yourself, you engage more critically with the material, leading to more nuanced insights than a simple AI summary could ever provide. Third, it reduces liability. In high-stakes fields like law and finance, relying on an unverified AI conclusion is a significant risk. Grounding your decisions in evidence pulled by an AI—but verified by you—is a much more defensible process. Finally, this method turns you into a more sophisticated AI user. You learn to leverage the tool's strengths (speed and data processing) while mitigating its weaknesses (lack of true comprehension and potential for fabrication).
A New Mindset for Human-AI Collaboration
This principle extends far beyond just comparing two documents. It represents a fundamental shift in how we should approach collaboration with AI. The most effective use of these powerful tools isn't about outsourcing our thinking, but about augmenting it. Whether you're analysing data, conducting research, or drafting a report, the goal should be to use the AI to gather, sort, and structure information. Ask it to identify key themes, extract all factual claims, or list arguments for and against a certain position. By prompting for evidence, you keep the human in the loop for the most important task: critical thinking. The AI becomes a transparent partner that shows its work, rather than an opaque oracle that demands blind faith.
