They Confidently Make Things Up
The first and most jarring surprise for many new practitioners is the LLM’s capacity for “hallucination.” You ask for a historical fact, and it provides a beautifully written, completely plausible, and entirely fictional answer. This isn't a bug; it's
a core feature of how they work. LLMs are not databases of facts. They are incredibly sophisticated pattern-matching engines, trained to predict the next most probable word in a sequence. When an LLM generates text, it's not retrieving information but composing what sounds like a correct answer based on the statistical relationships in its vast training data. The result is an output that has all the confidence of a seasoned expert but sometimes the factual accuracy of a daydream.
They Are Incredibly Literal and Lazy
For a tool that seems to understand natural language, an LLM can be frustratingly dense. Practitioners quickly learn that the slightest change in a prompt can lead to a dramatically different outcome. Asking it to “summarize this text” might yield a paragraph, while asking for “a summary in five bullet points” produces a perfectly structured list. This isn't because the model suddenly understands you better; it's because you've provided a more explicit pattern to follow. This phenomenon is why prompt engineering has become a critical skill. The model doesn’t infer your intent; it responds to your exact words. This literal-mindedness means users must learn to be incredibly specific, breaking down complex tasks into smaller, clearer steps to get the desired result.
Their 'Mistakes' Are Sometimes Brilliant
Just as hallucinations can be a source of frustration, they can also be a surprising wellspring of creativity. An LLM might misinterpret a prompt or connect two seemingly unrelated concepts, but the result can be a genuinely novel idea or a fresh perspective you hadn't considered. Researchers are increasingly exploring how to harness this tendency, viewing some hallucinations not as errors but as a form of computational brainstorming. By encouraging divergent thinking, these models can produce a wide range of ideas that a human might then filter for usefulness and accuracy. This turns the model's unpredictability from a liability into an asset, especially in creative fields where breaking from established patterns is the goal. For a practitioner, this means learning to embrace the occasional chaos and see the creative potential in the unexpected.
They Have a Very Short-Term Memory
One of the most counterintuitive surprises is how quickly an LLM can “forget” what you were talking about. This isn't a memory problem in the human sense but a technical limitation known as the “context window.” An LLM can only process a finite amount of text—both your input and its own output—at any given time. Once your conversation exceeds this limit, the earliest parts of the discussion are effectively dropped. A new user might be shocked when the model asks a question it was already told the answer to ten messages ago. This forces practitioners to design interactions that are either short and self-contained or use sophisticated techniques like retrieval-augmented generation (RAG) to continuously feed the model relevant information from external sources.
