Surprise 1: It's Not One Spotlight, but Many
The term 'attention' is misleading. It conjures an image of a single spotlight scanning a sentence, highlighting the one most important word. But the 'multi-head' part is the real story. Instead of one spotlight, imagine a panel of eight, twelve, or even
more specialist detectives all reading the same sentence simultaneously. One detective might only look for subject-verb relationships. Another is an expert in tracking pronouns back to their original nouns. A third might be obsessed with linking adjectives to the things they describe. This is the first major surprise: multi-head attention doesn't find the single 'most important' connection. It finds many different *types* of connections in parallel. For a practitioner, this means the model isn't just weighing words; it’s building a complex, multi-layered understanding of grammar, syntax, and semantics all at once. The model learns *what kind of relationships matter* on its own, and each 'head' becomes a specialist in finding them.
Surprise 2: The Computational Cost is Not Linear
Programmers are used to thinking about efficiency. If you double the input, you might expect the processing time to double. With multi-head attention, you’re in for a shock. The mechanism has a quadratic computational complexity. In plain English, this means that every single element in your input sequence has to 'attend' to every other element. If you have a 100-word sentence, you're running roughly 10,000 calculations (100 x 100). If you double that to a 200-word sentence, you're not doubling the work; you're quadrupling it to 40,000 calculations (200 x 200). This is a huge surprise for practitioners coming from older architectures like Recurrent Neural Networks (RNNs), which process sequences linearly. The practical implication is stark: feeding a Transformer model a very long document can lead to exorbitant memory usage and processing time, a trap many first-timers fall into. This quadratic scaling is the single biggest engineering constraint of standard Transformer models.
Surprise 3: It Often Focuses on 'Weird' Things
When you visualize what an attention head is looking at, you expect it to find profound semantic links—'king' paying attention to 'queen', for example. Sometimes it does. But often, you'll find a head intensely focused on punctuation, or all the articles ('the', 'a'), or the word that appeared two positions prior. The initial reaction is that the model is broken. This is a misunderstanding of its goal. The model isn't trying to impress you with human-like intuition on every layer. Some heads learn positional or structural patterns. A head might learn a simple rule like 'pay attention to the previous word,' which is useful for capturing local context. Another might learn to connect every word to the period at the end of the sentence, effectively creating a 'summary' token that aggregates information. These seemingly 'weird' attention patterns are often the model building a robust internal representation of sentence structure, which is just as important as understanding the meaning of the words themselves.
Surprise 4: There Are No Recurrent Loops
Before Transformers, the gold standard for processing sequences was the RNN. Its defining feature was a loop: it processed the first word, updated its internal memory (its 'hidden state'), then processed the second word using that memory, and so on. This sequential, step-by-step process felt intuitive. Multi-head attention throws this entire concept away. It processes all words in the input at the same time. There is no hidden state passed from one word to the next. The entire context is derived by this massive, all-at-once comparison of every word to every other word. For a new practitioner, this is mind-bending. How can it know about word order without processing words in order? The answer is a clever trick called 'positional encodings'—extra information added to each word that signals its position in the sequence. The surprise isn't just that it works, but that it works *better* than the sequential method for many tasks, primarily because processing everything in parallel is much faster on modern hardware like GPUs.
