The Old Math of AI Was Simple
Until recently, the dominant AI models were 'dense.' Think of a dense model as a single, massive brain where every single neuron fires to process any piece of information. If you wanted to make the model smarter, you had to add more neurons, making the whole
brain bigger. For the hardware market, this was a beautifully simple equation: a 175-billion-parameter dense model used all 175 billion parameters for every task. Scaling up meant a linear, predictable, and massive increase in demand for the GPUs needed to run it. This predictable growth fueled a gold rush for chipmakers, as every AI company needed more and more raw computing power, and investors could easily forecast the skyrocketing demand.
Enter the 'Mixture-of-Experts'
Mixture-of-Experts (MoE) models flip that script. Instead of one giant brain, an MoE model is like a team of specialized consultants. It has a 'router' or 'gating network' that looks at an incoming task and sends it to only the most relevant experts. For example, a query about Shakespeare might get routed to the literature and history experts, while the math and coding experts remain inactive. This is called sparse activation. The result is a model that can have a staggering number of total parameters—trillions, even—but only uses a small fraction of them for any single calculation. Models like DeepSeek or Mixtral might have over 600 billion total parameters but only activate 30 or 40 billion for a given token. This offers the knowledge of a huge model at a fraction of the computational cost.
The GPU Dilemma: Memory vs. Compute
This is where the forecasting headache begins. An MoE model creates a split between two different kinds of hardware needs: memory and compute. While only a few 'experts' are active at any given time (the 'compute'), the entire team of experts must be present and ready to go. This means all the model's parameters, active or not, must be loaded into the GPUs' high-speed memory (VRAM). So, you still need a large number of GPUs just to hold the massive model, creating a high floor for memory demand. However, the computational work required to run a query (known as inference) is dramatically lower than a dense model of the same total size. This efficiency in everyday use is MoE's main advantage, but it decouples the once-tight link between model size and moment-to-moment processing demand.
Why Forecasters Are Tearing Their Hair Out
The simple math is gone. Predicting future GPU demand is no longer about multiplying the number of AI users by the size of the models they use. With MoE, demand becomes a complex, multi-variable problem. Forecasters now have to ask: How much demand will come from training these giant models (which still requires immense, dense computation)? And how much will come from inference? For inference, the needs depend on the specific MoE architecture, how many experts are activated per token, and how efficiently the models are run. Furthermore, MoE models shift the bottleneck from pure GPU compute power to the network that connects the GPUs, as the router shuffles data between different experts living on different chips. This adds another layer of complexity, making it much harder for analysts and even chipmakers themselves to produce clean, long-term forecasts for GPU sales.













