The AI revolution runs on powerful chips called GPUs, but throwing more processors at a problem is no longer enough. A hidden bottleneck is throttling performance, and it's not the chips—it's the network that ties them all together.
From Faster Chips to Faster Chat
For the last few years,
the story of AI's explosive growth has been a story about GPUs (Graphics Processing Units). These specialized chips, primarily from Nvidia, are the engines of modern AI, and the race has been to acquire as many as possible. But infrastructure teams are now confronting a more complex problem: once you have thousands of GPUs, how do you get them to work together efficiently? When training a massive model like the ones that power chatbots or generate images, the task is distributed across a whole cluster of GPUs. These chips need to constantly exchange enormous amounts of data to synchronize their work. A single dropped data packet or a tiny delay can cause a pipeline stall across every single GPU, leaving thousands of pieces of expensive hardware sitting idle. This communication problem is the new frontier. It's not just about how fast a single GPU can compute, but how quickly the entire cluster can communicate. In many cases, the network is now the primary limiting factor in AI performance.
Inside the Box: The NVLink Advantage
To solve this, a hierarchy of networks has emerged. The first layer of communication happens inside a single server chassis. Here, Nvidia has a massive advantage with its proprietary interconnect technology called NVLink. Think of it as a private, ultra-high-speed highway connecting a small group of GPUs. Instead of sending data over the slower, more general-purpose PCIe bus that connects other computer components, NVLink provides a direct, GPU-to-GPU link. This allows a set of eight or more GPUs to act almost like a single, giant processor. The latest generations of NVLink provide tremendous bandwidth—multiple terabytes per second—which is ordersofmagnitude faster than the networks that connect different server racks. This is crucial for latency-sensitive tasks where GPUs must be in perfect sync. For any workload that can fit within a single server or rack, NVLink is the undisputed king of performance.
The Battle Outside: InfiniBand vs. Ethernet
The real bottleneck emerges when AI models become too large to fit on a single rack and need to span thousands of GPUs across a data center. This is where the 'scale-out' network comes in, and it's the site of a major industry battle between two competing standards: InfiniBand and Ethernet. For years, Nvidia's InfiniBand has been the gold standard for high-performance computing. It's a 'lossless' fabric by design, meaning it has built-in mechanisms to prevent data packets from being dropped, which is critical for AI workloads. However, InfiniBand is largely a single-vendor technology controlled by Nvidia. The alternative is high-speed Ethernet, the same technology that runs most corporate networks and the internet, but adapted for AI. Using a protocol called RoCEv2 (RDMA over Converged Ethernet), it can mimic much of InfiniBand's performance on more familiar, open hardware. While InfiniBand has a slight edge on absolute lowest latency, Ethernet is catching up and, according to Dell'Oro Group, surpassed InfiniBand in the AI networking market by early 2026.
The Rise of an Open Standard
The push for an open alternative to Nvidia's dominance has led to the formation of the Ultra Ethernet Consortium (UEC). This group, which includes major players like AMD, Intel, Microsoft, and Meta, is working to create a standardized, interoperable, and highperformance version of Ethernet specifically for AI and HPC workloads. The UEC's goal is to deliver a fullstack architecture that can compete with proprietary solutions on performance while offering the benefits of a multi vendor ecosystem, such as lower costs and more supply chain flexibility. The consortium released its first major specification in mid‑2025 and an updated version 1.1 in October 2026, which introduced improvements for reducing communication delays and managing network congestion. By creating a unified standard, the UEC aims to ensure that the future of AI networking isn't locked into a single company's ecosystem.
