What's Happening?
The High-Bandwidth Memory (HBM) industry is undergoing a significant shift with the introduction of HBM4, which allows for the customization of the base die. Traditionally, memory makers like Micron, Samsung, and SK Hynix designed and fabricated all layers
of an HBM stack, including the base die. However, with HBM4, the base die has transitioned to more advanced logic processes (likely 4nm or beyond) for improved power and performance. This change means that while memory makers still design standard base dies, logic foundries will now manufacture them. The new model also enables companies, particularly hyperscalers, to replace the standard base die with a customized one to optimize for specific workloads. This customization is driven by the need for differentiation and the rapid pace of computing evolution in areas like AI, which outpaces traditional standardization processes like JEDEC. The implementation of custom HBM (cHBM) involves complex negotiations regarding design, manufacturing, assembly, and testing responsibilities, with no single answer for who performs each step.
Why It's Important?
This evolution in HBM technology holds significant implications for the U.S. technology and business sectors, particularly for hyperscalers and companies involved in AI development. The ability to customize HBM base dies allows for greater optimization of computing systems for specific, narrow workload types prevalent in AI processing. This can lead to enhanced performance, reduced power consumption, and increased efficiency in data centers. For hyperscalers, who have the financial resources and in-house expertise in semiconductor design, this offers a crucial avenue for differentiation and competitive advantage. The shift in manufacturing to logic foundries also highlights the increasing interdependence within the semiconductor supply chain. While custom HBM is not expected to further burden the overall memory supply, which is currently experiencing shortages, it introduces complexities in supply planning due to the varied mix of standard and custom implementations. This could impact smaller companies or those without the resources to engage in custom design projects, potentially widening the technological gap between large and small players in the AI and data center markets.
What's Next?
Custom HBM stacks are anticipated to be deployed in data centers within the next one to two years. The design phase for these custom solutions will likely be undertaken by the companies consuming the custom memory, such as hyperscalers, or by specialized design houses. Manufacturing and wafer testing will occur at logic foundries, which may or may not be the same foundries producing standard base dies. The assembly process will vary by project, determined through agreements between memory houses, logic foundries, and assembly facilities. Final testing will proceed with custom test programs tailored to the specific designs. The industry will need to navigate the complexities of these new business engagement models, as each project will involve unique negotiations and collaborations. The ongoing memory shortage will continue to influence these developments, with memory suppliers having sold out their fabs for the foreseeable future, making the acquisition of both standard and custom HBM a critical challenge.
Beyond the Headlines
The emergence of custom HBM signifies a deeper trend towards specialized hardware solutions tailored for specific computational demands, particularly in the burgeoning field of artificial intelligence. This move away from purely standardized components reflects the increasing maturity and diversification of the semiconductor industry. Ethically, it raises questions about accessibility and equity in technological advancement, as only well-resourced entities like hyperscalers can currently afford the investment in custom designs. This could lead to a concentration of advanced computing capabilities among a few dominant players. Legally, the complex web of intellectual property and manufacturing agreements between memory makers, logic foundries, and end-users will require robust frameworks to manage. Culturally, it underscores a shift where hardware design is becoming as agile and adaptable as software development, pushing the boundaries of what's possible in high-performance computing and potentially accelerating the pace of innovation in AI and other data-intensive fields.
