A Rare Moment of Unity
In a series of recent appearances, including a high-profile session at the United Nations Security Council, OpenAI CEO Sam Altman and Anthropic CEO Dario Amodei have presented a surprisingly aligned message: the rapid advancement of artificial intelligence
demands unprecedented cooperation on safety. They argue that the potential for increasingly autonomous systems to cause unforeseen harm—from misuse by bad actors to concentrating too much power in too few hands—is a global risk that transcends corporate competition. This call to action has been echoed by other industry figures and marks a significant public acknowledgment that the race for capability cannot come at the cost of control. The core idea is to establish shared safety standards, testing protocols, and incident reporting mechanisms across the leading AI labs.
The Gap Between Support and Action
While there is broad verbal support for slowing down and prioritizing safety, translating this sentiment into a concrete, enforceable pact is proving immensely difficult. Currently, most AI safety measures are voluntary commitments made by individual companies, like OpenAI's Preparedness Framework or Anthropic's Responsible Scaling Policy. These are important signals, but they are not legally binding. Companies are collaborating with government bodies like the U.S. and U.K. AI Safety Institutes, which involves pre-release testing of new models, but these are still partnerships, not hard-and-fast rules governing the entire industry. The fundamental challenge is that voluntary pledges can buckle under intense competitive pressure to innovate and capture market share.
The Antitrust Hurdle
This is where the “careful wording” becomes critical. Any agreement between competitors to slow down development or limit product capabilities—even for safety reasons—can raise red flags for antitrust regulators. Such an agreement could be interpreted as an anti-competitive scheme to restrict output, protect the market position of incumbent leaders like OpenAI and Anthropic, and disadvantage smaller players or open-source models. This legal minefield is a major reason why labs have been hesitant to form deeper collaborations. Some, like Amodei, have suggested governments should grant a limited antitrust waiver specifically for safety cooperation. Others, like OpenAI's policy chief, believe it's possible to collaborate on reducing risks without one, but the legal uncertainty remains a significant chilling factor.
Defining the Terms of Safety
Even with legal clearance, the labs would face another enormous challenge: agreeing on what “safety” means and at what threshold development should be paused. Amodei has proposed a tiered approach, from easily agreeable bans on using AI for bioweapons to much harder agreements on setting a “speed limit” for models that can improve themselves. But critics worry that an insular group of powerful labs could define these standards in a self-serving way, effectively creating a cartel under the guise of public safety. This raises questions about who gets to be part of the decision-making process. Altman himself has stated that crucial decisions about AI's future cannot be made by a few labs in San Francisco alone and must involve democratic processes.
The Path Forward: A Hybrid Approach?
The most likely path forward is not a single, all-encompassing treaty but a multi-layered approach. This includes strengthening government-led bodies like the AI Safety Institutes, which can serve as neutral third-party evaluators. It also involves continued, albeit cautious, discussions between labs on establishing an industry standards body to handle testing, incident reporting, and auditor qualifications. This self-regulatory effort would exist alongside, and be scrutinized by, government regulators. The goal is to create a system where the responsibility for safety is shared, blending industry expertise with public accountability, without tripping antitrust wires or stifling innovation completely. The alternative is a fragmented, unregulated race where a major accident becomes not a question of if, but when.
