Click here to read the full article.
Anthropic spent much of its filing with the SEC ahead of its initial public offering warning potential investors about the risks its AI models could pose to humanity, acknowledging it could not entirely ensure its own
safety measures would be successful against the “existential” danger AI could pose.
The disclosures came as part of more than 90 pages of risk factors in Anthropic’s S-1 filing with the SEC, made public Monday as the company prepares to go public. The 90-page length of such risk factors, the disclosure of which is customary, is unusual, spurred by the potential dangers AI models have posed following disclosures from AI companies over agents acting autonomously, unaligned with human instructions, and the potential misuse of the technology by human actors themselves. Snap’s risk factors totaled roughly 30 pages, while Elon Musk’s SpaceX — currently the largest IPO of all time at an initial $1.77 trillion valuation — totaled 38.
“Concerns have been raised both within our organization and by other researchers, regulators, policymakers, and industry participants that sufficiently advanced AI systems could, if not developed, deployed, and operated with appropriate safeguards, oversight, and security measures, pose catastrophic or existential risks to humanity, particularly if such systems were misused, replicated, weaponized, or allowed to operate with a high degree of autonomy,” the company wrote, following warnings from a group of current and former Anthropic researchers this summer that the technology posed such risks.
To counter it, Anthropic laid out its sweeping safety-minded policies in its SEC prospectus, a priority for CEO Dario Amodei. Amodei, who founded Anthropic in 2021 with a group of former OpenAI colleagues following their defection from the Sam Altman-founded rival, has advocated for AI companies to band together and slow the development of AI models to ensure sufficient safety protocols.
But the company admitted that its own work contributed to such destructive potential: “Our development of highly advanced models, platforms, and applications and expansion of use cases could further increase the risk that our models cause harm.”
The company disclosed four separate instances in which its AI models accessed third-party systems without instructions to do so, including one in which an agent changed system settings and read a person’s personal information. The agents all acted independently, without collaborating with other agents, but the company warned that the continued deployment of AI agents risked contributing to more collective — and unauthorized — actions.
The company’s risk factors also included factors such as financial commitments, including its billions of dollars worth of compute payments to SpaceX and Microsoft, and technology constraints. Specifically, it warned of Taiwan’s lead in semiconductor manufacturing — and the hostility between Taiwan and China — and the political opposition to data centers as risks to how it performs as a business.
But these earthly economic and political factors may not matter much if Anthropic, and the broader AI industry, cannot prevent the models they’re working on from causing cataclysmic doom.
“Our safety-focused approach is designed in part to mitigate such risks, but there can be no assurance that our efforts will be successful or that the broader AI industry will adopt similar safeguards,” the company wrote.
In the S-1 filing, as is customary, Anthropic did not disclose the number of shares it plans to sell or at what price. Those details will be determined in the coming weeks. The company’s shares will trade under the ticker symbol “ANT” on the Nasdaq exchange.
