Leaders from OpenAI, Meta, Anthropic and Google said they could not quantify the risks imposed by AI in the worst case catastrophic scenario at a New York City Council hearing about AI on Monday, Oct. 5.
“I don't know. I also don't think it matters whether it's 1% or 10% or 20% chance that something catastrophic will go wrong. None of these levels is remotely acceptable,” OpenAI’s head of policy development and operation Morgan Dwyer said in response to Council Speaker Julie Menin’s question of how to quantify risk.
Menin said she was “incredulous at that answer” and called it “flippant, at best,” asking how the companies can know that the product is safe if they can't measure potential risk.
Meta’s director of its AI policy and legislation Shane
Cahill, Google’s global director of AI and emergency policy Alice Friend and an Anthropic head Logan Graham also testified during the at-times tense hearing. Each emphasized their company's commitment to safe AI.
But they could not guarantee that models will always follow safety guardrails or that the companies will accept legal responsibility for resulting harm, including risks ranging from sensitive data compromises to financial or health guidance.
AI pressure grows
The New York City Council hearing is the first time these companies have participated in public testimony under oath following reports throughout the summer of AI agents hacking companies and governments, according to Menin.
SpaceX was also asked to appear at the hearing but didn’t show, resulting in a subpoena issuedon Sept. 28 to compel the company to speak.
The hearing lasted multiple hours, with around 30 council members grilling the four leaders on issues such as job loss due to AI, protections for artists and data centers.
Referencing an interview with POLITICO over the weekend in which OpenAI CEO Sam Altman said that “the world should accept some bad things happening for the benefits of this technology,” Menin also asked why AI companies get to decide what level of risk society must bear.
“Clearly, some risks are unacceptable, but we also think it's important to get technology into the hands of as many people as possible, so that they can benefit from it,” Dwyer said of OpenAI.
Cahill from Meta countered that the company is committed to “serving people rather than racing towards recursive self-improvement” and that “other labs can do that also.”
All 51 members of the City Council gathered for the hearing to introduce 10 pieces of legislation that would oversee the development, deployment and use of AI in New York City.
One bill would introduce a whistleblower incentive program, where a whistleblower could receive up to 25% of the proceeds recovered from a filed complaint. Another would require a “kill switch” to exist in any AI system deployed in the city.
The meeting comes less than a week after executives from Anthropic, OpenAI, Meta, Google, Nvidia and SpaceX signed a White House accord on AI promising to self-regulate their development of the technology.
But many, such as Menin and other members of the New York City Council, thought that wasn't enough.
“Congress has failed to act, and the federal government has focused on the self-regulation of the leading AI firms-the very concept that Bill Gates entirely dismisses,” Menin said at a press conference before the hearing.
New York’s RAISE Act is one of few state-level regulations around AI, requiring developers to publish and maintain safety plans before deploying models and to report safety incidents to the state within 72 hours. New York City Mayor Zohran Mamdani has also enacted AI regulations, including the ban on AI in elementary and middle schools.
A majority of Americans are skeptical of AI and data centers, studies find. Only 6% believe that AI will mostly strengthen democracy, one study by nonprofit Murmuration found. And 69% of poll respondents oppose the construction of AI data centers in their local area, according to a NBC News Decision Desk survey.
In the face of criticism from some New York City leaders and Americans across the country, Google's Friend said at the hearing that the company wants to abide by "an approach to AI that is both bold and responsible, and we strongly support a pragmatic, technology-neutral, and evidence-based regulatory framework."
Researchers who resigned over AI concerns speak out
The council also heard from former OpenAI, Anthropic and Google DeepMind researchers Jacob Coxon, Daniel Kokotajlo and Alex Turner, all of whom resigned due to concerns about the power of AI technologies.
The researchers emphasized the speed at which frontier companies are reaching recursive self-improving AI, or the ability for AI models to train itself, rendering human input unnecessary.
Coxon’s public resignation from Anthropic in August sparked a wave of responses from industry employees and leaders agreeing with his warning that no one knows how to control the technology yet.
“This could transform the economy. It could make huge improvements in science and medicine, and it could make nearly everyone's life better. But on the current path, I think it is more likely than not that humanity loses control to these AIs, and it could end in human extinction,” Coxon said at the hearing. While he didn't cite specifics, one common fear is that superintelligence could autonomously deploy bioweapons.
Greta Reich covers the artificial intelligence industry for USA TODAY through a fellowship from the Tarbell Center for AI Journalism. Funders do not provide editorial input.
This article originally appeared on USA TODAY: AI leaders say they can't quantify the tech's risks during heated hearing
