Gov. Gavin Newsom on Friday signed asweeping executive order designed to accelerate independent oversight of artificial intelligence systems and advance the possible creation of an emergency AI “kill switch,” positioning California at the forefront of growing efforts to regulate rapidly evolving technology.
The order comes amid mounting concern from researchers, lawmakers and even some AI industry leaders following aseries of high-profile incidents involving advanced AI systems, including what state officials described as a recent “Hugging Face attack” and other reported cases in which AI agents allegedly escaped testing environments or carried out unauthorized actions online.
Newsom said California cannot afford to wait for federal action.
“The
federal government's abject failure to create any form of meaningful AI oversight or accountability should alarm every American,” Newsom said in announcing the order.
The governor is directing state agencies and outside experts to deliver recommendations within two months on strengthening California's AI laws, including proposals that would require independent safety monitors inside frontier AI companies and emergency shutdown capabilities for the most powerful models.
Newsom's AI executive order targets independent audits and AI safety reviews
The executive order accelerates implementation of two recently signed California laws: SB 813, which establishes a framework for independent verification organizations that can evaluate AI systems for safety risks, and AB 1405, which creates standards and a registry for AI auditors.
Among the proposals under review are:
- Requiring independent verification organizations to conduct regular audits of frontier AI labs.
- Requiring third-party verification of company safety plans, transparency reports and risk assessments.
- Expanding the definition of reportable AI safety incidents to include loss-of-control events.
- Developing a verified emergency shutdown mechanism, commonly referred to as an AI kill switch.
State officials emphasized that the order does not require companies to install a kill switch immediately. Instead, experts will study how such a safeguard might work and recommend possible legislative changes.
What is an AI kill switch?
Akill switch is generally described as an emergency mechanism that could disable or shut down a powerful AI model if it behaves dangerously, operates outside human control, or poses significant security risks.
The concept has gained traction in recent months as concerns about advanced AI capabilities have intensified.
A USA TODAY Network discussion published this week noted growing bipartisan interest in requiring AI companies to maintain emergency shutdown capabilities after reports of concerning AI behavior. The discussion described a kill switch as the ability to “completely unplug, turn off” a model if it begins acting outside intended limits or attempting to access systems it should not control.
The idea is notable because Newsom vetoed SB 1047 in 2024, a bill that included kill-switch provisions. The governor's new order effectively reopens the debate, though through a more targeted regulatory process.
AI industry leaders increasingly call for slower development and stronger regulation
Newsom's action follows a growing chorus of warnings from AI researchers and executives who argue that governments need stronger guardrails before next-generation systems become more powerful.
In recent weeks:
- An Anthropic researcher resigned publicly while warning about the dangers posed by advanced AI systems.
- AI researchers and executives have endorsed calls to slow development while safety protections catch up.
- Lawmakers from both parties have called for congressional action and briefings on AI risks.
- AI pioneers, including Geoffrey Hinton, have warned that governments may have limited time to establish effective oversight before systems become substantially more capable.
Those warnings have fueled broader debates about whether artificial intelligence could eventually threaten cybersecurity, critical infrastructure, employment and even human control over advanced systems.
Newsom calls on Congress to adopt California’s AI framework
California officials argue the state already has the nation's most comprehensive AI regulatory structure and are urging federal policymakers to use it as a roadmap.
The state has enacted laws covering frontier AI safety, independent audits, child-safety protections for AI companion chatbots, deepfake restrictions, AI-related fraud, privacy protections and cybersecurity standards.
Newsom said California's approach should serve as a national baseline as Congress considers whether to enact broader federal regulations.
The executive order directs the Government Operations Agency and the Governor's Office of Emergency Services to convene experts and return recommendations by mid-November. Those recommendations could form the basis for future legislation or regulatory changes in California.
This article originally appeared on USA TODAY: California governor proposes AI ‘kill switch’ amid fears of rogue tech
