Anthropic and OpenAI have given California different answers on how a dangerous AI model should be stopped, Computerworld reported Thursday in a comparison of the companies' letters to Gov. Gavin Newsom. Anthropic wants the state attorney general to be able to ask a court to suspend a model, while OpenAI wants developers legally required to keep shutdown controls and to use them. With state recommendations due Nov. 16, the answer could reach any organization that deploys frontier AI in California.
What Happened
Newsom's Executive Order N-9-26, signed Sept. 18, directs the Government Operations Agency, consulting the Governor's Office of Emergency Services, to recommend by Nov. 16 possible amendments to state AI safety laws. Topics include a verified "kill switch" for frontier models, outside reviewers embedded in AI labs and wider reporting of loss-of-control incidents. The order itself imposes no requirements.
Anthropic's letter, dated Oct. 15, proposes that if the attorney general concludes, informed by an independent evaluation, that a company's models pose a near-term risk of catastrophic harm, the office could seek a court order suspending a model's deployment, internal use or further development. It also proposes risk reports at least every six months, including on riskier models used only internally, with evaluators paid from a state-run pooled fund rather than directly by developers. The letter describes the pilot in which a METR team has employee-like access, which The Primary Wire reported Oct. 15, citing TechCrunch.
OpenAI's Oct. 20 letter argues that a single switch is the wrong model. It prefers "shutdown controls" spanning steps such as stopping a training run, isolating an agent, revoking tool or network access, or suspending a service. It wants SB 53 amended to require such controls and an enforceable duty to use them when unsafe conditions arise and nothing less would work. It says responsibilities should extend to inference providers, downstream deployers and users in proportion to what they control, and notes that developers cannot reliably shut off independently run copies of open-weight models.
OpenAI also seeks annual independent compliance audits and technical assessments, with the National Institute of Standards and Technology helping set standards.
Why It Matters
SB 53, signed in 2025, already requires large frontier developers to publish safety frameworks, report critical safety incidents and protect whistleblowers. The two letters mark out the next question: whether a court, acting for the state, or the companies themselves under a legal duty should hold the power to stop a model. OpenAI's version would also place duties on businesses that run or deploy models. Greyhound Research analyst Sanchit Vir Gogia said, according to Computerworld, that "every assigned duty needs an executable control." He advised that enterprise contracts spell out who can intervene and who can authorize a restart.
What to Watch
- The Government Operations Agency's recommendations, due Nov. 16.
- Whether lawmakers take up SB 53 amendments.
- Which approach, if either, the state's recommendations adopt.
