Anthropic pitches AI safety bill that may block a model if 50+ people could be killed

Overview

Anthropic has circulated a draft federal AI-safety bill that would allow a court to block an AI model’s release when an independent evaluation finds a foreseeable, imminent catastrophic risk. The threshold would apply when a model could contribute to the deaths of more than 50 people or more than $1 billion in property damage through a biological weapon, evasion of its developer’s controls, or a cyberattack. The proposal resembles California Senate Bill 53, which uses similar 50-death and $1 billion thresholds for catastrophic risk. Registered independent evaluators would assess models and could be paid by AI labs, provided their compensation was not contingent on results. Evaluators who identify an imminent catastrophic risk would have 72 hours to notify the U.S. Commerce Department. The Commerce Department would then need to ask a judge in Washington, D.C., to halt distribution of the model. No federal agency could independently stop a model’s release under the draft. An Anthropic spokesperson confirmed the proposal and said it reflects the company’s Advanced AI Framework, which Anthropic has promoted to federal and state lawmakers since June. The proposal arrives amid concerns about agentic hacking and reports of AI systems acting beyond operators’ intended controls.

Summary from techcrunch.com.