California Weakens AI Safety Bill SB 1047 Amid Industry Pushback

California Weakens AI Safety Bill SB 1047 Amid Industry Pressure

California's proposed legislation, Senate Bill 1047 (SB 1047), aimed at preventing AI-related disasters, has undergone significant amendments following opposition from Silicon Valley stakeholders, including AI firm Anthropic. The bill, which seeks to hold AI developers liable for catastrophic events caused by their models, passed through the California Appropriations Committee, moving closer to becoming law.

Key Amendments and Changes to SB 1047

Senator Wiener's office announced several key changes to SB 1047, incorporating suggestions from Anthropic and other industry players. These amendments aim to balance AI safety with the need for innovation, particularly for the open-source community.

Shift in Attorney General's Power: The bill no longer empowers California's Attorney General to sue AI companies for negligent safety practices before a catastrophic event occurs. Instead, the AG can seek injunctive relief to halt dangerous operations and can still pursue legal action if a disaster materializes.
Frontier Model Division (FMD) Restructuring: The bill eliminates the proposed Frontier Model Division (FMD) as a standalone agency. However, the core functions of the FMD are integrated into the existing Government Operations Agency through an expanded Board of Frontier Models, now comprising nine members instead of five. This board will continue to set compute thresholds, issue safety guidance, and regulate auditors for covered AI models.
Reduced Liability for Safety Certifications: AI labs are no longer required to submit safety test results "under penalty of perjury." They must now submit public "statements" on their safety practices, with no criminal liability attached to these submissions.
Modified Safety Standards: The requirement for developers to provide "reasonable assurance" that AI models do not pose a significant risk of catastrophe has been softened to "reasonable care."
Open Source Protection: A new provision protects open-source fine-tuned models. Developers who spend less than $10 million on fine-tuning a covered model are exempt from the bill's definition of a "developer," with responsibility falling on the original model developer.

Rationale Behind the Amendments

The amendments are seen as a strategic move to appease SB 1047 opponents and present Governor Newsom with a less contentious bill. While Governor Newsom has expressed commitment to AI innovation, these changes aim to mitigate potential backlash from the AI industry.

Anthropic is currently reviewing the revised bill before taking a formal stance, noting that not all of its proposed amendments were adopted.

Industry Reactions and Criticisms

While some, like Nathan Calvin of the Center for AI Safety Action Fund, believe the amendments support the bill's goal of advancing AI safety while fostering innovation, others remain critical.

Martin Casado, a general partner at Andreessen Horowitz, dismissed the edits as "window dressing," arguing they fail to address the core criticisms of the bill.

Furthermore, eight United States Congress members from California sent a letter to Governor Newsom urging him to veto SB 1047, citing potential negative impacts on the state's startup community, scientific development, and AI safety efforts.

Next Steps for SB 1047

The bill is now headed to the California Assembly for a final vote. If passed, it will return to the Senate for a vote on the amendments. Should it clear both chambers, it will land on Governor Newsom's desk for a final decision.

Image Credits

Melinda Podor / Getty Images

Event Promotion

TechCrunch All Stage Boston: July 15, Boston, MA. Founders and VCs of all stages are invited to connect and gain insights. Early bird registration offers a $450 discount.