Could California’s SB 1047 Impact AI Security?

September 6, 2024

Senate Bill SB 1047, also known as the Safe and Secure Innovation for Frontier Artificial Intelligence Models Act, is on the verge of being signed into law in California. If enacted, SB 1047 would require AI developers to implement safety measures to prevent egregious AI misuse resulting in “critical harm.” Such harms are defined in the bill as the use of covered AI models to develop weapons of mass destruction and cause mass casualty incidents or other “grave harms to public safety and security.” Covered AI models in this case use colossal computing power meeting one of two requirements: “trained on greater than 10^26 integer or floating-point operations, the cost of which exceeds...$100,000,000” or fine-tuning an above covered model at a cost >$10,000,000. Society has not yet reached this point where bad actors are leveraging high-powered AI models to cause mass casualties, but legislators are seeking to be proactive rather than reactive as AI innovation rapidly evolves.

Holding frontier AI model developers accountable for potential widescale disasters, SB 1047 proposes the following regulations:

The Race to Secure AI

To better understand the significance of SB 1047 on cybersecurity and for organizations outside of California, it is important to take a step back and review how technological innovations and safety regulations have been intertwined for decades. Whether it’s regulation to prevent the spread of biases in machine learning models or protect against data privacy violations associated with the Big Data revolution, we have often seen technology consumers and producers alike advocating for legislation to hold developers and corporations accountable for ensuring fairness, transparency and accountability. But as usual, technology advances faster than the government can regulate its production and use (cue the idiomatic “Wild West” headlines for crypto, Web3 and now AI).

In the rush to build and deploy new AI products and features, developers may not always think about the new risks that they are introducing into the environment if proper vetting, monitoring and training are not in place. From a cybersecurity perspective, this mindset can lead to attack surface expansion and exploitation of models to spread disinformation, disrupt critical infrastructure and launch cyberattacks or faster than ever. Regulations like SB 1047 will require developers to bake security into their models from the start.

As security-by-design approaches become a part of discussions surrounding AI regulations, Optiv clients find value in addressing the concept of secure AI. Optiv defines secure AI as a set of activities and solutions that work together across the AI lifecycle to unite technology, people and process against any risks. It involves embedding safety measures throughout the AI development process and ensuring transparency, accountability and ethics at every stage. This disciplined approach helps organizations to proactively keep up with rapidly changing AI technology solutions, while also meeting compliance requirements and protecting users from ethical violations or “critical harm.” Secure AI relies on foundational concepts of security by design, governance, data protection, risk management and threat modeling to achieve the goal of end-to-end security.

In the effort to help organizations adopt a secure AI approach, there is promise in the growing market landscape for AI security services designed to support developers and security teams throughout the journey of AI policy development, governance and risk management. When exploring such services, it is important to look for an investment incorporating both current and future regulations in AI governance advising to support compliance efforts and encourage responsible innovation. Such efforts can support an AI program maturation journey while ensuring security and governance is incorporated into the AI product and policy roadmaps early. Risk management is also a key component of AI security services. Mass casualties may seem obvious to prioritize as top risks. However, mature risk management practices, including threat models, risk assessments and risk registers, are needed to effectively address present-day and future AI use cases.

With an emphasis on security at the start, perhaps the battle between innovation and regulation does not need to be a battle at all. Both technical developers and lawmakers can focus on prioritizing ethics and safety.

What’s Next for AI?

In addition to SB 1047, Optiv governance experts reviewed several different proposed and passed AI regulations and frameworks, including the EU AI Act, the U.S. Executive Order on AI, the NIST AI Risk Management Framework and the U.S. Department of Health and Human Services (HHS) AI Rule. Common factors in these AI regulations include a prioritization of:

These central points will continue to be at the forefront of conversations surrounding AI and technology legislation. With a focus on safety protocols, annual audits and penalties against developers for compliance regulations, legislative efforts like SB 1047 are compelling businesses to think strategically about secure AI investments. California Governor Gavin Newsom could change the future of AI development with his final decision to pass or veto SB 1047. As the home of Silicon Valley technical innovation, California may set a precedent for other U.S. states looking to pass AI regulations. The debate intensifies over whether SB 1047 is a boon for humanity or a luddite dream. But ultimately a more nuanced approach is needed to carefully consider the impacts of regulating AI today for a more secure tomorrow.

Update: On September 29, 2024, it was announced that California Governor Gavin Newsom had vetoed SB 1047. In his veto message, Governor Newsom indicated some concerns about limiting the focus of the proposed legislation to frontier models: "By focusing only on the most expensive and large-scale models, SB 1047 establishes a regulatory framework that could give the public a false sense of security about controlling this fast-moving technology." Newsom also expressed concern that the proposed controls in the bill were not flexible enough, noting: "While well-intentioned, SB 1047 does not take into account whether an Al system is deployed in high-risk environments, involves critical decision-making or the use of sensitive data. Instead, the bill applies stringent standards to even the most basic functions - so long as a large system deploys it." Despite the veto, Newsom expressed a continued commitment to exploring AI safety initiatives. Optiv will continue tracking important decisions surrounding AI legislation to better guide our clients on governance and compliance best practices.