California sues OpenAI, alleging AI development lacked adequate safeguards

(ABC NEWS) — California sued OpenAI on Monday, alleging the artificial intelligence giant endangered the public by developing advanced AI without adequate safeguards, according to a release from the office of California Attorney General Rob Bonta.
OpenAI, the lawsuit alleges, “misled and endangered” people by developing AI products “without the safeguards and monitoring necessary to adequately protect the public.”
The lawsuit centers in part on a self-directed AI hack into a separate tech company called Hugging Face, which OpenAI disclosed over the summer. The hack allegedly resulted from “an internal culture at OpenAI that prioritizes growth and rapid model development over safety and accountability,” according to the lawsuit.
The Hugging Face incident took place as OpenAI tested the capabilities of a pair of its AI models, the company said in July, describing how the technology had escaped and gained access to the open Internet.
A swarm of about 700 AI agents, in turn, accessed Hugging Face and attempted to cover their tracks as they sought to complete the test, according to reports issued by research organizations METR and Redwood Research.
“While public reporting and OpenAI’s self-serving statements about the Hugging Face attack have evoked science fiction fantasies of ‘rogue’ AI systems run amok, the simple truth is that the attack was entirely foreseeable in light of OpenAI’s decision to test models it knew to be dangerous in a flawed and exploitable testing environment without proper oversight,” the lawsuit says.
In response to ABC News’ request for comment, OpenAI spokesperson Drew Pusateri said the company has taken steps to “understand and address misaligned model activity” since the Hugging Face incident.
“We’ve published a detailed technical report, commissioned and published a separate independent report, notified affected organizations, strengthened safeguards across our research systems, and created a framework for additional identification and disclosure of misalignment,” Pusateri said.
“This lawsuit appears to single out OpenAI over a challenge that’s been identified by every major AI lab. We take misalignment seriously, and we have engaged — and will continue to engage — constructively with the Attorney General’s office to share information and advance strong, practical safety standards for the entire industry,” Pusateri added.
Weeks after OpenAI disclosed the Hugging Face incident, the company in August said it would pause some training of its latest AI models to ensure the technology’s safety.
“As models become more capable, the risks associated with developing and testing them internally also grow. Our standards for monitoring, alignment, and security must stay ahead of those risks,” OpenAI said in a blog post.
“We wanted to take the time necessary to meet those standards, so we temporarily slowed the pace of scaling,” the company added.
OpenAI CEO Sam Altman echoed the sentiment, voicing commitment to the safety of the company’s products.
“We care very deeply about AI safety,” Altman wrote on X at the time. “We believe the entire field will have to coordinate on shared safety standards, but will act unilaterally in the meantime.”
Rival companies, including Anthropic, have also revealed AI-related security incidents in recent months.
Public concern over the possible risks posed by AI has grown this year, resulting in part from self-directed AI hacks as well as warnings from AI researchers.
In September, Anthropic CEO Dario Amodei announced a precautionary slowdown of AI development to ensure sufficient risk prevention. That month, Altman also said he would dial back the pace of his company’s AI development.
The lawsuit comes about two weeks after a midterm election landslide for Democrats, many of whom criticized potential negative impact from AI.
About 6 in 10 House voters nationally said AI is a bad thing for society and about three-quarters oppose the construction of new data centers in their communities, according to the ABC News Voter Poll.
Two days after the midterms, OpenAI issued a new set of data center policies. The company said it would abide by fresh commitments, including a vow to pay for all electricity and infrastructure used at its facilities.
“Building more computing infrastructure capacity in the United States can help lower the cost of delivering AI, put these tools in more people’s hands, and ensure America stays at the forefront of the technology,” OpenAI said in a blog post. “The communities that host this infrastructure should have a clear say in shaping their futures, too.”