Ask Onix
California launches investigation into xAI's Grok model
California Attorney General Rob Bonta has opened a formal inquiry into xAI after reports surfaced that its Grok artificial-intelligence model produced and distributed sexually explicit deepfakes without consent.
Allegations of non-consensual imagery
In a statement released Wednesday, Bonta described the volume of complaints as "shocking." The material, he said, depicts women and minors in nude and sexually explicit scenarios and has been used to harass individuals online.
The Democratic prosecutor called on xAI to take immediate corrective measures.
xAI and Elon Musk respond
xAI, the company behind Grok, previously stated that any user prompting the model to generate illegal content would face the same legal consequences as if they had uploaded the material themselves.
"I am not aware of any naked underage images generated by Grok. Literally zero."
Elon Musk, Wednesday post on X
Musk added that Grok only creates images in response to explicit user requests and does not generate content spontaneously. He also accused critics of using the controversy as a pretext for censorship, alleging political motivations.
Broader industry and legal context
Last month, Wired reported that tools from other major AI firms, including OpenAI and Google, have also been misused to create non-consensual nude images.
Three Democratic U.S. senators recently urged Apple and Google to remove X and Grok from their app stores. Within hours, X restricted its image-generation feature to paying subscribers, though both apps remain available for download.
Section 230 debate intensifies
Legal experts are divided over whether Section 230 of the Communications Decency Act shields AI companies from liability for content their models generate. The 1996 law grants online platforms immunity for user-posted material, but its applicability to AI-created content is contested.
"This isn't a case where users are making the images themselves and then sharing them on X. xAI itself is making the images. That's outside of what Section 230 applies to."
James Grimmelmann, Cornell University law professor
Senator Ron Wyden, a co-author of Section 230, told the BBC that the law was never intended to cover AI-generated imagery. He praised California's investigation, calling Grok a "horrific child sexual abuse material generator."
International pressure mounts
In the UK, Prime Minister Sir Keir Starmer warned that X could lose its self-regulatory privileges if it fails to control Grok. The country is advancing legislation to criminalize the creation of non-consensual intimate images.
UK communications regulator Ofcom has also launched a probe into Grok. If violations are found, xAI could face fines of up to 10% of its global revenue or £18 million, whichever is higher.
Next steps
The BBC has reached out to xAI for comment but has not yet received a response. California's inquiry is expected to proceed alongside parallel investigations in the UK and potential federal scrutiny in the U.S.
