Elon Musk’s xAI used child porn to train Grok models, lawsuit says

xAI is now facing allegations that it trained its AI model, “Grok,” using child sexual abuse material (CSAM), even as regulators and courts continue to investigate the scope of the issue—a situation complicated by the fact that some “Grok” users have previously been arrested.

In a complaint filed on Wednesday, the plaintiff—identified by the pseudonym “Jane Doe”—explained that she was of preschool age in the early 2000s when she was repeatedly raped by adult men for the purpose of producing child sexual abuse material intended for sale to online sexual predators. Since then, organizations such as the National Center for Missing and Exploited Children (NCMEC) and the Canadian Centre for Child Protection (CCCP) have recorded the “digital fingerprints” (hashes) of these images to facilitate tracking and detection.

To ensure her safety, Doe had opted to receive alerts from the U.S. Department of Justice’s victim notification system in the event she was involved as a victim in a new criminal investigation. Although she had received countless alerts, she was deeply traumatized when the CCCP informed her that it had detected AI-generated child sexual abuse material on the xAI platform depicting her in such scenes. This discovery caused a resurgence of psychological trauma for Doe. Her complaint cites the discovery, on online forums, of messages exchanged “between abusers discussing the creation of child sexual abuse material (CSAM) using artificial intelligence, featuring the plaintiff as well as other known victims who had previously suffered such abuse.”

“Doe” now fears that xAI is not merely facilitating the production of new abusive images documenting the most painful periods of her life; She also believes that the company retains the images generated by “Grok” and uses these outputs to further train the model itself. Consequently, she contends that “Grok” was trained on both the original set of images that has haunted her for over twenty years and the new AI-generated images.

This marks the first case accusing xAI of training its models on child sexual abuse material (CSAM), although the complaint does not provide specific details regarding this particular allegation. Previous reports by Ars had highlighted a controversial dataset—subsequently withdrawn after researchers discovered CSAM within it—but there is no indication that xAI specifically used that dataset for its training. A press release from the plaintiff’s (referred to as “Doe”) attorneys stated that her images appeared on the hash list maintained by the National Center for Missing and Exploited Children (NCMEC) and that “this same content” had—allegedly—”been incorporated into the dataset used by xAI to develop Grok’s image and video generation capabilities.” Similarly, the complaint was limited to the allegation that “CSAM depicting the plaintiff—and associated with known, long-documented hash values—was used as part of the dataset upon which xAI relies.”

However, the complaint provided further details regarding allegations that the “Grok” model is being trained using AI-generated CSAM.

The plaintiff’s complaint (referring to her as “Doe”) stated: “Given that Grok’s terms of use treat public posts on X and Grok’s own outputs as training data by default, posting an image does not merely present it to users; it also feeds the process xAI uses to train and refine its model, thereby generating new images.”

The lawsuit also highlighted that, although xAI filters violent content from Grok’s outputs to exclude it from training data, the company’s terms do not specify—notably—whether child sexual abuse material (CSAM), non-consensual intimate imagery (NCII), or content unsuitable for work (NSFW) fall under the “excluded categories.”

It follows that, “given the technical difficulty of completely eliminating the influence of a training example on an already trained model—and the fact that xAI has never publicly claimed to have done so—any child sexual abuse material (CSAM) ingested during training prior to any potential deletion will likely continue to influence the model’s outputs, even after the original images have been removed from the public domain,” according to the complaint filed by Doe.

Destroy all Grok CSAM, Doe says

Through this proposed class action, Doe aims to put a stop to the harmful content generated by Grok. If the class action is certified, it will represent all victims whose childhood images were used to generate child sexual abuse material via Grok.

The complaint accuses xAI of violating federal laws regarding child sexual abuse material as well as “Masha’s Law“; these statutes grant survivors of such abuse the right to sue over the production, possession, and dissemination of such content. Doe’s attorney, Margaret E. Mabie, emphasized in a press release that these three acts constitute criminal offenses and that xAI is the party responsible for them.

If Doe prevails, xAI could be required to pay damages to any victim who can prove that Grok generated child sexual abuse material based on their images. Doe has also asked the court to order xAI to destroy all child sexual abuse material generated by Grok that is stored on its servers or used to train its models. Furthermore, she demands that xAI prevent Grok from generating such content; …a measure that, according to her complaint, would require banning Grok from generating any sexually explicit content, including “bikini photos,” non-consensual intimate imagery (NCII), and NSFW content promoted by Musk.

Sarah London, one of Doe’s attorneys, stated in the release that her client “has lived for nearly twenty years knowing that images of the worst event of her life are circulating among online predators and could resurface at any moment. xAI must be held accountable for knowingly training its models on images of the horrific abuse she endured—as well as on images involving other survivors included in this class action.”

X did not respond to a request for comment from Ars.