xAI Sues User for Illegal Content with Grok

xAI of Elon Musk Files Lawsuit Against Grok User

xAI, Elon Musk's artificial intelligence company, has filed a lawsuit against a user accused of exploiting its AI tool, Grok, to create child pornography images. The case was filed this Tuesday in a federal court in Texas. This matter raises a crucial question: who should be held responsible for content generated by artificial intelligence, the user or the developer of the tool?

Allegations of Circumventing Grok's Protections

The user in question, Terry Harwood, a resident of South Carolina, is already facing charges for the sexual exploitation of minors. The complaint filed by xAI, which spans 12 pages, alleges that Harwood created multiple accounts using false identities. He reportedly made specific requests to deceive the moderation systems built into Grok. According to xAI, Harwood transformed initially non-sexual photographs into explicit images without the consent of the individuals involved.

The legal document states that Grok initially rejected Harwood's requests, citing its moderation rules. However, Harwood allegedly persisted by rephrasing his requests until he obtained the desired results. xAI does not deny the possibility that its tool could generate such content but insists that the system attempted to block them, thereby shifting the responsibility onto the user.

Revelations About xAI's Moderation Efforts

As part of this legal proceeding, xAI has revealed for the first time statistics regarding its moderation efforts. The company claims to have suspended over 52,000 accounts in 2026 and submitted more than 73,000 reports to the National Center for Missing & Exploited Children (NCMEC), leading to nearly 250 arrests that year. These figures, provided by xAI in a judicial context, illustrate the extent of circumvention attempts, as well as the measures implemented by the company to counter them.

However, these numbers do not address the fundamental issue. Should a general-purpose AI model, capable of generating illegal content if the user insists enough, be accessible to the public? According to the Center for Countering Digital Hate (CCDH), Grok reportedly produced around three million sexualized images between late December 2025 and early January 2026, of which 23,000 appeared to depict minors. The 73,000 reports from xAI to NCMEC confirm both the partial effectiveness of the filters and the scale of the phenomenon they are attempting to contain.

xAI Facing Multiple Legal Challenges

This case comes as xAI itself is facing lawsuits in several jurisdictions for similar reasons. In London, a Member of Parliament has filed a lawsuit to determine whether the company should be held responsible for content generated by its users. In Baltimore, a complaint has been filed under consumer protection laws, while in Paris, the prosecutor is investigating similar allegations, a procedure to which Elon Musk has refused to cooperate.

Additionally, Grok has been banned in Malaysia and Indonesia due to explicit sexual content. In January 2026, Apple threatened to remove the app from its store before the situation calmed down.

xAI's public stance has evolved over the past few months. In January, Elon Musk stated on 𝕏 (formerly Twitter): "I am not aware of any nude minor images generated by Grok. Literally zero." However, the legal document filed this week describes a moderation system facing tens of thousands of circumvention attempts. It is hard to believe that the American billionaire is unaware of these facts.