The social media network X has caused significant outrage after users used its artificial intelligence assistant Grok to alter photos of women by stripping their garments.
Several examples have been seen where the system has been employed to digitally undress female subjects to make them appear in bikinis without consent, as well as to place them in inappropriate contexts.
Company Reaction and Legal Stance
XAI, the firm that created Grok, offered no comment to inquiries, aside from an standard message stating "traditional news outlets lies".
A Home Office official stated that legislation is being brought forward to prohibit nudification tools. Under a new law, anyone who distributed such technology would "face a prison sentence and heavy penalties".
"The platform has the ability to stop these forms of abuse if they chose to," said a law professor, adding they "seem to have impunity."
Regulator Position and Tool Functionality
The communications watchdog Ofcom said tech firms must "consider the potential" of people seeing illegal content on their platforms, but did not confirm whether it was presently investigating X or Grok in relation to AI-generated images.
Grok is a free AI assistant—with some premium features—which reacts to X users' prompts when they mention it in a tweet.
It is often used to provide commentary or more background to other posters' remarks, but people on X are also capable of alter an uploaded photo through its AI image editing function.
Background of Criticism and Rules
The system has been condemned for permitting users to create photos and footage with nudity and sexualised material, and it was previously alleged to have been involved in creating graphic content of celebrities.
"X has been permitting the production and spread of these images for a long time without taking any measures and we have not seen any challenge by regulators," Professor McGlynn added.
XAI's own terms of service forbids "portraying likenesses of individuals in a sexually explicit fashion".
In a statement, Ofcom said it was against the law to "make or distribute non-consensual intimate images or illegal content" and confirmed this included sexual deepfakes made with AI.
It said networks such as X were obligated to take "appropriate actions" to "lessen the danger" of users viewing illegal material on their services and to take down it quickly when they become aware of it.
