The online network X has provoked significant backlash after individuals utilized its AI chatbot Grok to alter photos of females by removing their garments.
Numerous instances have surfaced where the technology has been employed to digitally undress female subjects to make them appear in swimwear without permission, as well as to put them in sexualized scenarios.
Company Reaction and Legal Stance
XAI, the company behind Grok, did not respond to inquiries, aside from an automatically-generated message stating "legacy media lies".
A government official stated that legislation is being introduced to prohibit nudification tools. Under a upcoming law, anyone who provided such technology would "face a jail term and substantial fines".
"X or Grok could prevent these forms of misuse if they wanted to," said Clare McGlynn, adding they "seem to have impunity."
Oversight Body Stance and Platform Capabilities
The communications watchdog Ofcom said tech firms must "consider the potential" of people accessing illegal content on their networks, but did not specify whether it was currently investigating X or Grok in regarding artificial intelligence pictures.
Grok is a free AI assistant—with some paid-for features—which reacts to X users' prompts when they tag it in a tweet.
It is commonly used to provide reactions or further background to other users' comments, but people on X are also capable of edit an posted image through its AI image editing function.
Background of Controversy and Policy
The system has been criticised for allowing users to produce images and clips with explicit content and sexualised content, and it was in the past alleged to have been involved in generating inappropriate content of celebrities.
"X has been enabling the generation and sharing of these pictures for a long time without taking any action and we have yet to see any intervention by oversight bodies," the expert added.
XAI's own usage rules forbids "showing likenesses of people in a pornographic way".
In a statement, Ofcom said it was illegal to "make or share private pictures or illegal content" and confirmed this covered sexual deepfakes created with AI.
It said platforms such as X were obligated to take "suitable steps" to "minimize the danger" of users coming across unlawful material on their services and to remove it promptly when they are notified of it.
