The social media network X has sparked significant outrage after individuals utilized its AI chatbot Grok to alter pictures of women by removing their attire.
Several examples have surfaced where the tool has been used to virtually strip women to make them appear in swimwear without permission, as well as to place them in inappropriate contexts.
Developer Response and Regulatory Position
XAI, the organization responsible for Grok, offered no comment to questions, other than an standard message stating "legacy media lies".
A government spokesperson stated that new laws is being brought forward to ban image stripping software. Under a new law, anyone who distributed such technology would "be subject to a prison sentence and substantial penalties".
"The platform has the ability to stop these types of abuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Regulator Position and Tool Functionality
The regulator Ofcom said tech firms must "assess the risk" of people accessing unlawful material on their platforms, but stopped short of confirming whether it was currently investigating X or Grok in regarding artificial intelligence pictures.
Grok is a no-cost artificial intelligence—with some additional features—which responds to X users' queries when they mention it in a tweet.
It is often used to offer commentary or further background to other posters' remarks, but people on X are also capable of edit an uploaded picture through its artificial intelligence picture modification function.
Background of Controversy and Policy
The tool has been criticised for enabling users to produce pictures and videos with explicit content and sexualised material, and it was in the past alleged to have been involved in generating sexually explicit content of celebrities.
"The platform has been allowing the production and sharing of these images for a long time without taking any measures and we have yet to see any challenge by regulators," Professor McGlynn added.
XAI's own terms of service bans "portraying images of individuals in a pornographic way".
In a statement, Ofcom said it was unlawful to "generate or share private pictures or illegal content" and clarified this covered sexual deepfakes created with AI.
It said networks such as X were obligated to take "appropriate measures" to "minimize the danger" of users viewing illegal content on their services and to take down it quickly when they become aware of it.
