The social media network X has sparked widespread outrage after individuals employed its AI chatbot Grok to manipulate photos of females by taking off their attire.
Numerous cases have been seen where the technology has been employed to digitally undress women to render them in bikinis without permission, as well as to put them in sexually explicit contexts.
Company Reaction and Regulatory Stance
XAI, the company behind Grok, offered no comment to requests for comment, other than an standard message stating "traditional news outlets lies".
A Home Office spokesperson stated that new laws is being proposed to ban nudification tools. Under a new law, anyone who supplied such technology would "be subject to a jail term and heavy penalties".
"The platform has the ability to stop these forms of abuse if they desired to," said a law professor, adding they "appear to enjoy a lack of consequences."
Regulator Stance and Tool Functionality
The regulator Ofcom said tech firms must "assess the risk" of people viewing illegal content on their platforms, but did not specify whether it was at this time investigating X or Grok in regarding artificial intelligence images.
Grok is a no-cost artificial intelligence—with some paid-for functions—which answers X users' prompts when they tag it in a post.
It is often used to provide commentary or further background to other posters' remarks, but people on X are also able to alter an posted picture through its artificial intelligence picture modification feature.
History of Controversy and Policy
The chatbot has been criticised for allowing users to generate images and videos with explicit content and sexually suggestive material, and it was previously accused of generating inappropriate material of public figures.
"X has been enabling the generation and sharing of these images for months without taking any steps and we have not seen any intervention by oversight bodies," the expert added.
XAI's own acceptable use policy bans "portraying images of individuals in a sexually explicit way".
In a comment, Ofcom said it was against the law to "create or share private pictures or illegal content" and confirmed this covered explicit AI images generated with AI.
It said networks such as X were obligated to take "necessary actions" to "minimize the danger" of users encountering illegal content on their services and to take down it swiftly when they are notified of it.
