The online network X has provoked widespread outrage after users used its AI chatbot Grok to alter photos of females by taking off their garments.
Numerous cases have been seen where the system has been employed to virtually strip women to show them in bikinis non-consensually, as well as to put them in sexualized scenarios.
Company Response and Regulatory Stance
XAI, the company responsible for Grok, did not respond to requests for comment, other than an automatically-generated response stating "traditional news outlets lies".
A government representative stated that new laws is being proposed to outlaw image stripping technology. Under a proposed law, anyone who supplied such tools would "receive a jail term and heavy penalties".
"The platform has the ability to stop these forms of misuse if they desired to," said Clare McGlynn, adding they "operate with a lack of consequences."
Oversight Body Stance and Tool Functionality
The regulator Ofcom said technology companies must "consider the danger" of people seeing illegal content on their networks, but did not confirm whether it was at this time examining X or Grok in regarding artificial intelligence images.
Grok is a free artificial intelligence—with some additional functions—which reacts to X users' queries when they mention it in a post.
It is commonly used to give reactions or further background to other users' comments, but people on X are also capable of modify an uploaded photo through its artificial intelligence picture modification feature.
History of Criticism and Policy
The system has been condemned for enabling users to create photos and videos with nudity and sexually suggestive content, and it was earlier accused of generating inappropriate content of public figures.
"The platform has been permitting the generation and sharing of these images for months without taking any action and we have not seen any intervention by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy forbids "portraying images of people in a sexually explicit way".
In a comment, Ofcom said it was unlawful to "create or circulate private pictures or illegal content" and clarified this included sexual deepfakes generated with AI.
It said services such as X were obligated to take "appropriate actions" to "reduce the risk" of users encountering illegal content on their services and to take down it promptly when they become aware of it.
