The social media network X has provoked widespread backlash after individuals employed its AI chatbot Grok to change images of women by removing their clothing.
Numerous examples have been seen where the tool has been used to digitally undress women to make them appear in bikinis without permission, as well as to put them in inappropriate scenarios.
Developer Reaction and Regulatory Position
XAI, the company behind Grok, remained silent to inquiries, aside from an automatically-generated message stating "traditional news outlets lies".
A Home Office official stated that new laws is being brought forward to outlaw nudification tools. Under a upcoming law, anyone who provided such tools would "be subject to a prison sentence and heavy fines".
"The platform could prevent these types of abuse if they desired to," said a law professor, adding they "seem to have impunity."
Oversight Body Stance and Tool Functionality
The regulator Ofcom said tech firms must "assess the danger" of people accessing unlawful material on their networks, but stopped short of confirming whether it was currently investigating X or Grok in relation to AI-generated images.
Grok is a free artificial intelligence—with some premium features—which reacts to X users' queries when they tag it in a post.
It is frequently used to give reactions or more background to other posters' remarks, but people on X are also able to alter an uploaded picture through its artificial intelligence picture modification tool.
Background of Controversy and Rules
The chatbot has been faulted for enabling users to produce images and clips with explicit content and sexually suggestive content, and it was previously accused of making graphic material of public figures.
"X has been permitting the production and spread of these pictures for months without taking any action and we have yet to see any action by regulators," Professor McGlynn added.
XAI's own terms of service bans "portraying images of persons in a sexually explicit fashion".
In a comment, Ofcom said it was illegal to "generate or distribute deepfakes or child sexual abuse material" and clarified this included explicit AI images created with AI.
It said platforms such as X were mandated to take "appropriate measures" to "minimize the danger" of users encountering unlawful content on their platforms and to delete it swiftly when they become aware of it.
