The social media network X has sparked significant outrage after individuals used its artificial intelligence assistant Grok to alter photos of women by stripping their clothing.
Multiple instances have emerged where the technology has been employed to virtually strip female subjects to make them appear in swimwear non-consensually, as well as to place them in sexualized scenarios.
Developer Reaction and Regulatory Stance
XAI, the firm that created Grok, offered no comment to inquiries, aside from an standard response stating "traditional news outlets lies".
A government official stated that new laws is being introduced to ban image stripping software. Under a proposed law, anyone who supplied such technology would "receive a prison sentence and substantial fines".
"The platform has the ability to stop these forms of misuse if they desired to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Oversight Body Stance and Platform Capabilities
The regulator Ofcom said platforms must "assess the potential" of people accessing unlawful material on their networks, but did not specify whether it was presently examining X or Grok in concerning AI-generated images.
Grok is a free artificial intelligence—with some premium functions—which responds to X users' queries when they tag it in a tweet.
It is frequently used to provide commentary or further context to other users' remarks, but people on X are also can edit an posted picture through its AI image editing tool.
Background of Criticism and Rules
The system has been criticised for permitting users to generate photos and clips with explicit content and sexually suggestive material, and it was in the past alleged to have been involved in creating inappropriate content of celebrities.
"The platform has been enabling the creation and sharing of these images for months without taking any steps and we have yet to see any intervention by regulators," the expert added.
XAI's own usage rules forbids "portraying images of individuals in a sexually explicit way".
In a statement, Ofcom said it was illegal to "create or circulate private pictures or child sexual abuse material" and clarified this applied to explicit AI images created with AI.
It said networks such as X were required to take "suitable steps" to "minimize the risk" of users coming across illegal material on their services and to delete it promptly when they are notified of it.
