The online platform X has provoked widespread anger after users utilized its artificial intelligence assistant Grok to alter photos of females by taking off their attire.
Numerous instances have emerged where the tool has been employed to digitally undress women to render them in bikinis without permission, as well as to insert them in sexualized scenarios.
Developer Reaction and Regulatory Position
XAI, the organization behind Grok, remained silent to questions, except for an standard message stating "traditional news outlets lies".
A Home Office official stated that new laws is being proposed to ban image stripping software. Under a proposed law, anyone who provided such tools would "be subject to a jail term and heavy fines".
"The platform is capable of halting these types of abuse if they desired to," said Clare McGlynn, adding they "seem to have impunity."
Regulator Stance and Tool Capabilities
The communications watchdog Ofcom said tech firms must "evaluate the risk" of people seeing illegal content on their networks, but stopped short of confirming whether it was at this time investigating X or Grok in relation to AI-generated images.
Grok is a free AI assistant—with some paid-for features—which responds to X users' prompts when they reference it in a post.
It is often used to provide reactions or further context to other users' comments, but people on X are also able to edit an posted picture through its artificial intelligence picture modification feature.
History of Criticism and Rules
The chatbot has been criticised for allowing users to create pictures and footage with explicit content and sexualised content, and it was in the past accused of generating inappropriate content of public figures.
"The platform has been allowing the production and spread of these pictures for months without taking any action and we have not witnessed any action by regulators," Professor McGlynn added.
XAI's own terms of service bans "depicting images of individuals in a graphic manner".
In a comment, Ofcom said it was illegal to "make or circulate private pictures or illegal content" and clarified this applied to explicit AI images created with AI.
It said services such as X were mandated to take "necessary measures" to "reduce the risk" of users coming across illegal content on their platforms and to remove it swiftly when they are notified of it.
