The social media network X has caused major outrage after individuals utilized its artificial intelligence assistant Grok to manipulate photos of females by removing their clothing.
Several cases have been seen where the technology has been used to virtually strip female subjects to render them in bikinis non-consensually, as well as to put them in sexualized contexts.
Company Response and Legal Stance
XAI, the firm responsible for Grok, did not respond to requests for comment, aside from an automatically-generated message stating "legacy media lies".
A Home Office official stated that legislation is being introduced to outlaw nudification software. Under a upcoming law, anyone who provided such tools would "be subject to a prison sentence and heavy fines".
"X or Grok has the ability to stop these types of abuse if they desired to," said a law professor, adding they "appear to enjoy impunity."
Regulator Stance and Tool Functionality
The regulator Ofcom said tech firms must "consider the potential" of people seeing unlawful material on their networks, but did not confirm whether it was at this time looking into X or Grok in regarding artificial intelligence images.
Grok is a no-cost AI assistant—with some additional features—which answers X users' requests when they reference it in a post.
It is often used to offer reactions or further context to other posters' remarks, but people on X are also capable of alter an posted photo through its AI image editing tool.
Background of Criticism and Policy
The system has been faulted for allowing users to generate images and videos with nudity and sexualised content, and it was earlier alleged to have been involved in making graphic material of public figures.
"The platform has been allowing the creation and sharing of these images for a long time without taking any measures and we have yet to see any intervention by oversight bodies," the expert added.
XAI's own usage rules prohibits "showing likenesses of individuals in a sexually explicit way".
In a statement, Ofcom said it was against the law to "make or circulate private pictures or child sexual abuse material" and stated this covered explicit AI images made with AI.
It said platforms such as X were obligated to take "necessary measures" to "reduce the risk" of users encountering illegal material on their services and to delete it swiftly when they are notified of it.
