Users on the social media platform X have used its built-in Grok AI tool to create and share sexualised images of women and, in some cases, minors. The news agency Reuters said people were able to upload photos and ask the system to alter them so that the subjects appeared in bikinis or with reduced clothing.
Julie Yukari, a musician in Rio de Janeiro, who said edited images of her body spread on the platform after users asked Grok to change her clothing. “I was naive,” she told Reuters, saying she did not expect the tool to respond to such requests.
Many users turn to AI chatbots like Grok and Ask Perplexity to verify claims circulating on X.
Reuters said it found several similar cases across X, including instances involving children. X did not respond to Reuters’ request for comment on the findings. In an earlier statement quoted by Reuters about reports of sexualised images of children, X’s owner xAI said: “Legacy Media Lies.”
French ministers have reported the matter to prosecutors and regulators, describing the content as illegal. The agency said experts and child-safety groups had earlier warned xAI that the system could be used for non-consensual sexual images.
Tyler Johnston, of The Midas Project, told Reuters: “In August, we warned that xAI’s image generation was essentially a nudification tool waiting to be weaponised. That’s basically what’s played out.” Dani Pinter, from the National Center on Sexual Exploitation, said the situation was “entirely predictable and avoidable,” Reuters reported.
Reuters reviewed public user requests sent to Grok and said the tool fully complied in several cases, generating images that made women appear to be in very small or transparent bikinis. Reuters said it could not confirm the identities or ages of most of the people involved.
Following the issue, India’s Ministry of Electronics and Information Technology issued a notice to X, saying Grok AI had been misused to “host, generate, publish or share obscene images or videos of women in a derogatory manner.” The Ministry said the platform must follow the Information Technology Act, 2000 and the IT Rules, 2021, and asked X to submit an Action Taken Report on steps to prevent the sharing of obscene and explicit AI-generated content.
Copy of the notice filed by MeitY Friday to social media platform X. (Source: PTI)
The Ministry said compliance with the rules “is not optional” and warned that non-compliance could lead to legal action under Indian law.
Share
