X admits Grok AI objectionable image issues, blocks 600 offending accounts: Sources

The government, though, was unconvinced by the company’s response, owing to a lack of any technical explanation behind the issue, and any steps to prevent Grok from generating such images in the first place, The Indian Express had learnt.

In its formal response, sent on Wednesday, X had acknowledged the severity of the issue and had committed to taking action against accounts creating such inappropriate prompts on its artificial intelligence platform. The company had assured the government that it will strictly comply with India’s Information Technology Act, 2000 and associated rules governing digital content.

On January 2, the IT Ministry had taken note of Grok being abused to generate objectionable pictures and videos of women, without their consent, while increasing the risk of exposure and harm, and had sent a missive to the company saying that it was not adhering to the country’s laws, raising red flags over “serious failure” of safeguards enforcement.

The issue became a global concern, with regulators in the European Union, United Kingdom and Malaysia also scrutinising the social media company.

The notice said that due to Grok’s actions, X was not adequately adhering to provisions of the Information Technology (IT) Rules, 2021, and the Bharatiya Nagarik Suraksha Sanhita, 2023, particularly in relation to obscene, indecent, vulgar, pornographic, paedophilic, or otherwise unlawful or harmful content which are potentially violative of current laws.

Users on X were increasingly tagging Musk’s AI chatbot Grok to generate sexualised and objectionable images of women without their consent, raising fresh concerns over AI misuse and platform safeguards. In several instances, users replied to photos of women, often taken from public posts, and prompted Grok to alter their appearance by adding revealing clothing or creating suggestive visuals, which the AI-bot promptly did. The AI-generated images appear publicly in the same threads, exposing the women to harassment without their approval or knowledge.

The government had also directed the company to undertake a comprehensive technical, procedural and governance-level review of Grok AI, including its prompt-processing, output generation, and image handling and safety guardrails, “to ensure that the application does not generate, promote or facilitate content which contains nudity, sexualisation, sexually explicit or otherwise unlawful content”.