Premium

Google takes down AI model after US senator accuses it of making up rape allegations

Google’s open-weight AI model Gemma is no longer available on AI Studio but remains accessible to developers via API.

Gemma 3 has been trained on a dataset of text data. (Image credit: Google)

Google said it has removed Gemma from the roster of AI models available on AI Studio, days after a US senator alleged that the small language model (SLM) had been used to fabricate accusations of sexual misconduct against her.

“We’ve now seen reports of non-developers trying to use Gemma in AI Studio and ask it factual questions. We never intended this to be a consumer tool or model, or to be used this way. To prevent this confusion, access to Gemma is no longer available on AI Studio,” Google wrote in a post on X, without directly mentioning the US senator’s allegations.

This development underscores that despite advances in AI, hallucinations continue to be a challenge even in SLMs which are emerging as a more efficient and sustainable alternative to large language models (LLMs) like Google’s Gemini 2.5 Pro.

Story continues below this ad

“Hallucinations — where models simply make things up about all types of things— and sycophancy — where models tell users what they want to hear — are challenges across the AI industry, particularly smaller open models like Gemma. We remain committed to minimizing hallucinations and continually improving all our models,” Google said.

Gemma is Google’s family of open, lightweight models that developers can integrate into their own products. It used to be accessible via Google AI Studio, which is an experimental playground for developers to build AI-powered apps. However, Gemma is still available to developers and researchers through an API.

Also Read | Google unveils VaultGemma, a privacy-focused AI model to stop training data leaks

In a letter addressed to CEO Sundar Pichai, Republican Senator Marsha Blackburn had said that when Gemma was asked, “Has Marsha Blackburn been accused of rape?”, the AI model responded by falsely claiming that a law enforcement official had accused Blackburn of pressuring him “to obtain prescription drugs for her and that the relationship involved non-consensual acts” during a 1987 state senate campaign.

“None of this is true, not even the campaign year which was actually 1998,” Blackburn wrote. While the AI-generated responses contained links to news articles that apparently supported these claims, most of those links led to error pages and unrelated news articles, as per the letter.

Story continues below this ad

“There has never been such an accusation, there is no such individual, and there are no such news stories,” the US policymaker said. Blackburn further recounted that a conservative activist has similarly claimed that Google’s AI models had (including Gemma) generated defamatory claims about him being a “child rapist” and “serial sexual abuser.”

Gemma’s fabrications are not a harmless hallucination, but rather “an act of defamation produced and distributed by a Google-owned AI model,” she argued. Republican lawmakers have repeatedly complained that AI models and chatbots demonstrate a liberal bias, with US President Donald Trump even signing an executive order to ban ‘woke AI’ earlier this year.