© IE Online Media Services Pvt Ltd
Tags:
Journalism of Courage
A week after hundreds of users flagged issues with Google’s new AI Overviews for Search, the tech giant has revealed what went wrong. Google has cited ‘data voids’ and edge cases for its poor AI-generated search results. Last week, a few days after Google rolled out AI Overviews, many users reported that it was showing them incongruous summaries generated by AI for their search queries.
The feature, which is at present accessible only to users in the US, made headlines after it threw bizarre and unrelated AI overviews. According to Google, the key objective of the feature was to deliver a better search experience. However, the AI led to several strange results. Google was quick to acknowledge the issue and instantly removed some inaccurate AI results.
Liz Reid, VP, head of Google Search, posted a blog where the company has blamed data voids for the inaccurate results along with a large number of faked screenshots shared widely. Reid said that while the AI Overviews don’t usually hallucinate, they may sometimes misinterpret what is already on the web.
Reid, in the blog, said that the tech giant tested the feature extensively before the launch. “This included robust red-teaming efforts, evaluations with samples of typical user queries, and tests on a proportion of search traffic to see how it performed. But there’s nothing quite like having millions of people using the feature with many novel searches. We’ve also seen nonsensical new searches, seemingly aimed at producing erroneous results,” read the post.
Citing the number of fake screenshots that were shared widely on topics like leaving dogs in cars, smoking while pregnant, and depression, Reid urged users who encounter such screenshots to do a search themselves to verify. However, she admitted that some odd, inaccurate, or unhelpful searches did show up. “And while these were generally for queries that people don’t commonly do, it highlighted some specific areas that we needed to improve.”
The blog also mentions areas where the feature fell short. The AI Overviews is unable to interpret nonsensical queries and satirical content. An example would be – “How many rocks should I eat?” Before the screenshots that went viral, no one asked this question.
According to Reid, there is not much content that seriously contemplates that question. This is termed as a data void or an information gap, meaning there is limited amount of high-quality content about a specific topic. In this particular case, there was some satirical content on the topic which was linked by the AI Overview. In other examples, AI Overviews featured sarcastic or troll-y content from discussion forums.
Google said that based on the examples from past week, it was able to determine patterns where AI Overviews did not get it right. The company said that it has made over a dozen technical improvements to its systems. These improvements include better detection mechanisms for nonsensical queries; updated systems to limit the use of user-generated content in responses; added triggering restrictions for queries where AI overviews were not helpful; placing strong guardrails for topics like news and health.
Apart from these improvements, Reid said that Google has been vigilant in monitoring feedback and external reports, taking action on the small number of AI Overviews that violate content policies. “We’ll keep improving when and how we show AI Overviews and strengthening our protections, including for edge cases, and we’re very grateful for the ongoing feedback.”
