‘AI detects 96% of policy-violating YouTube content’: Google India Trust and Safety lead

Over 2.36 million policy-violating apps were further blocked on Google Play Store by leveraging AI for real-time detection. 

Kartikeya Mitra, the Trust and Safety lead at Google India, spoke to indianexpress.com via email to share more insights into the company’s ongoing efforts to combat emerging online scams in India. Below are edited excerpts from the interview:

Could you break down the most common types of attempted fraud you’re seeing? How have your detection patterns evolved over the past year?

Mitra: Online fraud is evolving rapidly, with scammers leveraging major events and advanced technology to exploit users. Fake donations and travel scams spike during festivals like Diwali, as fraudsters create counterfeit charity sites and fraudulent bookings. AI-generated celebrity investment scams use deepfake technology to impersonate public figures promoting fake cryptocurrency schemes. Counterfeit e-commerce and travel websites deceive victims with unrealistically low prices, while remote access tech support fraud enables scammers to pose as support representatives and gain control over personal devices. Job scams trick applicants into paying fees or engaging in illegal activities, and predatory loan apps trap users with exorbitant interest rates while misusing personal data for harassment.

Google is actively combating online fraud through AI-driven detection, strict enforcement, and large-scale user awareness initiatives. With fraudsters constantly evolving their tactics, we have strengthened our systems to proactively detect threats, remove malicious content, and protect users from financial scams. Our efforts go beyond detection, focusing on educating users and fostering digital resilience to ensure a safer online experience.

From a detection standpoint, we’ve significantly strengthened our protective measures:

– AI-driven systems now detect 96% of policy-violating content on YouTube.

– Google Play Protect scans 200 billion apps daily, blocking 13.9 million harmful installations and identifying 13 million new malicious apps.

– Strict enforcement led to the removal of 2.36 million policy-violating apps and the banning of 158,000 developer accounts.

– Google Pay’s fraud prevention systems protected ₹13,000 crores in transactions and issued 41 million security warnings to users.

Empowering Users Through Digital Literacy:

– The DigiKavach campaign has reached 177 million Indians, spreading awareness about financial fraud.

– Google’s philanthropic arm, Google.org has provided grant support to two nonprofits CyberPeace Foundation, which has equipped over 197,000 individuals with fact-checking and responsible online behavior skills, and The Asia Foundation, which is working with local implementing partners, Foundation for MSME Clusters and CUTS International to equip 45,000 MSMEs with the tools and knowledge to combat cyber threats.

Google remains committed to enhancing security, enforcing strict policies, and educating users to build a safer digital ecosystem.

Predatory loan apps have been a particular concern in India. Has Google’s AI detection system been specifically trained to identify and remove these apps? What are the unique challenges in this space?

Mitra: Google has taken significant steps to combat predatory loan apps in India, leveraging AI-driven detection systems to identify and remove harmful applications before they reach users. Machine learning algorithms analyze app behavior, requested permissions, and user feedback to flag suspicious apps. 

To address these challenges, Google enforces strict compliance requirements for personal loan apps on the Play Store, mandating regulatory documentation. Our proactive measures include real-time AI interventions, close collaboration with financial regulators and law enforcement, and ongoing efforts to shut down fraudulent networks. Despite the high volume of emerging threats, Google remains committed to safeguarding users by ensuring financial apps adhere to strict safety policies and do not exploit vulnerable borrowers.

The pilot for the Enhanced Play Protect program, a security feature for Android devices, was launched in late 2024 to protect users from harmful apps by blocking their installation. It has already prevented over 309,000 malicious app installations. The program scans apps from both the Google Play Store and external sources, analyzing app permissions in real-time to identify potential threats. It specifically targets permissions often misused for financial fraud, such as those granting access to SMS messages, notifications, or screen content. If an app from outside the Play Store requests these sensitive permissions, Play Protect will likely block its installation to safeguard users against scams and fraud. Despite these measures, scammers continue to adapt their strategies, rebranding apps, blurring lines between legitimate and deceptive practices, and overwhelming the market with misleading financial services. Many of these apps exploit users through unsolicited loans, excessive interest rates, and misuse of personal data for harassment and extortion.

Festival-related scams spike during certain times in India. How does Google adjust its detection systems to account for these seasonal patterns?

Mitra: Festival-related scams surge during major events like Diwali, Dussehra, Navratri, and Dhanteras, as scammers exploit increased shopping and charitable giving. To counter this, Google enhances its detection systems, deploying advanced algorithms to monitor and remove fraudulent websites, phishing attempts, and impersonation scams—particularly those targeting UPI payment systems. Google Play Protect, which scans 200 billion apps daily, enables real-time fraud detection, while YouTube removed 4.1 million channels and 85,000 videos in Q3 2024 for violating scam policies, many linked to festival-related fraud.

Beyond detection, Google prioritizes user education and proactive protection. Google Pay issued 41 million fraud alerts, preventing financial scams during peak festive seasons. Strict policies prohibit products or services that exploit sensitive events, while tools like “About this Result” help users verify website credibility. Large-scale initiatives like DigiKavach further educate users on identifying scams, reinforcing Google’s commitment to safeguarding digital transactions and ensuring a safer online experience.

Collaboration with policymakers seems to be central to Google’s strategy. Could you share a specific example of how regulatory input has shaped Google’s approach to fraud prevention?

Mitra: Collaboration with policymakers is the cornerstone of Google’s strategy to combat online fraud effectively. A notable example of this collaboration is our partnership with the Indian Cyber Crime Coordination Centre. By onboarding Google Pay onto the National Cybercrime Reporting Portal, we facilitate the exchange of critical information regarding fraudulent financial activities. This partnership enhances our ability to investigate and mitigate scams, ensuring that we can respond quickly to emerging threats.

Furthermore, we are actively participating in the Safer Internet India Coalition, which brings together various stakeholders, including digital service providers, telecom companies, banks, and other organizations specializing in online trust and safety. This coalition aims to promote user welfare, trust, and safety in the Indian digital space. Regulatory input from these collaborations helps shape our policies and practices, ensuring they align with the latest insights into emerging threats and best practices for fraud prevention.