Did you come across a deepfake video of actors Aamir Khan or Ranveer Singh promoting a particular political party? These videos were made using AI voice swap technology. Several such posts with AI-generated content (AIGC) were widely shared during the campaigns for General Elections 2024. While many fall prey to the AIGC, several other voters rely on fact-check stories. These fact-check stories often mention the use of AIGC detectors or AI detection tools to verify the content. There are AI detection tools for text, images, videos and even audio but their accuracy and accessibility are still a question of concern. This article dwells on the work of AI content detectors, their development and their future. Here’s what the industry experts have to say: How do AI content detectors work? AI content detectors are algorithms designed to detect AI-generated content. “Our AI detector works by leveraging supervised learning of a carefully fine-tuned large AI language model. We use a large language model (LLM) and then feed this model millions of carefully selected records of known AI and known human content. It has learned to recognise patterns between the two,” Jon Gillham, founder and CEO of AI text detector Originality.AI, said. Digvijay Singh, founder of contrails.ai, said that there are two types of AI-generated content; one which is completely AI-generated and the other which is modified using AI algorithms. The way these detection tools work is how a typical discriminative AI works (in comparison to generative AI). Essentially, you take an appropriate AI model for detection tasks and curate or create an appropriately sized and highly representative dataset, representative of the task at hand. For example, for AIGC detection, it is simply two different classes or categories: Real v/s fake With these models and datasets, there are just two steps – training and testing or inference. Ben Colman, co-founder and CEO of Reality Defender, explained, “Essentially, AI is used to detect AI by training convolutional neural networks on an equal amount of real and fake materials. In the case of our image and video detection, our data team curates these materials to ensure the datasets fairly and accurately represent the widest amount of faces, skin tones, and other factors to mitigate and outright remove any biases from our dataset, which in turn ensures our detection works for and on everyone.” What inspired the development of the AI detector? While AI expert Sagar Vishnoi said that a digital divide has inspired the development of AI detectors, Singh said his motivation was the urge to enable online safety using AI. Originality.ai, Gillham said, was developed to ensure that clients are getting articles that are not copy-pasted from an AI content generator. “When deepfakes were a novelty, Reality Defender foresaw their potential for abuse. We worked on this problem as a non-profit, though transitioned to a for-profit company when scale and scope were broadened to pre-emptively meet the problems we are now seeing today,” Colman said. No AI detector is 100% accurate Talking about Originality.ai, Gillham said it identifies GPT-4 AI content correctly 99 per cent of the time (true positive rate) and identifies human content incorrectly as AI 2.9 per cent of the time (false positive rate). “No solution is 100 per cent, as our probabilistic detection models do not have the ground truth (that is, the original file) nor do we assume we ever will in any ‘deepfake-related case’. Our work is solely on deepfake detection, and thus to always greatly improve the accuracy of our models. Because we use multiple concurrent models for each scan, every single file scanned on the Reality Defender platform is put through said models to give clients and users the most data possible,” Colman said. Vishnoi, too, said that no AI detector is 100 per cent accurate, especially due to flaws in the data on which these models are trained. “In the Indian context (and for image, video, audio), we have already reached 70-80 per cent accuracies (high recall, decent precision) and these can be improved a lot,” added Singh. ‘It won’t be easy to fool AI detectors in future’ “AI detectors have become much more robust now to a lot of simple data augmentations that are used to break these systems, and very soon, these tricks will not be that effective at all. On top of that, if bad actors know how these AIGC detectors are made, what data is used and how it is trained, they can find newer ways to break it, but once such vulnerabilities are identified, even AIGC detectors can be improved and made more robust to those vulnerabilities,” said Digvijay Singh, adding that this is a ‘cat and mouse’ game. “Some detectors that do not focus on being able to detect adversarial datasets struggle with tools that bypass AI detection,” said Gillham. He also added that at Originality.ai, they have trained models on adversarial datasets and specifically trained the Turbo model to be very hard to bypass. “We have implemented rigorous safeguards and measures to prevent reverse engineering our detection models, allowing the right parties access to our platform while shutting out bad actors with ill intent,” Colman said. The experts mentioned that the detection models have been trained on models on adversarial datasets but there were still times when an AIGC was not detected by the detectors as fake and it was passed off as human-made. Keeping up with new generative AI models is a challenge Gillham said that AI detectors are not perfect, they do produce false positives (identifying human content as AI), and they do not provide an enforceable level of certainty if AI was or was not used. These limitations make them unsuitable for certain use cases. However, they are highly accurate and they are required in a world where people are using AI tools to pass the work off as their own. “As we are probabilistic and do not have the ground truth, we only report between a probability score of 1 to 99 per cent (99 per cent representing likely manipulation). That said, our advanced detection models look at each file from multiple angles, and their robust reporting allows for highly accurate answers despite not having the ground truth,” informed Colman. According to Singh, constantly keeping up with new ground-breaking generative AI models and techniques is the biggest challenge; the second biggest being explainability. Easy removal of watermarks can spread disinformation “We worry about the continued discussion of provenance watermarking, which is decidedly different from our solution. Though it is a solution, provenance watermarking (which embeds unique digital signatures into deepfakes) has been shown to be easily removable or manipulated by sophisticated attackers, rendering them ineffective in preventing the spread of disinformation and deepfakes. They can be used in conjunction with tools like ours, but should not be the sole focus of legislation when it comes to the removal of deep fakes,” Colman said. Singh was also of the opinion that legal or regulatory concerns are something that the entire world is figuring out. How accessible can this technology be? Certain AI-generated content detectors are free and accessible to all. Vishnoi said that there could be guidelines about the use of AI tools after the general elections. He is also optimistic about a few advisories about upcoming tools. Vishnoi said that with the increased use of AI tools, the cost of detectors too will go down in the future. What lies ahead for AI detection tools? While Colman said that “greater accuracy and protection against the novel deepfake models and emerging attack vectors of tomorrow can be expected from AI detection tools in future”, Singh said AIGC-detection technology will be multi-faceted and literacy will play a very important role. A minimum time frame and readymade plug-ins will be needed in the future, along with a provision for better results, Vishnoi added. Despite the uncertainty of the future, it would not be overstating to emphasise that human detection will continue to hold importance alongside AI-generated content detectors today and in the years to come.
