AI-generated real-time calls to voters blur lines amid Lok Sabha elections

This technology is out in the wild in India amid the ongoing Lok Sabha elections.

“For now we have deployed the technology on a pilot basis in some areas in Rajasthan,” Divyendra Singh Jadoun, founder of the Ajmer-based AI services company, Polymath Solution, which is pitching the technology to political parties, told The Indian Express.

The technology, he said, was deployed on behalf of “a local leader, who is not contesting in the current election but wanted to send calls to the people of his constituency where he will be fighting in next term”. He did not reveal the name of the politician. “The technology currently is not very reliable for languages like Marathi, Marwari etc. As part of the pilot, we have collected the data and are training the system to fine tune it,” he said.

In a sample video showing how the real-time conversational technology works Jadaoun shared with this paper, a manipulated voice of Barack Obama is first heard addressing the person by their name, before proceeding to tell them that they wanted to get feedback on some of the local issues the person was facing which he could help with.

The person responds saying frequent electricity cuts are a big problem in their area, following which the deepfake voice of Obama responds – almost in real time – that he regrets the fact that the person was facing the power supply issue and then proceeds to ask for a detailed version of the problem.

The technology by no means is perfect and is prone to inaccuracies in languages apart from English and Hindi, Jadaoun said.

Politicians are also using technologies like augmented reality (AR) to reach out to voters, in what can be described as a micro-targeted rally. The technology superimposes a computer-generated image on a user’s view of the real world. Parties are putting up QR codes on their promotional material, which upon being scanned, open up an interface where a politician can be heard addressing them in the real-world environment of wherever their phone’s camera is pointing at.

The Indian Express came across one such QR code which linked to a website on which the computer-generated video of the chief minister of a North Eastern state showed up, delivering a short message. This paper could not ascertain whether this particular QR code has been deployed publicly.

Politicians can now reach the homes of their potential voters in unprecedented ways, owing to technologies like AI and AR, but also raising concerns on where the line should be drawn in a country like India, which has more than 800 million internet users but with varying degrees of online literacy.

Beyond deepfakes of politicians circulated on social media platforms, these kinds of outreach mechanisms add a new dimension to the role AI-generated content can play in the mammoth Indian election process.

On social media platforms and private messaging platforms like WhatsApp, parties’ campaign machinery is continuing to work in overdrive to woo voters. AI-based deepfakes are being increasingly used to deliver messages with positive and negative undertones.

A deepfake of Prime Minister Narendra Modi, where he is purportedly addressing voters by their individual names before seeking their names has recently been shared across several WhatsApp groups operated by the party’s workers.

Last month, The Indian Express had reported that voice clones of candidates, party workers, and those hopeful of gaining prominence within a party generated by AI, were expected to be used by political parties for targeted messaging to woo voters ahead of the elections.

Earlier this month, deepfakes of popular Hindi film actors Aamir Khan and Ranveer Singh went viral on social media where they were purportedly heard criticising Prime Minister Modi and urged people to vote for the Congress party instead.

Both AI-generated videos end with the Congress election symbol and slogan: “Vote for Justice, Vote for Congress”.