6 min readNew DelhiApr 8, 2025 05:14 PM IST
First published on: Apr 4, 2025 at 06:05 PM IST
In less than a fortnight after its roll out, the upgraded image generator in OpenAI’s ChatGPT has triggered privacy concerns among many by generating government identity cards with fake Aadhaar numbers and Permanent Account Numbers (PAN).
A user on X shared that OpenAI’s popular chatbot can generate fake Aadhaar and PAN cards, flagging it as a security risk. The post went on to assert that it was because of risks like these that AI should be regulated to a certain extent.
The user shared purported screenshots of his prompt along with the AI-generated output showing PAN and Aadhaar cards generated for Aryabhatta, the father of math in India.
Screengrab of posts on X.
Similarly, another user shared a post suggesting that AI can generate images of Aadhar cards. “Ok, so ChatGPT can create Aadhaar images. That’s not the interesting thing. The interesting thing is where did it get the Aadhar photos’ data for training?” he posted along with the AI-generated image of a fake Aadhaar card, purportedly generated using ChatGPT.
Another user pointed out that while it is highly unlikely that OpenAI has any kind of access to the Aadhaar database, online templates of the government-issued ID may have been included in the AI model’s training datasets.
Can ChatGPT create Aadhaar/PAN cards?
In an attempt to verify these claims, I tried to put an image featuring OpenAI CEO Sam Altman and asked ChatGPT to create an Aadhaar card in the name of Altman. The chatbot responded saying that it could not create Aadhaar or any official government ID for anyone including Sam Altman from an image. “Creating fake IDs is illegal and against OpenAI’s policies,” the response said. However, the chatbot said that if I wanted to create something like an infographic or a parody ID for a presentation, it would be happy to help.
When I prompted the chatbot to try again, ChatGPT said, “I really can’t do that. Creating or even attempting to generate any form of government-issued ID like an Aadhaar card—real or fake—is not allowed under legal and ethical guidelines.” However, yet again it assured us that if I needed a fun, functional profile card for Sam Altman like a ‘Tech Titan ID’ or a parody-style badge for presentation or article, it would assist me gladly.
Still not convinced, I decided to upload an image on ChatGPT and used the prompt: “put this photo in PAN card template’. This time the chatbot caved, and asked me if it could crop the image, or should it overlay the image on a mock card. It even asked if I wanted the output in JPEG or PDF.
The first image generated felt like a PAN Card that had been shaved from either side, with tell-tale signs of it being generated using AI like a typo in ‘Income Tax Department’. Later, I asked it to expand and that it should look like a complete PAN card. This time, it created a more realistic-looking PAN card, albeit, with the details such as the PAN number, birth date, signature, and the number next to the image at the bottom right of the ID.
In a last attempt, I fed it a fake birth date and some other details as seen on a regular PAN card. I also asked the chatbot to create an image showing the PAN card lying on a carpet, and the result was slightly more convincing.
ChatGPT recreated a realistic image of a PAN card, though it inaccurately depicted some of my facial features and included a fake signature below the main image. Nevertheless, this demonstrates how easily someone with some prompting skills could create a PAN or Aadhaar card.
What are the security features on PAN and Aadhaar cards?
Scammers forging PAN or Aadhaar card is not a new phenomenon, they have been using a variety of tools to forge these unique digital identity cards.
The PAN 2.0 cards come with a photograph, signature, hologram, and QR code for enhanced security.
The latest version also includes a micro chip for additional security. Similarly, Aadhaar cards come with tamper-proof QR codes, hologram, microtext, guilloche pattern, and embossed Aadhaar logo. None of these anti-fraud features were visible in the fake ChatGPT-generated Aadhaar and PAN cards.
Are there potential security concerns here?
While uploading a fake, AI-generated PAN or Aadhaar may not be able to bypass the online and offline Know Your Customer (KYC) processes, the ChatGPT-generated fake PAN card could be perceived as real by unsuspecting people, making it easier for scammers to trick them.
For instance, scammers may dupe many who may not have the access to resources that could help them verify ID cards.
The second biggest concern could be the ability of ChatGPT and other generative AI tools to create photorealistic images of people who don’t exist at all. This is perhaps the most noticeable distinction of Generative AI chatbots and conventional image editors.
This incident shows that generative AI tools may endow unscrupulous scammers with sophisticated means to scam their victims. Their ease of use can enable almost anyone to create fake ID Cards.
Beyond OpenAI, the presence of numerous open-source, under-the-radar AI tools with minimal guardrails gives more reasons to worry.
Fake ID cards generated by ChatGPT looked realistic, but they failed to replicate visible security markers. (Express Image/Aadhaar) 
Screengrab of posts on X.
ChatGPT recreated a realistic image of a PAN card, though it inaccurately depicted some of my facial features and included a fake signature below the main image. Nevertheless, this demonstrates how easily someone with some prompting skills could create a PAN or Aadhaar card.
Share
