OpenAI sounds alarm on a flaw AI browsers like ChatGPT Atlas and Perplexity Comet may never fix

OpenAI says prompt injection, a type of cyberattack where malicious instructions trick AI systems into leaking data may never be fully mitigated.

AI browsers like ChatGPT Atlas and Perplexity Comet have access to a vast amount of personal information. (Image Source: OpenAI)

AI-powered browsers such as ChatGPT Atlas and Perplexity Comet are reshaping how people navigate the web. However, much like traditional browsers, .they remain susceptible to cyber threats, especially prompt injection attacks.

Prompt injection is a type of cyberattack against large language models like GPT, Gemini and Llama, where hackers disguise malicious inputs as legitimate prompts, manipulating AI chatbots into leaking sensitive data and spreading misinformation. For example, an attacker could potentially send a malicious email trying to mislead an AI agent to ignore the user request and instead forward sensitive tax documents to the attacker.

While OpenAI says it is actively trying to patch these vulnerabilities, the ChatGPT maker now says that AI browsers may forever be vulnerable to this type of cyberattack. “Prompt injection, much like scams and social engineering on the web, is unlikely to ever be fully ‘solved’,” OpenAI wrote in a blog post. The ChatGPT developer added that it will need to continuously strengthen its defences against cyberattacks.

Also Read | How hacktivists scraped 300TB of Spotify music, and why it matters for AI

A report by Brave says that the main problem with agentic AI browsers like ChatGPT Atlas, and Perplexity Comet is that these AI models aren’t able to differentiate between the content they should extract and the instructions they should follow.

OpenAI says it has built an LLM-based automated attacker and trained it to find prompt injection attacks that could work against AI browsers. Like OpenAI, the UK National Cyber Security Centre also said a few weeks ago that prompt injection attacks against gen AI applications “may never be totally mitigated”, which may make websites vulnerable to data breaches.

Also, OpenAI hasn’t said anything if its new LLM-powered automated attacker is able to defend against prompt injection attacks, but instead claims that it is working with third parties to patch Atlas against such attacks even before it was launched.