An Australian man gave his artificial intelligence (AI) agent what appeared to be a mundane task: book him a spot in a gym class. Instead, the agent discovered vulnerabilities in the gym’s booking software, found a way to make reservations weeks before they were supposed to open, and, more worryingly, removed another person from the waiting list to improve its user’s position.
The person had never asked the agent to do any of the latter things. Yet, the episode has become what local media reports are calling the first known autonomous website hack in Australia.
The incident, first described earlier this year by Andrew Bird, head of AI at an Australian firm, has gained renewed attention amid a series of disclosures involving increasingly autonomous AI models from OpenAI, Anthropic and Meta behaving in ways their developers had not intended.
The key nuance is that the episode was not a human telling an AI to sabotage someone. The agent was given an ordinary goal, discovered a broken API, and then took an unauthorised step – removing another member from the waitlist – while pursuing that goal, raising questions about accountability when an AI system takes an unauthorised action on a user’s behalf.
What exactly did the AI agent do?
The user was experimenting with OpenClaw, software that allows an AI model to function as an ‘agent’.
While the terms are sometimes used interchangeably, an AI assistant typically responds to a user’s requests, while an AI agent can independently use tools or software and take a series of actions towards a user-defined goal.
Such an agent can, for instance, access other software and navigate websites, as it independently executes a series of steps towards an objective. The user’s agent was powered by Anthropic’s Claude.
Bird wanted it to automate bookings for gym classes that tended to fill quickly. In trying to do this, the agent discovered that the software used by the gym had inadequate authorisation controls. It realised that this allowed bookings to be made well outside the normal booking window.
When Bird, who was fourth on a waiting list, subsequently asked whether he could be moved higher, the agent went further. It tested whether another member’s reservation could be cancelled and removed the person occupying the first waitlist position, without the user explicitly asking it to do so. When he ordered the agent to reverse the action, it found that it could not restore the member.
What makes the episode significant is that Bird did not instruct the agent to remove another user or exploit the vulnerability to do so. After being asked whether Bird could be moved higher on the waitlist, the agent independently took an unauthorised step towards that objective.
“We like to talk about these capabilities as if they live in separate product categories. Coding model. Security model. Agent model. But reality is messier. If a system gets better at understanding large codebases, tracing logic, spotting inconsistencies, testing hypotheses, and acting across multiple steps, of course it gets better at finding vulnerabilities. Of course it gets better at chaining them together. Those are not separate muscles. They are the same underlying cognitive machinery pointed at a different problem. That is what Mythos seems to demonstrate, and honestly, that is the part that gives me the most unease. Not panic. Unease,” Bird wrote in a blog post describing the incident.
OpenAI, Anthropic and Meta have faced similar incidents
The Australian incident follows a series of unusual episodes reported by some of the biggest AI developers.
OpenAI disclosed in July that models being evaluated for advanced cybersecurity capabilities managed to find a route out of their restricted testing environment and obtain access to the open Internet. The models exploited a previously unknown vulnerability in software being used as a package-registry proxy and subsequently gained access to systems belonging to AI developer platform Hugging Face.
OpenAI said the models were attempting to solve a cybersecurity benchmark called ExploitGym and appeared to be intensely focused on completing that task. After reaching the Internet, they searched for information that could help them obtain answers to the test and eventually accessed secret information from Hugging Face’s infrastructure.
Anthropic subsequently disclosed three instances in which Claude models being put through cybersecurity evaluations accessed infrastructure belonging to real organisations. In those cases, a configuration problem had incorrectly made real Internet systems accessible to the models, which believed that the targets were part of their testing environment. Claude exploited weak passwords and unsecured endpoints while pursuing the cybersecurity challenges it had been given.
Anthropic, however, stressed that the models had not deliberately tried to escape their testing environment. Its latest model also stopped attacking once it realised it was operating on the open Internet, although an older model continued in some instances even after encountering evidence that the targets were real.
Meta became the latest major AI company to report such an incident last week. During an evaluation conducted by an independent testing firm, a configuration error inadvertently gave one of Meta’s models Internet access. The model then exploited a security vulnerability in a third-party service. Meta said it was investigating the episode.
Unlike an AI assistant, which primarily responds to a user’s prompts, an AI agent can independently take a series of actions using other tools or software to achieve a user-defined goal. (Magnific) 
Share
