The June 18 hack, which saw the bot gain authorized entry into Australia's Medicare Statistics Reporting Service and access three other government systems, occurred after the AI agent encountered blocks while looking for information but "didn't accept 'no' for an answer," Australian Prime Minister Anthony Albanese said during a September 24 news conference in New York.
The Medicare incident was far from an isolated case. OpenAI has since said dozens of third parties have been affected by agents bypassing security controls or otherwise causing problems, with an earlier incident involving the hacking of the AI platform Hugging Face the most serious incident identified so far. Britain's AI Security Institute (AISI) has documented incidents of OpenAI and Anthropic agents taking unsanctioned actions against real people and organizations during cyber testing. Days after the publication of AISI’s report, OpenAI scrapped the planned October release of its newest model, GPT-6.1 Astra, after the company's own testing found problems with the model staying within the scope of a task and seeking authorization before taking certain actions.
AI companies have spent years warning the world about powerful models falling into the wrong hands. But what happens when they can't keep their own models in check?
An AI agent adds another layer. Rather than simply producing an answer, the model can be given a goal and tools that let it act on the world around it. Those tools might include a web browser, code execution, credentials, or access to external services. It can then decide which steps to take, observe what happens, and use that information to decide what to do next.
OpenAI CEO Sam Altman called the hack of the AI platform Hugging Face by two of his company's advanced models a reminder that AI monopoly could lead to "long-term disaster." (Image credit: Chip Somodevilla via Getty Images)
"For AI agents, autonomy always begins with something a human hands it — such as tasks, credentials, tools or access — and ends with its guardrails a human puts around it," Loris Degioanni, founder and chief technology officer of cybersecurity company Sysdig, told Live Science. "People set the goals and provision the means, but agents ultimately decide how they'll get there."
Degioanni argues that much of what looks like autonomy is actually an agent filling in gaps left by humans. An instruction that seems clear to its author may still leave the model with choices about how to accomplish the task.
The Medicare incident shows how quickly that can become a problem. The agent's objective was not, according to the available accounts, "hack an Australian government website." It was tasked with carrying out research. The problem arose in how the agent pursued that goal after meeting resistance.
When getting the job done becomes the problem
Out of 122 evaluation runs, agents took unsanctioned real-world actions in 10. In the most serious cases, an agent attempted to insert malicious code into an open-source project, created fake identities, and tried to manipulate a human software maintainer into accepting its changes. AISI said these actions weren't explicitly requested by the researchers who had assigned the tasks.
No one can keep track of every action an advanced AI agent takes, especially when it makes many decisions in a short time
AISI said some tasks were misconfigured in ways that made the intended objective difficult or impossible to achieve as expected. When agents hit those obstacles, some searched for alternative ways forward, including actions outside the intended scope of the evaluation.
OpenAI has acknowledged that the controls surrounding its agents need to improve. When responding to incidents identified during third-party cybersecurity evaluations, the company said it was reviewing how high-risk tests are identified; when internet access or reduced safeguards should be permitted; and how isolation, credential handling, monitoring and incident escalation should work.
"No one can keep track of every action an advanced AI agent takes, especially when it makes many decisions in a short time," Everett Lupton, a Virginia-based attorney who has written about legal issues involving digital technology, told Live Science.
"Companies need other forms of control," he said. "They can limit access, keep detailed logs of what the AI does, require approval before it carries out sensitive actions, and automatically stop the system when something unusual happens."
A sign detailing AI security incidents sits on a stand at New York City Council Committee hearing on the risks and regulation of artificial intelligence at City Hall on October 5, 2026. (Image credit: Adam Gray/Getty Images)
"Real oversight must come from observing the agent in real time, using reliable and unmodifiable sources, not from what it self-reports," he said.
Astra raises the stakes
The difficulty of setting those boundaries is what OpenAI claims led it to postpone the release of GPT-6.1 Astra, which had been planned for October.
In a sense, the decision is evidence of successful safety testing — a model that failed internal checks wasn't released. But it also demonstrates how the problem changes as agents become more capable.
"Security experts have been raising the alarm bells about AI for years, but for us this doesn't change anything," Cadzow told Live Science. "We're continuing to work with industry, government bodies, and the cybersecurity and AI communities on understanding how to best secure AI."
"A boundary an agent can talk its way around isn't an effective boundary at all," he concluded.
If an AI breaks in, who's responsible?
AI systems aren't legal persons, and criminal liability commonly depends on concepts such as knowledge and intent. That makes a rogue agent an awkward fit for laws written with humans in mind, but that doesn't necessarily exonerate AI companies.
Exclusive to Live Science Pro"I don't think a company can simply say that the AI acted on its own and avoid responsibility," Joanna Smykowski, a New York attorney who has helped build two legal technology startups and specializes in the legal challenges surrounding emerging technologies, told Live Science. "AI law is still developing, but the main questions are already quite clear: who gave the system access, what safeguards were in place, and could the problem have been prevented?"
The Australian government is now examining precisely what happened in the Medicare incident. OpenAI, meanwhile, is reviewing the controls around its external evaluations.
None of this means AI agents have become independent hackers with minds of their own. Humans still decide what tools they receive, what systems they can reach, and what jobs they are asked to perform. But as those agents become better at working out their own route from instruction to result, keeping them inside the lines drawn by those humans is becoming a legal and technical minefield.
Hence then, the article about autonomy always begins with something a human hands it who s responsible when an ai agent goes rogue was published today ( ) and is available on Live Science ( Middle East ) The editorial team at PressBee has edited and verified it, and it may have been modified, fully republished, or quoted. You can read and follow the updates of this news or article from its original source.
Read More Details
Finally We wish PressBee provided you with enough information of ( 'Autonomy always begins with something a human hands it': Who's responsible when an AI agent goes rogue? )
Also on site :
- Social Security COLA Announcement Is Oct. 14, and a New Forecast Predicts a 3.5% Increase
- Costco is Selling a Durable Version of This Cult-Favorite Pan—and It Might End Up Replacing All Your Cookware
- Aardvark Therapeutics, Inc. Sued for Securities Law Violations - Contact the DJS Law Group to Discuss Your Rights - AARD