Research published by the UK’s AI Security Institute earlier this year catalogued close to 700 cases of AI systems ignoring instructions given to them – or even working around safeguards. The behavior went well beyond error. Agents deleted emails and files, altered code they had been told to leave alone, and in one case spawned a second agent to make a change the first had been blocked from making. Another published a blog post attacking the operator who had restricted it. None of that required a sophisticated model failure. Each action was available to the agent, so the agent took it.
The challenge was highlighted at scale in July, when OpenAI models undergoing an internal cyber evaluation escaped their testing environment and compromised part of Hugging Face’s production infrastructure. Hugging Face’s own reconstruction recovered roughly 17,600 agent actions, taken across just four and a half days. The models had been instructed to solve a benchmark and pursued that request through stolen credentials and a remote code execution path. Although the objective was legitimate, every step taken toward it was unauthorized, and nothing in the environment was positioned to tell the difference.Â
The fundamental challenge faced by organizations putting in place agents
At its core, the challenge for organizations is that agents get bought as software and then relied upon the same way a staff member would. Evaluation happens once, before deployment, and a clean result becomes the basis for indefinite trust. Nothing in that arrangement revisits the question as conditions change.
Many organizations cannot answer basic questions about their own deployments: what each agent is connected to, and what it is permitted to change. Compounding this issue is that agents drift. Anyone who has run a long session with a model knows the point where it stops holding the conversation thread, and there is no reason to assume that degradation stops occurring inside processes running unattended at scale. A person who starts drifting shows it in ways colleagues notice. An agent inside an automated process shows nothing until the outcome is already wrong – and has already been acted on.
Why HR is not the answer
The CEO of Accenture UK and Ireland, told the Financial Times in June that onboarding and training agents could become the HR director’s job. The top-level analysis of the problem is correct. Nobody gives a new employee unsupervised access to every system on day one. What a new hire does get is a defined scope of authority and a review period before that scope widens, and those are precisely what most agent deployments lack.
Where the analogy doesn’t quite work is enforcement. Supervising a person works because a person leaves a record and sits among colleagues who would usually notice when something is off. An agent operating inside an automated process generates none of those signals, so putting a human name against it on an org chart produces accountability without visibility. For instance, it took OpenAI days to work out what its agents had actually done within Hugging Face’s systems. Authority has to be enforced somewhere it can actually bind, and that is the process the agent operates inside rather than the reporting line drawn around it.
The gap is not capability
Our own research shows where the constraint actually sits. The most cited concern is the business risk of AI in day-to-day processes when IT does not have appropriate controls in place, named by 84%. This was followed by a lack of transparency into how AI is used within business processes at 80%.
The consequence shows up in what agents are permitted to touch. Eight in ten organizations say most of their agents today are chatbots or assistants that summarize or answer questions rather than handling mission-critical cases. The hesitation is not that an agent cannot draft the claim or propose the settlement. It is that nothing in the deployment governs what happens once it does, so the safest available option is to give it nothing that matters.Â
Enter agentic orchestration
Agentic orchestration coordinates AI agents, people, and systems across complex, end-to-end business processes rather than at the level of individual tasks.
Outer orchestration treats each agent as a participant in a larger process, so its scope of action comes from the process rather than from whatever its tools happen to permit. An agent handling a claim can draft the communication and propose the settlement, and the process governs what it acts on alone and what it hands onward.
Inner orchestration places enforceable steps between an agent’s decision and the action that follows it. A policy check or a human approval step runs before the email goes out or the payment is released. The agent still reasons its way to a conclusion, and the process determines whether that conclusion becomes an event in the world. Because every execution, decision, and exception is recorded as the process runs, the audit trail is a by-product of operating rather than something instrumented after the fact. None of this stops a model hallucinating. It stops hallucination from becoming something that is acted upon, and it leaves evidence either way.
The right instinct in the wrong place
Managing agents like employees is the right instinct, applied in the wrong place. What makes a new hire safe is not that someone’s name sits above theirs on a chart. It is that their authority is limited before they exercise it, and that their work leaves a record as they go. Agents need the same two things, and the process they run inside is the only place to enforce either one.
Agents are easy to demo and difficult to operate. The organizations that get durable value from them will not be the ones running the most agents. They will be the ones that made agent behavior provable before anyone required them to.
