When an AI agent makes a call in production at 3 a.m., whose name is on it?
That question makes rooms quiet. Not because people haven’t thought about it, but because most of them don’t have a clean answer. They’ve deployed agents into existing team structures, added governance language to a policy document, and called it done. We took a different approach.
On the teams I lead, every new engagement starts with at least 50% agents working alongside human practitioners. Not as a pilot you graduate out of. As the actual delivery model, from day one. In practice that means the first week of any engagement includes defining agent roles alongside human roles, not after. Scope, handoff points, escalation protocols and accountability structures get built into the delivery model before any delivery work begins or a single output is reviewed. That design decision forces questions most organizations are still deferring, and how you answer those questions is what separates agentic AI that works in production from agentic AI that works in a demo.
The boundary between human and agent has to be a decision, not an assumption
The most common failure mode I see isn’t a model problem. It’s a clarity problem. Organizations deploy agents without specifying exactly what the agent owns and what the human owns. They use words like collaboration and partnership, which sound good but give you nothing to stand on when something goes wrong.
On teams of humans and agents, that boundary gets drawn before the engagement starts. Agents handle execution: content generation, data processing, workflow execution, routine decision-making, anything requiring speed and scale beyond what a human team can sustain. Humans handle direction and judgment. They define what good output looks like, intervene where context matters in ways the agent can’t yet read, and own every output before it reaches a client.
Getting that right means asking specific questions about every workflow before you build it. What information does the agent need to make a sound decision here? Where does the risk of getting it wrong outweigh the efficiency gain? What does escalation to a human actually look like in this context, in practice, at 3 a.m.? Skip these questions and you answer them later, expensively, under pressure.
Design the handoff before you need it
There’s a moment in every agentic workflow where the agent should stop and hand off to a human. We call it the graceful handoff. Getting it right takes more time than most teams expect, and it’s almost never where they expect.
You have to map every workflow before deployment and tag the points where agent confidence is likely to degrade: ambiguous inputs, situations outside the agent’s training distribution, outputs with significant downstream consequences if they’re wrong. At each of those points, the protocol has to be explicit before you go live. What does the agent hand to the human, in what format, with what context attached so the human can make a fast, informed call? That context should include what the agent was trying to do, what it knew when it made the call, and where it lost confidence. A human who has to reconstruct that from scratch isn’t being handed off to. They’re being handed a problem.
Most organizations discover these moments in production. A client escalation, a compliance flag, an output that went somewhere it shouldn’t have. That’s a solvable problem. It’s just a lot more expensive to solve it that way.
Accountability has to be a named person, not a framework
This is the question that separates organizations that have actually thought this through from those that have not.
In this model there is always a named human who owns every output the system produces before it reaches a client or customer. Not a team. Not a governance committee. Not a process. A person, whose name is attached to the output before it ships. That person exists before deployment starts. If you can’t name them, you’re not ready to deploy.
An EY survey of 202 senior AI executives at large U.S. companies, released in September, found that 49% of agentic AI users haven’t updated their governance frameworks, and 85% say some systems act without real-time human involvement. Retrofitting that later is expensive. The reputational cost is worse.
What you actually get on the other side
The step change we have seen on our strongest engagements doesn’t come from the technology. It comes from the organizational design underneath it. When the boundary is clear, the handoffs are designed rather than discovered, and the accountability is named rather than assumed, agents stop being a source of operational risk and start being a genuine force multiplier.
Most enterprises are treating agentic AI as a technology problem. The ones getting to production-grade outcomes figured out it’s an organizational design problem. That gap is closing. The question is which side of it you want to be on when it does.