The demo works. The pilot works. Then the agent meets production — real volume, real systems, real adversaries, real absence of the engineer who was watching it during the pilot — and a class of failures appears that nothing in the evaluation phase predicted.
This is not an argument against deploying AI agents. It is an argument for deploying them with eyes open. Industry research keeps landing on the same conclusion: the constraint on enterprise AI value is no longer model capability but deployment maturity — Deloitte's 2026 enterprise AI work describes the pilot-to-production gap as the defining challenge, and McKinsey frames agents as an operating-model problem rather than a technology purchase. What follows is a field catalog of how agent deployments actually break, and what the organizations that run agents successfully do about each one. (For the delegation and accountability groundwork, see What Executives Need to Know About AI Agents.)
Break #1: The agent does something nobody decided it could do
The most common production surprise is not a malfunction — the agent does exactly what it was built to do, against a scope nobody wrote down. It emails a customer in a tone reserved for internal notes. It updates a record that was supposed to be read-only. Everyone assumed someone else had drawn the boundary.
The fix is boring and non-optional: a written delegation contract — systems the agent may touch, actions it may take, approval points, escalation path — agreed by the process owner before go-live. If the scope only exists as shared understanding, production will find the gaps in that understanding.
Break #2: The agent meets the real system of record
Pilots run against clean sandboxes. Production runs against fifteen years of accumulated reality: duplicate customer records, fields repurposed by a team that left in 2019, permissions that differ between staging and production, APIs that time out under load. Agents are better than scripted automation at absorbing this messiness — that's their point — but "better" means they fail less often, not never, and when they misread a messy system they act on the misreading.
The fix: treat integration debt as part of the agent project, not a separate concern. Run the agent in shadow mode against production data before it acts on production data, and give it explicit "I'm not sure" behavior — an agent that escalates ambiguous records beats one that resolves ambiguity confidently and wrongly.
Break #3: Adversarial content arrives
In the pilot, the agent read documents your team wrote. In production, it reads whatever arrives: emails from strangers, attachments from vendors, web content, form submissions. Some of that content will eventually be crafted to manipulate it — instructions embedded in a document that attempt to redirect the agent's behavior or extract data it can access. Prompt injection is not exotic; it is the spam problem for the agent era.
The fix: assume untrusted input. Isolate what the agent reads from what it is instructed to do, constrain its tools so the worst credible manipulation has bounded consequences, and monitor for behavioral anomalies. The permission combination to audit hardest: read access to sensitive data plus write access to any external channel. The broader control set belongs to your governance program — covered in How to Govern AI Agents.
Break #4: The approval gate becomes a rubber stamp
Deployments that survive review often ship with a human-approval step on every action. It feels safe. Then volume arrives: the reviewer faces forty approvals a day, each one almost always fine. By week three they approve by reflex, and the control that justified the deployment now provides the appearance of oversight rather than oversight.
The fix: fewer, sharper gates. Reserve human confirmation for consequential, hard-to-reverse, or customer-facing actions, and keep the reviewer's queue short enough that attention is real. Measure the override rate — a reviewer who never rejects is a sign the gate is either unnecessary or asleep, and both are findings.
Break #5: Silent drift
The agent that passed evaluation in March is not the agent running in July. The underlying model was updated, a connected system changed its data shapes, the mix of incoming cases shifted. Nothing announced the change; quality just moved. Without measurement, drift is discovered by its consequences.
The fix: evaluation is regression testing, not a launch gate. Keep a benchmark set of real cases (including the ugly ones), re-run it on every model or integration change and on a schedule, and alert on movement in escalation rates, correction rates, and output quality. Snapshot confidence decays; only measured confidence persists.
Break #6: Supervision debt and the quiet cost curve
Two slow failures compound in the background. First, supervision debt: the action logs exist but nobody reads them, boundaries never get tuned, and small recurring errors persist because no one owns noticing them. Second, the cost curve: per-task inference costs that were trivial at pilot volume become a real budget line at production volume, often discovered by finance rather than engineering.
The fix: staff supervision as a role with time allocated, not a hope; and give every agent unit economics — cost per resolved case, tracked from day one — so scale decisions are made with numbers instead of surprises.
What production-ready actually means
A deployment is production-ready when it can answer yes to six questions:
Is there a written delegation contract with a named accountable owner?
Has the agent passed evaluation on real cases, with an error rate someone accepted in writing?
Do complete, reviewable action logs exist — and does a named person review them?
Have approval gates been placed only where consequence is real, with override rates measured?
Is there a benchmark set that re-runs on every change, with drift alerts?
Has the kill switch been tested — pause, rollback where possible, notification?
None of this is exotic. It is the same operational discipline enterprises already apply to critical systems and new hires, applied to a new kind of worker. The organizations that scale agents are not the ones that avoid these failures — everyone hits some of them. They are the ones that instrumented for the failures in advance, so each one was a finding instead of an incident.
This guide is part of AITJ’s AI Agents in the Enterprise coverage — governance, deployment, security, and ROI of delegating enterprise work to AI agents.
