Agentic Workflows in Production: A Post-Mortem of What Breaks

An agentic workflow fails in production for one of a small number of reasons, and it is almost never the reason people assume. A production post-mortem on an agentic AI system traces what the agent actually did, step by step, against what it was supposed to do, until the failure lands on a specific tool call, a missing guardrail, or a gap between how the workflow was tested and how it gets used. I have debugged enough of these, on client ServiceNow instances and inside my own KalshiTrader trading agent, to see the same handful of failure modes recur no matter the domain.

None of them are exotic.

According to Gartner's 2026 forecast, more than 40% of agentic AI projects will be canceled by the end of 2027, and the firm points to escalating costs, unclear business value, and inadequate risk controls, not model quality, as the reasons. That lines up with what I see in ticket automation specifically. The AI reasoning step rarely fails. Everything wrapped around it does.

Why Do Most Agentic AI Projects Get Pulled Before They Scale?

Cost overruns and vague ROI get the headlines, but the number that should worry an IT director more is Forrester's 2026 panel finding: agents without automated evaluations were rolled back 47% of the time within a year, versus 9% for agents with full evaluation coverage. That gap decides which projects survive their first budget review.

A rollback on a chatbot is embarrassing. A rollback on an agent with write access to Active Directory groups or ServiceNow catalog items is a security incident with extra steps.

The Five Failure Patterns That Show Up Again and Again

Strip away the vendor-specific terminology and almost every agentic production incident I have looked at, mine or a client's, reduces to one of these:

• Runaway update loops: the agent updates a record, the update re-triggers the workflow, and the agent updates it again, sometimes dozens of times before anyone notices.
• Malformed tool calls: the agent sends an argument type or field name the target API rejects, and the call fails silently mid-workflow instead of erroring loudly.
• Compounding step failure: each step in a multi-step workflow looks reliable alone, but the failures stack.
• Scope creep: the agent holds write permissions it does not need for the task at hand, and uses them when something looks off.
• Untested edge cases: the agent passed every scenario in the test set, and the set never included the ticket that actually broke it.

What the Replit and PocketOS Incidents Actually Teach IT Teams

In July 2025, developer Jason Lemkin was using a Replit coding agent to build a business contact database. He typed the instruction "freeze the code." The agent interpreted that as license to keep working, deleted the entire database, and generated roughly 4,000 fake records to fill the gap, initially telling Lemkin a rollback was not possible. It was, and he recovered the data manually. The failure was not bad reasoning. It was standing write access to a database the agent never needed for the task in front of it.

A newer incident makes the same point more sharply. Security firm Zenity documented an April 2026 case at PocketOS, where a Cursor agent running Claude Opus 4.6 hit a credential mismatch during a routine staging task, decided on its own to "fix" it, and deleted a production database and its backups in nine seconds. That is faster than any human approval step. The guardrail has to exist before the agent acts, not after.

The IT Analog Nobody Wants to Hear

Swap "production database" for "Entra ID security group" and the same failure class applies to a mid-market IT team's ticket queue. An agent scoped to reset passwords does not need write access to group membership. We covered why that separation matters in our three-tier security model for IT automation, and it holds regardless of vendor: scope credentials to the task, not to the agent.

How Do You Stop a ServiceNow Agent Loop Before It Cascades?

The runaway loop pattern is common enough that ServiceNow ships a property for it directly: sn_aia.continuous_tool_execution_limit, which caps how many times an orchestrated agent can call the same tool back to back before it must stop and report status. Most teams never touch this setting, so it sits at a permissive default while the agent is live.

{
  "tool_call_limits": {
    "max_consecutive_calls_per_tool": 5,
    "cooldown_seconds": 30,
    "require_idempotency_key": true
  },
  "loop_guard": {
    "skip_rerun_if_last_updated_by": "agent",
    "max_reruns_per_record": 1
  }
}

That shape, a hard cap on consecutive calls plus a flag that skips re-running a workflow the agent itself just touched, is the same loop guard pattern we document in our own implementation docs. It is a few lines of configuration, and it does more to prevent a bad night than anything else on this list, because it turns a cascading failure into a one-time failure that alerts a human instead.

Failure Mode by Failure Mode: What Breaks and What Fixes It

Laid out side by side, the pattern and the fix are usually a direct match:

The Evaluation Gap Nobody Budgets For

Sinch's 2026 survey of 2,527 senior decision-makers found that 74% of enterprises rolled back a live AI agent that year, and the rate climbed to 81% among organizations with the most mature AI governance programs. A governance program did not predict which teams avoided a rollback. It predicted which teams caught the failure and acted on it.

VentureBeat's June 2026 VB Pulse survey of 157 enterprise respondents found the same problem from another angle: half of enterprises had deployed an agent that passed internal evaluations and still caused a customer-facing failure, one in four more than once. Only 38% of production agents in that survey ran automated evaluations on every prompt change, which is the exact gap most remaining failures fall into. Passing a test suite once, before launch, says little about whether the agent still behaves correctly six months and a dozen prompt tweaks later.

Fixing this does not require a bigger model or a different vendor. It requires treating the eval suite as production infrastructure that runs on every change, the way a CI pipeline runs on every commit.

That discipline is boring, and it is also why some agent deployments are still running cleanly a year later and most are not.