Blog/Who's Accountable When Your AI Agent Breaks Something?
AI GovernanceAI AgentsIT SecurityServiceNowRisk Management

Who's Accountable When Your AI Agent Breaks Something?

August 20, 20266 min readBy Brad McCorkle, Founder & CEO, Lesos AI

AI agent accountability is the practice of naming a specific person, not a team or a policy document, who owns an agent's actions in production and answers for what it does wrong. Most mid-market IT organizations skip this step. According to Gravitee's State of AI Agent Security 2026 report, only 37.8 percent of organizations have named a person accountable for agent behavior, which means most agents running today have no one whose job it is to answer for them when something breaks.

That gap does not show up on a slide. It shows up in a meeting after an incident, where four people each say "I thought that was someone else's job" and nobody has an answer for the CFO.

Nobody built this gap on purpose.

The Data Behind the Accountability Gap

The numbers get worse the closer you look. Kiteworks' 2026 Data Security and Compliance Risk survey found that 80 percent of organizations experienced at least one security or AI-related incident in the past year, and 63 percent of those incidents produced a compliance outcome: an audit finding, a remediation plan, a board escalation, or a regulatory inquiry. Sixty percent of organizations told Kiteworks they could not terminate a misbehaving AI agent even after identifying it mid-incident.

  • 80% of organizations reported a security or AI-related incident in the past 12 months (Kiteworks, 2026)
  • 63% of those incidents produced a compliance outcome such as an audit finding or board escalation (Kiteworks, 2026)
  • 60% of organizations cannot terminate a misbehaving agent once it is caught mid-action (Kiteworks, 2026)
  • Only 21.9% of teams treat AI agents as independent, identity-bearing entities rather than shared service accounts (Gravitee, 2026)
  • 45.6% of teams still authenticate agent-to-agent calls with shared API keys (Gravitee, 2026)

Put those together and the picture is a workforce of agents that mostly cannot be killed on demand, mostly cannot be traced to an individual identity, and mostly have no named owner. That is not a security gap. It is a governance gap wearing a security incident as a costume.

Why "The AI Team Owns It" Isn't an Answer

Every IT leader I talk to has some version of an AI team, a platform group, or a center of excellence that supposedly owns agent governance. On paper, that group owns it. In practice, when a ServiceNow-connected agent grants access it should not have, the AI team did not build the workflow, the business unit that requested it did not configure the permissions, and IT operations did not review the tool package before it shipped.

Shared ownership is unowned ownership.

A named individual does not mean one person does all the work themselves. It means one person's name is on the change record, one person signs off before an agent gets write access to a system of record, and one person gets paged first when the agent does something unexpected. Everything else can still be a team effort. The accountability cannot.

Who Should Actually Own an AI Agent's Behavior?

The answer depends on what the agent touches, not on which department happened to build it. An agent that only reads from a knowledge base and drafts a suggested reply is a different accountability problem than one that resets passwords, closes tickets, or writes to your ERP. Match the owner to the blast radius, not to the org chart.

Agent TypeRecommended OwnerWhy
Read-only, suggests actions for a human to approveService desk managerLow blast radius; the owner already reviews the tickets it touches
Writes to ServiceNow or M365 (resets, provisioning)IT operations lead, named individualDirect write access to a system of record needs one accountable name, not a queue
Cross-system, financial or ERP-adjacentNamed individual plus change-advisory sign-offBlast radius crosses departments; one owner plus a second check
Agent-to-agent orchestrationPlatform or architecture ownerFailure chains are hard to audit without one person tracing the call graph

What This Looks Like on a ServiceNow + Microsoft 365 Stack

For most of the mid-market teams we work with, the practical fix is smaller than it sounds. We built a layered permission model for identity operations in our three-tier security model for IT automation, and the same pattern applies to accountability: scope what the agent can do, log who approved that scope, and name the person who reviews the log. If you are running ServiceNow's Action Fabric or a similar MCP-based integration, we covered the specific access controls to lock down in our Action Fabric security guide, and the same rule applies there. Every tool package needs a named approver before it ships, not just a role that could theoretically approve it.

  • Assign one named individual per agent category before it goes live, not after an incident
  • Put that name in the change record for every tool package, OAuth grant, or role the agent receives
  • Build a kill switch the named owner can trigger without a change ticket, then test it before launch
  • Route agent-to-agent calls through a real identity, not a shared service account, so a failure traces to one call chain
  • Review the owner list quarterly; agents accumulate scope creep the same way employees do

Can You Actually Turn Off Your Riskiest AI Agent?

Here is a fast way to check where your organization actually stands. Ask whoever owns your highest-privilege agent to disable it right now, without a change ticket, without waiting for a maintenance window. If they cannot do it in under five minutes, you do not have an accountable owner. You have a name on a slide.

Run this test on your most-trusted agent, not your newest one. Newer agents get watched closely. The one that has been running quietly for eight months is the one nobody remembers how to stop.

Frequently Asked Questions

Who is legally responsible when an AI agent makes a mistake?

Legal responsibility sits with the organization that deployed the agent, not the vendor whose model it runs on, in nearly every current enterprise contract. That is a separate question from operational accountability, which is the internal person who signs off on scope and gets paged when something breaks. Most incident postmortems stall because a company can answer the legal question but not the operational one.

Does every AI agent need a named human owner?

No. A read-only agent that drafts suggested replies for a human to approve carries a small enough blast radius that a team, rather than one person, can reasonably own it. Once an agent gets write access to a system of record, ServiceNow, an ERP, or an identity provider, it needs one name attached, because a group cannot get paged at 2 a.m.

How much does it cost to fix an AI agent accountability gap?

Assigning ownership itself costs nothing beyond the time to have the conversation and document it. The real cost shows up in the access review and kill-switch testing that should come with it, which for a mid-market IT team we typically scope as a one-to-two week engagement rather than a platform purchase.

Is this different from AI governance dashboards like ServiceNow AI Control Tower?

Governance dashboards give you visibility into what agents exist and what they touched, which is necessary but not sufficient. Visibility answers what happened. Accountability answers whose job it was to prevent it, and no dashboard assigns that name for you.

Not Sure Who Owns Your AI Agents Today?

Our AI readiness assessment maps every agent and automation touching your ServiceNow and Microsoft 365 environment, then flags the ones with no accountable owner before they become an incident.

Start Your Readiness Assessment

How AI-ready is your organization?

Free 2-minute assessment. Get an industry-specific score and action plan — no call required.

Get My Readiness Score