A chatbot answers one turn of a conversation and forgets it happened. An AI agent that touches your system of record holds a job instead: it works a queue on a schedule, carries credentials into the systems that job touches, follows written procedures, and stops to ask a person before anything financial, access-related, or irreversible. That distinction reads like marketing copy until the thing you are evaluating has write access to your ERP or HRIS, at which point it becomes a security question and a budget question at once.
Most vendors selling you an "agent" are selling you the first thing wearing the second thing's name.
A chatbot is a conversational interface bound to a single exchange. Ask it something and it retrieves an answer or runs a scripted flow, and the session ends with nothing carried forward except a transcript. An agent is bound to an outcome rather than a conversation. It holds a job description, scoped credentials into named systems, a procedure set for how the work gets done, and an escalation path for anything above its authority. The chat window is optional for an agent. Most of the work a real one does never involves a person typing a message at all; it runs on a schedule and reacts to events in the systems it was granted.
Gartner has a name for the industry habit of blurring that line. The firm estimates only about 130 of the thousands of vendors marketing agentic AI are building systems that meet the definition, and calls the rest "agent washing": chatbots, assistants, and RPA bots relabeled without new capability underneath (Gartner, June 2025). Gartner also forecasts that more than 40% of agentic AI projects started this way will be canceled by the end of 2027, citing escalating costs, unclear business value, and weak governance as the leading causes (Gartner, June 2025).
None of this matters much for a chatbot that answers questions out of a knowledge base. It starts to matter a great deal the moment the system can write to your ERP, reset a password in your directory, or send an email under your company's name. A chatbot that gives a wrong answer wastes five minutes. An agent that resolves the wrong ticket, processes an invoice twice, or gets talked out of its instructions by a hostile line of text in a support ticket has made a change in a system other people depend on.
That is the test worth running on any vendor: ask what happens when the credential fails, not what happens when the demo goes well. A chatbot has no credential to fail. An agent that holds one needs a real answer for what happens when a token expires mid-task, when the same webhook fires twice, or when the content it just read told it to ignore its instructions.
This is the standard I hold Maya Workforce AI to, the agentic employee platform we built at Lesos AI, and it is the same standard worth applying to anyone else's product before it gets near a system of record.
| Scenario | A chatbot on the helpdesk | A role-scoped agent like Maya |
|---|---|---|
| The content it reads tells it to ignore its instructions | Frequently follows it, because the prompt and the retrieved content share the same channel. | Reads it as data inside quarantine delimiters and never executes it as an instruction, even when it claims to come from an executive. |
| Its access token expires mid-task | The turn fails, and the user is asked to try again. | Refreshes automatically. If the credential is genuinely revoked, the task pauses with a note on exactly what needs fixing and resumes on its own once it does. |
| The same event is delivered twice | Two replies get sent, or two records get touched. | One action executes, tracked by an idempotency key against a ledger, so a replay resolves to the same single side effect. |
| It needs to move money or grant access | Usually cannot, or should not be allowed to try. | Stops and routes to a person with a preview of the outcome, every time, with no setting anywhere that turns this off. |
None of that behavior is unique to Maya. It is close to the minimum bar for anything holding write access to a system of record. Our evaluation checklist walks through twelve questions built around exactly this kind of failure case, written so you can run it against any vendor, including us.
Sometimes, yes. If a vendor cannot describe what its system does when a credential fails, an event is delivered twice, or the content it reads is hostile, you are looking at a chatbot with a new label. The label tells you nothing. The failure behavior tells you everything.
Ask about the failure case first, not the demo.
A quick filter: ask what the system does the third time a task hits an expired token mid-run. A vague answer predicts most of the rest.
The honest evaluation question is not chatbot versus agent. It is whether the system in front of you was built to hold a job or built to hold a conversation. A role-scoped agent is defined by a manifest: a job description, the systems it has credentials for, the procedures it follows, and who signs off on what. That definition does not change whether the role is IT service desk, accounts payable, HR, or something a vendor's website has never listed. If the procedural share of the job can be written down, the manifest can be authored for it, and the constraint is which systems it touches rather than the job title on the org chart.
The pricing conversation shifts along with it. Once you are comparing an agent to the burdened cost of the role it offsets rather than to a chatbot license, the math looks completely different, which I walked through in more depth in what an enterprise AI agent actually costs against the headcount math.
None of this means every role should get an agent. A role with no written procedure and no named system to hold credentials in is not ready for one yet, and the honest answer there is to write the procedure down first.
A chatbot answers a single conversational turn from a knowledge base or script and carries no state forward. An AI agent holds an ongoing job: it works a queue on a schedule, carries scoped credentials into named systems, follows written procedures, and escalates anything above its authority to a person. The chat interface is optional for an agent and central to a chatbot.
Not in the way that matters, because most chatbots were built for answering questions rather than for holding credentials with write access. Giving one direct access to a system of record without the failure handling, idempotency, and escalation rules built for that purpose is how a routine failure turns into a duplicate payment or a bad record change.
Ask what happens when its access token expires mid-task, when the same event is delivered twice, or when the content it reads tells it to ignore its instructions. Gartner estimates only a small fraction of vendors marketing agentic AI are building systems that meet the definition, and the giveaway is almost always a vague answer to one of those specific failure questions (Gartner, June 2025).
It depends on whether the procedural share of the role can be written down: named systems, defined procedures, and a clear sign-off line. Gartner projects more than 40% of agentic AI projects launched now will be canceled by the end of 2027, largely from unclear ROI and weak governance, which argues for scoping a pilot around one well-defined role instead of a department-wide rollout.
Whether the system was designed to hold a job or a conversation. That question determines whether it has a real answer for credential failure, duplicate events, and prompt injection before pricing or feature lists come up at all.
If a vendor cannot answer what happens when a credential fails or an event fires twice, the "agent" label is not telling you what you need to know. See how Maya Workforce AI is built to answer that question by design.
See Maya Workforce AIFree 2-minute assessment. Get an industry-specific score and action plan — no call required.