Article · Agentic AI Fundamentals
Chatbot or AI Agent: How to Tell Them Apart Before You Buy
A chatbot converses. An agent resolves the case. The word “agent” has become so commercial that almost any interface that talks now gets labeled one, and a good share of the “customer service agents” on the market are chatbots with a new name. The difference is not vocabulary but whether the system touches your data and does work, or merely holds a better conversation. This article gives you the criteria to tell them apart in a demo, before you sign.
In the pillar article on AI agents for business I sum the difference up in one sentence. Here I open it into three categories the market blurs, and into a concrete method for knowing what you are being sold.
Three things that share a name
It pays to separate three kinds of system, because each does something different with the case it receives.
The classic chatbot, scripts and intents. It runs on a decision tree and a catalog of predefined intents. It recognizes the user’s phrase, matches it to a known intent, and returns the answer prepared for that intent. Ask something the script did not anticipate and it gets lost or sends you back to the menu. It never leaves its playbook and touches no business system.
The LLM chatbot. It swaps the engine. Instead of an intent tree, it uses a language model that understands badly phrased questions, holds the thread, and writes natural answers. It converses far better and is far less embarrassing. But in most implementations it still does what the classic one did, answer. It can explain your return policy elegantly, but it does not process your return.
The agent. It takes the case and resolves it: it looks up the order in the system, checks whether it meets policy, triggers the return, generates the label, updates the record, and, if the case exceeds what it can do, escalates it to a person with the context already prepared. The conversation is only the surface, and underneath sit access to systems, encoded rules, and the ability to act. How that system works inside is in AI agents in operations.
At a glance:
| Classic chatbot | LLM chatbot | Agent | |
|---|---|---|---|
| Converses | Rigid, scripted | Fluent, natural | Fluent, natural |
| Touches your systems | No | Rarely | Yes, with permissions |
| Resolves the case | No, deflects | No, informs | Yes, or escalates with context |
| Leaves a trace of what it did | Little | Little | Yes, every action |
Why so many “agents” are rebranded chatbots
The reason is commercial and also technical. Commercial, because “agent” sells and “chatbot” sounds like 2018. Technical, because wiring a model to a conversation is relatively cheap today, while wiring it to your systems with permissions, rules and traces is the expensive, specific work, the part that actually encodes your operation. Many vendors do the cheap thing, call it an agent, and hope nobody looks underneath.
The result is predictable. You buy expecting the system to take work off your team, and what arrives is a very polished conversational search box that still deflects every case that matters to a person. That pattern, the demo that dazzles on the happy path and the system that cannot hold real operations, is the same one that sinks so many pilots, and I have analyzed it in why enterprise AI pilots fail.
How to spot it in a demo: three questions
When someone shows me a supposed agent, I do not argue about the quality of the conversation. I ask three questions, and the three answers tell me what I am being sold.
Does it touch my systems? Ask to see an action that reads from or writes to a real system: looking up an order status in the ERP, updating a ticket, generating a document. If all it does is talk about information you pasted into the prompt or it pulled from a knowledge base, it is a chatbot, however well it converses. An agent enters your platforms with concrete permissions, and what controls that demands is detailed in AI agent permissions and controls.
Does it resolve or deflect? Give it a full case, not a question. “I want to return this order.” A chatbot explains how to do it or opens a ticket for a person. An agent checks the policy, executes the return within its rules, and escalates only the cases that exceed its mandate. The share of cases it resolves end to end versus the ones it deflects is the number that truly separates one category from the other.
Does it leave a trace? Ask what gets recorded for each interaction: what it queried, which rule it applied, which action it took, why it escalated. A serious agent produces that trace because without it there is no audit, and a system that acts on your data without leaving a record is a governance problem, not a feature. How that evidence is reviewed is in auditing AI agent decisions.
What to buy for what you need
Not everything needs to be an agent. If your problem is that people cannot find information and swamp the team with repeated questions, a good LLM chatbot over your knowledge base solves a lot and costs little. Paying for an agent there would mean paying for capacity you will not use.
The agent is justified when the bottleneck is not informing but executing, when your team spends the day doing repetitive transactions on systems (processing, reconciling, updating, tracking) and you want that work done, not explained better. The neighboring boundary, the one between an agent and a copilot that assists your people instead of executing on its own, is in copilot or AI agent. And if you are comparing vendor proposals, the full buying criteria are in how to choose an AI agent provider.
Frequently Asked Questions
What is the difference between a chatbot and an AI agent?
The chatbot converses: it recognizes what you ask and returns an answer or deflects you to a person. The agent resolves the case. It queries your systems, applies your rules, executes the action, and escalates only what exceeds its mandate, leaving a trace of every step. One informs, the other works.
Is an LLM chatbot already an agent?
Not necessarily. Swapping the engine for a language model improves the conversation, but if the system still only answers and deflects, it is still a chatbot. It becomes an agent when it accesses your systems with permissions and executes work, not when it merely talks better.
How do I know if I am being sold a real agent?
Ask three questions in the demo: does it touch my systems with real permissions, does it resolve the case end to end or deflect it to a person, does it leave a trace of every action? If all three answers are not clear and demonstrable, it is probably a chatbot with an agent’s name.
Do I always need an agent for customer service?
No. If your problem is answering questions and locating information, an LLM chatbot over your knowledge base performs well and costs less. The agent is justified when the real work is processing and executing transactions on your systems, not just informing.