What Makes a Conversational Agent, Not a Chatbot

Search for conversational AI agents and you get comparison pages, nearly all of them published by companies that sell one. They tend to define a chatbot as the old thing, a conversational agent as the new thing, and the difference as understanding. That framing is not useful for building anything, because every product on that page understands language perfectly well.

The distinction that survives contact with production is narrower and less flattering. A chatbot answers. An agent acts. Everything that makes agents harder to build, more expensive to run and more dangerous to deploy follows from that one difference, and none of it follows from how naturally the thing writes.

The question you evaluate it on changes

A chatbot is judged on whether its answer was correct and appropriate. If it was wrong, the user reads something wrong and moves on. The blast radius of a bad response is the response.

An agent is judged on whether the action it took was the right one. A refund was issued, a ticket was reopened, a record was updated, an email went out. Being wrong now leaves a trace in a system somebody else depends on, and that trace outlives the conversation that produced it. This is why an agent needs an audit trail and a chatbot does not, and it is the real reason the two are different products rather than two settings of one product.

State is the first thing that gets expensive

A chatbot can be close to stateless. Each turn is a question with enough context attached to answer it, and when the conversation ends nothing needs to persist.

An agent carries state across turns, and language models are stateless underneath, so carrying state means resending it. Every turn ships the accumulated conversation back to the model as input. The user experiences one continuous conversation of a fixed length. Your bill experiences a prompt that grows every time somebody speaks, which is why the last turn of a long session costs a multiple of the first.

That shape is worth measuring rather than assuming. Take a realistic transcript, count what one late turn actually sends with a token counter, and put the result through the LLM cost calculator at your expected conversation length. The number that comes back is usually not the one people estimate from the length of a single message, and it decides whether you summarise history, cache it, or cap the session.

Tools are where accountability appears

Giving a model tools is the step that converts a chatbot into an agent, and the important property of a tool is not what it does but which direction it points.

A read-only tool is a retrieval feature. It can be wrong, it can be slow, and the worst case is a bad answer, which is the chatbot failure mode again. A tool that writes is different in kind: it makes the model an actor inside your business with whatever permissions you handed it. The design decision is therefore not which model to use, it is which of those tools exist at all and what has to be true before one of them fires.

The useful discipline is to draw the permission boundary before choosing anything else. Which actions may happen with no human involved, which require confirmation, and which the agent may only ever propose. That boundary is a product decision and a liability decision, and no amount of model quality substitutes for having made it.

Latency is spent in a loop, not on a reply

A chatbot spends its latency budget on one generation and the user watches it stream. An agent spends its budget on a loop: decide, call a tool, read the result, decide again. Some of those steps produce nothing the user can see, so the same total wait feels considerably longer.

Text interfaces absorb this reasonably well. Voice does not, because silence on a call reads as a dropped line rather than as thinking, which is a large part of why voice agents are architecturally stricter than their text equivalents.

What to settle before choosing a category

The vendor taxonomy stops mattering once these four are answered, and they are answerable without picking a platform first.

• What may this do without a person in the loop, and what must it only ever suggest?
• What does it remember, for how long, and who is allowed to read that later?
• When it acts wrongly, who is accountable, and how do they find out?
• How do you know afterwards what it actually did, rather than what it said it did?

An agent that cannot answer the last one is not an agent you can operate, however well it converses. For more on building on these systems, see the rest of our writing.