AI agent for customer support vs. the chatbot you already tried: what's actually different
Your old support bot routed people in circles. An AI agent reads tickets, calls your APIs, and closes loops. Here's the real gap, with examples.
- ↳A chatbot picks from a script. An agent reads context, calls tools, and changes state in your systems.
- ↳If your support bot can't issue a refund, look up an order, or escalate with a real summary, it's a decision tree wearing a hat.
- ↳The hard part isn't the LLM. It's the tools, the guardrails, and the eval set you build from real tickets.
- ↳Voice agents are real now, but latency budgets are brutal. Under 800ms round-trip or it feels broken.
- ↳Measure deflection, CSAT, and time-to-resolution. If you can't measure it, you're just paying OpenAI for vibes.
A client showed me their old support chatbot last month. You typed “refund” and it asked you to pick from six buttons. You picked one. It asked you to pick from four more buttons. Then it gave you a phone number. That’s not an AI agent. That’s a phone tree with CSS.
I want to be specific about what changed, because the word “agent” is getting laundered the same way “AI” got laundered in 2023. Most things called agents are still chatbots. Some chatbots are quietly becoming agents. And the difference matters if you’re about to spend money on one.
the chatbot you already tried
The customer service ai chatbot wave from 2020-2023 was mostly intent classifiers wired to decision trees. You’d train it on 50 intents (“refund_request”, “shipping_status”, “password_reset”), and a classifier would route the message to a hardcoded flow. Intercom Fin v1, Drift, Ada, the early Zendesk Answer Bot. They worked in a narrow band and fell off a cliff outside it.
The failure mode was predictable. Customer types “hey my order from last tuesday hasn’t shipped and also can i change the address”. Bot picks one intent. Usually the wrong one. Customer escalates. Bot has no memory of the conversation when a human picks it up. Human starts from zero.
The LLM-wrapper chatbots that came next (mid-2023 onward) were better at understanding the question but mostly worse at doing anything about it. They could explain your return policy in fluent prose. They could not actually start a return.
what an actual agent does differently
An ai agent for customer support has three things a chatbot doesn’t:
- It reads the full conversation and the customer’s record, not a single classified intent.
- It calls tools. Real functions against your real systems. Look up the order, check inventory, issue the refund, create the Jira ticket, send the Resend email.
- It decides when to stop and hand off, with a useful summary, not a transcript dump.
Here’s roughly what a tool call looks like in practice. This is a trimmed version of something we shipped for a client running on Cloudflare Workers with Anthropic’s API:
const tools = [
{
name: "get_order",
description: "Fetch an order by ID or by customer email + recent date",
input_schema: {
type: "object",
properties: {
order_id: { type: "string" },
customer_email: { type: "string" }
}
}
},
{
name: "issue_refund",
description: "Refund an order. Requires manager approval over $200.",
input_schema: {
type: "object",
properties: {
order_id: { type: "string" },
amount_cents: { type: "integer" },
reason: { type: "string" }
},
required: ["order_id", "amount_cents", "reason"]
}
},
{
name: "escalate_to_human",
description: "Hand off with a summary. Use when policy unclear or customer asks.",
input_schema: { /* ... */ }
}
];The agent gets the conversation, picks tools in a loop, and writes back to the customer between calls. The model isn’t doing magic. It’s reading, choosing, calling, reading the result, choosing again. The Anthropic and OpenAI docs both describe this loop the same way.
The hard work isn’t the prompt. It’s writing issue_refund so it can’t be tricked into refunding an order that isn’t the customer’s. It’s the eval set. It’s the guardrails that block prompt injection attempts buried in a forwarded email.
where ai chatbots for customer service still win
I’ll be honest: if your support volume is 80% “what are your hours” and “do you ship to Canada”, you don’t need an agent. A decent FAQ bot with semantic search will deflect those tickets at a tenth of the cost. We’ve built both. We tell clients which one they actually need, and sometimes it’s the cheap one.
The agent shows up when the questions get messy. Multi-turn. Account-specific. Requires reading three systems and writing to one. That’s where the chatbot collapses and the agent earns its keep.
ai agent for technical support is a different beast
Tech support agents have a harder job. The customer’s problem usually isn’t in your CRM. It’s in their logs, their config, their version mismatch. A good ai agent for technical support has read access to whatever the customer can share: error messages, stack traces, sometimes a screenshot via vision models.
We built one for an internal tool that handles deployment issues. It reads the last 100 lines of build output, cross-references against a knowledge base of past incidents, and either suggests a fix or opens a ticket with the right engineer already tagged. About 40% of incoming issues never reach a human now. The other 60% reach a human with context, which is the more valuable number.
voice agents, briefly
AI voice agent for customer support is real now and it’s also a trap if you rush it. The latency budget is brutal. Speech-to-text, LLM call, text-to-speech, network. You need to be under about 800ms end-to-end or the conversation feels broken. Deepgram and ElevenLabs have made this possible. OpenAI’s Realtime API collapses the pipeline further. But the moment your agent needs to call a tool that takes 2 seconds, you need filler audio or the customer thinks you hung up.
We’re shipping voice agents now but I’m still cautious recommending them for high-stakes support. Text is forgiving. Voice is not.
what to measure
If you can’t measure it, you’re paying for vibes. The numbers we track on every deployment:
- Deflection rate (% of tickets resolved without a human)
- CSAT on agent-handled conversations vs human-handled
- Time to resolution
- Tool call success rate (this one catches most regressions before customers do)
- Escalation quality (did the human get useful context, or a wall of text)
For one e-commerce client we went from 12% deflection on their old Zendesk bot to 58% on the agent we built, with CSAT actually going up half a point. The CSAT part surprised me. Turns out people prefer a fast correct answer to a slow human one, even when they say they want a human.
what to ask before you buy or build
If a vendor is selling you an “AI agent” ask them: which tools does it call against my systems, what happens when it doesn’t know, and show me the eval set you ran against support tickets like mine. If they hand-wave any of those, it’s a chatbot with better marketing.
If you’re building it yourself, start with one workflow. Order lookups. Returns. Password resets. Get that single loop tight before you expand. The teams that try to ship a general-purpose agent on day one usually ship nothing.
We do this work at EdsDev, mostly for businesses where the support queue is the bottleneck. If you’ve got one and want a second opinion on whether an agent or a chatbot fits, send us a note.
Common questions
▸What's the actual technical difference between a chatbot and an AI agent?
A chatbot classifies your message into an intent and runs a scripted flow. An AI agent reads the full conversation, decides which tools to call (look up an order, issue a refund, search docs), executes them, reads the results, and decides what to do next. The agent changes state in your systems. The chatbot mostly routes you to one.
▸Are there open source AI agents for customer support on GitHub worth using?
Frameworks yes, drop-in products no. LangGraph, Vercel AI SDK, and Anthropic's tool-use examples are solid starting points. Mastra and CrewAI are getting decent. But they're frameworks, not products. You still write the tools, prompts, evals, and guardrails. Anyone who says they have an open source plug-and-play support agent is selling a demo, not a deployment.
▸How much does it cost to run an AI agent for customer service?
Variable cost per conversation runs roughly 5 to 30 cents in model calls, depending on model choice and average turns. Claude Sonnet and GPT-4o class models are usually overkill for FAQ-style traffic and necessary for complex resolutions. The real cost is engineering time: building tools against your systems, writing evals, and monitoring drift. Budget more for that than for tokens.
▸Can an AI voice agent handle support calls reliably?
For scoped tasks yes, for open-ended support I'm still cautious. Latency is the killer. You need sub-800ms response or the call feels broken, and tool calls eat that budget fast. Deepgram plus OpenAI Realtime plus ElevenLabs can hit it for simple flows. For anything involving multiple system lookups, build filler audio and set expectations clearly with the caller.
▸How do we stop the agent from hallucinating policies or making up refund amounts?
Two things. First, never let the model output the answer to a factual question without a tool call backing it. If it claims to know an order status, it called get_order. Second, constrain tool inputs server-side. Validate that the order belongs to the authenticated customer, cap refund amounts, require approval over a threshold. The model proposes. Your code disposes.
▸How do I pick the best AI agent for customer support for my business?
Start with your top 10 ticket types and ask: how many require reading or writing to your systems? If most are FAQ-style, buy a cheap chatbot with semantic search. If most require account lookups, refunds, or multi-step workflows, you need a real agent, either built custom or via a platform that supports proper tool calls. Ignore demos. Ask vendors to run a pilot on your actual ticket data.
- [1]Anthropic: Tool use with Claudedocs.anthropic.com
- [2]OpenAI: Function calling and toolsplatform.openai.com
- [3]OpenAI Realtime APIplatform.openai.com
Related posts
AI chatbots for customer service: what actually works after 90 days in production
Ninety days of running customer service bots in production. What survived, what got ripped out, and the design choices that decide whether your bot helps or annoys.
An AI lead generation agent that books meetings while you sleep
How we build AI lead generation agents that actually book meetings: the workflow, the prompts, the guardrails, and the bits that quietly break at 3am.
Building an AI chatbot for small business that actually knows your products
The RAG stack we ship for small business chatbots: ingestion, embeddings, retrieval, guardrails, and the boring parts that decide whether it answers correctly or hallucinates a refund policy.