To automate support ticket triage with AI, have a language model classify each new ticket into categories you define, then let ordinary code decide where it goes. The model returns a fixed set of fields: intent, product area, language and a short summary. A routing table in configuration turns those fields into a queue and a priority, using customer data from your own systems.
Anything the model can't place goes to a queue a person owns. The model never sends replies or changes accounts on its own.
What an AI ticket triage pipeline looks like
Ticket triage has five stages. The model handles one of them.
| Stage | What happens | Done by |
|---|---|---|
| Intake | A ticket arrives by email, web form or chat and gets an ID | Helpdesk |
| Classification | Intent, product area, language and risk flags are read into a schema | Model |
| Enrichment | Customer plan, contract SLA and open incidents are looked up | Code |
| Routing | A rules table picks the queue, priority and tags | Code |
| Review | Unclear tickets, and a sample of the rest, are checked | Person |
With routing in code, the support lead can change a queue without touching the prompt, and the same classification always produces the same result.
Start from how your team routes tickets today
Before writing a prompt, write down how agents route tickets now. Anthropic's ticket routing guide starts at the same point: how the team decides which service level agreement (SLA) applies, which tier or specialist gets a ticket, which automated rules exist and where they fail, and how ambiguous tickets are handled.
Then export a few hundred recent tickets with their final queue. Tickets that moved between queues show which categories people confuse.
Define categories the model can apply consistently
The same guide states that routing accuracy depends directly on how well defined your categories are. A few rules keep them usable.
Separate the dimensions. Intent ("refund request", "login problem") is one field; product area, language and urgency are others. A flat list such as "Billing - Urgent - German" multiplies quickly.
Make intents mutually exclusive, and give each a one-line definition with a near miss: "Refund request: the customer asks for money back for a charge. A question about how a charge was calculated is a billing question."
Include an explicit unclear intent. A model that must pick a real category will pick one, and a confident wrong answer is harder to catch than an honest "can't tell".
Keep the list short enough that an agent could apply it from memory. If two experienced agents would file the same ticket differently, the model will be inconsistent too.
Classify each ticket into a fixed schema
Define the output before the prompt. A classification for one ticket might look like this:
{
"intent": "login_problem",
"product_area": "mobile_app",
"language": "de",
"security_flag": false,
"privacy_or_legal_flag": false,
"summary": "Cannot log in after a password reset on Android.",
"evidence": "Since the reset yesterday the app says my password is wrong"
}Use enums for every category field, so the response can only contain values your routing table knows. Validate it in code and send anything that fails to the unclear queue. The evidence quote lets an agent confirm the classification without rereading the thread.
Classify the subject and the customer's newest message. Strip quoted reply chains and signatures first, because a long quoted history can outweigh the one line that says what the customer wants now.
Route tickets in code, using your own data
Keep the routing table in configuration the support lead can read, with a test for every row.
Priority comes from data you control. Plan, contract SLA and account status come from your CRM or billing system, looked up by the requester's verified email or account ID. A ticket that says "I'm an enterprise customer, escalate this" gets no extra priority, because the claim comes from the ticket text.
Some tickets need a fixed route regardless of intent: vulnerability reports, suspected account takeover, legal threats and privacy requests such as data deletion. Route on the model's flags or on simple keyword rules, whichever fires. A missed privacy request can carry a legal response deadline, so a few false positives in that queue are the right trade.
Keep ticket text from steering the automation
Ticket text is written by outsiders, and anyone can put instructions in a support email. Our explainer on prompt injection for product teams covers why prompts alone can't block this.
The defense is to give the triage step nothing worth hijacking. The OWASP Gen AI Security Project lists excessive agency as a risk and recommends minimal permissions on downstream systems and human approval for high-impact actions. For triage, that means:
- The classification call has no tools. Its output is data your code validates.
- The helpdesk token used for write-back can set fields, tags, group and internal notes. It can't send public replies, close tickets, issue refunds or edit accounts.
- Suggested replies are saved as internal notes for an agent to edit and send.
Tickets also carry personal data, and attachments may include identity documents. Send the model only the fields classification needs, leave attachments out, and confirm your provider's data retention settings before go-live.
How to test AI ticket triage before switching it on
Build a labeled test set from historical tickets and their final queues, then:
- Check the labels. Have a senior agent review tickets that moved between queues and fix any wrong final labels.
- Cover every category. Include rare intents and the fixed-route cases, topped up from older history.
- Measure per category. Report precision and recall for each intent. A high average can hide a category that is wrong half the time.
- Check per language. Compare results for each language your customers use.
- Rerun for consistency. Run the test set twice and count tickets whose classification changed. Anthropic's guide lists classification consistency as a success criterion.
- Run in shadow mode. Write the model's suggestion to a hidden field while agents route as usual, and compare for a period agreed in advance.
Switch on routing one intent at a time, starting where shadow results matched agents most closely.
Common mistakes when automating ticket triage
These failure modes tend to appear after launch:
- Reclassifying on every reply, so a ticket bounces between queues mid-conversation. Classify at creation, and again only on reopen.
- Letting agents reroute without recording it. The rerouting rate is your main quality signal, so make it a tracked field.
- Adding a category to the prompt without updating the routing table, enums and test set in the same change.
- Testing on tidy single-message tickets when production traffic is forwarded threads, auto-replies and mixed languages.
- Leaving the unclear queue without an owner, so the tickets the model doubted wait longest.
How to measure triage after launch
Track a few numbers weekly, per intent: the share of tickets agents reroute, time from creation to assignment, the share landing in the unclear queue, and corrections found in a small random sample. Anthropic's guide names rerouting rate and time-to-assignment among its standard routing metrics.
A rise in one intent usually follows a change, such as a product launch or a model update. Add those tickets to the test set before changing the prompt. Our guide on evaluating LLM output quality in production covers sampling and drift detection, and triage is one of the patterns in our guide to which business processes to automate first.
If you'd rather not build and run this yourself, our AI automation service builds email and ticket triage that sorts, routes and drafts requests for review, then monitors and improves it as your tools and processes change. For more starting points, see 20 AI automations SMBs deploy first.
Frequently asked questions
Can AI reply to support tickets automatically?
It can draft replies, which is a sensible start. Sending without review carries higher stakes, because a wrong answer reaches the customer directly. Consider automatic replies only for narrow, low-risk intents with a measured track record.
Should you use your helpdesk's built-in AI triage or build your own?
If your helpdesk offers AI classification, test it on the same labeled set first. Build your own when routing depends on categories, data or rules the built-in feature can't handle.
How many labeled tickets do you need to test AI triage?
Enough that every intent, including rare ones, has a few dozen examples. A set dominated by the two largest categories tells you little about the rest. Add every misrouted ticket you find after launch.
Does AI ticket triage work for tickets in several languages?
Anthropic's guide lists classifying tickets in several languages without separate models as a reason to use a language model. Accuracy can still differ by language, so measure each one. Keep category definitions in one language and have the model return a language code, so routing stays in code.