Text classification & routing (emails, tickets, requests)

Shared inboxes, service desks and citizen portals get thousands of free-text requests in many languages. Someone has to read each one, work out what it is about and how urgent it is, and forward it to the right team. A classifier does that first read in milliseconds. It predicts topic, intent, priority and language, attaches a confidence score and either routes the message automatically or puts it in a short review queue. The result is fewer misrouted tickets, faster first response and dispatchers freed for real work. This is the backbone that answer agents (AGENT-001) and chatbots (CONV-001) build on.

Problem & value

• Today: dispatchers or first-level agents triage by hand, often with brittle keyword rules in the helpdesk or mail system. Misrouted tickets bounce between teams, which hurts SLAs. Urgent items such as complaints, cancellations, legal threats and GDPR requests get buried. Multilingual input makes it worse.
• AI value: consistent labels 24/7, routing in near real time, prioritisation by urgency and customer value, structured data for reporting (which topics drive volume), and a clean trigger for downstream automation.
• Evidence: helpdesk vendors now ship classification out of the box. Zendesk intelligent triage predicts topic, sentiment and language (about 150 languages) with a confidence value, and these fields feed routing and prioritisation workflows [1]. ServiceNow Predictive Intelligence predicts incident category and reports precision and coverage per solution [2]. In a 2024 JKU master's thesis on IT tickets at the Austrian formwork maker Doka, fine-tuned GPT models beat fine-tuned BERT on 4 of 5 ticket attributes, and data augmentation narrowed the gap [3]. Published vendor "routing accuracy" figures are rarely comparable, so benchmark on your own data.

Variants

• Customer service (B2C/B2B): intent and priority classification in Zendesk, Freshdesk, Salesforce Service Cloud, HubSpot or Microsoft Dynamics. Typical labels are order status, return, invoice question, cancellation, complaint and technical fault.
• IT service management: category, assignment group and priority for incidents and requests (ServiceNow, Jira Service Management, TOPdesk). Autonomous resolution is covered in AGENT-016.
• Shared mailboxes in the back office: info@, accounting@ and purchasing@ get split into order, invoice, reminder, delivery note and spam. Attachments go through DOC-008 input management.
• Insurers, utilities, telcos: high-volume written customer communication (address change, meter reading, claim notification, cancellation) with legal deadlines. Classification feeds case management.
• Public sector: citizen requests that arrive through OZG portals, email and letters get routed to the responsible office (Zuständigkeitsfinder logic). The front-end chatbot is CONV-011, and emergency-call triage is the separate high-risk use case CONV-021.
• Whistleblowing & ethics hotline intake (regulated variant): reports arriving via a HinSchG / EU Directive 2019/1937 channel are translated, summarised, deduplicated, classified (fraud, harassment, data protection, competition law) and routed to the competent case handler. The specific requirements are confidentiality of the reporter's identity, EU-only processing, anonymisation before wider circulation (DOC-018), strict role-based access, and the statutory deadlines: acknowledgement within 7 days and feedback within 3 months (§17 HinSchG) [4]. Vendors (e.g. EQS Integrity Line, Hintbox, LegalTegrity, all EU-based) advertise AI-assisted triage, but measured results have not been independently published. Use AI for preparation only, and keep the case decision with a human.
• Developers / open-source maintainers: labelling GitHub issues and pull requests. Bug reproduction and fixing is CODE-014.
• Private use: rule-plus-AI mail sorting in Gmail, Outlook or Thunderbird add-ons. Rarely needs a dedicated solution.

How it works

1. intake (IMAP/Graph API, helpdesk webhook, portal form, chat)
2. pre-processing (strip signatures/quoted history, detect language, PII masking if needed)
3. optional: translate to pivot language (LANG-009)
4. classification ├─ fast path: fine-tuned encoder (ModernBERT / mmBERT / XLM-R) or SetFit model └─ hard/new cases: LLM with label definitions + few-shot examples retrieved from similar historic tickets (kNN over embeddings), structured JSON output
5. confidence calibration & business rules (VIP, SLA, keywords for legal/GDPR)
6. routing: set queue/assignment group/priority, trigger workflow (n8n, Power Automate)
7. human review of low-confidence items
8. corrections flow back as training labels

• Label design matters more than the model. Keep labels mutually exclusive or explicitly multi-label, write a one-line definition plus examples for each, and add an "other/unclear" class.
• Hybrid cascade: a small encoder handles the bulk cheaply. An LLM handles the long tail, new categories and messages that need reasoning (e.g. "is this a complaint or a cancellation?"). Small LLMs (3–8B) can also act as the router directly.
• Human-in-the-loop: per-label thresholds; anything below goes to a dispatcher. Agents can correct labels in the helpdesk, and these corrections become the next training set. Periodic sampling of auto-routed items catches silent drift.

Data preparation

Data comes from the shared mailbox (Exchange, IMAP), the helpdesk or ITSM system, web forms and chat logs. The key asset is the history: past tickets with their final queue, category and priority. Knowledge about routing rules often exists only in the heads of the service team.

Typical gaps are inconsistent or too many categories, labels that reflect the first guess rather than the final outcome, tickets reassigned several times, signatures, quoted threads and disclaimers inside the text, duplicates, and personal data in free text. Several languages and attachments add noise.

Preparation steps: 1. Fix a category scheme of 10 to 30 classes with clear definitions (PREP-019). 2. Export tickets via API or mail export (PREP-005) and harvest labels from final queue and resolution (PREP-020). 3. Clean text: strip signatures and quoted replies, remove duplicates, mask names and IBANs (PREP-022). 4. Have experts label a test set of 300 to 500 tickets (PREP-018). 5. Capture agents' corrections of AI labels continuously (PREP-021).

Effort: high, about 40 to 50 percent of the project, mainly category design and label cleaning. Without this step accuracy stays well below what the model could deliver.

Implementation paths

Which path for whom

• SMB: Path A, because helpdesk and shared-inbox products already offer EU-hosted auto-triage and the routing rules are standard.
• Enterprise: Path B, because volume, multiple languages and links to ITSM or CRM favour an own classifier on cloud APIs trained on historical tickets.
• Developer: Path B, because LLM or embedding APIs with a labelled test set let developers build a routing service fast.
• Public: Path C, because citizen requests contain personal data and a self-hosted classifier on EU infrastructure keeps control and meets procurement rules.

Path A — Buy (recommended for most SMBs)

• Built into helpdesk/ITSM: Zendesk intelligent triage (Copilot add-on) [1], Freshdesk Freddy AI, Salesforce Einstein for Service, HubSpot Service Hub, Microsoft Dynamics 365 Contact Center, ServiceNow Predictive Intelligence / Now Assist [2], Jira Service Management, TOPdesk (NL).
• EU / German email-response & input-management suites: novomind iAGENT (Hamburg) [5], Insiders Technologies Response Management / OvAItion (Kaiserslautern, part of Proalpha) [6], ITyX (Cologne), Userlike / Cognigy for chat-centric flows. Check for EU hosting, an on-prem option, German-language quality and connectors to SAP, DMS and CRM.
• Choose A when: your helpdesk already offers it, labels are standard, and volume is moderate. Expect configuration (custom topics, routing rules) rather than development.

Path B — Build on cloud APIs

• LLM classification: prompt a frontier or mid-size model with label definitions and examples, and enforce JSON schema output. Options include Mistral (EU provider, e.g. Mistral Small/Medium), Claude via AWS Bedrock or Google Vertex EU regions, GPT via Azure OpenAI EU Data Zone, and Gemini on Vertex AI EU. Anthropic's ticket-routing guide is a useful reference for prompt design and evaluation [7].
• Classic managed classifiers: Azure AI Language custom text classification (single/multi-label) still works. Microsoft has announced its retirement on 31 March 2029 and directs new projects to Foundry models, so don't start new long-lived projects on it [8]. Mistral's former Classifier Factory fine-tuning is marked deprecated [9].
• Cost control: classify with a small/cheap model, escalate to a larger one only if confidence is low, and cache by thread. Prompt caching of the long label definition helps.
• Integration: helpdesk webhooks or Microsoft Graph for mailboxes, plus n8n / Power Automate / Camunda for routing actions.

Path C — Self-host (open source)

• Encoder classifiers: ModernBERT (Answer.AI/LightOn, 8k context) [10] for English, and mmBERT (multilingual, 1,800+ languages, 8k context) [11] or XLM-RoBERTa / German BERT variants for German and mixed input. Fine-tune with Hugging Face Transformers.
• Few-shot / zero-shot: SetFit (a few labelled examples per class, competitive with full fine-tuning in the authors' tests) [12]. GLiClass (Apache-2.0, zero-shot labels at inference time, about 10× faster than cross-encoders per the authors) [13].
• Small LLMs as router: Qwen3 (4B/8B), Gemma 3, Mistral Small or Llama 3.x served via vLLM or Ollama with constrained JSON decoding (Outlines / vLLM structured outputs).
• Tooling: Argilla or Label Studio for labelling; embeddings (e.g. multilingual-e5, BGE-M3) with Qdrant/pgvector for kNN few-shot retrieval; n8n self-hosted for routing.
• Hardware: encoders run on CPU, at hundreds of messages per second on a standard server (HW1). A small LLM router needs HW2 (one 16–32 GB GPU). Ops effort is low to moderate: monitoring, periodic retraining.

Path D — Custom training

Worthwhile when you have thousands of historic tickets with reliable final labels (the queue that actually resolved them, not the first guess) and high volume. Then a fine-tuned encoder is usually the cheapest and fastest option, and on a stable label set often at least as accurate as a prompted LLM. LoRA fine-tuning of a small LLM is the alternative for multi-attribute outputs. Clean labels first. Merge duplicate categories and relabel a stratified sample. Retrain quarterly or on drift.

State of the art (2026)

• Encoders are back. ModernBERT (Dec 2024) and mmBERT (Sep 2025) brought 8k context and higher throughput to the BERT class [10][11]. Long emails with quoted history no longer need truncation to 512 tokens.
• LLMs for the long tail: zero/few-shot LLM classification removes the cold-start problem and handles label changes without retraining [7]. Fine-tuned models still tend to win on stable, high-volume label sets [3].
• Retrieval-augmented classification: retrieving similar labelled tickets as in-context examples (or kNN voting on embeddings) is a cheap, strong baseline and adapts as new labels arrive.
• Helpdesk-native triage is commodity in major platforms [1][2]. Differentiation is moving to routing optimisation (OPT-035) and autonomous resolution (AGENT-001, AGENT-016).
• Limits: ambiguous or multi-topic messages, sarcasm, mixed-language threads, label drift after product launches, LLM over-confidence (self-reported confidence is poorly calibrated, so use log-probs or agreement checks), and prompt injection in incoming email text. Never let an email's content change routing rules or trigger privileged actions.

Shared resources & combinations

• Same encoder/small-LLM stack (text-classifier, llm-small, embedding-text): LANG-013 sentiment, LANG-014 moderation, LANG-015 entity extraction, LANG-024 compliance review and DOC-008 document classification can share one CPU/HW2 inference server, one labelling tool and one evaluation harness.
• Pipeline neighbours: DOC-008 (attachments, scanned mail) → LANG-012 → AGENT-001 (draft or send reply), AGENT-016 (IT resolution), CONV-007 (agent assist), LANG-008 (thread summary for the handler), OPT-035 (which agent gets the contact).
• Translation & privacy: LANG-009 for pivot translation and DOC-018 for anonymisation, which is mandatory in the whistleblowing variant.
• Knowledge: the same embeddings index can back KNOW-002 answer suggestions.
• Workflow engine shared with AGENT-002.

Effort, cost & KPIs

• Path A: days to a few weeks of configuration. Usually a per-agent add-on licence.
• Path B: 4–8 weeks including evaluation set, prompt tuning and integration. Per-message API cost is small for short emails. The cost drivers are long threads and calls to large models.
• Path C/D: 1–3 months including labelling and MLOps. Inference cost is negligible on CPU.
• KPIs: routing accuracy / macro-F1 per label on a held-out set, auto-routing rate at target precision, reassignment (bounce) rate, time to first assignment, SLA breach rate for urgent classes, review-queue size, drift alerts.

People & change

Involve management and the department lead (service, IT desk) first: they own the categories, SLAs and the escalation rules. End users (agents) must help define classes and review low-confidence cases, otherwise nobody trusts the labels. IT integrates mailbox and ticket system; data owners release the history. The DPO is needed because emails and tickets contain personal data: check legal basis, AVV with every processor, and a DPIA (Art. 35 GDPR) if volumes are large or special categories occur (GUIDE-005).

Works council: yes. Routing systems log who handled what and how fast, so they are objectively suited to monitor performance and trigger §87(1) no. 6 BetrVG (BAG). Agree that classification data is not used for individual performance evaluation (GUIDE-004). §90(1) no. 3 requires information in good time; Personalrat in the public sector.

AI literacy: support the team with training on confidence scores, bias and the habit of checking low-confidence items, see GUIDE-006. Communicate that routing removes sorting work and not jobs (GUIDE-008). Run a pilot in shadow mode with one team: the AI suggests, agents decide, and you compare (GUIDE-009). Keep humans in control for complaints, legal threats and data-subject requests, which are always escalated to a person.

Risks, regulation & compliance

• EU AI Act: typical topic/queue routing is minimal risk, with only the AI literacy duty (Art. 4). It depends on context:
• If classification decides access to essential public services or benefits (Annex III 5(a)), insurance or credit, or prioritises emergency calls (Annex III 5(d), see CONV-021), the system can be high-risk. A purely preparatory or narrow procedural routing step can fall under the Art. 6(3) derogation, but never when it profiles natural persons [14].
• Classifying employees' messages for performance or conduct moves towards Annex III 4 (see LANG-032).
• Inferring emotions of employees is prohibited under Art. 5(1)(f). Customer-sentiment tags are not emotion recognition in the Art. 5 sense, but keep them away from staff evaluation.
• GDPR: emails and tickets contain personal data and sometimes special categories (health, union membership). You need a legal basis (usually contract or legitimate interest), a DPA with the vendor, EU hosting, retention limits for training data, and purpose limitation when reusing historic tickets for training. Automatic routing alone is not an Art. 22 decision, but auto-rejection or deprioritisation with significant effect can be.
• Whistleblowing: confidentiality under §8 HinSchG, deadlines under §17 HinSchG [4], and access logging. No US-cloud processing without a transfer assessment.
• Mitigations: confidence thresholds, a mandatory "unclear" queue, keyword safety nets for legal and GDPR requests, a monthly audit sample, a bias check across languages and customer groups, and prompt-injection filtering.

Getting started

1. Export 3–12 months of tickets/emails with their final queue and category. Count volume per label and measure today's reassignment rate.
2. Clean the label set: 10–30 well-defined classes, an "other" class, and written definitions. Relabel a stratified sample of about 500 items as a gold test set.
3. If your helpdesk has native triage (Path A), test it first against the gold set. Otherwise run a prompted LLM with few-shot examples (Path B) as a baseline.
4. If volume is high and labels are stable, train a ModernBERT/mmBERT or SetFit model (Path C/D) and compare accuracy, latency and cost.
5. Go live in "suggest" mode, then enable auto-routing per label once precision meets target. Feed corrections back monthly.

Sources

1. Zendesk — Automatically classifying tickets with intelligent triage — https://support.zendesk.com/hc/en-us/articles/4550640560538
2. ServiceNow — Predictive Intelligence (incident categorisation, precision & coverage) — https://www.servicenow.com/docs/bundle/zurich-intelligent-experiences/page/administer/predictive-intelligence/concept/predictive-intelligence.html
3. Engelbrechtsmüller, S. (2024): Fine-Tuning Large Language Models for Ticket Classification at Doka GmbH, JKU Linz — https://research.jku.at/en/publications/fine-tuning-large-language-models-for-ticket-classi%EF%AC%81cation-at-dok/
4. Hinweisgeberschutzgesetz §17 (deadlines) — https://www.gesetze-im-internet.de/hinschg/__17.html
5. novomind — iAGENT customer service software — https://www.novomind.com/en/
6. Insiders Technologies — Response Management, OvAItion — https://insiders-technologies.com/en/
7. Anthropic — Ticket routing use-case guide — https://platform.claude.com/docs/en/about-claude/use-case-guides/ticket-routing
8. Microsoft Learn — Custom text classification overview (retirement notice) — https://learn.microsoft.com/en-us/azure/ai-services/language-service/custom-text-classification/overview
9. Mistral AI — Classifier Factory (deprecated) — https://docs.mistral.ai/capabilities/finetuning/classifier_factory/
10. Hugging Face — Finally, a replacement for BERT: ModernBERT — https://huggingface.co/blog/modernbert
11. Hugging Face — mmBERT: a modern multilingual encoder — https://huggingface.co/blog/mmbert
12. SetFit — efficient few-shot learning with Sentence Transformers — https://github.com/huggingface/setfit
13. Knowledgator — GLiClass zero-shot classification — https://github.com/Knowledgator/GLiClass
14. EU AI Act Article 6 (classification rules, Art. 6(3) derogation) — https://artificialintelligenceact.eu/article/6/
15. TeleDoCTR: domain-specific ticket routing with BERT and LLMs (arXiv 2601.00691) — https://arxiv.org/abs/2601.00691