Document classification & mailroom / input management

Every organisation receives documents in mixed batches: a scanned stack with an invoice, a delivery note and a customer letter in one PDF; an email with six attachments; a portal upload with an unknown form. Input management (the "digital mailroom") recognises what each document is, splits batches, captures key index data and routes it to the right process. Modern systems replace rule- and template-based sorting with zero-shot or few-shot classifiers (VLMs/LLMs) and trained page classifiers. It is the front door for all other document-AI use cases.

Problem & value

• Today: staff open post and email, sort by eye, scan, name files and drop them into folders or queues. Mixed batches need manual separator sheets or barcodes; misrouted documents miss deadlines (e.g. objection periods, discount dates).
• AI value: automatic type recognition and page splitting without per-layout templates, extraction of routing keys (customer number, case ID, contract number), confidence-based exception handling, and one consistent intake for all channels. Classification is the gate that lets specialised extractors (invoice, form, ID, medical) run only on matching documents.
• Figures: vendor-reported results exist (e.g. DocuWare states that GenAI-based extraction can cut manual data entry by roughly 80% [8]), but these are marketing claims, not independent measurements. Measure your own baseline.
• Legal driver (Germany): the GoBD require that incoming business documents are captured completely, traceably and without alteration, which makes a documented, auditable intake process (Verfahrensdokumentation) part of the value case [7][14].

Variants

• SMB: email-in plus scan-to-folder with a cloud DMS/accounting tool that sorts documents into invoice, contract, correspondence and tax documents.
• Enterprise shared-service centre: central scanning/mailroom, multi-entity routing, SAP/Dynamics integration, e-invoice channels in parallel (XRechnung/ZUGFeRD, Peppol).
• Insurance, banking, utilities: high-volume customer correspondence, claims and application packages; classification plus routing to case systems (see DOC-003, DOC-024).
• Public sector: e-Akte/DMS intake, assignment to the filing plan (Aktenplan), deadline-relevant document detection (e.g. Widerspruch); lifecycle governance of the existing corpus is a different case, see KNOW-025.
• Legal / HR: separating mixed case files, classifying application documents (the document type only; candidate evaluation would be a different, high-risk use case).
• Private use: a household inbox that sorts scanned letters into tax, insurance and warranty folders (see DOC-002).

How it works

1. channels (post scan / email / portal / upload / EDI)
2. normalisation (convert Office/images to PDF/PNG, de-skew, remove blank pages, virus scan)
3. OCR + layout parsing, per-page text + page image
4. page classification (zero-shot VLM/LLM or trained classifier) + confidence
5. batch splitting (page ranges
6. logical documents; continuation-page detection)
7. light extraction of routing keys (sender, customer/contract/case no., date, deadline)
8. matching against master data (CRM/ERP/case system)
9. routing: specialised extractor (DOC-001, 003, 004, 006, 007) | queue | person | archive
10. human review queue for low-confidence / unknown ("other") documents
11. GoBD-compliant archiving + audit log

• Classification: two families coexist. (1) Trained classifiers on layout plus text features, which need only a few labelled examples per class and give calibrated confidence (e.g. Azure Document Intelligence custom classifier: min. 5 samples per class, up to 1,000 classes, returns page ranges and confidence; PDF, images and Office files [1]). (2) Generative zero-/few-shot classification with a VLM or LLM that gets the class list with short descriptions (Google Document AI custom classifier v1.5 works zero-shot, fine-tuning improves it [2]). Hybrid designs use the cheap classifier first and escalate uncertain pages to a large model.
• Splitting: the hardest part in practice. Services return page ranges per class (Azure splitMode = auto, perPage or none [1]); LLM-based splitters judge "is this page a new document or a continuation?" from first-page cues, page numbering and sender/letterhead.
• Human-in-the-loop: confidence thresholds per class; an explicit "other / unknown" class so unseen documents are not forced into a wrong bucket [1]; reviewer corrections feed back as training data.

Data preparation

Data: scanned post and batch scans, e-mail attachments, portal uploads and e-invoices, plus a document taxonomy, filing plan and routing rules. In an SMB these live in the mailbox, a scanner folder, the DMS or file shares and, still, paper trays; master data (customers, contracts, cases) sits in the ERP or CRM.

Typical gaps: no agreed document types, inconsistent folder structures, mixed batches without separator sheets, bad scans, duplicates, missing metadata and unclear access rights.

Preparation steps: 1. Define the taxonomy and the "other" class together with the departments (PREP-019). 2. Harvest labelled examples from existing filed documents; 20-50 per class are usually enough (PREP-020, PREP-010). 3. Clean up the file structure and naming where it is the source of labels (PREP-009). 4. Digitise paper and convert legacy formats consistently (PREP-006, PREP-007). 5. Export master data for matching (PREP-005). 6. Fix routing rules and retention classes, and check legal basis (PREP-025). 7. Keep a held-out test set from real post (PREP-018).

Tools: scanners with batch separation, OCRmyPDF, DMS import tools, spreadsheet for the taxonomy.

Effort share: about 25-35 percent; the taxonomy discussion often takes longer than expected. See DOC-014, DOC-018 and GUIDE-003.

Implementation paths

Which path for whom

• SMB: Path A. Mature input-management and DMS products with built-in AI classification are available from EU vendors at SMB prices, and incoming mail is a standard process.
• Enterprise: Path B. Many channels, own filing plans and routing into several specialised extractors call for configurable APIs and orchestration.
• Developer: Path C. An open-source pipeline (OCR, small classifier, workflow tool) is easy to build and reuse across projects.
• Public: Path C. Authorities handle citizen data and need sovereign operation and integration with e-file systems on German or own infrastructure.

Path A — Buy (recommended for most SMBs and mid-market)

• DMS/ECM with built-in input management (EU/German vendors marked): DocuWare (DE/US; IDP with GenAI zero-shot extraction, intelligent indexing; cloud or on-premises [8]), d.velop (DE), ELO Digital Office (DE), Fabasoft (AT), SER/Doxis (DE), ecoDMS (DE), paperless-ngx (open source, see C).
• IDP and capture platforms: ABBYY Vantage, Kofax/Tungsten Automation, Hyland, OpenText, Rossum, Hypatos (DE), Klippa (NL), Parashift (CH), plus scanning-service providers that run the mailroom as a service (check processor contract and location).
• Productivity-suite classification: Microsoft Purview trainable classifiers label content by type for retention and DLP, but custom classifiers are limited to English and cannot be retrained once published [3]; they classify stored content for compliance, not mailroom routing.
• Choose A when: standard document families, an existing DMS/ERP, no data-science team. Check: EU hosting, split quality on your mixed batches, unknown-class handling, connectors, GoBD/IDW PS 880 archive support.

Path B — Build on cloud APIs

• Specialised classifiers: Azure Document Intelligence custom classification (West Europe available as a copy target) [1]; Google Document AI custom classifier with EU processing for v1.5, fine-tuning in EU only as Preview at the time of writing, and newer v1.6 release candidates without data-residency support [2]; Amazon Comprehend custom classification for native PDF/image documents (verify EU-region availability) [4].
• Generative classification and routing: multimodal LLMs (Claude, Gemini, GPT, Mistral) with structured output: {doc_type, confidence, page_range, routing_keys}; Mistral Document AI (OCR plus annotations; EU provider) [5].
• Orchestration: n8n, Power Automate or Camunda for channels, routing and review tasks; connectors to DATEV, SAP, d.velop or DocuWare.
• Choose B when: you need custom taxonomies, unusual routing logic or integration into your own platform, and cloud processing is acceptable under a DPA with EU region.

Path C — Self-host (open source)

• Parsing/OCR: Docling (MIT; reads PDF, Office, email EML/MSG, images; runs locally for air-gapped use; includes the Granite-Docling 258M VLM, Apache 2.0) [6][9], PaddleOCR, MinerU.
• Classification: fine-tuned encoder (e.g. a multilingual BERT-class model or layout-aware model) or an open VLM such as Qwen3-VL (2B to 235B, Apache 2.0, multilingual OCR) [10] served with vLLM or Ollama; embedding-based nearest-neighbour classification with a vector DB for fast few-shot extension.
• DMS: paperless-ngx (auto-tagging with a classic classifier), Mayan EDMS, Nextcloud; workflow with n8n.
• Hardware: HW2 (one 24-32 GB GPU) handles a mid-sized mailroom with an 8B-class VLM plus a small page classifier; classification-only workloads often run on CPU (HW1) with a fine-tuned encoder.

Path D — Custom training

Worth it for high volumes with stable, company-specific taxonomies (hundreds of classes, special forms, multilingual post). Fine-tune a layout/text classifier or a small VLM on a few hundred to a few thousand labelled pages per hard confusion pair; reviewer corrections are the labelled data. Public benchmark data such as RVL-CDIP (400,000 images, 16 classes, 25,000 per class) is useful for pre-training and sanity checks but its classes (letter, memo, form, invoice, etc.) are far coarser than real company taxonomies and its licence terms need checking [11].

State of the art (2026)

• Zero-shot is good enough to start: vendors now ship classifiers that work without training (Google v1.5 zero-shot [2]) and general VLMs classify pages from image plus short class descriptions. Trained classifiers still win on calibrated confidence, cost per page and latency for stable taxonomies [1][2].
• Incremental updates: Azure supports adding classes or samples without keeping the original training set (incremental training) [1].
• Small document VLMs: compact models such as Granite-Docling (258M) convert pages to structured text on commodity hardware, lowering the cost of local first-pass parsing [9].
• Limits / failure modes: (1) splitting errors of continuation pages and attachments without headers; (2) near-identical classes (credit note vs invoice, order confirmation vs offer); (3) low-quality scans, fax, handwriting, stamps; (4) forced assignment of unseen document types if there is no "other" class or threshold [1]; (5) language mix; (6) prompt injection inside documents (a letter saying "route this to the CEO / ignore previous instructions") when an LLM decides routing; (7) benchmark accuracy on public sets (RVL-CDIP) does not predict accuracy on your taxonomy.

Shared resources & combinations

• Same stack as DOC-001/002/003/004/006/007/014: OCR + layout parsing + VLM server (HW2/HW3). One shared page-image service and review UI can serve the whole intake chain; classification output selects the downstream extractor.
• Downstream: DOC-001 invoices, DOC-002 receipts, DOC-003 forms/claims, DOC-004 ID documents, DOC-006 logistics documents, DOC-007 medical records, DOC-014 parsing for RAG.
• Related classification and routing: LANG-012 (text/ticket classification), AGENT-001 (email inbox agent) and AGENT-004 (order entry from email/PDF) share the email-ingest component; KNOW-025 applies the same classes to the existing corpus (retention, filing plan).
• Privacy layer: DOC-018 (anonymisation/redaction) when documents are shared further; guardrails for prompt injection.
• Shared building blocks: text embeddings and a vector index for few-shot class extension, workflow engine (AGENT-002), DMS/archive, master-data lookup (CRM/ERP).

Effort, cost & KPIs

• Path A: days to weeks; typically per-page, per-document or per-user licences. Scanner/MFP integration and process design dominate.
• Path B: 1-3 months for channels, classifier, review UI and DMS integration; cost per page is small compared to manual sorting, but grows with LLM calls on every page (use a cascade).
• Path C: 2-4 months plus operations; one GPU server or CPU-only for classifier-only setups.
• Path D: adds labelling and evaluation effort (weeks) on top of B or C.
• KPIs: classification accuracy and macro-F1 per class (not only overall), split accuracy (page-range exact match), straight-through rate, share routed to "other"/review, misrouting rate found downstream, time from receipt to the right queue, deadline-relevant documents processed within SLA, cost per document.

People & change

Involve: management (sponsor), the heads of the receiving departments (they own the document types and routing), the mailroom or back-office staff as daily users, IT (scanners, mailbox, DMS), the data owner of the archive and the DPO, because letters contain personal data of customers, applicants and employees.

Works council: depends. Classification itself does not monitor staff, but review queues with per-person correction or throughput data are suited to monitor performance and trigger §87(1) no. 6 BetrVG (BAG: suitability suffices). Keep such statistics at team level or regulate them in a works agreement, and inform the council under §90(1) no. 3. Public sector: Personalrat. See GUIDE-004.

Data protection: an AVV with any cloud service is required; a DPIA is normally not needed for pure routing but should be checked if sensitive categories (HR, health) are classified (GUIDE-005).

AI literacy: support the mailroom team with a short introduction to confidence scores and the "unknown" class (AI Act Art. 4; GUIDE-006).

Pilot and communication: start with one inbox or three to five document types, run in suggestion mode, and widen once accuracy per class is stable (GUIDE-009, GUIDE-008).

Concerns: "my job disappears" (staff move to exception handling and process improvement), misrouting of confidential post (confidential classes always go to a human) and transparency about what is logged.

Risks, regulation & compliance

• EU AI Act: typically minimal risk (document sorting, no Annex III purpose); AI literacy (Art. 4) applies. Classification becomes part of a higher-risk system if its output drives decisions listed in Annex III, e.g. an intake that automatically ranks or rejects job applications or benefit applications (employment, access to essential services), so keep intake classification separate from decisions about people and have humans decide [12].
• GDPR: mail contains personal data, sometimes special categories (health, union, religion; Art. 9) and correspondence with authorities. Needs a DPA, EU hosting or adequate safeguards, role-based access to routed documents (misrouting is itself a data breach risk), retention and deletion rules. Automated decisions with legal effect (Art. 22) must not result from routing alone.
• Bookkeeping / archiving: GoBD (completeness, traceability, immutability of the original, process documentation for scanning that replaces paper originals), HGB/AO retention; keep the original file plus AI metadata with the audit log [7][14].
• Confidentiality: professional secrecy (lawyers, doctors, tax advisers) and works-council co-determination for staff-related mail; mailboxes marked private must be excluded.
• Security: malware in attachments, prompt injection in document text; use sandboxed conversion, treat document text as untrusted data, never let the classifier call tools directly based on document content.
• Mitigations: confidence thresholds, "other" class, sampling-based QA of auto-routed documents, drift monitoring when new document types appear.

Getting started

1. Collect 2-4 weeks of real mixed intake (post, email, portal); define 10-30 document types and routing rules with the receiving departments, including an explicit "other" class.
2. Label a test set (a few hundred pages, with split boundaries) and measure a zero-shot baseline from 2-3 candidates (a DMS vendor, a cloud classifier, a VLM prompt).
3. Add routing-key extraction and master-data matching; start with auto-routing only for classes above a high confidence threshold, review everything else.
4. Connect to the DMS/archive and the downstream extractors (DOC-001, DOC-003); document the process for GoBD.
5. Track per-class accuracy and the review share weekly; feed corrections back into training or few-shot examples, and widen automation step by step.

Sources

1. Microsoft Learn — Azure Document Intelligence custom classification model (v4.0 2024-11-30; splitMode, min. 5 samples/class, incremental training) — https://learn.microsoft.com/en-us/azure/ai-services/document-intelligence/train/custom-classifier
2. Google Cloud — Document AI custom classifier (zero-shot v1.5, fine-tuning, EU/US regions) — https://docs.cloud.google.com/document-ai/docs/custom-classifier
3. Microsoft Learn — Microsoft Purview trainable classifiers — https://learn.microsoft.com/en-us/purview/trainable-classifiers-learn-about
4. AWS — Amazon Comprehend documentation (custom document classification; EU availability to be verified) — https://docs.aws.amazon.com/comprehend/latest/dg/idp-document-classification.html
5. Mistral AI — Document AI / OCR documentation — https://docs.mistral.ai/capabilities/document_ai/
6. Docling project (MIT licence; format support incl. email, Office, images) — https://github.com/docling-project/docling
7. § 147 AO, Ordnungsvorschriften für die Aufbewahrung von Unterlagen (legal basis behind the BMF GoBD letter on electronic records) — https://www.gesetze-im-internet.de/ao_1977/__147.html
8. DocuWare — Intelligent document processing, classification and indexing (vendor claims) — https://start.docuware.com/
9. IBM — Granite-Docling-258M model card (Apache 2.0, released 2025-09-17) — https://huggingface.co/ibm-granite/granite-docling-258M
10. QwenLM — Qwen3-VL repository (2B-235B, Apache-2.0, OCR in 32 languages) — https://github.com/QwenLM/Qwen3-VL
11. RVL-CDIP dataset card (400,000 images, 16 classes) — https://huggingface.co/datasets/aharley/rvl_cdip
12. EU AI Act Service Desk — Annex III high-risk areas — https://ai-act-service-desk.ec.europa.eu/en/ai-act/annex-3
13. Regulation (EU) 2024/1689 (AI Act) — https://eur-lex.europa.eu/eli/reg/2024/1689/oj
14. BMF — GoBD letter of 28 Nov 2019, 2nd amendment of 14 July 2025 (e-invoices, applies from 14 July 2025) — https://www.bundesfinanzministerium.de/Content/DE/Downloads/BMF_Schreiben/Weitere_Steuerthemen/Abgabenordnung/2025-07-14-GoBD-2-aenderung.html