Industrial OCR: labels, serial numbers, package tracking

Production lines, warehouses and parcel hubs must know exactly which part, batch or parcel passes a point. Barcodes cover most of this, but many items carry only human-readable or marked text: engraved serial numbers on castings, dot-peened VINs, curved or reflective labels, damaged or smudged print, handwritten parcel addresses. Deep-learning OCR on industrial cameras reads these reliably at line speed, verifies them against master data and feeds traceability and tracking systems, replacing manual keying and fragile rule-based character libraries.

Problem & value

• Today: traceability relies on 1D/2D codes plus manual reading and keying for everything else. Classic machine-vision OCR needs carefully trained fonts, controlled lighting and breaks on variation (corrosion, texture, curvature, low contrast, changing fonts). Failed reads mean line stoppages, manual re-scans, mislabelled goods and missing audit trails.
• AI value: font-free, pretrained or few-shot character reading that tolerates distortion and background texture; higher read rates on difficult marks; fewer rejected-for-unreadable items; automatic verification against order or batch data; searchable image evidence per item.
• Regulatory pull in the EU: medicines carry a unique identifier (2D Data Matrix with product code, serial number, batch, expiry) plus human-readable text under Delegated Regulation (EU) 2016/161 [1]; the Battery Regulation (EU) 2023/1542 and the Ecodesign for Sustainable Products Regulation (EU) 2024/1781 introduce digital product passports and item-level identification [2][3]. Vendors state that deep-learning OCR targets exactly the hard-to-read serial numbers on such products [4]. No sourced, vendor-neutral read-rate figures exist; always measure on your own parts.

Variants

• Part and serial-number traceability (manufacturing, automotive): engraved, laser-marked or dot-peened text on metal and plastic; tyre DOT codes; VIN plates; PCB markings.
• Label verification (consumer goods, pharma, food): check lot number, best-before date and print quality on packs at line speed; compare to artwork rules → see DOC-027.
• Parcel and pallet tracking (logistics): tunnel or overhead cameras read shipping labels, addresses (including handwritten) and SSCC/GS1 text when the barcode is damaged; link to scan events.
• Container, wagon and trailer IDs: gate cameras read ISO 6346 container codes and wagon numbers.
• Meter and gauge reading (utilities, facilities): digits from analogue/digital displays.
• Private use: phone apps scanning serials or warranty labels; not an industrial focus.

How it works

1. trigger (PLC / conveyor sensor)
2. image acquisition (controlled light, correct focal length, line-scan for fast belts)
3. region detection (find label / marking zone)
4. pre-processing (de-skew, unwarp curved surface, contrast normalisation)
5. text detection + recognition (deep-learning OCR, optional VLM for messy/handwritten text)
6. post-processing (regex/format mask, check digits, GS1 syntax, dictionary of valid values)
7. verification against master data (expected part, batch, shipment)
8. result to PLC (accept / reject / divert) + MES / WMS / TMS event + image archive
9. exception queue (low confidence
10. operator screen with image)

• Models: a text detector (e.g. DBNet-style) plus a sequence recogniser (CRNN/SVTR-style), or an edge-optimised deep-learning OCR tool on a smart camera. VLMs are used offline or for the exception queue (handwriting, damaged labels, multilingual addresses), not in the sub-100 ms loop.
• Constraint-based decoding is the main accuracy lever: if the serial must match [A-Z]{2}\d{8} or a known batch list, reject all other readings.
• Human-in-the-loop: unreadable or low-confidence items go to manual review and the corrected value becomes new training data; reject decisions on regulated products stay auditable.

Data preparation

The data is mostly images of labels, nameplates, serial numbers and parcel stickers, plus the master data they must be matched against (part numbers, order and shipment data in ERP or WMS). Images are often not stored at all; they only exist live at the line or in a driver's phone. Typical gaps are poor lighting, reflections on metal or foil, damaged or curved labels, mixed fonts and languages, and no reference list for validating results (check digits, format rules per supplier).

Preparation steps: capture 200-500 real sample images per label type under production conditions, fix lighting and camera position first (this gains more than any model tuning), write down the format rules for each field (regex, check digit, allowed values), export the master data for matching (see PREP-005), and build a small test set with verified ground truth (PREP-018). For paper archives with historic labels see PREP-006. Tools: a labelling tool, a spreadsheet for format rules, and the vendor's test mode. Data work is typically 10-20 percent of effort; the larger share is camera mounting and integration.

Implementation paths

Which path for whom

• SMB: Path A, because mature OCR and code-reading products (smart cameras, scanner apps) exist at SMB prices and label reading is a standard process.
• Enterprise: Path B, because the results must feed MES, WMS and ERP and volume makes API-based or custom pipelines more economical than seat licences.
• Developer: Path C, because open OCR engines (such as PaddleOCR or Tesseract) run on modest hardware and can be embedded in your own product.

Path A — Buy (recommended for most SMBs)

• Machine-vision smart cameras with deep-learning OCR: Cognex (In-Sight series with deep-learning OCR and serial-number reading) [4], Zebra (Aurora Vision / AI OCR on fixed industrial scanners), Keyence (IV and CV-X series), SICK (EU, Germany), MVTec HALCON (EU, Germany; deep OCR in the library), Basler and Balluff (EU), Omron, Datalogic (EU, Italy), Teledyne DALSA.
• Logistics: tunnel scanners and label-reading systems from Cognex, Zebra, Datalogic, SICK, Fizyr (EU, NL) for parcel handling; carrier-side systems at hubs are usually vendor-integrated.
• Choose A when: a fixed station with controlled lighting, standard PLC/MES integration and a small number of mark types. Buy a station-level pilot with your own samples before ordering.

Path B — Build on cloud APIs

• Document/image OCR APIs: Azure AI Document Intelligence Read, Google Cloud Vision OCR / Document AI, AWS Textract and Rekognition text detection (EU regions), Mistral OCR (EU provider).
• Multimodal models (Claude, Gemini, GPT, Mistral Pixtral/Medium class) for exception handling and handwritten or multilingual parcel labels, with structured JSON output and format validation.
• Fit: goods-in photos, mobile capture, back-office label reading; not for hard real-time line control (latency, connectivity). Store images in EU regions and strip people from frames.

Path C — Self-host (open source)

• Libraries: PaddleOCR (PP-OCR, PP-OCRv5) [5], docTR from Mindee (EU, France; PyTorch/TensorFlow, detection plus recognition) [6], EasyOCR, Tesseract (weak on industrial marks), MMOCR; open-vocabulary detectors or YOLO-class models to locate label regions.
• Serving: export to ONNX / OpenVINO / TensorRT; run on industrial PC or Jetson-class edge device (HW1). Small VLMs (Qwen-VL family) on one GPU (HW2) for the exception queue.
• Operational need: image-capture hardware (camera, lens, light, trigger) matters more than the model; budget for MLOps to retrain when packaging or fonts change.

Path D — Custom training

Worth it for unusual marks (cast-in numerals, dot-peen on rough metal, thermal-transfer smears) or when a generic model plateaus. Typical need: a few hundred to a few thousand labelled crops per mark type, plus synthetic data (rendered fonts with noise and blur) to cover rare characters. Fine-tune a recogniser only; keep the detector generic. Keep a frozen test set from a different shift or batch.

State of the art (2026)

• Deep-learning OCR on smart cameras is mainstream: pretrained character models need little or no font training and run on-device; vendors position them for serial numbers on difficult surfaces [4].
• Open-source recognisers (PaddleOCR family, docTR) are production-grade for printed text and are the usual base for custom industrial models [5][6].
• VLMs read messy scenes (handwriting, curved labels, multiple languages) but are slower, can hallucinate plausible-looking characters and should never be the sole source for serial numbers on regulated goods; combine with format checks, check digits and master-data lookup.
• Failure modes: glare on shiny metal, low-contrast engraving, motion blur, wrong focus, look-alike characters (0/O, 1/I, 5/S, 8/B), partial occlusion, new label layouts after supplier changes, drift in lighting. Most errors are optical, not model errors: fix lighting first.
• Barcodes first: where a 2D code can be applied, read the code and keep OCR as backup; GS1 standards define the human-readable interpretation alongside the data carrier.

Shared resources & combinations

• Same stack (ocr-docparse + edge-runtime + object-detection): VIS-001 quality inspection and VIS-002 assembly verification share cameras and stations; read the serial number and inspect the part in one shot.
• VIS-011 licence-plate recognition uses the same detect-then-recognise pattern; VIS-019 warehouse inventory counting reads rack and pallet labels; VIS-026 measurement shares calibration.
• DOC-006 logistics documents and DOC-027 packaging artwork proofing handle the paper and artwork side; DOC-001 shows the VLM-extraction pattern for exception cases.
• Downstream: OPT-006 picking optimisation, ROBO-002 AMRs that rely on trustworthy location and item IDs.

Effort, cost & KPIs

• Path A: days to a few weeks per station; cost is camera, lens, light and integration rather than software. Cost drivers: number of stations, line speed, ruggedisation, PLC/MES interface.
• Path B: 1–2 months for a goods-in or back-office pipeline.
• Path C/D: 2–4 months incl. data collection, labelling and edge deployment; ongoing retraining.
• KPIs: read rate (first-pass), misread rate (critical: wrong value accepted), false-reject rate, items sent to manual review, throughput per minute, time to resolve exceptions, traceability completeness (share of items with image + ID record), rework or mis-shipment cost.

People & change

Involve the department lead and the line or warehouse staff who currently read and type labels, and the IT team for integration with ERP, MES or WMS. Management decides scope; the data owners of part and order master data clear the matching rules. Suppliers matter when labels come from them, since label quality and format agreements are cheapest to fix at the source.

The works council question depends: a camera that only reads labels is normally not aimed at people, but if it logs which employee scanned what and when, or captures workers in view, §87(1) no. 6 BetrVG applies because objective suitability for monitoring suffices. Inform the works council in good time under §90(1) no. 3 anyway; it usually costs little and avoids later conflict. Personal data is rarely involved (names on parcel labels, driver data); where it is, involve the DPO, check whether a DPIA is needed and have an AVV with any cloud vendor.

Support AI literacy per AI Act Art. 4 with a short briefing on typical misreads and how to correct them. Pilot on one line or one dock with a human confirming low-confidence reads. Typical concerns are job loss and blame for errors; explain that staff move from typing to exception handling. See GUIDE-008, GUIDE-009 and GUIDE-004.

Risks, regulation & compliance

• EU AI Act: minimal risk for the typical deployment; AI literacy duty (Art. 4) applies. It can become part of a high-risk product only if the OCR is a safety component of a regulated machine or device (Annex I product legislation, e.g. Machinery Regulation); then conformity assessment of that product applies. Not an Art. 5 practice; do not extend it to identifying people.
• GDPR: mostly no personal data. Exceptions: handwritten parcel addresses and recipient names, cameras that capture workers, vehicle plates at gates (VIS-011). Mitigate with narrow field of view, masking, short retention for raw images, DPA with vendors, EU hosting; works-council agreement (Germany) if cameras can capture staff performance data.
• Sector rules: medicines verification (Delegated Regulation 2016/161) [1], DPP-related duties [2][3], GS1 syntax rules; keep audit trail of every accept/reject decision.
• Risks: misreads accepted as valid (counterfeit or mix-ups), silent drift, over-reliance on VLM output. Mitigations: constrained formats, check digits, dual reads (OCR + code), periodic golden-set tests, reject-by-default for low confidence.

Getting started

1. Collect 200–500 real images per mark type under production conditions, including worst cases (dirty, worn, glare) and note required read rate and cost of a misread.
2. Fix optics and lighting first (diffuse or dark-field light, trigger, exposure); test two smart-camera vendors and one open-source model (PaddleOCR or docTR) on the same set.
3. Define format rules and a master-data check so that only valid values can be accepted.
4. Pilot at one station in shadow mode (compare to the current process) for 2–4 weeks, then enable PLC reject logic.
5. Set up an exception review screen and a retraining loop; track misreads, not only read rate.

Sources

1. Commission Delegated Regulation (EU) 2016/161 (safety features on medicinal products) — https://eur-lex.europa.eu/eli/reg_del/2016/161/oj
2. Regulation (EU) 2023/1542 (Batteries Regulation) — https://eur-lex.europa.eu/eli/reg/2023/1542/oj
3. Regulation (EU) 2024/1781 (Ecodesign for Sustainable Products) — https://eur-lex.europa.eu/eli/reg/2024/1781/oj
4. Cognex — Serial number scanning for product traceability (optical character recognition) — https://www.cognex.com/en/applications/optical-character-recognition/serial-number-scanning
5. PaddleOCR — https://github.com/PaddlePaddle/PaddleOCR
6. docTR (Mindee) — https://github.com/mindee/doctr
7. GS1 General Specifications — https://www.gs1.org/standards/barcodes-epcrfid-id-keys/gs1-general-specifications
8. Regulation (EU) 2024/1689 (AI Act) — https://eur-lex.europa.eu/eli/reg/2024/1689/oj