Assembly verification & completeness checks

A single missing clip, wrong screw length or unplugged connector can cause a field failure or recall. Vision systems verify at the station that every component of the bill of materials is present, of the correct variant and correctly placed, and block the workpiece or alert the operator if not. Modern deep-learning tools replace brittle rule-based setups and can be configured with a few dozen images.

Problem & value

• Today: completeness is checked by operators (visual check, checklists), by mechanical or sensor poka-yoke (limit switches, light barriers) or by rule-based machine vision with hand-tuned templates. Humans tire on repetitive tasks; sensors only cover one feature each; rule-based vision breaks with variant changes, lighting drift and glossy or deformable parts.
• AI value: one camera covers many features at once; variant handling through BOM-driven recipes; tolerance for lighting and position variation; image evidence per serial number for traceability and complaint handling; faster changeover for high-mix, low-volume production.
• Typical benefit: fewer escapes to customers, less rework and warranty cost, less manual end-of-line inspection. Vendors publish case studies, but figures are application-specific; measure your own baseline (escape rate, rework minutes) before and after a pilot rather than relying on vendor claims.

Variants

• Station-level (in-process): check after each step (e.g. gasket seated, screw present) so errors are caught before they are covered by later parts.
• End-of-line completeness: final check of the finished product (all connectors, labels, accessories).
• Kitting and packing verification: boxes, trays and pallets contain the right items in the right quantity (logistics, medical-device kits, e-commerce). Counting overlaps with VIS-026.
• Electronics: THT/SMT component presence, polarity, connector seating, cable routing and harness checks (wire-harness boards, clip positions).
• Guided assembly with verification: projector/screen instructions plus camera confirmation of each step; see VIS-039 for field installations. assembly checks.

How it works

1. trigger (PLC/sensor: workpiece in position)
2. image acquisition (fixed camera, controlled lighting, optionally 3D/depth)
3. read serial number/variant
4. fetch BOM + inspection recipe from MES/ERP
5. detection / segmentation of expected components (+ OCR of labels, VIS-018)
6. rules: expected vs. found (presence, class, count, position, orientation, colour/length)
7. pass/fail + evidence image stored with serial number
8. PLC/MES: release, stop line, reject, or route to rework; operator display with overlay
9. review of uncertain cases + feedback loop into labelled data

• Presence/position: object detectors (YOLO-class, RT-DETR) or segmentation models trained on tens to hundreds of images per feature; geometric rules compare the result against the recipe.
• Anomaly-based checks: unsupervised models (PatchCore, Anomalib family) trained on good samples flag "anything unusual" where defects are rare or unknown; they localise but do not name the defect.
• Variant logic: deterministic, from the BOM — the AI finds objects, the rules decide pass/fail. This keeps decisions auditable and avoids letting a generative model "judge".
• VLMs: useful for set-up and long-tail cases (draft labels, ad-hoc questions such as "is the second cable routed through the clip?") but too slow and non-deterministic for cycle times below a second; use them offline for annotation or on the exception queue.
• Human-in-the-loop: low-confidence results go to an operator review screen; override decisions are logged and become training data. Safety-critical joints (e.g. airbag, brake components) stay on validated, process-controlled checks (torque monitoring etc.); vision complements, not replaces.

Data preparation

Needed are station images of correct and incomplete assemblies per product variant, bills of materials and work orders from ERP or MES, assembly instructions or CAD, and PLC cycle signals. They typically live in the ERP, in PDM/CAD folders, on file shares and as paper work instructions. Typical gaps: variants without reference images, outdated BOMs, work instructions only on paper or as scans, no examples of rare errors, and no link between image and serial number.

Preparation steps: (1) clean and release BOM and variant master data (DATA-003); (2) digitise paper instructions (PREP-006, DOC-014); (3) capture golden images for every variant under fixed lighting; (4) stage typical errors (missing part, wrong part, wrong orientation) and label them (DATA-023, PREP-018); (5) store images with serial number, variant and timestamp; (6) route operator corrections back as labels (PREP-021).

Tools: CVAT or Label Studio, simple scripts for the ERP export (PREP-005). Rough share of total effort: 30-50 percent, driven by the number of variants.

Implementation paths

Which path for whom

• SMB — Path C: open-source detection and anomaly models run on HW1 edge hardware and the same platform can be reused for quality inspection (VIS-001), so a Baukasten approach beats per-station licences when product variants change often.
• Enterprise — Path B: variant data and BOMs live in MES/ERP and need tight integration, which cloud-trained models plus an internal edge runtime handle well.
• Developer — Path C: open-source frameworks give full control over models, station integration and latency.

Path A — Buy

• Smart cameras / vision software with deep learning: Cognex (In-Sight, VisionPro Deep Learning), Keyence (CV-X / XG-X with AI modes), Zebra (Aurora Vision), MVTec (HALCON, MERLIC — EU, Germany), SICK (Inspector / AppSpace — EU), Basler (EU, cameras and pylon AI tooling), Balluff, Omron, Teledyne DALSA/Sherlock, Siemens (SIMATIC and Industrial Edge apps).
• Cloud/low-code platforms: Landing AI (LandingLens), also ISRA VISION (Atlas Copco, EU), Neurala (VIA, US), Elementary (US) and system integrators for turnkey stations.
• Choose A when: fixed products, standard lighting, the vendor ecosystem already matches the PLC/camera hardware, no in-house ML team. Insist on on-prem or edge operation of inference.

Path B — Build on cloud APIs

• Train in the cloud, run at the edge: Azure AI Foundry / custom vision-type services, Google Vertex AI, AWS SageMaker with Greengrass edge deployment — use EU regions.
• Cloud VLM APIs (Claude, Gemini, GPT, Mistral) for labelling assistance, defect-description generation and exception review; not for in-cycle decisions or latency-critical paths.
• Integration: OPC UA, MQTT, REST to MES/ERP; GigE Vision / GenICam cameras via a vendor-neutral SDK.
• Choose B when: you have a software team, many sites, and want central model management.

Path C — Self-host (open source)

• Detection/segmentation: Ultralytics YOLO (AGPL-3.0 or commercial licence — check licensing for product use), RT-DETR, RF-DETR (Apache-2.0), SAM 2 / SAM 3 for label assist, open-vocabulary detectors (Grounding DINO, OWL-ViT-class, YOLO-World) for zero-shot starts.
• Anomaly detection: Anomalib (Intel/OpenVINO Toolkit, Apache-2.0) with PatchCore, EfficientAD.
• Annotation and MLOps: CVAT, Label Studio, MLflow; inference via ONNX Runtime, OpenVINO, TensorRT.
• Hardware: HW1 — an industrial PC with a mid-range GPU or a Jetson Orin / Hailo accelerator handles several cameras at 10–30 fps; HW2 for multi-camera high-resolution or 3D processing.
• Ops effort: model versioning, drift monitoring, validated release process — plan a part-time ML engineer.

Path D — Custom training

Worthwhile for high-volume lines, unusual components or when vendor tools miss tolerance. Needs hundreds of annotated images per component class across lighting, shifts and supplier variants, plus images of realistic faults (staged "bad" samples). Synthetic data from CAD renderings (NVIDIA Omniverse/Replicator, Blender) can bootstrap rare faults. Validate with a held-out set of real production images and a documented acceptance test.

State of the art (2026)

• Few-shot and prompt-based set-up: vendors now advertise training from roughly 5–50 images via foundation-model assistance (visual prompting, SAM-based labelling), shrinking set-up time from weeks to days; verify on your own parts.
• Foundation models for labelling: SAM-family and open-vocabulary detectors cut annotation cost; production inference still uses small, distilled detectors for determinism and speed.
• Edge accelerators: inference on smart cameras and industrial PCs (NPU/GPU) at millisecond latency.
• 3D and multi-view: depth cameras and structured light solve height, seating and orientation checks that 2D struggles with.
• Limits and failure modes: reflective or black parts, occlusion by tools or hands, tiny fasteners in clutter, new supplier variants (look-alike parts), lighting drift, camera displacement after maintenance, and over-reliance on "pass" when the feature is simply out of view. Anomaly models drift with legitimate process changes. Always run periodic golden-sample and known-bad tests ("challenge parts") to prove the system still catches errors.

Shared resources & combinations

• Same stack (object-detection + segmentation + edge-runtime): VIS-001 surface defect inspection, VIS-026 counting/measurement, VIS-053/VIS-057 process and NDT inspection, VIS-018 label/serial OCR — one annotation pipeline and one edge platform serve them all.
• Downstream: ROBO-001 (vision-guided picking and placing), VIS-034 (cycle-time analytics from the same cameras, with separate legal review), predictive-quality models using the stored inspection results, and MES/QMS integration for traceability.
• Hardware: industrial cameras, lighting and PLC interface are shared across stations; GPU or NPU edge boxes can serve several lines.

Effort, cost & KPIs

• Path A: weeks to a few months per station; cost drivers are camera, optics and lighting (often more than software), integration with PLC/MES, and validation.
• Path B/C: 2–6 months for a first production station incl. data collection, annotation, acceptance testing; later stations reuse pipelines.
• KPIs: escape rate (defects reaching customers), false-reject rate (pseudo-scrap), detection rate on challenge parts, inspection cycle time, rework minutes per unit, first-pass yield, customer complaints per million, time to add a new variant.

People & change

Involve management for budget and quality targets, the production lead as process owner, IT and IT security for camera, PLC and network integration, and data owners for BOM and variant data. The works council belongs in from the start: station cameras and cycle logs can capture workers' hands and pace.

Works council: yes, in practice. Objective suitability to monitor behaviour or performance is enough for §87(1) no. 6 BetrVG, and station-level logs can be linked to shifts and people. Agree on a works agreement that rules out individual performance evaluation, defines retention and masks or avoids faces. The works council must also be informed under §90(1) no. 3, and may call in an AI expert (§80(3)). Public bodies: staff council. The DPO assesses whether a DPIA is needed when people are visible.

AI literacy: the AI Act (Art. 4) requires measures that support staff's AI literacy; train station workers on what the system checks and what it cannot see. Communicate clearly that it checks the product, not the person. Pilot with one station and volunteers, let workers flag false alarms, keep a human release for rework decisions, and keep a manual fallback. See GUIDE-004, GUIDE-008 and GUIDE-009.

Risks, regulation & compliance

• EU AI Act: typically minimal risk for quality inspection of products. It becomes high-risk only if the system is a safety component of a regulated product (Annex I, e.g. machinery or medical devices) or is used for worker management (Annex III); the AI-based inspection of a product on a line is normally neither. AI literacy (Art. 4) applies. Art. 5: do not use the cameras for emotion recognition of workers.
• GDPR / works council: station cameras may capture operators' hands, faces or behaviour. Frame the field of view on the workpiece, blur or mask people, document purpose limitation and involve the works council (Germany: BetrVG § 87(1) Nr. 6). Do not repurpose inspection footage for performance monitoring.
• Machinery and product safety: integration into PLC safety chains must follow the Machinery Regulation (EU) 2023/1230 and functional-safety standards; vision-based release signals are generally not safety functions unless validated as such.
• Quality systems: ISO 9001, IATF 16949, ISO 13485 or GAMP-style validation for regulated sectors — model changes need change control and revalidation.
• Mitigations: deterministic rules around the model, challenge-part tests, human override logging, fail-safe default (no image / low confidence = reject or hold), image retention aligned with product liability periods.

Getting started

1. Pick one station with measurable escapes; record the baseline (escape rate, rework time).
2. Fix the physics first: camera position, lighting, fixturing — most failures are optical.
3. Collect 200–500 real images including variants, plus staged faults; label with CVAT or vendor tools.
4. Pilot a Path A tool and an open baseline (YOLO/RF-DETR) side by side; compare on a held-out set.
5. Integrate with the PLC/MES in shadow mode (no stopping) for 2–4 weeks, then go live with challenge-part tests and a weekly review of false rejects.

Sources

1. Regulation (EU) 2024/1689 (AI Act) — https://eur-lex.europa.eu/eli/reg/2024/1689/oj
2. Regulation (EU) 2023/1230 (Machinery Regulation) — https://eur-lex.europa.eu/eli/reg/2023/1230/oj
3. Anomalib (Open Edge Platform / Intel) — https://github.com/open-edge-platform/anomalib
4. Ultralytics YOLO documentation — https://docs.ultralytics.com/
5. RF-DETR (Roboflow) — https://github.com/roboflow/rf-detr
6. Segment Anything (Meta) — https://github.com/facebookresearch/sam2
7. MVTec HALCON / MERLIC — https://www.mvtec.com/
8. Cognex VisionPro Deep Learning — https://www.cognex.com/products/deep-learning
9. Landing AI LandingLens — https://landing.ai/
10. CVAT annotation tool — https://github.com/cvat-ai/cvat