AI-based safety functions for machines & robots (person detection, speed & separation monitoring)

Fences and light curtains keep people away from dangerous machines but cost floor space and block flexible work. Safety-rated sensors with AI-based perception detect and classify people around robots, AGVs, forklifts and mobile machinery and trigger a slow-down or stop, so humans and machines can share space. This is a safety function that acts on the machine, not an analytics dashboard, and it is regulated as such: the EU Machinery Regulation, functional-safety standards and the AI Act apply together.

Problem & value

• Today: protection is by physical guards, interlocked gates, light curtains and safety laser scanners with geometric protective fields. These are reliable but cannot tell a person from a pallet, so fields are conservative (frequent false stops) or the area is fenced off entirely.
• AI value: distinguish humans from objects, track their position and velocity, and adapt the protective distance dynamically (speed and separation monitoring, SSM) instead of stopping at a fixed boundary. Result: fewer nuisance stops, smaller footprints, fenceless cells, safer mixed traffic of people and forklifts/AMRs.
• Why now: the Machinery Regulation (EU) 2023/1230 applies from 20 January 2027 and explicitly addresses machinery with self-evolving behaviour/logic and safety components using machine learning, with such safety components subject to stricter conformity assessment (Annex I) [1].
• Benefit figures: vendors report productivity and floor-space gains in case studies; there is no neutral industry-wide figure, so measure your own baseline (stops per shift, cell footprint, cycle time).

Variants

• Collaborative robot cells: SSM and power-and-force limiting per ISO 10218 and ISO/TS 15066 (now integrated into the revised ISO 10218-2) with 3D safety cameras or LiDAR [2].
• Mobile machinery and forklifts: radar, camera or LiDAR person detection around trucks, loaders and excavators; slow-down or stop; relevant to ROBO-002, ROBO-014, ROBO-009.
• AGV/AMR safety fields: safety laser scanners with switchable fields; AI classification as a non-safety layer that reduces speed earlier.
• Presses, packaging and processing lines: replacing fixed guards with sensor-based area protection.
• Not in scope: analytics of near-misses or PPE compliance (see VIS-007).

How it works

1. safety-rated sensor (3D LiDAR / radar / stereo / ToF)
2. point cloud or image on a safety controller or edge module
3. detection + classification (person vs. object) + tracking (position, velocity)
4. distance and stopping-time computation (protective separation distance, ISO 13855 / ISO 13849)
5. decision: continue | reduce speed | protective stop | restart permission
6. safety output (OSSD or safe fieldbus: FSoE, PROFIsafe, CIP Safety) to robot / drive / PLC
7. diagnostics, logging, periodic proof test

• Two-layer architecture is the norm: a certified, deterministic safety channel (geometry, distances, diverse redundant channels, PL d / SIL 2 typical targets) and, where AI is used, either an ML classifier that is itself validated as part of the safety function or a non-safety AI layer that only reduces risk (earlier slowdown) while the certified layer still guarantees the stop.
• AI components: 3D person detection on point clouds, pose and motion prediction, sensor fusion of camera and radar, scene segmentation for dynamic zone adaptation.
• Human-in-the-loop: restart after a protective stop is a defined, often manual, acknowledgement; safety configuration, validation and any change to the AI model go through documented sign-off by a qualified safety engineer.

Data preparation

Safety AI needs representative data of the real hazard situation: sensor and camera recordings of people, vehicles and obstacles in the actual work area, risk assessments, safety requirement specifications, incident and near-miss reports, and test and validation evidence. In an SMB these live in machine documentation, risk-assessment files on file shares or paper, the safety officer's folders, incident logs (often paper or Excel) and plain video archives.

Typical gaps: incident and near-miss data that is not systematically recorded, no labelled examples of dangerous situations, risk assessments that are outdated or not linked to the machine's configuration, scanned documents without structure, and no documented operating domain (lighting, clothing, speeds, layouts).

Preparation steps: digitise and structure risk assessments and incident logs (PREP-006, PREP-008, PREP-010), define hazard and event categories with the safety officer (PREP-019), record and label scenario data across conditions and build a fixed test set that also covers rare cases (PREP-018), and document the operating domain. Pseudonymise or restrict recordings of people (PREP-022).

Rough share of effort: high, about 40 percent, because data coverage and traceable validation are central to the safety case. See GUIDE-003.

Implementation paths

Which path for whom

• SMB: Path A, because safety functions require certified components and a conformity assessment, which only certified products (safety scanners, safety controllers, vendor safety packages) provide at SMB scale.
• Enterprise: Path A, because certified products with a safety case are the practical route; own AI safety layers remain additional, non-safety-rated functions.
• Developer: Path B, because developers building machine or robot products integrate certified safety components and cloud tooling for training and validation, while the safety-rated function itself stays certified.

Path A — Buy (recommended)

• Certified safety sensors and systems: SICK (EU; safeVisionary2, microScan3, nanoScan3, safety software such as Safe Robotics Area Protection), Pilz (EU; SafetyEYE camera system), Leuze (EU), Omron, Keyence, Pepperl+Fuchs (EU), Blickfeld (EU) and other LiDAR vendors for mobile use.
• Safety PLC / controller side: Siemens (EU) Safety Integrated, Beckhoff (EU) TwinSAFE, B&R/ABB, Rockwell GuardLogix; robot makers (KUKA, ABB, Fanuc, Universal Robots, Franka/Agile Robots) ship certified safety functions such as SSM, speed limits and safe zones.
• Choose A when: you need a CE-marked, certified function. Essentially every SMB should buy a certified sensor and let the vendor carry the AI-in-safety certification burden.

Path B — Build on cloud APIs

• Cloud APIs are unsuitable for the safety channel (latency, availability, no determinism).
• Cloud is useful for non-safety work: fleet monitoring, model training/retraining pipelines, simulation and test-scenario generation (ROBO-010), and analysing stop events to tune layouts.

Path C — Self-host (open source)

• Perception for the non-safety layer or research: OpenPCDet / MMDetection3D (3D detection), YOLO-class detectors (Ultralytics, RT-DETR), Open3D, ROS 2 with Nav2 and MoveIt, OpenVINO / TensorRT / ONNX Runtime on edge devices. NVIDIA Isaac platform includes tools for perception and ROS integration, and NVIDIA's Jetson Thor / IGX Orin target safety-oriented edge computing [3].
• Caution: open-source stacks do not carry a safety certificate. Using them as the sole safety channel would require you to become the manufacturer of a safety component and follow the full conformity route.
• Hardware: HW1 (Jetson Orin-class or industrial edge PC) for inference; deterministic RTOS and certified safety controller for the final stop.

Path D — Custom training

• Needed for unusual environments (high-vis clothing, wheelchairs, carts, crowded aisles, dust/steam) or custom sensors. Requires representative, diverse labelled data including edge cases and adversarial conditions (occlusion, reflective surfaces, unusual postures). Validated statistically with documented residual-risk argument; see ISO/IEC TR 5469 and ISO/PAS 8800 (road vehicles) as guidance for AI in safety-related functions [4].

State of the art (2026)

• Standards are catching up: the revised ISO 10218-1/-2 (2025) consolidate collaborative operation requirements; IEC/ISO work (ISO/IEC TR 5469, IEC 61508 maintenance, ISO 13849 revision) addresses AI in functional safety. Harmonised standards under the new Machinery Regulation are still being developed, so manufacturers rely on state-of-the-art arguments and notified bodies [1][4].
• Certified 3D safety sensors with person-classification exist for mobile and stationary use; most certified products still keep ML out of the final safety decision or restrict it to bounded, validated classifiers.
• Open question: how to evidence the failure rate of a neural network to the level of a PL d / SIL 2 function. Typical approach: diverse redundancy, conservative fallback to geometric fields, monitoring of input distribution, and runtime plausibility checks.
• Limits and failure modes: occlusion, unusual postures, reflective or transparent surfaces, dust, sensor contamination, model drift after layout or lighting changes, and missed detection of children/visitors not in training data. Never rely on AI alone for stop decisions.

Shared resources & combinations

• Same stack: object-detection + 3d-vision + tracking on edge hardware, shared with ROBO-002 (AMRs), ROBO-014 (unloading), ROBO-015 (inspection robots), ROBO-001 (bin picking).
• Simulation and test: ROBO-010 for scenario-based validation (digital twins, synthetic edge cases).
• Adjacent: VIS-007 consumes the same camera streams for non-safety analytics (separate system, separate data path); ROBO-004 humanoids and ROBO-003 programming-by-demonstration both depend on a certified safety layer; CODE-020 for safety-critical embedded software; ROBO-011 for TinyML sensing.

Effort, cost & KPIs

• Path A: weeks to months for risk assessment, sensor selection, integration and validation; sensor and safety-controller cost dominates, plus notified-body or in-house validation effort.
• Path C/D: many months to years if building a certified component; mostly validation and documentation effort, not model training.
• KPIs: nuisance-stop rate, availability of the cell, cycle time, floor-space saved, detection performance (miss rate in validation scenarios, measured, not assumed), response time vs. required stopping time, near-miss and incident rate, proof-test results.

People & change

Involve management (liability, release), the machine or plant lead, the safety officer and the workers whose area is protected, IT security, legal/compliance (Machinery Regulation, AI Act, product liability), suppliers of safety components and the works council from the start (GUIDE-001). Notified bodies or inspection organisations come in early, not at the end.

Works council: depends. Safety sensors detect persons, but are meant to protect, not to monitor. Still, systems that log presence, positions or behaviour are objectively suited to monitor employees and trigger co-determination (§87(1) no. 6 BetrVG); occupational safety matters can add further co-determination rights. Inform the works council in good time about AI-planned processes (§90(1) no. 3), and an AI expert may be engaged (§80(3)). Agree that safety data is not used for performance evaluation (GUIDE-004). Public operators involve the Personalrat.

Data protection: recordings of persons are personal data; involve the DPO, minimise storage and consider a DPIA (GUIDE-005). High-risk classification may apply for safety components (GUIDE-014).

AI literacy: support staff with training on what the system detects, its limits and what to do on a stop or fault (GUIDE-006). Typical concern is false trust; stress that AI is an added layer on certified safety functions. Run a supervised pilot in one area (GUIDE-009), communicate openly (GUIDE-008) and keep humans in control: protective stops cannot be overridden by AI, and changes need formal release.

Risks, regulation & compliance

• EU AI Act: AI used as a safety component of a product covered by Annex I Union harmonisation legislation (including the Machinery Regulation) that requires third-party conformity assessment is high-risk under Art. 6(1); obligations apply from 2 August 2028 for Annex I products (postponed from August 2027 by the Digital Omnibus on AI, Reg. (EU) 2026/1744) [5]. Not an Art. 5 prohibited practice. Do not combine with emotion recognition or worker performance scoring: workplace emotion inference is prohibited (Art. 5(1)(f)).
• Machinery Regulation (EU) 2023/1230: applies 20 Jan 2027; essential health and safety requirements (Annex III) cover control systems and evolving behaviour; conformity assessment by notified body for listed safety components; cybersecurity of safety functions required [1].
• Standards: ISO 12100 (risk assessment), ISO 13849-1 and IEC 62061 (functional safety), ISO 13855 (positioning of protective equipment), IEC 61496 (ESPE), ISO 10218-1/-2 and ISO/TS 15066 (robots), ISO 3691-4 (driverless trucks), ISO 13482 (personal care robots) [2].
• GDPR / labour law: safety sensors should not store identifiable video; use point clouds or on-device processing without recording; if cameras are used, works-council agreement (BetrVG §87 in Germany), purpose limitation and data minimisation apply.
• Typical risks and mitigations: systematic false negatives (diverse redundancy, geometric fallback), change of layout without revalidation (change management), tampering or bypass (access control, muting audits), cyber attacks (IEC 62443).

Getting started

1. Run a formal risk assessment (ISO 12100) of the existing cell and define which hazards need a safety function, with required performance level.
2. Check whether a certified sensor or robot-native safety function already covers your use case (Path A) before considering any AI-specific component.
3. Define protective separation distance, stopping times and restart logic; validate with test persons and defined scenarios.
4. Keep AI-only layers (efficiency, earlier slowdown) separate from the certified stop channel.
5. Document everything in the technical file, plan change management, and talk to a notified body early.

Sources

1. Regulation (EU) 2023/1230 on machinery — https://eur-lex.europa.eu/eli/reg/2023/1230/oj
2. ISO 10218-1:2025 Robotics, safety requirements for industrial robots — https://www.iso.org/standard/73933.html
3. NVIDIA Isaac robotics platform — https://developer.nvidia.com/isaac
4. ISO/IEC TR 5469:2024 Artificial intelligence, functional safety and AI systems — https://www.iso.org/standard/81283.html
5. Regulation (EU) 2024/1689 (AI Act) — https://eur-lex.europa.eu/eli/reg/2024/1689/oj
6. EU-OSHA — Digitalisation and OSH: robotics and AI at work — https://osha.europa.eu/en/themes/digitalisation-work
7. SICK AG — Safe robotics area protection — https://www.sick.com/de/en/
8. Pilz — SafetyEYE safe camera system — https://www.pilz.com/en-INT/products/sensors/camera-systems/safetyeye