Driver & in-cabin monitoring (drowsiness, distraction, fleet safety dashcams)

Fatigue and inattention remain leading causes of serious crashes, especially in long-haul freight and bus transport. Driver-monitoring systems (DMS) use a near-infrared camera and compact vision models on an embedded chip to track eyelid closure, gaze and head pose in real time and warn the driver. In the EU this is now a regulatory baseline for new vehicles; in fleets, AI dashcams add event capture and coaching. The design challenge is as much about GDPR, works councils and the AI Act's emotion-recognition limits as about accuracy.

Problem & value

• Today: drivers rely on self-assessment and tachograph rest rules; managers learn about near-misses only after incidents or complaints. Legacy telematics sees speed and braking, not the driver's attention.
• AI value: immediate in-cab warnings (seconds matter), objective evidence for claims and liability, targeted coaching instead of blanket scolding, and measurable fleet risk reduction (fewer collisions, lower insurance premiums).
• Regulatory pull: Regulation (EU) 2019/2144 (General Safety Regulation) mandates driver drowsiness and attention warning (DDAW) and advanced driver distraction warning (ADDW). ADDW applied to new vehicle types from July 2024 and applies to all newly registered vehicles from 7 July 2026; technical requirements are in Delegated Regulations (EU) 2021/1341 and 2023/2590 [1][2][3]. Euro NCAP rewards robust driver monitoring in its ratings [5].

Variants

• OEM type-approval DMS (DDAW/ADDW): embedded in the vehicle; warning-only, no data leaves the car; tested against the delegated regulations and Euro NCAP protocols.
• Retrofit truck/bus/van fleets: aftermarket in-cab video telematics with cloud portal, event upload and coaching workflow; often combined with road-facing camera (tailgating, lane departure, near-miss).
• Child-presence and occupant monitoring: cabin sensing (camera or radar) to detect a child left in the vehicle and to check seat occupancy/belt use; Euro NCAP-driven feature.

How it works

1. NIR in-cab camera (+ road camera, CAN/GPS/IMU)
2. on-device face/eye/head-pose & hand/object detection (embedded NPU/SoC)
3. state estimation (eyelid closure/PERCLOS, gaze zone, head turn, phone, seatbelt, occupancy)
4. rule/threshold logic fused with vehicle context (speed, indicator, manoeuvre)
5. in-cab alert (audio/haptic) | event clip + metadata (fleet variant)
6. cloud review (optional VLM triage, human safety analyst)
7. coaching workflow, dashboard, claims evidence

• Perception: lightweight CNN/transformer models for landmarks, gaze and pose run at the edge; latency must be well under a second and work with sunglasses, at night and under vibration.
• Fusion: a glance away is tolerated while overtaking or checking mirrors; the same glance at motorway speed is not. Vehicle context cuts false alerts that drivers otherwise learn to ignore.
• Cloud triage: events are scored and often reviewed by human analysts; vision-language models can pre-label clips ("phone in hand", "tailgating") to reduce review volume.
• Human-in-the-loop: coaches decide on conversations; no automatic disciplinary action from model output. Drivers can dispute events.

Data preparation

The data are in-cabin video or sensor signals (eyelid closure, gaze, head pose), vehicle data (CAN, GPS, speed) and incident records. These typically live on the dashcam device, the vendor's portal and the telematics system, while context such as shift plans, driver master data and accident reports sits in the TMS, payroll or HR system and on paper. Typical gaps: no ground truth for what counts as drowsy, wrong camera positions, drivers with glasses or masks, missing consent and retention rules, and no link between alerts and actual incidents.

Preparation steps: first decide the minimum data needed; process on the device and store only events, not continuous video (PREP-022). Set retention periods and access rights per role (PREP-023, PREP-024). Check the legal basis for processing employee data (PREP-025). Build a test set of annotated events from pilot vehicles with driver cooperation (PREP-018) and harvest incident labels from accident reports (PREP-020). Link vehicle and driver master data via exports (PREP-005). Calibrate cameras per vehicle type.

Rough share of effort: 40 percent or more, because legal basis, retention design, calibration and ground-truth collection dominate; the detection models themselves are comparatively mature.

Implementation paths

Which path for whom

• smb (A): Certified fleet dashcam systems with in-cabin sensors are mature and ready to install; the legal and employee-data side matters more than the technology.
• enterprise (B): Large fleets need integration with telematics, TMS and HR systems, and employee data is highly sensitive, so a configurable cloud service with strict data minimisation fits.
• developer (C): Open-source face-landmark and gaze models run on HW1 edge devices, keeping video in the vehicle and giving control over what leaves the cab.

Path A — Buy (recommended for most fleets)

• Fleet video telematics: Samsara, Lytx, Motive, Netradyne, Nauto, Stoneridge/MirrorEye ecosystem, Mobileye-based aftermarket units, and EU-based providers such as Trimble/Transics, Webfleet (Bridgestone), Brigade and Streamax-based devices. Check EU data residency, retention controls, on-device-only mode and works-council templates.
• OEM-installed systems: for new trucks and buses the factory DMS (Smart Eye, Seeing Machines, Valeo, Continental, Bosch supply chains) is already present; use it rather than adding a second camera.
• Choose A when: a fleet of roughly 10 to several thousand vehicles needs deployment in weeks without an ML team.

Path B — Build on cloud APIs

• Rarely a good fit for the real-time loop (latency, connectivity, cost). Cloud is suitable for the back-end: event ingestion (AWS Kinesis Video Streams, Azure IoT/Event Hubs in EU regions), clip triage with multimodal APIs (Gemini, Claude, Mistral vision models) and coaching summaries.
• Upload event clips only, never continuous cabin video (see VIS-030 for blurring).

Path C — Self-host (open source)

• Models: MediaPipe Face Mesh / Iris, OpenFace, YOLO-family detectors for phone/cigarette/seatbelt, open datasets for training such as DMD (Driver Monitoring Dataset) and DrivFace-style sets; Ultralytics YOLO or RT-DETR exported to ONNX/TensorRT.
• Runtime: NVIDIA Jetson Orin (Nano/NX), Qualcomm, Hailo-8 or Ambarella SoCs; OpenVINO on Intel.
• Hardware: HW1 edge device per vehicle; back-end on CPU server or HW2 GPU for clip review.
• Ops effort: high; you own safety-relevant calibration, camera hardware, OTA updates and validation across lighting, ethnicities, glasses and head coverings.

Path D — Custom training

• Needed for OEM/Tier-1 development or for unusual cabins (buses, cranes, trains). Requires large, diverse, consented in-cabin data across skin tones, ages, eyewear and illumination, plus ground truth (PERCLOS, Karolinska Sleepiness Scale labels, gaze targets). Synthetic data is a common supplement. Euro NCAP and Delegated Regulation test protocols define the validation targets [3][5].

State of the art (2026)

• Regulation-driven maturity: after July 2026 every new EU vehicle carries DDAW and ADDW, so DMS is commoditising at the OEM level while differentiation moves to fleet analytics and coaching.
• Multi-sensor cabin sensing: camera plus 60 GHz radar for child-presence and vital-sign-adjacent detection; combined occupant and driver monitoring on one SoC.
• Fusion with road-facing AI: dashcams correlate driver state with tailgating, lane departure and harsh events, enabling "distraction preceded the near-miss" evidence.
• Limits: sunglasses, extreme head pose, camera occlusion, glare, driver diversity bias, false-alarm fatigue, and spoofing. Real drowsiness is a physiological state; eyelid signals are proxies. Inferring "emotion" or "stress" from the face is scientifically weak and legally problematic (see below).

Shared resources & combinations

• Same stack (pose-estimation, object-detection, tracking, edge-runtime): VIS-024 pose and gesture analysis, VIS-007 PPE compliance, VIS-010 surveillance; one edge SoC family and one labelling pipeline.
• Road side: ROBO-005 (ADAS perception), VIS-012 traffic monitoring, VIS-054 enforcement cameras (phone use at the wheel; different legal setting).
• Downstream: PRED-030 usage-based insurance scoring, VIS-031 natural-language video search over event libraries, VIS-030 anonymisation of clips before wider sharing.

Effort, cost & KPIs

• Path A: weeks for rollout; cost is per-vehicle hardware plus a monthly subscription; installation and driver onboarding dominate effort.
• Path B: 2-4 months for back-end; ongoing storage and egress cost for clips.
• Path C/D: 6-18 months, specialised team; OEM-grade validation can take years.
• KPIs: alert precision/recall on labelled clips, false alerts per 100 hours, collisions and near- misses per million km, share of drivers with positive trend after coaching, claim cost and cycle time, driver acceptance score, number of data-subject requests handled in time.

People & change

Management and the fleet or department lead set the purpose, which should be safety, not performance control. The works council must be involved from the start, before any camera is bought. Works council: yes. In-cabin monitoring is a technical system suited to monitor employees' behaviour (§87(1) no. 6 BetrVG; objective suitability suffices), and §90(1) no. 3 requires timely information about AI-planned work processes. Conclude a works agreement that fixes purposes, bans use for performance ratings or dismissal, sets retention and defines who sees what. §80(3) allows the works council to engage an AI expert. Public-sector fleets involve the staff council (Personalrat). Where no works council exists, drivers should still be consulted and must be informed transparently.

The DPO is mandatory in practice: a DPIA (Art. 35 GDPR) is very likely because of systematic monitoring of employees, plus an AVV with the vendor and a documented legal basis. IT security reviews device and cloud access; legal and HR check employment-law limits and the AI Act position on emotion recognition in workplaces. AI literacy (Art. 4): train drivers and dispatchers on what is recorded, what alerts mean and how to contest them. Pilot with volunteer drivers and a few vehicles, report results openly and let drivers see their own data. Typical concerns (surveillance, dismissal, privacy at the wheel) are addressed by the works agreement, local processing and human review before any consequence. See GUIDE-004, GUIDE-005 and GUIDE-008.

Risks, regulation & compliance

• EU AI Act: classification depends on use. Art. 5(1)(f) prohibits AI systems that infer emotions of employees in the workplace, except for medical or safety reasons. Fatigue and attention detection is generally regarded as a safety use and not as emotion recognition, but any "mood", "stress" or "anger" scoring of drivers is at serious risk of being prohibited; keep outputs to physical states (eyes closed, gaze off road, phone in hand). Vehicle safety components approved under the type-approval regime fall under the sectoral rules in Annex I; fleet monitoring that feeds employee evaluation, task allocation or disciplinary decisions can fall under Annex III (employment) as high-risk; biometric identification of drivers (face login) brings the biometrics rules [4]. Obtain a legal assessment.
• GDPR: in-cabin video of employees is personal data; biometric templates are special category (Art. 9). Need a lawful basis (usually legitimate interest with balancing test; consent is weak in employment), a DPIA (Art. 35), transparency, data minimisation, short retention, and access limits. The type-approval DMS is designed not to store or identify; keep that principle in fleets [1][6].
• Labour law: in Germany, works-council co-determination applies (BetrVG Sec. 87(1) no. 6) and a company agreement is customary; comparable rules exist in other member states. Never covert recording; consider a privacy-by-design mode (event-triggered recording only, no continuous upload, audio off).
• Mitigations: on-device processing, event-only upload, driver-visible indicator light, appeal process, bias testing across demographics, staged rollout with driver involvement.

Getting started

1. Check what the vehicles already contain: OEM DDAW/ADDW and their data export options.
2. Define the safety goal and a minimal data set; run the DPIA and sign a works-council agreement first.
3. Pilot 10-20 vehicles with two vendors for 3 months, measuring false alerts and driver acceptance.
4. Set coaching rules (positive feedback first, no automatic sanctions) and a retention schedule.
5. Review quarterly: KPI trend, bias checks, event disputes, and regulatory changes.

Sources

1. Regulation (EU) 2019/2144 (General Safety Regulation) — https://eur-lex.europa.eu/eli/reg/2019/2144/oj
2. Delegated Regulation (EU) 2021/1341 (DDAW/ADDW technical rules) — https://eur-lex.europa.eu/eli/reg_del/2021/1341/oj
3. Delegated Regulation (EU) 2023/2590 (ADDW amendments) — https://eur-lex.europa.eu/eli/reg_del/2023/2590/oj
4. Regulation (EU) 2024/1689 (AI Act), Art. 5 and Annex III — https://eur-lex.europa.eu/eli/reg/2024/1689/oj
5. Euro NCAP — Assessment protocols and driver monitoring — https://www.euroncap.com/en/for-engineers/protocols/
6. EDPB Guidelines 01/2020 on connected vehicles — https://www.edpb.europa.eu/our-work-tools/our-documents/guidelines/guidelines-012020-processing-personal-data-context_en
7. Smart Eye press release on ADDW becoming mandatory — https://www.inderes.fi/en/releases/advanced-driver-distraction-warning-systems-now-mandatory-across-all-new-eu-vehicles
8. Driver Monitoring Dataset (DMD) — https://dmd.vicomtech.org/
9. MediaPipe Face Landmarker — https://ai.google.dev/edge/mediapipe/solutions/vision/face_landmarker