Traffic monitoring & analytics

Cities, road operators and logistics sites need to know how many vehicles, bikes and pedestrians use which route, how fast, and where incidents or conflicts occur. Manual counts and inductive loops are sparse and expensive. Computer vision turns existing or purpose-built cameras into continuous, classified traffic sensors, running mostly on edge devices and exporting anonymous aggregates. The design question in Europe is less accuracy than privacy and legal scope.

Problem & value

• Today: traffic is measured with induction loops, pneumatic tubes, radar and short manual counting campaigns. Coverage is sparse, cyclists and pedestrians are poorly captured, and data arrives weeks late. Incident detection in tunnels and on motorways often relies on operators watching walls of monitors.
• AI value: continuous classified counts (including active modes), speed and queue estimates, automatic incident detection, adaptive signal input, and before/after evidence for measures such as new cycle lanes. Surrogate safety analysis (near-miss detection from trajectories) allows risk assessment without waiting for crashes.
• Policy drivers: EU road-safety and ITS policy (Directive 2010/40/EU and its 2023 revision) and national cycling and Vision Zero programmes need granular multimodal data [1].
• No invented benefit figures are given here; measure your own baseline (counts vs. manual reference).

Variants

• Urban counting and mode split: cyclists, e-scooters, pedestrians at junctions for planning.
• Motorway and tunnel incident detection: stopped vehicle, wrong-way driving, fire/smoke, debris.
• Junction and near-miss analytics: conflict detection between vehicles and vulnerable road users.
• Adaptive signal input: detector replacement feeding signal control, see OPT-025.
• Private sites and logistics: yard, port gate and campus traffic flow, queue monitoring.
• Event and parking: congestion around venues; plate-level use goes to VIS-011.
• Enforcement (speed, phone use) is a different, legally regulated use: see VIS-054.

How it works

1. camera / radar stream (RTSP, ONVIF)
2. edge decode + detection (vehicle classes, VRUs)
3. multi-object tracking (IDs, trajectories)
4. calibration / homography (pixels
5. metres) + zone and line logic
6. metrics (counts, speed, headway, queue, conflicts) + anomaly rules
7. on-device anonymisation (no frames or plates stored)
8. aggregates / alerts
9. traffic management centre, dashboard, API
10. human operator confirms incidents; planners review reports

• Detection and tracking: YOLO-class or RT-DETR detectors plus trackers such as ByteTrack or BoT-SORT; fisheye and night/rain conditions need dedicated training or thermal sensors.
• Speed and distance: require camera calibration; accuracy depends on mounting height and geometry, and should be validated against radar or reference vehicles.
• VLM layer: vision-language models can add natural-language search and incident descriptions over footage (see VIS-031), but are too slow and costly for per-frame counting.
• Human-in-the-loop: operators verify incident alerts before dispatching services; planners validate count accuracy on sample periods.

Data preparation

Traffic analytics needs camera or radar feeds plus context: camera calibration, lane polygons and counting lines, signal-controller status, incident feeds and reference counts. These data sit in the traffic centre, in the road authority's GIS, in old counting reports (PDF, Excel), in loop-detector databases and in paper site plans. Typical gaps are uncalibrated or moved cameras, no ground truth, inconsistent road and junction IDs, coverage holes, and footage that shows faces and plates.

Preparation steps: inventory cameras and sensors with position and view; calibrate each view and draw lane and counting geometry; unify junction and road IDs with the GIS (PREP-005, PREP-011); take manual reference counts for 1 to 2 hours per site as a test set (PREP-018); anonymise at the edge so only counts and trajectories leave the device (PREP-022); document purposes and retention (PREP-025).

Rough share of effort: about 25 to 35 percent, mostly calibration, reference counting and integration with existing traffic data. Cameras that move or get new lenses require recalibration.

Implementation paths

Which path for whom

• Public: Path C, because road authorities need data sovereignty, local anonymisation at the edge and procurement-neutral open tooling; Path A fits only where certified, anonymising products are available.
• Enterprise: Path B, because operators and logistics sites must connect counts and incidents to their own traffic centre, yard or planning systems.
• Developer: Path C, because open detectors, trackers and calibration tools run on HW1 edge boxes and allow tailored analytics.

Path A — Buy (off-the-shelf)

• Specialised traffic-analytics vendors: Miovision (junction analytics, Canada), Iteris (US), Vaisala (radar and sensors), Kapsch TrafficCom (AT, EU), Yunex Traffic (DE, EU), Swarco (AT, EU), FLIR/Teledyne radar and thermal sensors, Axis and Bosch cameras with on-board analytics, Citilog (FR, EU) for incident detection, Vivacity Labs (UK) and Telraam (BE, citizen-science low-cost sensors, EU) for counts, Hayden AI and Numina for city sensing.
• Platform stack: NVIDIA Metropolis with partners, Hexagon and Genetec for integration into operations centres.
• Choose A when: you want certified hardware, support, anonymisation by design and a ready traffic-centre interface.

Path B — Build on cloud APIs

• Managed vision services (Google Cloud Vertex AI / Video Intelligence, Azure AI Vision, AWS Rekognition and Kinesis Video Streams) with EU regions can detect and track objects, but per-stream streaming costs are high for 24/7 camera fleets.
• Cloud is better suited to the back end: stream aggregates into a data platform, use frontier VLMs (Gemini, Claude, GPT, Mistral) for incident summaries or ad-hoc video questions on short clips.
• Keep raw video on edge or in EU regions; send only aggregates and alert clips.

Path C — Self-host (open source)

• Models and tools: Ultralytics YOLO (AGPL, check licence for commercial use), RT-DETR, RF-DETR (Apache 2.0), Roboflow Supervision for zones and line counting, ByteTrack, DeepStream or Frigate/Savant as pipeline frameworks, OpenVINO, TensorRT, ONNX Runtime.
• Open-weight VLMs: Qwen-VL family and NVIDIA Cosmos Reason for clip-level reasoning.
• Hardware: HW1 (Jetson Orin, Hailo-8, Intel iGPU) handles 1 to several streams per device; HW2/HW3 server consolidates dozens of streams and VLM search.
• Effort: needs ML, video and edge-fleet operations skills.

Path D — Custom training

Worthwhile for local conditions (tram lines, cargo bikes, scooters, unusual cameras, night fisheye). A few thousand annotated frames per camera type, with active learning from operator corrections, is a typical starting point. Public datasets (e.g. UA-DETRAC, AI City Challenge, BDD100K) help pretraining [3].

State of the art (2026)

• Real-time detectors reach high accuracy on vehicles in good conditions; small, occluded or night-time vulnerable road users, dense queues, rain and glare remain the main error sources.
• Open-vocabulary detectors and VLMs reduce labelling effort for rare classes and enable text queries; the AI City Challenge benchmarks tracking, counting and anomaly detection on traffic video [3].
• Edge-first architectures keep video on the device and export only metadata, which is the main privacy pattern in the EU.
• Digital twins and trajectory data feed simulation and surrogate safety measures.
• Limits: counts drift when cameras move or lens/weather change; speed estimates need calibration; ID switches inflate counts if trackers are unmonitored; classification of vehicle sub-types varies by site. Always run periodic manual validation.

Shared resources & combinations

• Same stack (object-detection + tracking + edge-runtime): VIS-008 people counting, VIS-011 plate recognition, VIS-010 surveillance, VIS-028 in-cabin monitoring, VIS-054 enforcement cameras; one edge fleet and one MLOps pipeline can serve several.
• Privacy tooling: VIS-030 face and plate blurring for any retained footage.
• Downstream: OPT-025 adaptive signal control, PRED-019 mobility demand forecasting, VIS-004 road asset condition monitoring using the same cameras or dashcams.
• Search layer: VIS-031 natural-language search over camera archives.

Effort, cost & KPIs

• Path A: weeks to months per corridor (permits, mounting, power, connectivity dominate).
• Path B/C: 3 to 9 months for a robust, calibrated pipeline plus ongoing fleet operations.
• Cost drivers: installation and civil works, edge hardware per camera, annotation, validation campaigns, communications, data protection impact assessment.
• KPIs: count accuracy vs. manual reference per class (MAPE), speed error vs. radar, incident detection rate and false alarms per camera-day, time to detect incident, uptime, share of data processed anonymously on-device, coverage of vulnerable road users.

People & change

Involve management and the department lead (traffic or mobility planning) as owners, IT and IT security for network and sensor integration, and the DPO from the start. Legal reviews road-traffic and data protection law, and the AI Act classification where incident detection supports critical infrastructure. Citizens are affected indirectly, so communicate purpose and anonymisation openly. Data owners of GIS and signal data must be named early.

Works council: depends. Public traffic cameras rarely monitor employees, but operators in the traffic centre or field crews can be covered when logs show their work; then co-determination (Personalrat under BPersVG or state law, in companies §87(1) no. 6 BetrVG) applies. Inform it under §90(1) no. 3 where work processes change. DPO: a DPIA is likely for systematic observation of public space, and an AVV is needed with every processor (GUIDE-004, GUIDE-005).

AI literacy: support staff in the traffic centre with training on limits, miscounts and incident false alarms (GUIDE-006). Start with a few junctions as pilot and compare against manual counts (GUIDE-009). Typical concerns are surveillance and cost; address them with counts only, no storage of identifiable footage and published methods, while humans decide on signal timing and enforcement (GUIDE-008).

Risks, regulation & compliance

• EU AI Act: classification depends on use. Pure counting and planning analytics is typically minimal risk. AI used as a safety component in the management and operation of road traffic is a high-risk category (Annex III, point 2, critical infrastructure), so incident detection that directly controls signals or closures needs conformity assessment, risk management, logging and human oversight [2]. Art. 4 AI literacy applies.
• Prohibited practices (Art. 5): real-time remote biometric identification in public spaces for law enforcement is prohibited except in narrow cases; do not extend a traffic system with face recognition, behaviour scoring or tracking of individuals across the city. Keep the system anonymous and separate from policing functions [2].
• GDPR: vehicles, plates and people are personal data when identifiable. Rely on public-task or legitimate-interest bases, run a DPIA, apply data minimisation (aggregate on device, short or no retention, blur faces and plates), inform the public with signage, and follow EDPB video-surveillance guidelines 3/2019 [4]. National law may limit public-space camera use.
• Cybersecurity: camera fleets are attack targets; NIS2 may apply to operators. Use network segmentation, signed firmware and key management.
• Mitigations: privacy by design, sampled manual audits, bias checks across lighting and vulnerable-road-user classes, human confirmation before safety actions, documented purpose limitation.

Getting started

1. Define the decisions the data serves (planning, incident response, signal control) and the legal basis.
2. Run a DPIA and choose an anonymous, edge-first architecture before buying cameras.
3. Pilot 2 to 3 sites with a vendor or open stack (YOLO/RF-DETR + tracker) and compare to a manual count.
4. Calibrate, test at night and in rain, and set KPI targets for accuracy and alert false positives.
5. Integrate aggregates into your traffic or open-data platform (DATEX II) and scale gradually.

Sources

1. European Commission — Intelligent Transport Systems (Directive 2010/40/EU and revision) — https://eur-lex.europa.eu/eli/dir/2010/40/oj
2. Regulation (EU) 2024/1689 (AI Act), Annex III and Art. 5 — https://eur-lex.europa.eu/eli/reg/2024/1689/oj
3. AI City Challenge (traffic video benchmarks) — https://www.aicitychallenge.org/
4. EDPB — Guidelines 3/2019 on processing of personal data through video devices — https://www.edpb.europa.eu/our-work-tools/our-documents/guidelines/guidelines-32019-processing-personal-data-through-video_en
5. NVIDIA Metropolis — https://www.nvidia.com/en-us/autonomous-machines/intelligent-video-analytics-platform/
6. Roboflow Supervision — https://github.com/roboflow/supervision
7. Telraam — https://telraam.net/