People counting & footfall analytics

Retailers, venues, offices and city centres need to know how many people come, where they go and how long they stay. Camera and sensor based counting replaces manual clickers and unreliable infrared beams: detectors and trackers count crossings and zone occupancy at the edge, and only aggregated numbers leave the device. Results drive staffing, opening hours, store layout, HVAC and space planning. Done as anonymous counting it is among the least intrusive vision use cases; done carelessly (face analytics, re-identification) it becomes one of the most sensitive.

Problem & value

• Today: manual counts, infrared beams (miscount groups, prams, carts), badge data (offices only) or POS transactions as a proxy. None gives conversion rate, dwell time or queue length.
• AI value: accurate directional counts at doors, occupancy per zone, queue and dwell analytics, and conversion (visitors vs. transactions). Counts feed demand and staffing forecasts (PRED-001, PRED-011) and building controls (OPT-007).
• Benefit evidence: vendors publish counting accuracy claims for dedicated 3D sensors; treat these as vendor claims and verify on your own doors in a pilot. Typical gains are operational (rosters aligned to peaks, fewer unused office floors, energy saved in empty zones) and should be measured against your own baseline, not generic figures.

Variants

• Retail and shopping centres: door counts, conversion, zone heatmaps, queue alerts.
• Offices and workplaces: desk and room utilisation for space reduction; must be anonymous and agreed with the works council (no individual tracking of employees).
• Public transport, stations, airports: platform and gate crowding, passenger flow.
• Events, museums, libraries, swimming pools: capacity limits and safety.
• City centres and tourism: pedestrian flow for planning, often via anonymous sensors or radar/LiDAR, without video.
• Private use: rare (home occupancy for heating/presence), usually a simple sensor, not a camera.

How it works

1. sensor/camera (ceiling 3D or IP camera)
2. edge inference: person detection (head/shoulder or full body)
3. multi-object tracking (short-lived track IDs, no appearance features stored)
4. line-crossing / zone logic (entries, exits, occupancy, dwell)
5. aggregation per interval on the device (images discarded)
6. time-series store + dashboard / API
7. forecasts, alerts, integration with POS, rostering, building management

• Detection and tracking: YOLO-class or transformer detectors plus trackers such as ByteTrack; ceiling-mounted 3D/stereo sensors reduce occlusion and lighting problems.
• Privacy by design: processing on the device, only counts transmitted, track IDs deleted after the person leaves the field of view. This keeps the pipeline close to anonymous statistics.
• VLM role: open-vocabulary or VLM models help to set up zones, label staff vs. visitors or answer ad-hoc questions about footage (see VIS-031), but are too costly for continuous counting.
• Human-in-the-loop: manual validation counts during commissioning (e.g. one hour of ground truth per door), exclusion of staff and a periodic re-check after layout changes.

Data preparation

Data: counts from door sensors, Wi-Fi, 3D or thermal sensors or cameras, plus context data such as opening hours, sales or transactions from the POS or ERP, staff rosters, weather, events and floor plans. In an SMB these are spread across the POS, spreadsheets, rosters and facility systems.

Typical gaps: no comparable timestamps across systems, sensors that are never calibrated, staff and children double counted, missing closing days and events, and no definition of zones or entrances.

Preparation steps: 1. List entrances and zones and agree on definitions (PREP-012). 2. Calibrate counts against manual counts for a few days (PREP-018). 3. Export POS and roster data into a tidy table keyed by hour (PREP-005, PREP-011). 4. Add a calendar of holidays, promotions and events. 5. Store only aggregated counts and avoid storing video or identifiers (PREP-022, PREP-024).

Effort: low, about 10-20% of the project, mostly sensor placement and calibration. See GUIDE-003.

Implementation paths

Which path for whom

• SMB (shops, venues, small offices): A, because mature, EU-hostable counting sensors and cloud dashboards are available at SMB prices and the task is standard.
• Enterprise (chains, facility operators): C, because many sites make per-sensor fees expensive, data should stay in the own platform and edge counting on HW1 scales cheaply.
• Developer: C, because open-source detection and tracking models run on HW1 edge devices and allow anonymous-by-design counting.
• Public sector: C, because procurement rules and sovereignty favour on-premises or sovereign EU cloud operation and public spaces raise strict privacy requirements.

Path A — Buy (recommended)

• Dedicated sensors with analytics: Xovis (CH), Axis Communications (SE) people counters and Axis Object Analytics, Hanwha Vision (KR) and Bosch (DE) camera analytics, Hikvision/Dahua (CN; consider supply-chain and security policy of your organisation), Vivotek, Avigilon.
• Footfall analytics platforms: Sensormatic/Johnson Controls, RetailNext, Axper (EU), DILAX (DE, public transport and retail), Hella Aglaia (DE), Flame Analytics, VergeSense (workplace).
• Choose A when: the goal is door counts and zone analytics with a handful of sites; check on-device anonymisation, EU hosting, DPIA support and API access to raw counts.

Path B — Build on cloud APIs

• Existing camera network: Azure AI Video Indexer or a custom detector (note: Azure AI Vision Spatial Analysis was retired on 30 March 2025) or Google Cloud Video Intelligence person detection / Vertex AI for ad-hoc analysis; AWS Rekognition person tracking (use only the non-identifying features).
• Edge-plus-cloud: run detection on a gateway and send only counts to a cloud time-series database (Azure Data Explorer, BigQuery, Timescale); forecasting with a managed service.
• Choose B when: cameras already exist, you need custom dashboards, or integration with BI and POS. Prefer EU regions and never upload raw video unless unavoidable.

Path C — Self-host (open source)

• Models and tools: Ultralytics YOLO (AGPL, enterprise licence available) with its object counting solution, RT-DETR, Roboflow supervision (line and polygon zones), ByteTrack/BoT-SORT, NVIDIA DeepStream / Metropolis, Frigate NVR, OpenVINO for Intel edge.
• Stack: RTSP stream → inference → MQTT/Kafka → Timescale or InfluxDB → Grafana.
• Hardware: HW1 suffices: a Jetson Orin Nano, Hailo-8, Intel iGPU or Coral handles several 1080p streams with a small detector. HW2 only for dense crowds or many streams per server.
• Licence note: check AGPL implications of YOLO weights and code for commercial products.

Path D — Custom training

Rarely required. Fine-tune a detector on a few thousand frames from your own camera angles when the viewpoint is unusual (top-down, fisheye, very dense queues, thermal) or when children, wheelchairs, prams and staff uniforms are mis-detected. Label with CVAT or Roboflow; evaluate counts per door, not only mAP.

State of the art (2026)

• Detection and tracking of people is a solved commodity on good viewpoints; modern real-time detectors run on sub-10 W edge chips. Overhead 3D sensors remain the accuracy reference for door counting.
• Crowd counting (density-map models, e.g. on ShanghaiTech and NWPU-Crowd benchmarks) is used for very dense scenes where individuals cannot be separated; error grows with density.
• Open-vocabulary detectors and VLMs enable zero-shot setup and natural-language queries on footage, but not yet cost-effective for 24/7 counting.
• Radar and LiDAR counting sensors are growing for privacy-sensitive places (washrooms entrances, changing areas, city streets) because no image is ever produced.
• Limits and failure modes: groups walking close together, children, prams, shopping carts, reflections, glass doors, strong backlight, door-side staff loitering, camera shifts after maintenance. Re-identification to count unique visitors across days is technically possible but moves the system into personal-data and often unlawful territory.

Shared resources & combinations

• Same stack (object-detection + tracking + edge-runtime): VIS-007 PPE monitoring, VIS-010 intrusion detection, VIS-012 traffic analytics, VIS-009 shelf monitoring, VIS-031 video agents. One edge box with a shared detector can run several analytics if the legal purposes are separated.
• Privacy tooling: VIS-030 anonymisation for any retained footage.
• Downstream: PRED-001 demand forecasting, PRED-011 staffing forecasts, OPT-007 building energy control, PRED-048 building performance.
• Do not combine with VIS-016 face recognition or emotion/demographic profiling without a separate legal assessment.

Effort, cost & KPIs

• Path A: days to weeks per site; cost is hardware per door plus a subscription per sensor.
• Path B: 1–2 months for a pilot on existing cameras; costs scale with stream count and cloud use.
• Path C: 1–3 months; low hardware cost, but own ops for model updates and monitoring.
• KPIs: count accuracy versus manual ground truth per door (error %), conversion rate, occupancy peak vs. capacity, queue wait time, staffing hours per visitor, energy per occupied hour, system uptime.

People & change

Involve early: management (purpose, budget), the store, venue or facility manager as process owner, IT for network and sensors, the DPO for the anonymisation concept, legal or compliance for signage and information duties, and the works council where one exists. Customers are informed through clear signage.

Works council: depends. Counting is aimed at visitors, but cameras and sensors in staff areas can also capture employee presence and behaviour; because objective suitability for monitoring is enough under §87(1) no. 6 BetrVG, check this and inform the works council under §90(1) no. 3. Public sector: involve the Personalrat. Settle the point in a short agreement (visitor counts only, staff excluded, no individual evaluation) (GUIDE-004).

Data protection: truly anonymous counting on the device (no images stored, no re-identification) keeps risk low. Camera-based tracking, Wi-Fi or Bluetooth tracking with device IDs can be personal data; then involve the DPO, check the legal basis and DPIA and conclude an AVV with the provider (GUIDE-005).

AI literacy and communication: support managers with training on reading the dashboards and their limits (GUIDE-006); tell staff and visitors openly what is measured (GUIDE-008).

Pilot: one location for 4-8 weeks, compared with manual counts (GUIDE-009).

Concerns: "surveillance of staff" (staff excluded, aggregate data) and "staffing decisions made by a machine" (counts inform the roster, managers decide and staff keep their say).

Risks, regulation & compliance

• EU AI Act (Reg. 2024/1689): anonymous counting is generally minimal risk; AI literacy (Art. 4) applies. Art. 5 warning: emotion recognition in workplaces and education is prohibited (Art. 5(1)(f)), as is biometric categorisation inferring sensitive attributes (Art. 5(1)(g)) and untargeted facial-image scraping. Adding emotion, gaze-based mood or sensitive-trait inference to a footfall product can therefore make it prohibited. Remote biometric identification (face matching) in publicly accessible spaces is high-risk or prohibited depending on the actor and purpose, and is out of scope here (see VIS-016). Crowd counting used to manage critical infrastructure safety (e.g. in road, rail or energy facilities) may touch Annex III(2); check case by case.
• GDPR: video of identifiable people is personal data even if only counts are stored. Fully on-device processing that never stores or transmits images, and uses non-reversible short-lived tracking, reduces risk, but the capture itself is still processing; authorities differ on whether this is outside GDPR. Needs a legal basis (typically legitimate interest, Art. 6(1)(f)), necessity and balancing test, signage, retention limits, and a DPIA where large-scale systematic monitoring of public areas occurs. EDPB Guidelines 3/2019 on video devices apply [1].
• Employees: works council co-determination in Germany (§87(1) Nr. 6 BetrVG) and national labour law; office utilisation sensors should be anonymous and not usable for performance control.
• Mitigations: edge-only processing, no image storage, no re-identification, aggregated reporting with minimum group sizes, published privacy notice, vendor DPA, regular audits.

Getting started

1. Define the decision the counts will support (rosters, layout, energy) and the metric needed (door count, occupancy, dwell). If a non-video sensor (3D, radar, LiDAR) suffices, choose it.
2. Do a short DPIA screening and involve the data protection officer and works council early.
3. Pilot two vendors (Path A) or an open-source edge box (Path C) at one entrance, with one hour of manual ground-truth counting at peak and off-peak.
4. Verify that no images or identifiers leave the device; test staff exclusion and groups.
5. Connect counts to POS or building systems, then review the KPIs after 4–8 weeks.

Sources

1. EDPB — Guidelines 3/2019 on processing of personal data through video devices — https://www.edpb.europa.eu/sites/default/files/files/file1/edpb_guidelines_201903_video_devices_en_0.pdf
2. Regulation (EU) 2024/1689 (AI Act), Art. 5 and Annex III — https://eur-lex.europa.eu/eli/reg/2024/1689/oj
3. Ultralytics docs — Object counting solution — https://docs.ultralytics.com/guides/object-counting/
4. Roboflow supervision — line and polygon zone counting — https://github.com/roboflow/supervision
5. ByteTrack: multi-object tracking by associating every detection box — https://github.com/ifzhang/ByteTrack
6. Microsoft — Azure AI Vision Spatial Analysis (retired 30 March 2025; historical reference) — https://learn.microsoft.com/azure/ai-services/computer-vision/intro-to-spatial-analysis-public-preview
7. Xovis — people counting sensors — https://www.xovis.com/
8. Axis Communications — people counting analytics — https://www.axis.com/products/analytics
9. NVIDIA — Metropolis / DeepStream SDK — https://developer.nvidia.com/deepstream-sdk