Catalog / Computer Vision / VIS-048

Aircraft turnaround & logistics process milestone monitoring

Cameras at aircraft stands, loading docks and yards detect process milestones automatically, so ground handlers, airports and logistics operators get accurate timestamps, early delay alerts and evidence for SLA disputes.

Parts needed to run it yourselfDocument readingPrediction & time seriesVision

Turnaround and loading milestones are still logged by hand, by radio or by sparse sensors, so delay causes are disputed and delays are seen too late. Computer vision on existing apron, dock and yard cameras recognises the events that make up the process, timestamps each one, compares it with the plan and warns dispatchers while there is still time to react. The same data settles service-level disputes and feeds Airport CDM (A-CDM) and logistics systems.

Problem & value

  • Today: at airports, handlers and controllers enter milestones (on-block, fuelling, catering, boarding, off-block) manually or from partial sources. At docks and yards, staff and drivers key in arrival, door and loading times. Data is late, inconsistent or missing, so the actual cause of a delay is argued about after the fact.
  • AI value: objective, continuous timestamps for every event visible on camera; earlier detection of a late start; one shared version of the truth between airline, handler, airport and carrier; evidence clips for billing and penalty disputes; training data for delay prediction.
  • Evidence: deployments are reported at apron turnaround monitoring at airports such as London Heathrow, Toronto Pearson and Seattle-Tacoma (Assaia ApronAI); quantified benefits come from vendor case studies and should be validated in your own pilot [1][2]. The A-CDM concept itself targets shared milestone data and better predictability of off-block times [3].

Variants

  • Aircraft turnaround (apron/stand): detect aircraft in position, chocks, ground power, boarding bridge or stairs, baggage and cargo doors, fuel truck, catering truck, belt loader, pushback tug, and derive the turnaround timeline against target off-block time.
  • Dock-door monitoring: truck at door, trailer door open, loading start/end, dwell per appointment.
  • Yard management: trailer arrival, parking occupancy, dwell and detention time (VIS-011, VIS-019).

How it works

  1. cameras (existing CCTV / dedicated stand cameras, RTSP)
  2. edge or server inference: detection + tracking of aircraft, vehicles, equipment, doors
  3. zone logic per stand/door (geometry, calibration)
  4. event engine: state machine of the turnaround / loading process
  5. milestone + confidence
  6. plan comparison with flight / appointment data (A-CDM, AODB, TMS)
  7. delay prediction + alert rules
  8. dispatcher / ramp-control dashboard
  9. milestone export (API, A-CDM, message bus) + evidence clip storage
  • Detection and tracking: object detectors (YOLO-class, RT-DETR) trained on stand-specific classes such as GSE types, aircraft doors and bridges, plus a multi-object tracker.
  • Event engine: rule-based state machine over tracked objects and zones (e.g. "tug attached and chocks removed" implies pushback start). Rules are explainable and auditable; ML-only sequence models are used where rules are brittle.
  • VLMs: open-vocabulary detectors and VLMs speed up labelling and clip Q&A; too costly as the main per-frame detector.
  • Prediction: gradient boosting or sequence models on milestone timing, flight data, weather and stand context predict delay risk.
  • Human-in-the-loop: ramp controllers confirm or correct milestones, review low-confidence events and decide on operational action; AI never commands vehicles or aircraft.

Data preparation

Three data worlds must meet: video streams (CCTV, stand and dock cameras), operational schedule data (A-CDM, AODB, TMS, WMS, dock appointments) and geometry (stand, door and zone layouts, camera calibration). Schedule data typically lives in airport or logistics systems with different timestamps and codes; geometry exists in CAD or paper plans.

Typical gaps: no labelled video for GSE types and milestones, cameras that do not see all doors, uncalibrated geometry, inconsistent milestone definitions between handlers, missing links between flight numbers and stands, and retention rules that forbid keeping footage.

Preparation steps: 1. Agree one milestone dictionary with ramp control and handlers (PREP-019, PREP-012). 2. Capture stand and zone geometry and calibrate each camera. 3. Pull schedule and TMS data through interfaces rather than exports (PREP-005). 4. Label 2-3 weeks of video per stand type with domain experts and keep a held-out test set (PREP-018); open-vocabulary models speed up first labels. 5. Set retention, blurring and access rules (PREP-024, PREP-022). 6. Feed ramp-controller corrections back as labels (PREP-021).

Effort share: about 40-50 percent of the total effort; labelling and calibration dominate.

Implementation paths

Which path for whom

  • enterprise (C): turnaround monitoring is core to ground handlers and airports; stand video stays on site, many cameras make cloud and seat costs high, and stand-specific models and rules give control. Buy (A) remains a valid fast start if a vendor already covers your stands.
  • developer (C): open detectors and trackers with a rule-based state machine on edge GPUs give full control and auditability.
  • public (C): airport operators in public hands benefit from on-premises or sovereign processing of security-relevant video and from avoiding lock-in in procurement.

Path A — Buy (off-the-shelf)

  • Apron turnaround vision: Assaia (ApronAI, Switzerland-founded, serves European and international airports); other airport-vision and A-CDM suppliers include Searidge Technologies (airport vision and tower tools), Adb Safegate (EU, Belgium; apron and gate systems), and A-CDM/AODB platforms such as Amadeus Airport Suite and SITA, plus Copenhagen Optimization (EU) for turnaround planning. Verify current product scope with each vendor.
  • Dock/yard: yard management and dock scheduling suites (e.g. Blue Yonder, Manhattan Associates, Kaleris, Gatekeeper) and camera-analytics vendors such as Rhombus, Spot AI and Cogniac; add local EU integrators for camera work.
  • Choose A when: you want proven milestone libraries, integration with A-CDM or TMS and vendor-managed model updates. Check EU data residency, retention controls and face/person blurring.

Path B — Build on cloud APIs

  • Vision: managed video services (Azure AI Vision / Video Indexer in EU regions, Google Cloud Video Intelligence, AWS Rekognition Video) are weaker on GSE and aircraft classes; use them mainly for generic detection and clip search. Multimodal APIs (Gemini, Claude, GPT, Mistral) suit offline analysis of clips, labelling and incident summaries, not continuous per-frame use.
  • Integration: Kinesis Video Streams / Azure IoT / Kafka, results to A-CDM or TMS via API.

Path C — Self-host (open source)

  • Models: Ultralytics YOLO (AGPL, check licence), RT-DETR, RF-DETR, Grounding DINO and SAM-family for labelling; open VLMs (Qwen-VL family) for clip Q&A.
  • Pipeline: NVIDIA DeepStream, GStreamer, ByteTrack/BoT-SORT, Supervision for zone logic.
  • Hardware: HW1 edge devices (Jetson Orin) per few cameras, or HW2 (one 24-32 GB GPU) for roughly a stand cluster or dock hall; ops effort is real, as cameras, weather and night conditions need continuous tuning.

Path D — Custom training

Usually required in part: GSE and aircraft-type classes are specific to each airport and handler. Collect and label footage across weather, night, de-icing, snow, rain and lighting changes; a few thousand labelled frames per class and site is a typical starting point. Controller corrections serve as active-learning labels.

State of the art (2026)

  • Real-time detectors plus trackers reliably detect large equipment and vehicles; open-vocabulary detectors and VLMs now bootstrap labelling and cover rare equipment types.
  • Production systems combine vision events with schedule and A-CDM data; vision-only milestones are most reliable for visible, discrete events (chocks, bridge, tug, truck at door).
  • Hard cases: occlusion, night, weather, glare, de-icing spray, events inside the aircraft (boarding is partly inferred). Camera placement matters more than model choice.
  • A-CDM milestones (e.g. TOBT, TSAT, ARDT, AEZT per the Airport CDM manual) give a common vocabulary [3]. For docks no common schema exists, so define an event taxonomy early.

Shared resources & combinations

Effort, cost & KPIs

  • Path A: 2-4 months for a pilot stand cluster or dock hall incl. camera survey and integration; subscription per stand or camera, pricing on request.
  • Path B/C: 4-9 months; cost drivers are cameras, labelling, integration and 24/7 operations.
  • KPIs: milestone detection precision/recall and timestamp error against manual logs; share of turnarounds with complete data; on-time off-block and departure rate; accuracy of delay prediction at fixed lead times; dwell and detention time at docks; dispute resolution time.

People & change

Start with management and the ramp-control or operations lead, who own milestone definitions. Bring in IT and IT security early (camera network, edge devices, OT/IT separation at airports), the DPO, the works council and, where relevant, handler and carrier partners whose data and SLAs are affected. Ramp controllers and dispatchers are the end users; pilot on two or three stands or docks before scaling.

Works council: yes. Cameras that track vehicles, equipment and staff at stands can monitor behaviour and performance, so §87(1) no. 6 BetrVG applies even if monitoring is not the goal (BAG: objective suitability). Inform under §90(1) no. 3, and use an expert under §80(3) if needed. A works agreement should exclude individual performance evaluation and define evidence-clip use. Public operators: staff council.

Data protection: persons are visible, so run a DPIA (Art. 35 GDPR), blur persons by default and sign AVVs with vendors. Check the AI Act classification with legal; safety components of critical infrastructure may become high-risk.

AI literacy and control: train controllers on confidence levels and corrections (GUIDE-006); the AI never commands vehicles or aircraft.

Concerns: "ground staff are being timed" - transparent purpose, process-level KPIs only; "SLA disputes" - evidence clips with agreed access rules (GUIDE-004, GUIDE-005, GUIDE-014).

Risks, regulation & compliance

  • EU AI Act: depends. Pure process monitoring of aircraft and trucks is typically minimal risk. It may become high-risk if used as a safety component in the management and operation of critical infrastructure, including road traffic and utilities (Annex III no. 2), or if it is used to monitor and evaluate workers' performance and behaviour (Annex III no. 4). Art. 5 prohibitions: emotion recognition in the workplace and untargeted facial image scraping are banned; do not use emotion inference on ground crews or drivers [5].
  • GDPR: apron and dock cameras capture ground staff, drivers and visitors. Needs a lawful basis, purpose limitation (process milestones, not individual productivity), DPIA, retention limits, masking or blurring of faces and plates where not needed, and works-council involvement (Germany: BetrVG section 87(1) no. 6 co-determination) [4].
  • Aviation safety: outputs are decision support only; safety-critical functions stay under existing aerodrome and handling safety management systems.
  • Mitigations: edge anonymisation, role-based access, audit logs, aggregate by process not person.

Getting started

  1. Define the 10-15 milestones that matter for one process (e.g. one stand type or dock hall) and how they map to A-CDM or TMS fields.
  2. Survey cameras for coverage and night performance; add cameras only where needed.
  3. Record 2-4 weeks of footage, label a gold set and compare with manual logs to measure baseline data quality.
  4. Pilot a vendor (Path A) or a YOLO + tracker + state-machine prototype on one site, with controller review of every milestone.
  5. Complete DPIA and works-council agreement before go-live; then link alerts to a concrete operational action and measure on-time performance.

Sources

  1. Assaia - ApronAI turnaround monitoring - https://www.assaia.com/
  2. Assaia - customer and case-study pages - https://www.assaia.com/resources
  3. EUROCONTROL - Airport Collaborative Decision-Making (A-CDM) - https://www.eurocontrol.int/concept/airport-collaborative-decision-making
  4. Regulation (EU) 2016/679 (GDPR) - https://eur-lex.europa.eu/eli/reg/2016/679/oj
  5. Regulation (EU) 2024/1689 (AI Act), Art. 5 and Annex III - https://eur-lex.europa.eu/eli/reg/2024/1689/oj
  6. Ultralytics YOLO documentation - https://docs.ultralytics.com/
  7. NVIDIA DeepStream SDK - https://developer.nvidia.com/deepstream-sdk