Face recognition & biometric access control

Doors, gates, kiosks and workstations are opened by a badge or PIN that can be lent, lost or stolen. Face recognition compares a live face with an enrolled template (1:1 verification) or a small gallery (1:N identification) and adds liveness detection against photo, screen and mask spoofs. It speeds up entry and ties access to a person, but face templates are special-category data in the EU, so the design (on-device templates, opt-in, non-biometric alternative) matters as much as the model.

Problem & value

• Today: RFID badges, PINs and keys dominate. Lost badges, tailgating and badge sharing weaken security; reception and badge administration cost time. Time-and-attendance fraud ("buddy punching") is a common reason for interest in biometrics.
• AI value: hands-free entry, no credentials to lose, stronger link between person and access right, faster throughput at turnstiles, and step-up authentication for sensitive rooms (labs, server rooms, pharma cleanrooms where gloves make fingerprints impractical).
• Accuracy is measurable: NIST's ongoing Face Recognition Technology Evaluation (FRTE) publishes independent 1:1 and 1:N error rates per vendor algorithm, so claims can be compared on neutral data [1]. Figures depend heavily on image quality and demographics; do not rely on vendor marketing numbers.
• Counterweight: in the EU, a biometric solution must pass a necessity test: if a badge plus PIN reaches the purpose, regulators regard biometrics as disproportionate [2][3].

Variants

• 1:1 verification (preferred): the template lives on the person's own card, phone or wallet; the reader only compares. Lowest privacy impact.
• 1:N identification: the reader searches an enrolled gallery (small workforce, secure site). Needs a central or local template store and stronger justification.
• Device/workstation login: Windows Hello for Business, Face ID, Android biometrics; templates stay in secure hardware on the device.
• Remote onboarding / KYC selfie match: compare a selfie with an ID photo, see ANOM-014 and AGENT-015.
• Hospitality and travel: opt-in facial boarding, hotel or lounge entry; see the EDPB opinion on airports [2].
• Consumer/private: smart doorbells and phone unlock; household exemption applies only to purely private use.
• Out of scope here: policing, border control and surveillance identification (see VIS-010, VIS-050); age estimation (VIS-029).

How it works

1. enrolment (consent/contract, ID check, several face images)
2. template (embedding vector) stored on card/phone or in encrypted local store camera at door (NIR + RGB)
3. face detection + quality gate (pose, blur, light)
4. presentation attack detection (liveness: passive texture/depth/NIR, optional challenge)
5. embedding extraction (ArcFace-style CNN/ViT)
6. 1:1 compare with claimed identity | 1:N nearest-neighbour search in gallery
7. threshold decision + optional second factor (badge/PIN)
8. lock/turnstile release, pseudonymised audit log
9. fallback lane (badge/PIN/guard) and manual review of rejects

• Embedding model: maps a face crop to a vector (typically 512 dimensions); decisions use a similarity threshold tuned to a target false-accept rate (FAR) for the site.
• Liveness / presentation attack detection (PAD): tested against ISO/IEC 30107-3 methodology; independent labs certify products [4]. Injection attacks (virtual cameras, deepfakes) need device and camera attestation, see SEC-011.
• Human-in-the-loop: security staff handle rejects, spoof alerts and enrolment disputes; a non-biometric alternative of equal speed must always exist.

Data preparation

Needed data: enrolment photos or templates of authorised persons, the access-rights matrix (who may enter where), and logs of door events. Sources are HR systems, the access-control system, the visitor management tool, badge databases and paper visitor logs.

Typical gaps: outdated or missing HR records, people with several identities, no valid consent or legal basis for enrolment, poor enrolment image quality, unclear roles and access rights, and logs stored indefinitely.

Preparation steps: clean the person master data and access matrix with HR and the data owner (PREP-004), check purpose and legal basis before reuse (PREP-025), define a voluntary alternative such as card or PIN, enrol with standard lighting and pose, set retention rules, and keep templates only on the device or in an encrypted store. Tools: the identity system, a template vault, directory services. Data preparation is roughly 15-25 percent of the effort; the larger part is legal and organisational. See GUIDE-003 and GUIDE-005.

Implementation paths

Which path for whom

• SMB: Path A, because certified access-control products with on-device or on-premises template storage exist at SMB prices; biometrics should only be used where a card or PIN is clearly insufficient.
• Enterprise: Path C, because biometric templates are special-category data and a central on-premises platform integrated with access and identity systems keeps control and serves several sites.
• Developer: Path C, because open face-recognition libraries run on HW1 hardware, though model licences must be checked for commercial use.
• Public: Path C on sovereign infrastructure, because procurement rules, strict legal limits on biometric identification and sovereignty favour operated-by-us solutions.

Path A — Buy

• Access control with face readers: Idemia (FR), Thales (FR), dormakaba (CH), Nedap (NL), Suprema (KR), HID, Hikvision/Dahua-class terminals (check supply-chain and export policy), Axis and Bosch camera ecosystems with partner analytics. Prefer products with on-device template storage and encrypted card-based enrolment.
• Identity verification / liveness SDKs: IDnow (DE), Veriff (EE), Onfido/Entrust, Facephi (ES), Signicat (NO), Jumio, iProov.
• Device login: Windows Hello for Business, Apple Face ID, Android Class 3 biometrics.
• Choose A when: fewer than a few hundred enrolled users per site, standard door hardware, no data-science team. Check: iBeta/ISO 30107-3 PAD results, NIST FRTE listing, EU hosting, DPIA support, template deletion API.

Path B — Build on cloud APIs

• Face APIs: Azure AI Face (access to identification and verification features is gated via a Limited Access application), Amazon Rekognition (face comparison, search and Face Liveness; EU regions) [5]. Google Cloud offers face detection only, not recognition.
• Design for GDPR: send only crops, use EU regions, prefer comparison-only calls (1:1) over cloud galleries, delete images immediately, and sign a data processing agreement. Cloud galleries are the hardest part to justify.
• Integration: reader or camera → small gateway → API → lock controller via OSDP/Wiegand/REST.

Path C — Self-host (open source)

• Models/tools: InsightFace (SCRFD detection, ArcFace recognition; check model-weight licences, many are research-only) [6], AdaFace, MediaPipe/OpenCV YuNet detection, Silent-Face-Anti-Spoofing and similar PAD models. Vector search via FAISS or pgvector.
• Runtime: ONNX Runtime / TensorRT / OpenVINO on Jetson Orin, Intel NPU or a CPU mini-PC; HW1 is enough for a few doors.
• Ops effort: moderate; you own PAD quality, threshold calibration, template encryption and updates. Open-source PAD generally lags certified commercial PAD against modern attacks, so it suits low-security uses.

Path D — Custom training

Rarely worthwhile for recognition itself; vendor and open embeddings are trained on millions of identities. Custom work is justified for domain-specific PAD (unusual NIR sensors), masks/PPE (respirators, cleanroom hoods) or fairness tuning. Needs thousands of consented images across lighting and demographic conditions; scraped face data is unlawful in the EU (see Risks).

State of the art (2026)

• Recognition: margin-based embeddings (ArcFace lineage [7]) and large-scale ViT backbones lead the NIST FRTE leaderboards; error rates on good-quality frontal images are very low, and the gap between vendors is now mostly in low-quality, masked and cross-age conditions [1].
• Liveness: passive multi-spectral/NIR and depth PAD is standard; the active threat has moved to injection attacks and generative deepfakes, which are mitigated by hardware-attested cameras and challenge-response, not by the matcher.
• On-device and privacy-preserving: template on card/phone, secure enclaves, cancelable biometrics and homomorphic or split matching are maturing but not yet widely deployed.
• Limits and failure modes: demographic differentials (NIST measures them per algorithm; choose vendors with small, documented gaps), low light and backlight, twins and look-alikes, ageing of templates, masks and glasses, and threshold drift. Performance on your camera positions can differ markedly from lab numbers; pilot with your population.

Shared resources & combinations

• Same capability (face-biometrics, edge-runtime): VIS-010 (surveillance analytics, where face identification is typically restricted), VIS-029 (age estimation), VIS-030 (face blurring for the same camera network), VIS-011 (plate recognition at vehicle gates).
• Identity stack: ANOM-014 (document forgery & liveness), AGENT-015 (KYC onboarding), SEC-020 (identity governance), SPEECH-010 (voice biometrics as alternative factor).
• Threat side: SEC-011 deepfake detection for injection attacks.
• Hardware: a Jetson/NPU edge box per site (HW1) can also run VIS-007 or VIS-008 analytics; a vector index (pgvector) is shared with text RAG stacks.

Effort, cost & KPIs

• Path A: days to weeks per site; readers are priced per door plus software licence per user or site.
• Path B: 1-3 months; API cost per verification is small, integration and DPIA dominate.
• Path C: 2-4 months plus continuous PAD and threshold maintenance.
• KPIs: false-accept rate (FAR) and false-reject rate (FRR) at the chosen threshold, failure-to-enrol rate, PAD attack-acceptance rate in red-team tests, throughput per person, share of users choosing the biometric lane, fallback usage, time to delete a template after leaving.

People & change

Biometric access is sensitive, so involve people before choosing a product: management for the business reason, the department lead or facility manager as process owner, IT and information security for architecture and template storage, the DPO for legal basis and DPIA, legal and compliance, HR for enrolment of staff, and the works council from the start. End users must be heard early.

Works council: yes. Face recognition at doors is suited to monitor behaviour and presence, so co-determination under §87(1) no. 6 BetrVG applies, and the council must be informed under §90(1) no. 3; the council may engage an AI expert (§80(3)). Public bodies involve the staff council (Personalrat). Conclude a works agreement covering purpose, no use for performance control, retention and deletion, and an alternative for those who decline.

Data protection: biometric data for unique identification is special-category data (Art. 9 GDPR); in employment a necessity test under §26 BDSG is needed, and consent is hard to rely on due to dependence. A DPIA (Art. 35 GDPR) is required, plus an AVV with any processor. For AI literacy, support security staff with training on error rates and bias (Art. 4 AI Act). Communicate openly, pilot at one low-risk door with volunteers, and keep a human fallback for rejections. Guides: GUIDE-004, GUIDE-005, GUIDE-008.

Risks, regulation & compliance

• EU AI Act: classification depends on the function. Biometric verification whose sole purpose is to confirm that a person is who they claim to be is excluded from the Annex III biometric high-risk category; remote biometric identification (1:N) and biometric categorisation by sensitive attributes are Annex III high-risk, with duties applying from the dates in the Act (a postponement is under discussion in the Digital Omnibus; check current status) [8]. Art. 5 prohibitions (in force since 2 Feb 2025) must be checked clearly: real-time remote biometric identification in publicly accessible spaces for law enforcement (narrow exceptions), untargeted scraping of facial images to build databases, emotion inference at work or school, and biometric categorisation to infer race, beliefs or similar traits [8][9]. Access control at a private door with opt-in 1:1 matching is not among them; do not add emotion or attribute inference to it.
• GDPR: face templates used to uniquely identify are special-category data (Art. 9), both for 1:1 and 1:N according to the EDPB and the AEPD's updated guidance [2][3]. A lawful Art. 9(2) basis is needed: explicit consent that is truly free (employees are rarely free to refuse, so an equal non-biometric alternative is required) or a national employment-law basis (e.g. Germany's BDSG §26(3)). A DPIA is practically always required (Art. 35); privacy by design (Art. 25) points to on-device templates. The EDPB found airport schemes with central databases not compliant, and favours templates under the individual's control [2].
• National enforcement: data protection authorities in several member states have fined or banned biometric time-clocks and school entry systems; check your authority's guidance.
• Security: templates cannot be reset like passwords, so encrypt at rest, separate from identity data, and plan revocation. Deepfake and replay attacks need PAD and attested cameras.
• Fairness and works councils: document demographic error rates; in Germany, works council co-determination applies (BetrVG §87).
• Mitigations: minimise (1:1 over 1:N), retention limits, no secondary use, signage, audit logs, regular PAD red-teaming.

Getting started

1. Write the purpose and test necessity: could badge + PIN or a mobile credential do the job? Document the answer.
2. Run a DPIA, involve the DPO and works council, and define the non-biometric alternative lane.
3. Shortlist 2-3 vendors with NIST FRTE results and ISO 30107-3 PAD evidence; demand on-device or card-based templates.
4. Pilot at one door with volunteers; measure FAR, FRR, failure-to-enrol and fallback use in your lighting.
5. Roll out with enrolment/deletion procedures, logging and a quarterly red-team of spoof attempts.

Sources

1. NIST — Face Recognition Technology Evaluation (FRTE) 1:1 — https://pages.nist.gov/frvt/html/frvt11.html
2. EDPB — Opinion 11/2024 on the use of facial recognition to streamline airport passenger flows — https://www.edpb.europa.eu/our-work-tools/our-documents/opinion-board-art-64/opinion-112024-use-facial-recognition-streamline_en
3. EDPB — Guidelines 3/2019 on processing of personal data through video devices — https://www.edpb.europa.eu/our-work-tools/our-documents/guidelines/guidelines-32019-processing-personal-data-through-video_en
4. ISO/IEC 30107-3 — Biometric presentation attack detection, testing and reporting — https://www.iso.org/standard/79520.html
5. AWS — Amazon Rekognition Face Liveness and face comparison docs — https://docs.aws.amazon.com/rekognition/latest/dg/face-liveness.html
6. InsightFace project — https://github.com/deepinsight/insightface
7. Deng et al. — ArcFace: Additive Angular Margin Loss for Deep Face Recognition — https://arxiv.org/abs/1801.07698
8. Regulation (EU) 2024/1689 (AI Act), Art. 5 and Annex III — https://eur-lex.europa.eu/eli/reg/2024/1689/oj
9. European Commission — Guidelines on prohibited AI practices (Feb 2025) — https://digital-strategy.ec.europa.eu/en/library/commission-publishes-guidelines-prohibited-artificial-intelligence-ai-practices-defined-ai-act