AI-personalised phishing simulation & security-awareness training (human risk management)

Annual awareness videos and generic phishing tests teach little, while attackers now use GenAI to write fluent, targeted lures in any language. Human-risk-management (HRM) platforms turn the same technology around: an LLM drafts simulations matched to each employee's role, language and current threat trends, difficulty adapts to behaviour, and a failed click triggers a short, targeted lesson. Result: more realistic training, measurable per-team risk, and documented evidence for the training duties of NIS2 and DORA.

Problem & value

• Today: awareness is mostly a yearly e-learning course plus a monthly template phishing campaign. Templates are generic, translated badly, quickly recognised by staff and shared between colleagues. Reporting is "click rate", which says little about real risk.
• Why humans matter: the Verizon Data Breach Investigations Report consistently lists the human element (phishing, stolen credentials, errors, misuse) in a majority of breaches [1]. ENISA's Threat Landscape names phishing as a leading initial-access vector [2].
• AI value:
• scenarios in the employee's language, department and tools (e.g. a supplier-invoice lure for accounts payable, a fake HR portal for new joiners, a payroll change request for finance);
• difficulty adapts per person, so beginners are not demoralised and experts are not bored;
• just-in-time micro-training at the moment of failure, when learning sticks best;
• risk scoring from behaviour (reporting rate, time-to-report, repeat failures) instead of click rate alone;
• far less manual effort to author and localise campaigns.
• Regulatory pull: NIS2 Art. 20 requires management bodies to follow cybersecurity training and to offer similar training to employees regularly; Art. 21 lists basic cyber hygiene and training among mandatory risk-management measures [3]. DORA Art. 13(6) requires ICT security awareness programmes and digital operational resilience training for staff and management of financial entities [4].
• Vendor-published benefit figures are self-reported; benchmark your own baseline.

Variants

• SMB: hosted platform with automatic monthly campaigns, report button for Microsoft 365/Google Workspace and a ready-made course library; one part-time owner.
• Enterprise / regulated (finance, energy, health): role-based tracks (executives, finance, developers, admins), multilingual rollouts, integration with SIEM/IdP, works-council agreement, board reporting.
• Multi-channel: email, smishing, QR-code ("quishing"), Teams/Slack lures and voice-phishing simulation with synthetic voice for helpdesk and finance call-back scenarios. Deepfake-voice simulations of executives need explicit approval, consent rules and a clear debrief (see SEC-011 for detection).
• Public sector / MSP: low-budget open-source tooling, or multi-tenant platforms run as a managed service.
• Private use: family scam-call exercises exist but are mostly consumer education.

How it works

1. role/locale data + current threat themes
2. scenario generation (LLM: lure text, landing page, sender, difficulty, language)
3. safety review (guardrails, brand/impersonation limits, human approval of templates)
4. delivery via allow-listed sending infrastructure (email / SMS / chat / voice)
5. telemetry (open, click, credential entry, report, time-to-report)
6. human-risk scoring (per person, role, team; adaptive difficulty)
7. just-in-time micro-training + nudges
8. reporting (NIS2/DORA evidence, board dashboards) and feedback to SEC-002 controls

• Generation: an instruction-following LLM produces the lure within a scenario schema (pretext, urgency cue, red flags to teach, difficulty 1-5). Each template contains the "teachable red flags" so that the lesson can reference what was missed. Providers reduce abuse by restricting generation to authenticated customer tenants.
• Adaptive difficulty: a simple model (rules, gradient-boosted trees or bandit) selects the next scenario from past outcomes; no deep model is needed.
• Human-risk score: composite of simulation behaviour, reporting behaviour, training completion and, optionally, security signals (e.g. risky-login events). Prefer team-level scores for management; individual scores only to personalise training.
• Voice/smishing: TTS or speech-to-speech models run scripted call scenarios, ideally with a trainer in the loop.
• Human-in-the-loop: awareness lead approves scenario libraries and sensitive pretexts (bonuses, layoffs, health, ongoing crises are usually banned); works council/DPO sign-off on scoring; failures lead to learning, not discipline.

Data preparation

This use case needs little classic data, but it does need clean organisational data. The key source is the directory (Entra ID, AD, Google Workspace) and the HR system for department, role, language and location; email addresses and group memberships often live in several places. Typical gaps are stale accounts, missing language or role attributes, shared mailboxes, external staff and duplicates between HR and IT lists. A second input is the simulation telemetry itself, which is personal data and must be protected.

Preparation steps: - Export and reconcile the user list from directory and HR (PREP-005, PREP-011); fix role and language attributes. - Define role groups and banned pretexts before the first campaign. - Agree a pseudonymised or team-level reporting mode (PREP-022) and check purpose and legal basis (PREP-025). - Allow-list simulation sender domains and IPs with IT. - Collect a few real, anonymised phishing mails as seed themes.

Effort share is low, roughly 10-15 percent of the project; most time goes into content approval and agreements with the works council and DPO.

Implementation paths

Which path for whom

• SMB: Path A, because hosted EU platforms (e.g. SoSafe, Hoxhunt) cover simulation, report button and evidence at per-user prices with a standard process.
• Enterprise: Path A, because awareness training is a commodity function and vendor content libraries, languages and IdP/SIEM connectors outweigh build effort.
• Public sector: Path C, because GoPhish plus an open-weight model on own infrastructure keeps employee telemetry on-prem and fits tight budgets and procurement rules.

Path A — Buy (recommended for SMBs and enterprises)

• Global HRM / SAT platforms: KnowBe4 (AIDA agents for AI-driven simulations and training), Proofpoint Security Awareness, Mimecast Engage, Hoxhunt (Finland, EU; adaptive, gamified), Adaptive Security (AI-personalised simulations incl. deepfake voice and SMS), Cofense, Microsoft Defender for Office 365 Attack Simulation Training (included in higher M365 licences).
• EU-based: SoSafe (Germany; AI-assisted phishing simulation and risk dashboard), Hoxhunt (Finland). Verify EU data location and subprocessors.
• Selection checklist: EU hosting and DPA, pseudonymised reporting mode, works-council configuration, languages, channels (voice/QR), report-button integration, SIEM/IdP/SCIM connectors, compliance-evidence export, controls against abuse of generated lures.
• Choose A when you want evidence for NIS2/DORA quickly and have no in-house content team.

Path B — Build on cloud APIs

• Content: EU-hosted or EU-region LLM APIs (Mistral, Azure OpenAI in EU regions, Claude or Gemini via EU-region cloud endpoints) with structured outputs for scenario JSON; generation limited to internal tenants.
• Delivery: open-source campaign engine plus a sending domain isolated from production mail; allow-listing in the mail gateway for simulation headers.
• Scoring/analytics: warehouse plus simple ML; dashboards in existing BI.
• Choose B when you have special scenarios (industrial, clinical) or want to extend a purchased platform with company-specific lures. Expect more effort in legal review than in code.

Path C — Self-host (open source)

• Campaigns: GoPhish (open source phishing framework) for sending, landing pages and tracking.
• Content: open-weight LLMs such as Mistral Small / Mixtral-class, Llama, Qwen or gpt-oss served by vLLM or Ollama; open TTS (e.g. Piper, Coqui-style models) for voice drills.
• Training content: self-built micro-lessons, optionally generated and reviewed by the awareness team.
• Hardware: HW2 is sufficient; the models only write short texts. Operational effort is in deliverability, hosting and content upkeep.
• Choose C when data must stay on-prem (public sector, defence-adjacent suppliers) or budget is minimal.

Path D — Custom training

Rarely justified; a bespoke risk-prediction model needs long campaign histories and strong privacy review. Start with adaptive rules.

State of the art (2026)

• Agentic simulation: vendors let AI agents plan the campaign (who, when, which lure, which lesson) rather than only writing text; KnowBe4 markets its AIDA agents, Hoxhunt and SoSafe adaptive engines, Adaptive Security AI-personalised multichannel simulations [5][6][7].
• Beyond email: SMS, QR codes, collaboration tools and synthetic-voice calls reflect how real attackers now operate; deepfake-driven CEO fraud cases are widely reported and make voice drills a differentiator.
• From click rate to behaviour: leading programmes track reporting rate and time-to-report as the positive metric; a fast reporter is a human sensor feeding the SOC (see SEC-001).
• Evidence is mixed: academic work questions the value of mandatory, punitive embedded training. A large field study at a US health system found embedded training had little measurable effect [8]. Design for short, voluntary-feeling, contextual lessons, a positive reporting culture and technical controls (MFA, phishing-resistant FIDO2, mail filtering) as the primary defence.
• Limits and failure modes:
• LLM lures can be unrealistically perfect or culturally off; require review and local-language checks.
• Over-targeted scenarios (private life, health, bonus) damage trust and may violate employment law.
• Scoring individuals creates surveillance concerns and gaming.
• Simulation mail may be blocked or flagged as real, or trigger real incidents if not coordinated with the SOC.

Shared resources & combinations

• Shared stack: the generation model can be the same server used by SEC-008 (threat-intel summarisation).
• SEC-002 (phishing & email security): simulation results tune detection; real phish caught by employees feed new training themes.
• SEC-011 (deepfake detection): shared voice/video scenarios and awareness content on deepfake CEO fraud.
• SEC-013 (shadow-AI discovery): training on safe AI use; the same nudge channel.
• SEC-009 (compliance automation): training evidence for NIS2, DORA and ISO 27001 A.6.3 awareness controls.
• EDU-006 / EDU-010: general microlearning and role-play tooling; this use case differs by generating adversarial content and scoring security behaviour.

Effort, cost & KPIs

• Path A: days to weeks; typically per-user per-year subscription; major cost is change management and works-council agreement.
• Path B: 1-3 months; API cost negligible versus legal and content-governance effort.
• Path C: 2-6 weeks for GoPhish plus LLM; ongoing deliverability and content effort.
• KPIs: report rate, median time-to-report, repeat-failure rate, credential-submission rate, training completion, participation sentiment, number of real phishing incidents reported by staff, mean time from first real-phish report to containment, coverage of NIS2/DORA training obligations. Avoid click rate as the sole KPI.

People & change

• Involve early: management (mandate, no-blame stance), IT security (owner, SOC briefing), works council, DPO, HR (training duties, employee data), IT (allow-listing, directory) and end-users via a volunteer pilot.
• Works council: yes. Simulation telemetry and risk scores are suited to monitor behaviour and performance, so §87(1) no. 6 BetrVG applies (objective suitability suffices, per the BAG). Conclude a works agreement covering pseudonymised or team-level reporting, retention, banned pretexts and a ban on disciplinary use; inform the council in good time (§90(1) no. 3); an AI expert may be engaged (§80(3)). Public bodies involve the Personalrat.
• Data protection: DPIA likely for individual scoring; AVV with the vendor; DPO involved before the pilot; EU hosting.
• AI literacy: the programme itself supports Art. 4 literacy measures; add a short briefing on AI-generated lures and voice cloning (GUIDE-006).
• Communication: announce campaigns in general terms, explain the goal (protection, not testing people), publish team-level results and celebrate reporters.
• Pilot: 50-100 volunteers across roles and languages.
• Concerns: "being spied on" is met with aggregated reporting; "being tricked" with debriefs instead of sanctions. Humans approve all templates and see all scoring rules.
• See GUIDE-004, GUIDE-008 and GUIDE-005.

Risks, regulation & compliance

• EU AI Act: usually minimal or limited risk; the training content is not itself a high-risk system. Points of attention:
• Art. 5(1)(f) prohibition: emotion recognition in the workplace is banned; do not infer emotions or stress from voice, webcam or typing during training or simulations.
• Art. 50: synthetic voice or deepfake content must be disclosed where it is shown to people; closed internal exercises with a debrief should document the exemption reasoning.
• Annex III(4) employment: using human-risk scores for promotion, discipline, monitoring or termination could trigger high-risk obligations. Keep scores for training only and document that purpose.
• Art. 4 AI literacy is met in part by such programmes.
• GDPR: simulation telemetry is personal data about employees. Needs a lawful basis (typically legitimate interest with balancing test, or legal obligation under NIS2/DORA for the training itself), transparency, data minimisation, retention limits, DPIA for individual scoring, and a pseudonymised mode. In Germany, works-council co-determination (BetrVG §87(1) no. 6) applies; reports to managers should be aggregated.
• Employment and ethics: no punishment for failing; avoid exploitative pretexts; legal review of lures that impersonate real brands (trademark) or real executives.
• Security: isolate simulation infrastructure; whitelist properly; brief SOC and helpdesk; never collect real passwords (store a boolean "entered credentials").
• Mitigations: template approval workflow, ban-list of topics, role-based access to results, annual review of scoring and fairness.

Getting started

1. Define goals and baseline: report rate and time-to-report, plus a map of NIS2/DORA training obligations.
2. Agree scope with DPO and works council: pseudonymised reporting, no individual-based discipline, banned pretexts.
3. Pilot 2-3 vendors (or GoPhish plus an LLM) with 50-100 volunteers across roles and languages.
4. Deploy the report-phishing button and wire it into the SOC workflow (SEC-001/SEC-002).
5. Run monthly adaptive campaigns plus just-in-time lessons; review KPIs quarterly and add voice/QR scenarios once the basics are stable.

Sources

1. Verizon — 2025 Data Breach Investigations Report — https://www.verizon.com/business/resources/reports/dbir/
2. ENISA — Threat Landscape — https://www.enisa.europa.eu/topics/cyber-threats/threats-and-trends
3. Directive (EU) 2022/2555 (NIS2) — https://eur-lex.europa.eu/eli/dir/2022/2555/oj
4. Regulation (EU) 2022/2554 (DORA) — https://eur-lex.europa.eu/eli/reg/2022/2554/oj
5. KnowBe4 — AI-driven security awareness training and AIDA — https://www.knowbe4.com/products/aida
6. Hoxhunt — Human risk management platform — https://hoxhunt.com/
7. SoSafe — Security awareness and human risk management — https://sosafe-awareness.com/
8. Ho et al. — Understanding the Efficacy of Phishing Training in Practice (IEEE S&P 2025) — https://www.computer.org/csdl/proceedings-article/sp/2025/223600a076/26hiMybBLEs
9. GoPhish — open-source phishing framework — https://getgophish.com/
10. Regulation (EU) 2024/1689 (AI Act) — https://eur-lex.europa.eu/eli/reg/2024/1689/oj
11. Adaptive Security — https://www.adaptivesecurity.com/