Catalog / Data Readiness for AI / PREP-022

Pseudonymisation of datasets & exports before AI use

Replace names, e-mail addresses, IDs and other personal data in CRM exports, tickets, e-mails and spreadsheets with consistent placeholders before they reach an AI tool, so staff can use AI on real data with far less risk.

Quick win: proven, manageable effort, first results within weeks, low regulatory risk.

Parts needed to run it yourselfAgents & automationLanguage model

Someone exports 5,000 support tickets or a customer list, pastes them into an AI tool, and personal data leaves the company without a legal basis, contract or deletion concept. A small pre-processing step finds names, e-mail addresses, phone numbers, IBANs and customer numbers, replaces them with consistent placeholders and keeps the mapping table at home. The AI sees "PERSON_017 complained twice about invoice INV_204", the analyst can still work with it, and the result can be mapped back internally.

Problem & value

  • Today: employees paste raw exports into chatbots (shadow AI) or avoid AI because the DPO says "no personal data".
  • AI value: consistent placeholders keep relationships intact (same customer, same token), unlike deletion.
  • Be honest about the legal effect: pseudonymised data is still personal data under GDPR Art. 4(5) as long as the mapping exists. It reduces risk and supports data minimisation (Art. 5(1)(c), Art. 25), but it does not remove the need for a legal basis or an AVV when a processor is involved [1][2].
  • No automated detector finds 100 % of personal data; Presidio's own documentation says so [3]. Plan for human spot checks.

Variants

  • Tickets/e-mails: free text, hardest case. Tabular exports: column rules. Analytics sets: generalise (age bands, postcode areas). Private: local tools before pasting into chatbots; see DOC-018.

How it works

  1. export (CSV/XLSX/JSON/text)
  2. classify columns (direct identifier | quasi-identifier | free text | harmless)
  3. detect (regex/checksums for IBAN, phone, e-mail + NER for names, addresses)
  4. replace (consistent tokens via keyed hashing or lookup table)
  5. store mapping separately (restricted access, own retention)
  6. review sample + detection report
  7. release pseudonymised file to the AI tool
  8. map results back (optional, internal only)
  • Detection: rules for IBAN/phone/e-mail plus NER for names and places. Replacement: same input gives the same token per project; use a secret key, not an unsalted hash (guessable for e-mail addresses).
  • Human in the loop: review a sample (e.g. 100 rows) and all uncertain findings before first release.

Data preparation

Which data: the export you actually want to give to AI, nothing more. Start with a purpose statement ("analyse complaint reasons") and select only needed columns.

Where it lives: CRM/ticket tools (HubSpot, Zendesk, Odoo), DATEV/ERP exports, Microsoft 365 or Google Workspace mailboxes, Excel on shares.

Typical gaps: names in free text and signatures, quoted reply chains, inconsistent spellings, and harmless fields that identify someone in combination.

Steps (1-3 days for the first dataset): (1) mark identifiers per column, using PREP-003 and PREP-005; (2) decide per column: drop, replace, generalise, keep; (3) run detection on 200 rows and tune German patterns (phone, IBAN, Steuer-ID); (4) store the mapping table separately with restricted access; (5) document in the processing records (Art. 30) and AI rules (PREP-024, PREP-025).

Tools: Excel/Power Query, Python with Microsoft Presidio, n8n or Power Automate for scheduled runs, Nextcloud or SharePoint for restricted storage. See GUIDE-003.

Implementation paths

Which path for whom

  • SMB — Path C: customer, HR and trade-secret data is sensitive, and a simple open-source tool (Presidio plus spaCy) runs on a normal CPU server or laptop (HW1). One small service also serves several later use cases (ticket analysis, RAG, reporting), which gives a Baukasten effect.
  • Enterprise — Path C: a shared internal anonymisation service in front of all AI platforms is easier to govern than many ad-hoc scripts; commercial data-masking suites are an alternative if already licensed.
  • Developer — Path C: the libraries are open source and embed into pipelines and test-data workflows.

Path A — Buy (off-the-shelf)

Microsoft Purview (sensitivity labels, DLP) or Google Cloud Sensitive Data Protection mainly block or flag sensitive data; they are less suited to producing consistent tokens for a dataset. Sensible if already licensed. Check EU hosting and AVV (GUIDE-010).

Path B — Build on cloud APIs

Azure AI Language PII detection, Google Sensitive Data Protection or AWS Comprehend (EU regions) can be called from n8n or Power Automate. Catch: raw data goes to the cloud before protection, so the AVV and legal basis are needed for that step. Rarely right for the most sensitive data.

Path C — Self-host (open source)

  • Microsoft Presidio (analyzer + anonymizer, REST API, custom recognizers) [3], spaCy or transformer NER models with German support, optional Ollama with a small model (about 7-8B) for second-pass review.
  • Orchestration: n8n self-hosted (for example in an EU data centre such as Hetzner or IONOS) or a scheduled Python job; files on Nextcloud.
  • Hardware: HW1; a CPU server handles tens of thousands of rows per hour without a GPU.
  • Ops effort: a few hours per month (pattern updates, spot checks).

Path D — Custom training

Only for domain-specific entities (patient IDs, project codes): fine-tune a small NER model on a few hundred annotated examples (PREP-018).

State of the art (2026)

  • Hybrid pipelines (rules plus NER plus optional small LLM) are the practical standard; German free text and addresses are often weaker than English, so test on your own data.
  • The EDPB Guidelines 01/2025 on pseudonymisation (consultation version adopted January 2025; check for the final text) describe pseudonymisation as a safeguard, stress that "pseudonymisation secrets" (keys, mapping tables) must be kept separate and protected, and that pseudonymised data stays personal data [2].
  • Limits and failure modes: missed names (false negatives) are the real danger; over-masking (false positives) destroys analytic value; re-identification through combinations of attributes; token consistency across files can itself enable linking; LLM-based redaction can hallucinate or leak when run in the cloud.

Shared resources & combinations

Effort, cost & KPIs

  • Path A: days. Path B: 1-2 days plus API costs and DPA work. Path C: about 2-5 days to a tuned pipeline, then hours per month; a small VM costs tens of euros per month.
  • KPIs: recall on a manually labelled sample (target agreed with the DPO, typically above 95 % for direct identifiers), share of datasets passing through the pipeline vs. pasted raw, number of shadow-AI incidents, hours saved on manual redaction.

People & change

Involve early: DPO (defines sufficient and signs off), IT security (keys, mapping table), data owners, department leads.

Works council: depends. The tool itself processes customer and ticket data, but tickets and e-mails also contain employee names and handling times; if employee data is in scope or the pipeline logs who exports what, §87(1) no. 6 BetrVG co-determination can apply (objective suitability for monitoring is enough). §90(1) no. 3 requires timely information when work processes are planned with AI. Pseudonymisation usually makes agreement easier because it protects staff too (GUIDE-004). Public bodies involve the Personalrat.

Data protection: a DPIA (Art. 35) is likely only if downstream AI evaluates people at scale or uses special categories; AVV for every cloud AI tool receiving the output (GUIDE-005).

AI literacy: Art. 4 AI Act requires measures that support staff's AI literacy; teach staff what pseudonymisation does and does not guarantee, and which data may never be pasted raw (GUIDE-006, GUIDE-007).

Pilot: offer the tool as the easy approved route, not a ban; pilot with one team and 1-2 datasets for four weeks (GUIDE-008, GUIDE-009).

Humans in control: a person reviews the detection report before the first release of each new data type.

Risks, regulation & compliance

  • EU AI Act: the pre-processing step is minimal risk; the downstream AI use case determines the class (as amended by Regulation (EU) 2026/1744, Annex III high-risk duties apply from 2 Dec 2027; for example HR use cases may be high-risk) [4]. See GUIDE-014.
  • GDPR: pseudonymised data remains personal data; legal basis, purpose limitation, AVV and records of processing still apply [1][2]. Truly anonymised data falls outside GDPR but is hard to achieve for free text.
  • Security: protect the mapping table like a password database; guard any LLM step against prompt injection (GUIDE-013).
  • Mitigations: sample review, recall tests, retention limits for mappings.

Getting started

  1. Pick one concrete dataset (for example last year's support tickets) and one purpose.
  2. Agree with the DPO what level of detection recall is required and who may hold the mapping table.
  3. Install Presidio (Docker) on a small server, test it on 200 rows and add German-specific patterns.
  4. Wrap the pipeline in an n8n flow or script with a review step and a detection report.
  5. Publish the "what may go into which AI tool" rule (PREP-024) and point staff to this pipeline.

Sources

  1. Regulation (EU) 2016/679 (GDPR), Art. 4(5), 25, 28, 35 — https://eur-lex.europa.eu/eli/reg/2016/679/oj
  2. EDPB — Guidelines 01/2025 on Pseudonymisation (version for public consultation) — https://www.edpb.europa.eu/our-work-tools/documents/public-consultations/2025/guidelines-012025-pseudonymisation_en
  3. Microsoft Presidio — documentation and source — https://microsoft.github.io/presidio/
  4. Regulation (EU) 2024/1689 (AI Act) — https://eur-lex.europa.eu/eli/reg/2024/1689/oj
  5. n8n documentation (self-hosting) — https://docs.n8n.io/hosting/