Family historians spend most of their time deciphering handwriting, guessing at spelling variants and checking whether two records describe the same person. AI now transcribes old scripts, translates Latin and historic German, proposes record links and flags impossible trees. The main risk is not effort but trust: language models invent plausible ancestors, so every statement needs a cited source.
Problem & value
- Today: hobby researchers browse indexes (FamilySearch, Ancestry, MyHeritage, Archion, Matricula, regional archives), read church books and civil registers by eye, keep trees in desktop software and often copy unverified branches from other users' trees.
- AI value: (1) handwritten text recognition (HTR) makes scans searchable and readable, including Kurrent/Sütterlin, Latin and old French; (2) fuzzy name and place matching across spelling variants and renamed places; (3) consistency checks on dates and relations; (4) translation and readable summaries; (5) narrative writing for family books.
- Benefit evidence: providers publish HTR accuracy figures per model, but they depend strongly on hand and scan quality. No reliable cross-vendor time-saving figures exist; test on your own pages [2][3].
Variants
- Casual hobbyist: AI features inside a tree platform (hints, record summaries, photo restoration).
- Serious researcher / local historian: HTR training on one scribe or parish, own database, citation discipline (Evidence Explained style).
- Professional genealogist / heir finder: same tools plus client confidentiality; heir tracing touches legal rules (see Risks).
- Archives and societies: crowdsourced transcription with AI pre-fill, see DOC-005 and KNOW-019.
- Photo and oral history: old photos sorted and captioned (PERS-006), interviews transcribed with ASR.
How it works
- scans / photos / interviews
- image cleanup + layout analysis
- HTR / OCR (script-specific model)
- raw transcription with confidence
- LLM: normalise, translate, extract persons, events, places (structured JSON)
- candidate linking against own tree + index search (fuzzy names, phonetic, place gazetteer)
- reasoning check: dates, ages, sequence of births, same-name pitfalls
- human review: accept/reject each link, cite source
- GEDCOM / tree software, narrative export
- HTR: line-based neural recognition trained on pages by the same hand; dedicated tools beat general chat models on difficult scripts, while frontier vision-language models are good on clear 19th–20th century hands and for translation.
- Linking: record-linkage with embeddings plus rules (name variants, birth-year windows, place hierarchy). Genealogical conclusions need at least one original source per fact.
- Human-in-the-loop: every proposed identity merge and every transcription of a name or date is confirmed by the researcher; the LLM never writes into the tree unreviewed.
Data preparation
Genealogy data is the most preparation-heavy part of this use case. Sources are scattered: shoebox photos, certificates, church books, handwritten letters, old family trees in GEDCOM files, Excel lists, e-mails from relatives, and scans from archives. Typical gaps: low-resolution scans, old German script (Kurrent, Sütterlin) and Latin entries, inconsistent name spellings and place names, duplicate persons across trees, missing source citations, and no distinction between living and deceased persons.
Preparation steps: - Digitise paper at 300 to 600 dpi in lossless formats and name files consistently (see PREP-006, PREP-007). - Run OCR or handwriting recognition (Transkribus for old scripts, Tesseract or modern VLMs for print) and always keep the original image next to the transcript. - Consolidate trees into one GEDCOM and clean duplicates and place names against a gazetteer (PREP-011 for tabular lists). - Record the source for every fact; add a tag for living persons so they are excluded from uploads. - Define a naming and folder scheme before importing (PREP-010).
Effort share is high, around 50 to 70 percent of the project: scanning and checking transcripts takes far longer than the AI queries. Budget time for manual verification of every transcription, since models invent plausible names and dates.
Implementation paths
Which path for whom
- Private: Path A. Tree platforms (Geneanet, MyHeritage, FamilySearch) bring record databases and hints that no home setup can replicate.
- Developer: Path C. Gramps or a similar open-source tree plus local OCR and an Ollama model keeps living relatives' data private and avoids platform lock-in of years of research.
Path A — Buy (recommended for private users)
- Tree platforms: Ancestry (US), MyHeritage (Israel; offers photo animation and colourisation), FamilySearch (non-profit, free), Geneanet (France, EU), Findmypast (UK). Their AI features: record hints, summaries, AI-assisted indexing of images.
- HTR service: Transkribus by READ-COOP SCE (Austria-based cooperative, EU) with public and trainable models, plus eScriptorium (open source) hosted by institutions [2].
- Local software: Gramps (open source), RootsMagic, Ahnenblatt and Family Tree Maker with GEDCOM import/export.
- Choose A when: you want results within days and accept that tree and DNA data sits with the provider. Check the privacy terms, especially for DNA.
Path B — Build on cloud APIs
- Multimodal LLM APIs (Claude, Gemini, GPT, Mistral as EU provider) for transcription, translation and extraction with JSON schema outputs; Mistral OCR or Azure AI Document Intelligence for printed pages.
- Orchestration: a small script or n8n; tree as GEDCOM or Gramps database; optional MCP server giving an agent read access to the tree and archive finding aids.
- Use EU regions and no-training settings; send only the page, not the whole tree, whenever possible.
- Choose B when: you have hundreds to thousands of pages in standard hands and some scripting skill.
Path C — Self-host (open source)
- HTR/OCR: Kraken with eScriptorium, PyLaia, TrOCR-style models, Tesseract (weak on handwriting), Docling or PaddleOCR-VL for printed pages [3].
- LLM/VLM: open-weight Qwen-VL or Mistral/Llama-family models via Ollama or vLLM for translation and extraction; embeddings via BGE or multilingual-e5; Gramps or a graph database for the tree.
- Hardware: HW2 is enough for batch HTR and a 7–32B model; a laptop with HW1 handles Tesseract-class tasks. Runs locally, which suits sensitive family and health information (see PERS-005).
Path D — Custom training
Worth it for a whole parish or region in one hand: fine-tune an HTR model on 50 to a few hundred hand-corrected pages, then correct and retrain iteratively. Transkribus and Kraken support this workflow without ML expertise.
State of the art (2026)
- HTR for historic European scripts is usable for many hands; typical results are searchable text with errors that still need proofreading. General vision-language models read many clear hands without training but miss rare names and can silently "normalise" them.
- Hallucinated ancestry: LLMs invent parents, dates and sources when asked open questions. Constrain them to supplied documents and require quotations from the source image.
- Record linkage by embeddings plus rules finds spelling-variant matches that exact search misses; precision is limited with common names (Johann Müller).
- DNA: matching is done by the testing companies; AI helps to organise clusters, but results are probabilistic.
- Photo AI: restoration, colourisation and animation are common but alter evidence; label results as synthetic (AI Act Art. 50 applies to deepfake-like output published to others).
- Limits: poor scans, damaged pages, abbreviations, and local naming customs (patronymics, fixed given-name sets) cause systematic errors.
Shared resources & combinations
- Same stack (ocr-docparse + vlm + llm-general): DOC-005 (historical handwriting transcription), KNOW-019 (archive discovery).
- PERS-006 photo library search, PERS-009 family organisation, PERS-017 lifelogging memory, PERS-005 local home-server assistant (same GPU), PERS-016 to exercise GDPR rights against tree platforms.
- Embeddings and vector index are reusable for the family document archive.
Effort, cost & KPIs
- A: subscriptions, mostly per month for premium record access; HTR services usually use credits per page.
- B: a weekend for a first pipeline; API cost per page is small.
- C: one GPU PC; days of setup, minimal upkeep.
- KPIs: character/word error rate on a held-out sample of pages, share of links confirmed by sources, number of conflicts found, sourced facts per person.
People & change
The "team" is the researcher plus relatives. Involve relatives early: ask who has photos, letters and certificates, and who does not want their data in an online tree. Living persons have GDPR rights even in a private project, and publishing a tree on a platform leaves the household exemption. Agree rules with the family: living relatives are hidden by default, photos of children are not uploaded, DNA results (special category data) are never shared without consent.
Works council: not applicable (private use). A DPO and DPIA are not required for purely private research, but if a researcher works professionally (a genealogist, archive or society), AVV with tools (GUIDE-005) and a privacy notice become necessary.
AI literacy: the Act only asks providers and deployers to support literacy, so for a hobby a short checklist suffices: treat AI-suggested ancestors as hypotheses, verify against primary sources, and label AI-generated transcriptions and colourised photos as such (Art. 50 transparency applies to deepfake-like image content published by deployers).
Typical concerns: "is my family data being sold?" Check the platform terms and DNA sharing options (GUIDE-010 helps with vendor questions). Keep humans in control by recording a confidence level for every AI-derived fact and reviewing before merging into the main tree. Acceptance tips: GUIDE-008.
Risks, regulation & compliance
- EU AI Act: minimal risk for private use. Face recognition on family photos to identify people is biometric processing; household use is outside the AI Act's deployer duties as purely personal non-professional activity, but a provider operating it commercially must check Art. 5 and Annex III. Publishing synthetic or altered images: Art. 50 transparency.
- GDPR: the Regulation does not protect deceased persons (Recital 27), but family trees include living relatives; the household exemption (Art. 2(2)(c)) ends once a tree is published online, at which point controller duties apply. Genealogy platforms and DNA companies process health-related and genetic data (Art. 9); give consent deliberately and check export and deletion options.
- Archive rules: Germany protects civil register entries by retention periods (births 110 years, marriages 80, deaths 30 after entry, per Personenstandsgesetz) and archive laws; many other EU states have comparable rules. Reproduction terms of archives and image licences apply.
- Cloud upload: scans contain names of living people and sometimes health or religious data; use EU providers or local models for sensitive material.
- Quality risks: fabricated lineages, wrong merges of namesakes. Mitigate with source citations, confidence labels, and keeping originals.
- Heir tracing/ legal use: results are not evidence in inheritance matters without certified documents.
Getting started
- Digitise: scan at 300–400 dpi and name files by source and page.
- Put the existing tree into Gramps or export GEDCOM from your platform.
- Try Transkribus (or a VLM) on 10 representative pages; measure error rate and fix a few.
- Make a rule: the AI proposes, you cite. Add a source note to each fact.
- Review privacy settings for living people before sharing the tree.
Sources
- Regulation (EU) 2024/1689 (AI Act) — https://eur-lex.europa.eu/eli/reg/2024/1689/oj
- Transkribus (READ-COOP) — https://www.transkribus.org/
- Kraken OCR/HTR engine — https://github.com/mittagessen/kraken
- eScriptorium — https://escriptorium.readthedocs.io/
- FamilySearch — https://www.familysearch.org/
- Gramps open-source genealogy — https://gramps-project.org/
- GDPR (Regulation (EU) 2016/679), incl. Recital 27 and Art. 2(2)(c) — https://eur-lex.europa.eu/eli/reg/2016/679/oj
- GEDCOM specification — https://gedcom.io/
- Personenstandsgesetz (PStG), §5 retention periods — https://www.gesetze-im-internet.de/pstg/