Archive, library & cultural-heritage collection discovery

Archives, libraries and museums hold huge collections whose catalogue records are sparse, written in one language and organised by provenance rather than by the questions people ask. AI turns handwritten and printed pages into searchable text, embeds records and images in a shared semantic space, and lets users ask natural-language questions across languages, with answers that cite the exact record. The professional (archivist/librarian) stays the authority for description and access.

Problem & value

• Today: discovery relies on hierarchical finding aids, keyword search and reading-room expertise. Most digitised material is images without text; many items have only a series-level description. Users must know the right terms, spelling variants and historical place names.
• AI value: full-text search inside handwritten and printed pages, concept-level (semantic) and cross-language retrieval, search by example image, automatic proposals for subjects, persons and places, and conversational access with citations.
• Scale (examples, from the portals themselves): Archivportal-D lists about 270 archives, more than 170,000 holdings and over 36 million objects [3]; Europeana gives access to items from over 2,000 European institutions, with roughly 34 million image and 25 million text items listed [4]; Transkribus reports 200M+ pages transcribed [1]. Such volumes cannot be described manually.
• No audited cost-saving figures are cited here; institutions should measure their own baseline (see KPIs).

Variants

• State/municipal/church archives: finding-aid search plus HTR of registers, minutes, parish books.
• Libraries: semantic catalogue search, discovery over digitised newspapers and early prints.
• Museums/galleries: image and text search over object records (LIDO), "find similar object".
• Corporate archives & Mittelstand: company history, brand/product archives, technical drawings (see DOC-013) searchable for marketing, patents and service.
• Genealogy/private use: family historians searching registers; local history societies.
• Aggregators: Europeana / Deutsche Digitale Bibliothek style portals with multilingual retrieval [3][4][5].
• Handwriting transcription itself is DOC-005; this use case is the discovery layer on top.

How it works

1. collections (finding aids, catalogue records, scans, audio/video)
2. capture: HTR/OCR + layout parsing (ALTO / PAGE XML / TEI) for scans
3. normalise: map to standard schemas (EAD3, RiC-O, LIDO, EDM) and IIIF manifests
4. enrich: NER (persons, places, dates), authority linking (GND, Wikidata), translation, subject proposals
5. index: hybrid BM25 + dense embeddings (text and page/image), metadata filters, rights/access flags
6. retrieve: query rewrite
7. hybrid search
8. reranker
9. access-rights filter
10. present: result list with highlighted region in IIIF viewer; optional RAG answer with citations
11. feedback: archivist reviews AI-suggested metadata; corrections flow back

• Retrieval: hybrid lexical + semantic is essential because historical spelling, OCR/HTR noise and proper names defeat pure embeddings. Page-image retrievers such as ColPali-style multi-vector models embed page images directly and avoid brittle OCR pipelines for layouts with tables or stamps [7].
• Standards: Records in Contexts (RiC-O v1.1, an OWL ontology for archival description as linked data) [6], IIIF for image delivery [8], Europeana Data Model for aggregation [4].
• Human-in-the-loop: AI-generated descriptions are labelled and queued for professional approval; access restrictions (closure periods, personal-data protection periods) are enforced before retrieval.

Data preparation

This is the most data-heavy use case of the batch. The material is catalogue records and finding aids (archive or library system, often old databases or Word and Excel lists), scans and microfilm, audio and video, and in small institutions paper registers. Files typically live in the collection management system, on file shares and on external drives.

Typical gaps: no or incomplete digitisation, scans without text layer, HTR needed for handwriting, inconsistent catalogue schemas, duplicate and variant spellings of names, missing rights and protection-period flags, and personal data in non-public records.

Preparation steps: - Inventory holdings and decide a digitisation priority (PREP-001, PREP-006). - Scan to a standard (e.g. TIFF/JPEG2000 with IIIF delivery) and run OCR/HTR with Transkribus or Kraken; keep ALTO/PAGE XML. - Map catalogue data to EAD3, RiC-O or LIDO and clean authority names against GND (PREP-010, PREP-012). - Add access-rights flags before indexing (PREP-023, PREP-025). - Build a test set of real user questions with archivist-judged relevance (PREP-018); related: DOC-005, DOC-014, DOC-018.

Data preparation is the largest cost block, typically 50 to 70 percent of the total effort, because digitisation and quality control scale with collection size.

Implementation paths

Which path for whom

• Public: Path B on a sovereign EU or German cloud, because archives and libraries need custom portal integration and public procurement rules favour EU hosting; non-public records move to Path C.
• Enterprise: Path B, because company archives and media libraries need integration with existing DAM/DMS and a managed, EU-hosted API stack keeps ops effort low.
• Developer: Path C, because Kraken/eScriptorium, Docling and open retrieval models give full control and can be reused for institutions with restricted holdings.
• Private: Path A, because tools such as Transkribus make family and local-history transcription possible without technical skills.
• SMB: Path A, because small archives, societies and companies need searchable transcriptions fast and Transkribus (Austria, EU) is a mature product at modest prices.

Path A — Buy (off-the-shelf; recommended for small archives and societies)

• Transkribus (READ-COOP, Austrian cooperative, EU): HTR in 100+ languages, full-text search across collections, public sites; data processed on its own servers in Austria; API from the Scholar plan [1]. Strong default for handwritten material.
• Archive/library software with search: archive management systems (e.g. Arcinsys, FAUST, AUGIAS, scopeArchiv), library systems and discovery layers (e.g. Ex Libris Primo, EBSCO Discovery, VuFind-based), museum systems (e.g. museum-digital, MuseumPlus). Check roadmaps for AI search, EU hosting and EAD/LIDO/IIIF export.
• Enterprise search with RAG: general platforms (KNOW-001/KNOW-002) can index exports, but usually lack hierarchy-aware (fonds/series/item) retrieval and access-period handling.
• Choose A when: a small archive or society wants searchable transcriptions fast, without developers.

Path B — Build on cloud APIs (recommended for institutions with developers)

• Models: an EU-region multilingual embedding model and reranker, an LLM with structured output and citations (e.g. Mistral, Claude or Gemini on EU regions; Azure OpenAI EU data zones) for query rewriting, answer generation and metadata proposals; a VLM for photos and page layout.
• Index: OpenSearch/Elasticsearch hybrid, or Qdrant/Weaviate/pgvector with BM25 side by side.
• Pipeline: transcription via Transkribus API or a cloud document-AI service, then enrichment and indexing; expose results through the existing portal and IIIF viewer (Mirador, Universal Viewer).
• Choose B when: a mid-sized institution or aggregator has developers or an integrator and needs custom ranking and portal integration. Fully public, already-published material has few data restrictions; non-public records need a processing agreement and EU hosting.

Path C — Self-host (open source)

• Capture: Kraken with eScriptorium (trainable layout and HTR for historical and non-Latin scripts, Apache-2.0, ALTO/PAGE XML output) [2]; Docling (MIT) for modern PDFs [9]; open VLM-based OCR models.
• Retrieval: open multilingual embedding models (BGE-M3-class, multilingual-e5) with rerankers; ColQwen-family page-image retrievers (Apache-2.0 variants exist; check each model licence) [7].
• Serving: vLLM/Ollama with an open-weight LLM (Qwen, Mistral, Llama families); OpenSearch or Qdrant; Cantaloupe or similar IIIF image server; Mirador/Universal Viewer.
• Hardware: HW3 (one 48–141 GB GPU) for HTR batches, embeddings and a 30–70B LLM; HW2 suffices for retrieval-only deployments with a smaller LLM. Initial bulk transcription can be rented as a time-limited job.
• Choose C when: non-public records (personnel files, police or church records, restricted holdings), sovereignty requirements or very large volumes make cloud processing unacceptable.

Path D — Custom training

Worth it for HTR on a specific hand, script or period: train or fine-tune a recognition model on transcribed ground truth (typically a few dozen to a few hundred pages for a first usable model; more for difficult hands, no guarantee). Second target: domain-adapted embeddings for historical language and controlled vocabularies, using click logs and archivist relevance judgements. Community model repositories (Transkribus models, Kraken model repository) reduce the need [1][2].

State of the art (2026)

• Foundation of discovery is now recognition quality: HTR/OCR for historical documents is production grade for many scripts but varies strongly with hand, script and scan quality; always keep confidence scores and the page image next to the text.
• Page-image retrieval (ColPali family) and VLM-based parsing let institutions search layouts, stamps, maps and tables without perfect OCR [7].
• Hybrid retrieval + reranking + metadata filters outperforms pure vector search on historical language; hierarchical RAG that respects fonds/series/item context is an active practice area.
• Linked-data modelling is moving from EAD to RiC-O/RiC-CM (v1.1 published 2025) so that AI-extracted relations can be stored as a graph (KNOW-013) [6].
• Limits and failure modes: hallucinated provenance or dates in generated summaries; bias toward well-described, majority-language material; HTR errors on names (the key search terms); embedding models trained on modern language; colonial or outdated terminology in legacy descriptions being reproduced or "corrected" without context; cost and time of reprocessing millions of pages. I could not verify independent benchmark numbers for the 2026 generation of tools and therefore give none.

Shared resources & combinations

• Same capture stack: DOC-005 (handwriting recognition & historical transcription), DOC-014 (document parsing for RAG) — one GPU server (HW3) serves all.
• Same retrieval stack: KNOW-001 (enterprise semantic search), KNOW-002 (RAG over documents), KNOW-016 (multimodal image/video/audio archive search) share embeddings, vector DB, reranker and IIIF/DAM layer.
• Enrichment: KNOW-013 (knowledge graph / GraphRAG) for person–place–event networks; LANG-009 (translation) for cross-language access; DOC-018 (anonymisation/redaction) before publishing records with protected personal data.
• Hardware: embedding and reranker models plus an LLM on one HW3 node; batch HTR on the same GPUs outside opening hours.

Effort, cost & KPIs

• Path A: days to weeks to start; subscription/credit-based, cost driven by pages processed [1].
• Path B: typically 3–9 months for a production portal integration; cost drivers are bulk transcription, embedding of the full corpus (one-off) and per-query LLM cost.
• Path C: 6–12 months with in-house or partner team; GPU server plus operations; main cost is integration with archive systems and quality control, not hardware.
• KPIs: share of collection with full text; HTR character/word error rate on a held-out sample; recall@k and nDCG on a test set of real user questions with archivist-judged relevance; zero-result rate; citation correctness of generated answers; share of AI metadata proposals accepted; time to describe an item; reading-room and online request volume.

People & change

Who and when: Management and the department lead (head of archive or library) agree scope and priorities at the start. Archivists and librarians as end-users define metadata rules and review AI-proposed descriptions from the first pilot. IT builds the pipeline; the DPO and legal-compliance clarify access periods, copyright and personal data before any indexing of non-public records. Data owners (donors, depositors, source institutions) must be heard where contracts restrict use.

Works council: Usually no, because the system does not monitor staff; review if reading-room or staff search logs are evaluated per person (§87(1) no. 6 BetrVG). In public institutions the Personalrat is informed in good time of changed work processes (§90(1) no. 3 BetrVG analogue under BPersVG/state law).

Data protection: Personal data in non-public records is likely, so a DPIA (Art. 35 GDPR) is advisable, an AVV (Art. 28 GDPR) is mandatory with every processor, and protection periods must be enforced before retrieval.

AI literacy: Art. 4 AI Act requires measures that support staff AI literacy; train archivists on HTR errors and hallucinated citations (GUIDE-006, GUIDE-005).

Communication and pilot: Pilot with one well-described fonds and a small group of volunteers (GUIDE-009, GUIDE-008).

Concerns: "Will AI replace professional description?" No: AI proposals are labelled and approved by professionals, who stay in control of what becomes official metadata.

Risks, regulation & compliance

• EU AI Act: typically minimal risk — a search and description tool is not an Annex III use case. A chatbot over the collection triggers Art. 50 disclosure (user must know they interact with AI); AI-generated descriptions published to the public should be labelled [10]. Using the same tools for e.g. profiling persons from police or intelligence-service archives would change the picture (law-enforcement/Annex III) and is out of scope.
• GDPR: archives hold personal data (registers, personnel, court, health files, including special categories). Archiving in the public interest has privileges (Art. 89 GDPR and national archive laws), but access, protection periods (e.g. 30 years after death or 110 years after birth in many German archive laws; check the applicable federal/state law) and rights of third parties still apply. Enforce access rules in the index, not only in the front end; no sending of restricted records to non-EU APIs.
• Copyright and rights: orphan works, database rights and licence conditions (rights statements such as RightsStatements.org) must be carried through to results; check the EU copyright text-and-data-mining exceptions before training or fine-tuning models on third-party collections.
• Professional ethics & provenance: AI output must be distinguishable from archivist-created description; keep audit trail of who/what created each statement (archival authenticity).
• Mitigations: access-filter before retrieval, confidence display, human approval of enrichment, red-team queries on sensitive collections, EU/on-prem processing for non-public records.

Getting started

1. Pick a bounded pilot collection (e.g. one fonds or newspaper title) with mixed handwritten and printed material and a known set of 50–100 real user questions.
2. Digitise or export to IIIF + ALTO/PAGE XML; run HTR/OCR with Transkribus or Kraken and measure error rate on a hand-checked sample.
3. Build a hybrid index (BM25 + embeddings + reranker) with access-restriction metadata, and compare it with the current search on the question set, judged by archivists.
4. Add a cited-answer layer and AI-suggested metadata only after retrieval is good; label all AI output.
5. Document the legal basis, retention and access rules, then plan scale-up and integration with the archive information system.

Sources

1. Transkribus (READ-COOP) — https://www.transkribus.org/
2. Kraken OCR/HTR engine — https://github.com/mittagessen/kraken
3. Archivportal-D (Deutsche Digitale Bibliothek) — https://www.archivportal-d.de/
4. Europeana — About us — https://www.europeana.eu/en/about-us
5. Deutsche Digitale Bibliothek portal — https://www.deutsche-digitale-bibliothek.de/
6. ICA Records in Contexts Ontology (RiC-O v1.1) — https://www.ica.org/standards/RiC/ontology
7. ColPali (page-image document retrieval) — https://github.com/illuin-tech/colpali
8. IIIF — Why IIIF — https://iiif.io/get-started/why-iiif/
9. Docling — https://github.com/docling-project/docling
10. EU AI Act Article 50 (transparency) — https://artificialintelligenceact.eu/article/50/