Catalog / Data Readiness for AI / PREP-013

Curating an authoritative knowledge base for RAG assistants

Select, de-duplicate, fix and sign off the documents an AI assistant may answer from, so that an SMB assistant quotes current, approved content instead of outdated or contradictory versions.

Parts needed to run it yourselfAgents & automationDocument readingLanguage modelSearch & RAG

A retrieval-augmented assistant (RAG) is only as good as the documents behind it. If the file share holds five versions of the travel policy, an assistant will quote whichever one it finds first. Curation is the human-led step before any chatbot goes live: decide which documents count, remove outdated and contradictory versions, fill gaps and let named owners sign off a "golden" set. It is mostly an organisational task, supported by small automations, and it decides answer quality more than the choice of model.

Problem & value

  • Today: knowledge sits in file shares, SharePoint, mailboxes and personal folders. Old drafts and superseded price lists are never deleted. Staff know which document is right; a search index does not.
  • AI value: a small, curated, owned source set lets an assistant give consistent answers with a citation, reduces repeated questions to colleagues and makes wrong answers traceable to one owner.
  • Why it matters: retrieval treats every indexed file as equally true, and Microsoft 365 Copilot grounds on content the user can already access [1]. Better 200 verified documents than 20,000 unverified ones.

Variants

  • Single-topic library: HR handbook, IT helpdesk, product manuals, quality procedures - start here.
  • Customer-facing knowledge base: stricter sign-off because answers reach customers.
  • Public sector: regulations, service descriptions and circulars, with the staff council involved.

How it works

  1. scope the questions (what must the assistant answer?)
  2. inventory candidate sources (PREP-001, PREP-009)
  3. triage: keep | update | merge | archive | delete (owner decides)
  4. detect duplicates & near-duplicates, find contradictions
  5. fill gaps (interview experts, write short answers)
  6. owner sign-off, version + valid-until date, status "approved"
  7. publish to the assistant's source library only
  8. feedback loop from wrong answers (PREP-017)
  • AI helps, humans decide: embeddings or hashing find near-duplicates; an LLM can list pairs of documents that disagree (e.g. different notice periods) and draft a summary of each. The owner decides what is true. No document goes live without a human sign-off.

Data preparation

Where it lives: file shares, SharePoint/OneDrive, Google Drive, Nextcloud, Paperless-ngx or another DMS, the ticket system, mailboxes and paper binders.

Typical gaps: several versions of one document, no owner or date, contradictory rules between departments, scans without text layer, badly parsing tables, personal data mixed into the same folders, knowledge only in people's heads.

Preparation steps: 1. Write down 30-50 real questions the assistant should answer; they define the scope (PREP-018). 2. Export a file list with owner, date and path (Excel or PowerShell is enough, PREP-005). 3. Cut hard: archive everything older than your review cycle unless an owner confirms it. 4. Find duplicates (file hash, then embedding similarity) and have owners pick the master. 5. OCR relevant paper (PREP-006); fix badly parsing tables and headings (PREP-014). 6. Fill gaps: experts write short, dated answers (PREP-016). 7. Exclude personal data, HR files and trade secrets (PREP-023, PREP-022). 8. Set owner, version and review date per document (PREP-010, PREP-017).

Tools: Excel/Power Query for the triage list, SharePoint or Nextcloud metadata columns, Paperless-ngx (tags, correspondents, OCR), Power Automate or n8n for reminders and status flags, an embedding model via Ollama for duplicate detection.

Rough share of effort: 60-70 percent of the total assistant project is realistic; the AI configuration is the smaller part. Mostly owner time, not IT time. See GUIDE-003.

Implementation paths

Which path for whom

  • SMB - Path C: the content is often sensitive (HR rules, price calculations, customer terms) and the tooling is light: Paperless-ngx or Nextcloud, n8n and a small Ollama model run on HW1/HW2 and serve several PREP use cases (Baukasten effect). Path A is a valid alternative if the company lives in Microsoft 365 and accepts the tenant boundary.
  • Enterprise - Path B: many sources and owners; a governed pipeline on cloud APIs in an EU region with the central AI platform and DMS scales better than manual lists.
  • Developer - Path C: open-source ingestion and deduplication libraries give full control of the pipeline.
  • Public - Path C: documents include internal and citizen-related content; sovereign or German-cloud hosting and procurement rules favour self-operation.

Path A - Buy (off-the-shelf)

  • Built into the suite: SharePoint with Copilot Studio knowledge sources and Microsoft Purview sensitivity labels; Google Workspace with Gemini and NotebookLM-style source sets; Confluence/Notion with AI search. Microsoft offers the EU Data Boundary for customer data [2]. Curation functions are mostly document libraries, versions, approval workflows and labels - not AI.
  • Knowledge-base products: Guru, Document360; EU-based options: Nextcloud, d.velop, DocuWare. Check approval workflows and review reminders.
  • Choose A when: the company is fully inside one suite, content is not highly sensitive and a library owner exists.

Path B - Build on cloud APIs

  • n8n or Power Automate with embeddings and LLM via EU-hosted APIs (Azure OpenAI in an EU region, Mistral, IONOS AI Model Hub, STACKIT AI Model Serving) for duplicate and contradiction hints.
  • Choose B when: you need custom approval logic or many sources and have developer capacity.

Path C - Self-host (open source)

  • Document store: Paperless-ngx (OCR, tags, workflows) or Nextcloud; wiki such as BookStack, Outline or Wiki.js for written answers.
  • Parsing: Docling, Apache Tika or Unstructured for converting PDFs and Office files [3].
  • Deduplication and hints: hashing, MinHash or embeddings with a multilingual model (e.g. bge-m3 or multilingual-e5) through Ollama; a small LLM (Qwen, Llama, Mistral families) to summarise differences between two versions.
  • Workflow: n8n self-hosted for status flags, owner reminders and export to the vector store (Qdrant, pgvector).
  • Hardware: HW1 handles embeddings and batch triage overnight; HW2 gives faster contradiction checks.

Path D - Custom training

Almost never. Fine-tuning does not replace curation; stale facts in model weights are harder to remove than a file in a library.

State of the art (2026)

  • Hybrid search plus reranking is standard, but nothing repairs contradictory sources.
  • Suites now offer curated knowledge sources and label filters, so sign-off is the real control point.
  • LLMs can reliably flag candidate duplicates and conflicts; they miss conflicts that need domain context (exceptions, local rules) and may invent a conflict. Treat output as hints.
  • Limits: curation does not stay done. Without owners and review dates, quality decays within months.

Shared resources & combinations

Effort, cost & KPIs

  • Path A: 1-3 weeks for a first library of 50-200 documents, mostly owner time (estimate: 10-20 minutes per document; verify in your pilot).
  • Path B: 3-6 weeks incl. pipeline and approval flow.
  • Path C: 2-4 weeks for a first small setup (1-3 days server and n8n flows), then about half a day per month operations.
  • KPIs: share of documents with owner and review date, number of duplicates removed, share of the 30-50 test questions answered correctly with the right citation, share of answers citing outdated content (target zero), documents overdue for review.

People & change

Involve early: management (scope, owner time), department heads as document owners, IT for exports and permissions, and the data owners of each source. IT security joins for confidentiality classes. The DPO is needed because personal data in minutes, HR files and tickets may end up in the index; an AVV (Art. 28 GDPR) is required with every hosted processor, a DPIA only if employee or special-category data is evaluated at scale (GUIDE-005, GUIDE-010).

Works council: depends. The curated library alone does not monitor staff, but the assistant that uses it usually logs queries per user, which makes it objectively suited to monitor behaviour under §87(1) no. 6 BetrVG (BAG: suitability is enough). §90(1) no. 3 requires timely information when work processes are planned with AI. Agree early on: no individual evaluation of questions asked, short log retention. Public bodies involve the Personalrat (GUIDE-004).

AI literacy: Art. 4 AI Act requires providers and deployers to take measures that support staff's AI literacy. Train document owners in what the assistant can and cannot do, and users in checking citations (GUIDE-006).

Communication and pilot: present curation as "making our knowledge findable", not as audit. Start with one topic, one owner per document and an AI champion (GUIDE-008, GUIDE-009). Typical concerns: extra work for owners (time-box it) and fear of being replaced (experts become the authorities).

Humans in control: only owners approve; the assistant never edits approved content on its own.

Risks, regulation & compliance

  • EU AI Act: minimal risk for an internal knowledge assistant; Art. 4 AI literacy applies. A customer-facing chatbot must disclose that users talk to AI (Art. 50, applying since 2 Aug 2026). If the assistant is used for hiring or employee evaluation, Annex III high-risk duties apply from 2 Dec 2027 (GUIDE-014).
  • GDPR: personal data in minutes, tickets and HR files; purpose limitation and data minimisation argue for leaving such files out; erasure requests must reach the index (PREP-025).
  • Confidentiality: an assistant can surface documents to people who should not see them; fix permissions first (PREP-023, GUIDE-013).
  • Risks: outdated answers, over-trust, sensitive files indexed by accident, prompt injection via external documents (GUIDE-013).
  • Mitigations: approved-only index, owners and review dates, citations in every answer, logging of unanswered questions, quarterly spot checks.

Getting started

  1. Pick one topic and write 30-50 real questions with correct answers.
  2. Export the file list for that topic and mark each document keep, update, merge or archive within one workshop (GUIDE-001, GUIDE-003).
  3. Let owners fix or write the missing answers and sign off, with a review date.
  4. Put the approved set in one library and connect only that library to a pilot assistant (GUIDE-009).
  5. Test with the question list, log wrong answers and feed them back into the library.

Sources

  1. Microsoft - Data, privacy and security for Microsoft 365 Copilot - https://learn.microsoft.com/en-us/copilot/microsoft-365/microsoft-365-copilot-privacy
  2. Microsoft - EU Data Boundary - https://learn.microsoft.com/en-us/privacy/eudb/eu-data-boundary-learn
  3. Docling project - https://github.com/docling-project/docling
  4. Paperless-ngx documentation - https://docs.paperless-ngx.com/
  5. n8n documentation - https://docs.n8n.io/
  6. Regulation (EU) 2024/1689 (AI Act) - https://eur-lex.europa.eu/eli/reg/2024/1689/oj
  7. BSI - Generative AI models: opportunities and risks - https://www.bsi.bund.de/EN/Themen/Unternehmen-und-Organisationen/Informationen-und-Empfehlungen/Kuenstliche-Intelligenz/kuenstliche-intelligenz_node.html