Personal knowledge management ("second brain")

People collect far more than they can find again: notes in five apps, PDFs in folders, bookmarks never reopened. An AI-assisted "second brain" keeps everything in one personal store, indexes it semantically and lets you ask questions of it, get links and summaries suggested, and have forgotten material resurfaced. The decisive design choice is where the data and the model live: a cloud SaaS, a local-first app with an optional API, or a fully self-hosted stack.

Problem & value

• Today: notes sit in apps with keyword search only; manual tagging and linking (Zettelkasten, PARA) take discipline most people do not sustain. Reading highlights are rarely revisited.
• AI value: meaning-based search instead of exact keywords, "ask my notes" with citations to the source note, automatic link/related-note suggestions, summarisation of long sources, and drafting from one's own material.
• Evidence level: adoption is clearly widespread (many mainstream apps ship AI features), but there are no robust independent studies on productivity gains for personal use; treat vendor claims on time savings with caution.
• Scope boundary: this use case is about deliberately written and collected material. Continuous ambient recording is PERS-017; meeting notes are SPEECH-001; shared company knowledge is KNOW-002/KNOW-014; photo libraries are PERS-006; flashcard-based learning is covered under education use cases.

Variants

• Private / student: NotebookLM-style source-grounded notebooks, Obsidian vaults, Apple Notes/OneNote with built-in assistants.
• Freelancer / consultant: client notes, research and snippets; client confidentiality drives the local-first choice.
• Researcher / writer: literature notes with highlights and citations (see KNOW-008), drafts from own corpus.
• Developer: Markdown/Git-based vault, MCP server exposing notes to coding agents and chat clients.
• Small team (SMB): shared workspaces (Notion, Confluence) as lightweight team brain; beyond that see KNOW-002.

How it works

1. capture (typing, web clipper, share sheet, email-in, voice memo
2. ASR, scan
3. OCR)
4. normalise to Markdown/blocks + metadata (source URL, date, tags)
5. chunk + embed (local or API embedding model)
6. vector + keyword index (hybrid)
7. retrieval (hybrid search
8. optional reranker)
9. LLM: answer with citations / summarise / suggest links / draft / weekly review
10. user accepts, edits or rejects suggestions (notes remain the source of truth)

• Source of truth: plain files (Markdown) or an open database, so the index can always be rebuilt and the user is not locked into one vendor.
• Embeddings: multilingual models matter for German/English mixed notes.
• Agentic layer: agents with tool access can create and edit notes, file clippings and run scheduled digests; grant write permissions narrowly and keep version history (Git, sync history) to undo errors.
• Human-in-the-loop: AI suggests tags, links and summaries; the person decides what becomes part of the vault. AI-generated text should be marked so that later retrieval does not mistake it for primary sources.

Data preparation

The data is whatever people already keep: notes in apps, Markdown files, saved web pages, PDFs, voice memos, e-mails and photographed paper. In an SMB it sits in personal folders, OneNote, mailboxes and chat histories, which is exactly the scattered knowledge problem.

Typical gaps: duplicates and half-finished notes, no titles or dates, PDFs and photos without text layer, mixed German and English, notes with customer data that must not leave the company, and no rule on who may see what.

Preparation steps: - Decide on one plain-file source of truth (Markdown folders) and a simple naming and tag convention; do not reorganise everything first, start with the last six months. - Move notes in by export or web clipper; convert old formats (PREP-007). - OCR scans and photos (Paperless-ngx or the app's own OCR) and transcribe voice memos with Whisper (see PREP-006 and DOC-014). - Mark confidential notes and keep them out of cloud indexes (PREP-024). - Write 20 to 30 test questions to check retrieval quality (PREP-018).

Preparation is a continuous habit, not a project; expect a weekend for the first import. Data preparation is roughly 10 to 20 percent of the effort, and most of it is the user's own tidying discipline.

Implementation paths

Which path for whom

• Private: Path A, because notes apps such as NotebookLM, Notion or Capacities are mature, cheap and need no technical skills.
• SMB: Path C, because business notes often contain customer and trade-secret content, and a local-first stack (Obsidian with Smart Connections, Ollama) runs on a laptop (HW1), needs no AVV with a notes vendor and is the same platform that can later serve KNOW-001 (Baukasten effect).
• Developer: Path C, because Khoj, Logseq or Obsidian with local models and MCP access give full control and fit developer workflows.

Path A — Buy (recommended for private users)

• Source-grounded notebooks: Google NotebookLM (upload sources, ask questions with citations; per the help page a source may be up to 500,000 words or 200 MB) [1].
• Notes apps with AI: Notion (Notion AI), Obsidian with community plugins, Mem, Reflect, Tana, Capacities (Germany-based), Apple Notes, Microsoft OneNote/Copilot. Anytype (Swiss association, local-first, end-to-end encrypted sync) is a privacy-oriented alternative [5].
• Choose A when: convenience matters more than data control and notes contain no client secrets or special-category data. Check: EU data residency, whether content is used for training, export to Markdown, DPA for business use.

Path B — Build on cloud APIs

• Glue a Markdown vault (Obsidian, Logseq) to a hosted LLM via plugins such as Copilot for Obsidian, which supports bring-your-own-key and OpenAI-compatible providers [3]; use EU-hosted endpoints (e.g. Mistral's La Plateforme, Azure OpenAI/Bedrock EU regions) when notes contain personal data.
• Own pipeline: n8n or a small Python service → embedding API → pgvector/Qdrant → LLM with citations; expose via MCP so chat clients and coding agents can query the vault.
• Choose B when: you want custom behaviour (e.g. nightly digests, email-in capture) without running models yourself.

Path C — Self-host (open source; recommended for SMBs)

• Apps: Obsidian (data stored locally; app is closed-source but file-based) with Smart Connections, which runs a local embedding model by default and works offline [2][6]; Logseq (AGPL-3.0, local Markdown/Org files; the database version is still beta, so keep backups) [7]; Khoj (AGPL-3.0, self-hostable, works with local or online LLMs, accessible from browser, Obsidian, Emacs, WhatsApp) [4]; Anytype (source-available licence) [5].
• Models: local runtime via Ollama or llama.cpp; open embedding models such as Qwen3-Embedding (0.6B/4B/8B, 100+ languages) [8]; a 7–14B instruction-tuned open-weight LLM for answers; Whisper-class ASR for voice memos.
• Hardware: a modern laptop with NPU/GPU or a small home server (HW1) suffices for embeddings and 7–8B models; 14–30B models for better answers want a 16–24 GB GPU (HW2). Shares infrastructure with PERS-005.
• Ops effort: low; the main chores are backups and re-indexing after changing embedding model.

Path D — Custom training

Rarely sensible for an individual. Fine-tuning an embedding model on one's own notes brings little over a good multilingual base model plus a reranker. Only a research group or tool vendor with large labelled corpora might train domain-specific embeddings or retrieval models.

State of the art (2026)

• Local-first plus open models: capable embedding models run on laptops, and apps ship with built-in local indexing (Smart Connections [2]); the multilingual embedding leaderboard (MTEB) is led by open models, with Qwen3-Embedding-8B reported at rank 1 in June 2025 [9]; check the live leaderboard before choosing.
• Agents inside notes apps: Obsidian Copilot now centres on an agent mode that can inspect notes, use tools, create files and work over several turns with permissions the user controls, and can connect to Claude Code, Codex or local models [3].
• Source-grounded workspaces: NotebookLM-style products restrict answers to uploaded sources and cite them, which reduces (but does not eliminate) hallucination [1].
• Open interfaces: MCP servers let any compatible assistant read and write a Markdown vault.
• Limits: retrieval misses relevant notes if chunking is poor or notes are terse and cryptic; summaries may contain errors; auto-links can create plausible but meaningless connections; vendor products change pricing, features and licences quickly (e.g. Khoj and Anytype are not classic OSI-approved in all parts — check licences); AI-written notes can pollute the vault with unverified claims; prompt injection via clipped web pages is a real risk for agents with write access (see SEC-010).

Shared resources & combinations

• Same stack (embedding-text + vector-db + llm-general): KNOW-001 (enterprise search), KNOW-002 (RAG over company documents), KNOW-011 (KB generation) and KNOW-014 (organisational memory) use the same retrieval pipeline at team scale; a self-hosted embedding + LLM server (HW1/HW2) can serve personal and small-team use together.
• Document ingestion: DOC-014 (PDF parsing for RAG) and OCR for scans feed the vault.
• Voice and meetings: SPEECH-001 transcripts and voice memos become notes; PERS-017 (wearable lifelogging) is the ambient-capture extension with higher legal risk.
• Research: KNOW-008 (literature review) and KNOW-013 (knowledge graph / GraphRAG) extend linking and synthesis.
• Local assistant: PERS-005 (private local AI on own hardware) provides the model server; PERS-006 handles photo archives.
• Automation: AGENT-001 (email triage) and AGENT-002 (workflow automation) can file emails and clippings automatically.

Effort, cost & KPIs

• Path A: hours to set up; subscription typically per user per month; some tools free for personal use.
• Path B: a weekend to a few weeks for custom pipeline; API cost scales with usage and is small for personal volumes.
• Path C: one to several days; hardware you may already own; main cost is your time.
• KPIs: retrieval hit rate on 20–30 own test questions, share of answers with correct citation, time to find a known note, number of notes revisited per week, share of AI suggestions accepted, backup/restore test passed.

People & change

Who and when: Management and IT-security set the rules first: which notes may go into which tool (GUIDE-007). End-users are the main actors and should choose their own tool within these rules. IT supplies the approved local setup and backups; the DPO is involved if notes contain customer or employee data.

Works council: Depends. A purely personal vault on a local laptop is not suited to monitor behaviour. Once the company rolls out a shared tool with usage logs or admin reporting, §87(1) no. 6 BetrVG applies (the BAG holds that objective suitability suffices), and the works council should be informed under §90(1) no. 3 BetrVG. In the public sector, involve the Personalrat.

Data protection: Notes about customers or colleagues are personal data. Cloud tools require an AVV (Art. 28 GDPR); a DPIA is usually not needed for personal use, but check with the DPO for systematic employee data.

AI literacy: Art. 4 AI Act requires measures that support staff AI literacy; short hands-on sessions on citations, hallucinations and not pasting secrets are enough (GUIDE-006).

Communication and pilot: Offer it as voluntary; start with 3 to 5 volunteers who act as AI champions (GUIDE-008).

Concerns: "Is my private note collection surveilled?" State plainly that no usage evaluation takes place. AI suggestions stay suggestions; the person decides what enters the vault.

Risks, regulation & compliance

• EU AI Act: minimal risk for personal use; no Annex III category. Article 4 AI literacy applies to organisations deploying it for staff. If AI-generated content is published, Article 50 transparency duties may apply to the deployer of public-interest text or deepfakes [10].
• GDPR: a purely private vault falls under the household exemption, but business use (client data, HR notes, contact details of third parties) triggers GDPR: need a legal basis, a DPA with the SaaS or API provider, data minimisation, and deletion/retention concepts. Special-category data (health, union membership) belongs in local or EU-hosted setups only.
• Confidentiality: lawyers, doctors, tax advisers and journalists have professional secrecy duties (e.g. §203 StGB); cloud AI use needs careful contractual and technical safeguards.
• Security: vault plus API keys are valuable targets; encrypt devices and backups, use end-to-end encrypted sync where available (Obsidian Sync states file names and content are E2E encrypted [6]); beware malicious plugins and prompt injection from clipped content.
• Mitigations: local-first by default, EU endpoints, narrow agent permissions, version history, regular export to open formats.

Getting started

1. Pick the source of truth: a plain Markdown folder (portable) or one app with good export; migrate your notes in.
2. Try Path A with a non-sensitive subset (NotebookLM or Notion AI) and write down 20 questions you really ask.
3. Test a local-first setup: Obsidian + Smart Connections (local embeddings), then add Ollama for answers.
4. Decide a data policy: what may go to cloud APIs, what stays local; enable end-to-end encrypted sync and backups.
5. Establish a weekly review where AI proposes links/summaries and you accept or reject them; measure hit rate.

Sources

1. Google NotebookLM Help — adding sources, source limits — https://support.google.com/notebooklm/answer/16215270
2. Smart Connections (Obsidian plugin) — https://github.com/brianpetro/obsidian-smart-connections
3. Copilot for Obsidian — https://github.com/logancyang/obsidian-copilot
4. Khoj — open-source AI second brain — https://github.com/khoj-ai/khoj
5. Anytype (anytype-ts) — local-first, encrypted, Swiss association — https://github.com/anyproto/anytype-ts
6. Obsidian privacy policy — https://obsidian.md/privacy
7. Logseq — https://github.com/logseq/logseq
8. Ollama library — Qwen3-Embedding — https://ollama.com/library/qwen3-embedding
9. Qwen3-Embedding repository (MTEB multilingual result, June 2025) — https://github.com/QwenLM/Qwen3-Embedding
10. Regulation (EU) 2024/1689 (AI Act) — https://eur-lex.europa.eu/eli/reg/2024/1689/oj