Public consultation & objection (Einwendungen) analysis

A consultation on a law, a plan or a grid-expansion project can draw from a few hundred to tens of thousands of submissions, many of them copies of campaign templates. Procedural law requires that each relevant submission is read, weighed and answered. AI proposes themes, groups duplicates, assigns every submission to themes with evidence and drafts the response matrix. Analysts review the themes, verify the mapping on samples and sign off the answers. Public evidence shows a usable level of agreement with human coders, but it is still pilot-stage technology.

Problem & value

• Today: staff of ministries, planning authorities, Bezirksregierungen or project developers (Vorhabenträger) read every submission, sort it by hand into topics in spreadsheets and write a table of statements and responses. Deadlines are fixed by law (e.g. objections within a month after the end of the display period for grid expansion under § 22 NABEG; six weeks after display under § 73 VwVfG), so peaks of submissions create bottlenecks and backlogs [3][4].
• Legal core: the authority must consider timely submissions and communicate the result (§ 3 BauGB); for mass submissions of more than 50 identical texts, § 17 VwVfG and § 3 BauGB allow bundled handling via a representative or inspection of the result. The risk of a court challenge for missed objections is what separates this use case from market-research coding (LANG-019) and generic topic modelling (LANG-016) [4][5][6].
• AI value: faster first overview, consistent theme assignment, detection of near-duplicates and campaign letters, nothing is overlooked (every submission is mapped), transparent statistics for the final report.
• Measured evidence (UK): in the first live test of the UK government's Consult tool on a Scottish Government consultation (about 2,000 responses, six open questions), AI and human reviewers agreed most of the time; the government reports an F1 score of 0.76 and negligible effect on theme ranking. Its own estimate (not a measurement) is that roughly 500 consultations a year need about 75,000 analyst days, worth about £20m [1][2]. These are government statements from a trial, not independent audits.

Variants

• Legislative / policy consultations (ministries, EU-level "Have your say"-type exercises): open questions plus many organisational position papers; focus on stakeholder-group breakdown and positions.
• Formal planning objections (Planfeststellung, Raumordnung, NABEG, Bauleitplanung under §§ 3, 4a BauGB, UVP): each Einwendung is legally relevant; output is the Abwägungs-/Erwiderungstabelle for the Erörterungstermin.
• Campaign and mass-letter handling: clustering of form letters, extraction of individual additions on top of a template.
• Participation platforms / citizen dialogues (budget, mobility plans): ideas and comments from online platforms; informal, so less formal legal pressure.
• Corporate stakeholder consultations (permitting for wind farms, grid or logistics projects; association comment letters).
• Private use: residents or NGOs structuring their own objection to a nearby project; light-weight use of LLM chat, no platform needed.

How it works

1. intake (web form / e-mail / post scan / platform export, with receipt timestamp)
2. OCR + layout parsing, language detection, personal-data minimisation (pseudonymise names/addresses)
3. duplicate & campaign clustering (embeddings + near-duplicate hashing; keep individual additions)
4. segmentation into statements (one submission
5. several issues)
6. theme discovery (LLM proposes themes in batches
7. condense
8. refine) [analyst edits the themes]
9. theme mapping (each statement
10. one or more themes + quote + confidence)
11. human sampling & correction; legal check for "relevant to the decision" vs. out of scope
12. response drafting (matrix: theme
13. summary
14. plan reference
15. proposed answer) [legal/technical reviewer signs]
16. reporting (statistics, report, audit log)
17. archive in e-file (E-Akte)

• Theme discovery follows the staged pattern of the open-source ThemeFinder: sentiment/position, theme generation from batches, condensation, refinement, then mapping with multiple themes per response [7].
• Human control points: (1) analysts accept, merge or rewrite the theme catalogue before any mapping; (2) stratified sampling of mappings with error rate tracking; (3) every answer in the response matrix is approved by a named official. The decision on the submission itself is never taken by the model.
• Evidence-first design: every assignment points to the quoted passage so reviewers can verify it in seconds; this also gives the audit trail.

Data preparation

Input is typically a mix of scanned and handwritten objections, e-mails, web-form exports (CSV/XLSX) and PDFs, held in the registry (E-Akte/DMS), a mailbox or on file shares. Typical gaps: low-quality scans, hundreds of campaign duplicates with small edits, missing sender metadata (name, municipality, affected property), inconsistent file names, and attachments such as maps or signature lists that carry no text.

Preparation steps: (1) inventory all channels and fix one intake folder per procedure; (2) OCR scans and check a sample for errors (see PREP-006); (3) convert mail archives and odd formats (see PREP-007); (4) deduplicate and cluster form letters, keeping the count per cluster; (5) record sender, date, channel and affected object as metadata (see PREP-010); (6) agree the category scheme with the case team before classifying (see PREP-019); (7) pseudonymise names where analysts do not need them (see PREP-022). Build a small gold set of hand-labelled submissions to evaluate the theme assignment (PREP-018).

Tools: Paperless-ngx or an existing DMS, OCR engines such as Tesseract or a cloud document parser, Python or n8n for dedup and export. Rough share of total effort: 40-50 percent, mostly digitisation and cleaning.

Implementation paths

Which path for whom

• public: Path C — consultation texts contain personal data and the procedure must be auditable, so a sovereign or self-hosted stack (HW3) fits procurement and data-protection rules best.
• enterprise: Path B — project developers with recurring permitting or stakeholder procedures can combine EU-hosted APIs with their own case system.
• developer: Path B — build the pipeline on EU-region APIs and keep the theme catalogue and audit trail as your own code.
• smb: Path B — consultations are rare for SMBs and no standard product fits, so a small pipeline built per procedure on EU-hosted APIs is the realistic route.

Path A — Buy (off-the-shelf)

• Participation platforms with analysis modules: open-source or vendor platforms for online consultation such as Adhocracy+ / meinBerlin (Liquid Democracy e.V., Berlin, AGPL-3.0) and Decidim (AGPL-3.0, originally Barcelona) cover collection, moderation and export; their text analytics are comparatively basic [8][9]. Pol.is (AGPL-3.0, hosted free for non-profits and government) clusters opinions via voting rather than analysing free-text objections [10].
• Specialised AI analysis for formal objections: the German BMV/mFUND project BIM4People (RWTH Aachen with Carpus-Partner, albert.ing, WIN.DN, Interactive Pioneers; August 2023 to April 2026) built an AI-supported Einwendungsmanagement prototype (topic pre-sorting, similarity detection) on open BIM standards [11]. Check with the consortium whether a productive offer exists; I did not find a verified commercial product list.
• E-file / DMS vendors in German administration (e.g. d.velop, PDV, Fabasoft - verify current AI modules) and consultancies for participation processes (e.g. Zebralog, Bonn) deliver the process; verify what analytics they use.
• Choose A when: low volume, no in-house data team, platform already in place. Always ask for: EU hosting, DPA, evidence-linked output, export of the audit trail, no training on your data.

Path B — Build on cloud APIs (recommended default)

• Models: an EU-hosted frontier or EU-provider LLM through API with structured outputs and long context (e.g. Mistral Large 3 / Mistral Medium 3.5 via Mistral's platform, Claude, Gemini or GPT via EU regions of the hyperscalers); temperature 0 and JSON-schema output for theme mapping [12]. Verify the processing region and zero-retention terms in the contract.
• Pipeline: ThemeFinder (MIT) as the theme discovery and mapping engine, plus embeddings (e.g. BGE-M3, multilingual, MIT) for near-duplicate clustering and a vector/hybrid index for search through submissions; a spreadsheet or web review UI for analysts [7][13].
• Consult (UK i.AI, MIT, Django/Svelte app on AWS) is a reference architecture with human theme sign-off, but its README does not recommend it for critical use yet [14].
• Integration: OCR for scanned letters (see DOC-003), pseudonymisation before sending text (DOC-018), export to the E-Akte and the Erörterung documents.
• Choose B when: a project team can integrate, volumes are 1,000+ submissions, and the data may legally go to a processor in the EU.

Path C — Self-host (open source)

• Models: open-weight LLMs such as Mistral Small 4 / Ministral 3 (Apache 2.0) or Qwen3-30B-A3B-Instruct-2507 (Apache 2.0, 30.5B total / 3.3B active parameters, 262k native context), served with vLLM; BGE-M3 for embeddings [12][15][13].
• Stack: ThemeFinder or BERTopic (MIT; hierarchical topics, LLM-generated labels) plus a hybrid index (OpenSearch or Qdrant/pgvector) and a self-hosted review UI [7][16].
• Hardware: HW3 (one 48-96 GB GPU) for a ~30B MoE class model with long context; HW2 is enough for small models and pure clustering, with lower mapping quality on legal prose.
• Choose C when: objections contain special-category data or rules forbid external processors (Bundeswehr, security-relevant infrastructure, some Länder IT rules); effort is higher because you operate the stack.

Path D — Custom training

Rarely required. A fine-tuned encoder classifier (text-classifier) is worthwhile only for recurring, stable theme catalogues (e.g. an authority handling many similar grid or wind projects) where thousands of earlier, human-verified statement-to-topic assignments exist. Corrections made by analysts in earlier procedures are the training data. For one-off consultations, prompting plus human review is more economical.

State of the art (2026)

• Two-stage "find themes, then map" pipelines with humans signing off the themes are the pattern validated in the public sector: themes first generated from batches, condensed and refined, then every response mapped, with dashboards for verification [1][7][14].
• Evaluation: published agreement is moderate-to-good rather than perfect (F1 0.76 in the first Consult test); quality depends on the theme catalogue, response length and domain. Run your own evaluation: double-code a stratified sample (e.g. 5-10% of submissions) and measure agreement, rather than relying on vendor claims [1][2].
• Long-context LLMs (hundreds of thousands of tokens) can read entire plan documents alongside the objections, improving the link "objection ↔ plan section", but long-context accuracy varies by model; test on your own corpus.
• Limits and failure modes: (a) minority or unusual arguments are folded into larger themes and lost; (b) models may assign a theme to a submission that only mentions it in passing; (c) legally relevant individual concerns (property, health, specific parcels) hidden inside form letters; (d) multilingual and dialect inputs, handwriting and poor scans; (e) prompt injection in submissions (text addressed to the AI), see SEC-010; (f) false confidence in counts: theme frequency is not importance, and the weight of an argument is a legal judgement.
• Note: the tools from the UK are labelled incubation / not for critical use; treat any quantitative claim as a pilot result.

Shared resources & combinations

• Same building blocks (llm-general, embedding-text, vector-db): LANG-019 (qualitative coding), LANG-016 (topic modelling), LANG-013 (open-ended feedback analysis) and LANG-008 (summarising document sets) can run on one GPU server (HW3) and one embedding/index service; the theme-mapping prompt and review UI can be reused.
• Upstream: DOC-003 (form and application processing, scans), DOC-018 (anonymisation before analysis), LANG-012 (routing submissions to subject-matter departments), LANG-015 (extraction of parcels, addresses, plan references).
• Downstream: LANG-010 (plain-language public summary of results), LANG-034 (legislative drafting from consultation outcomes), LANG-020 (policy horizon scanning), CONV-011 and AGENT-026 (citizen service and case processing in the same administration).
• Governance: SEC-018 (privacy operations) for records of processing and retention of submissions.

Effort, cost & KPIs

• Path A: weeks, mostly procurement and DPA work; licence or per-project fee.
• Path B: 1-3 months for a pilot including the review UI and a validation study; model cost is small relative to analyst time for large consultations, but data protection review dominates the schedule.
• Path C: 2-4 months plus GPU server and operations.
• KPIs: agreement with double-coded sample (F1 / Cohen's kappa), share of submissions mapped without edits, analyst hours per 1,000 submissions, time from deadline to report, number of court or ombud challenges for overlooked concerns, duplicate-cluster precision, share of themes added after sampling (a recall proxy).

People & change

Involve the department lead (process owner) and the legal team at the start, because the duty to consider every submission makes them accountable for the result. IT and the data protection officer (DPO) join during design: submissions contain personal data and often special categories, so a DPIA under Art. 35 GDPR is likely, and every cloud provider needs an AVV (Art. 28 GDPR). Works council or staff council: in authorities the Personalrat (BPersVG or state law) usually has a say; in companies the works council is relevant if analysts' use is logged (§87(1) no. 6 BetrVG, which the BAG applies to any objectively suitable system) and must be informed early about AI-supported work processes (§90(1) no. 3). Without an elected council this does not apply.

AI literacy: support the case team with short training on what the model can miss (rare arguments, irony, minority views); the AI Act asks deployers to support staff literacy (Art. 4). Communicate that the AI only proposes themes and clusters; analysts decide on every mapping and sign off the catalogue. Start with one closed procedure as a pilot and compare AI results with the manual table. Typical concerns are missed arguments and legal challenge risk: address them with sampling checks, quoted evidence per theme and a documented audit trail.

See GUIDE-004, GUIDE-005 and GUIDE-009.

Risks, regulation & compliance

• EU AI Act: a typical analytical assistant is minimal risk. Annex III point 8 covers AI used by judicial authorities and AI intended to influence the outcome of an election or referendum, not consultation analysis; point 5(a) covers eligibility for essential public benefits [17]. It becomes high-risk (or at least needs a case-by-case check) if the system is used to decide individual entitlements, to profile or rank individuals, or if used by a court. Art. 4 AI literacy applies. Art. 50(4) requires disclosing AI-generated text published to inform the public on matters of public interest unless it has been human-reviewed and someone holds editorial responsibility, so publish AI-drafted reports only after human review [18].
• Administrative law: the authority cannot delegate the weighing (Abwägung) or the decision to a model; it must be able to show that each timely submission was considered. Late submissions and Präklusion rules (§ 73 VwVfG, § 21 UVPG, § 22 NABEG) require reliable receipt timestamps, which the AI pipeline must not alter [3][4][19]. Whether and how AI use must be disclosed in the procedure should be clarified with the legal department (state law differs).
• GDPR: submissions contain names, addresses, property and health information and political opinions (special categories, Art. 9). Basis typically Art. 6(1)(e) with the procedural statute; minimise and pseudonymise before analysis, EU processing, DPA and no vendor training, DPIA for large procedures. Publication of objections with names needs separate care.
• Other risks: bias towards well-written, organised submissions; loss of minority views; public trust (communicate that humans decide); prompt injection in submissions; vendor lock-in for the theme catalogue.

Getting started

1. Take a finished consultation (or 300 submissions) with its existing human theme table as ground truth.
2. Run ThemeFinder with an EU-hosted model on it and measure agreement per theme; inspect missed minority arguments.
3. Add duplicate/campaign clustering and check that individual additions inside form letters survive.
4. Define the sign-off workflow: who approves themes, what sampling rate, how AI suggestions and decisions are logged.
5. Complete the DPIA and procedural-law review, then pilot on a live procedure with parallel manual review of a sample.

Sources

1. UK Government (DSIT) - "Government-built Humphrey AI tool reviews responses to consultation for first time" (14 May 2025) - https://www.gov.uk/government/news/government-built-humphrey-ai-tool-reviews-responses-to-consultation-for-first-time-in-bid-to-save-millions
2. Civil Service World - "Civil service AI tool for consultation analysis passes first public test" - https://www.civilserviceworld.com/news/article/civil-service-ai-tool-for-consultation-analysis-passes-first-test
3. § 22 NABEG (Einwendungen, Stellungnahmen, Fristen) - https://gesetze.legal/bund/nabeg/22
4. § 73 VwVfG (Anhörungsverfahren) - https://www.gesetze-im-internet.de/vwvfg/__73.html
5. § 17 VwVfG (Gleichförmige Eingaben) - https://www.gesetze-im-internet.de/vwvfg/__17.html
6. § 3 BauGB (Beteiligung der Öffentlichkeit) - https://www.dejure.org/gesetze/BauGB/3.html
7. i.AI ThemeFinder (MIT) - https://github.com/i-dot-ai/themefinder
8. Liquid Democracy e.V. - adhocracy4 / Adhocracy+ / meinBerlin - https://github.com/liqd/adhocracy4
9. Decidim - https://github.com/decidim/decidim
10. Pol.is - https://github.com/compdemocracy/polis
11. BMV mFUND - BIM4People project page - https://www.bmv.de/SharedDocs/DE/Artikel/mFUND/Projekte/bim4people.html
12. Mistral AI - models overview - https://docs.mistral.ai/getting-started/models/models_overview/
13. BAAI BGE-M3 - https://huggingface.co/BAAI/bge-m3
14. i.AI Consult (MIT, incubation) - https://github.com/i-dot-ai/consult
15. Qwen3-30B-A3B-Instruct-2507 - https://huggingface.co/Qwen/Qwen3-30B-A3B-Instruct-2507
16. BERTopic - https://github.com/MaartenGr/BERTopic
17. EU AI Act Annex III (via artificialintelligenceact.eu) - https://artificialintelligenceact.eu/annex/3/
18. EU AI Act Art. 50 (via artificialintelligenceact.eu) - https://artificialintelligenceact.eu/article/50/
19. § 21 UVPG - https://www.gesetze-im-internet.de/uvpg/__21.html

Accessed 2026-10. Regulation (EU) 2024/1689 full text: https://eur-lex.europa.eu/eli/reg/2024/1689/oj