Sentiment & customer feedback analysis (reviews, NPS, surveys)

Companies collect thousands of free-text comments in reviews, NPS surveys, support tickets and complaints, but most go unread or get sampled by hand. AI reads every comment. It detects overall and aspect-level sentiment ("delivery slow, product great") and codes each comment against a stable theme taxonomy, so you can see which themes drive the score. It also flags new problems before they hit the KPI. LLMs have changed the field. Codeframes no longer need months of rule-writing or labelling: a usable taxonomy now takes days, with explanations and cited verbatims instead of a bare polarity score.

Problem & value

• Today: NPS/CSAT dashboards show that scores move but not why. Open ends are read selectively, coded by hand in spreadsheets or by market-research agencies, or matched with keyword rules that miss negation, irony and new vocabulary. Reviews on dozens of platforms and locations are only read when someone complains loudly.
• AI value: coverage of 100 % of feedback instead of samples; consistent coding over time; faster detection of product defects, delivery problems or price perception shifts; closing the loop with detractors; evidence for product and service prioritisation.
• Evidence: feedback-analytics vendors publish large deployments. These are vendor claims, not independent studies: Caplena reports IKEA analysing more than 3 million feedback comments in 40 languages, and DHL using it for the voice of its employees [5]. Medallia says its text analytics supports 1,000+ programmes with hundreds of prebuilt industry models [6]. No independent study reliably measures the ROI of sentiment analysis itself. Value comes from the actions taken, so measure it in your own pilot.

Variants

• SMB / local business (hospitality, practices, trades, retail): review aggregation across Google, Tripadvisor, Booking.com, Jameda, ProvenExpert with sentiment + theme per location; replying to reviews is a separate use case → LANG-038.
• E-commerce & marketplaces: product-review mining by attribute (fit, quality, packaging) to feed product pages, returns reduction and supplier quality; fake-review screening → ANOM-010.
• Enterprise VoC / CX programmes: NPS/CSAT relationship and transactional surveys combined with tickets, chats and call transcripts (from SPEECH-002) in one taxonomy; driver analysis and closed-loop workflows into CRM.
• Software/app companies: app-store reviews, in-app feedback and feature requests clustered into the product backlog.
• Employee feedback (engagement surveys, exit interviews): same technology, but report only aggregated results with minimum group sizes. Works-council co-determination applies in Germany (§87(1) no. 6 BetrVG) [10]. Individual performance evaluation is out of scope; see LANG-036.
• Public sector: citizen satisfaction surveys, complaint management and service-portal feedback; mass consultations and objections → LANG-026.
• Market research agencies: open-end coding of quantitative surveys; deep qualitative coding of interviews → LANG-019.
• Private use: summarising pros/cons across hundreds of product reviews before a purchase, which general assistants and marketplaces now provide as built-in review summaries.

How it works

1. collect (review APIs, survey tool export/webhook, ticket system, CRM)
2. normalise & deduplicate (language detection, spam/fake filter, PII redaction)
3. segment (sentence / clause)
4. sentiment: document + aspect-level (aspect, opinion, polarity, intensity)
5. theme coding against codeframe (multi-label; new-theme discovery via embeddings + clustering)
6. enrich with metadata (score, segment, product, location, channel)
7. analytics: driver analysis vs. NPS/CSAT, trend & anomaly alerts
8. outputs: dashboards, LLM insight summaries with cited verbatims, close-the-loop tasks

• Classification engine: a high-volume production setup usually has two tiers. A cheap classifier handles polarity and frequent themes: a fine-tuned encoder model or a small LLM. A larger LLM handles aspect extraction, ambiguous cases, codeframe proposals and summaries. With structured outputs, the LLM returns JSON with aspect, polarity and a supporting quote.
• Codeframe management: an LLM proposes candidate themes by clustering embeddings (BERTopic-style), and analysts approve, merge and name them. The approved codeframe then stays stable, so trends can be compared month over month. LANG-016 covers topic modelling in depth.
• Human-in-the-loop: analysts own the codeframe. They review low-confidence and newly discovered themes and validate a monthly sample against human coding. Close-the-loop contact with detractors is done by people.

Data preparation

Data needed: review texts (Google Business Profile, Trustpilot, app stores), NPS/CSAT verbatims with score and metadata, open survey answers, support tickets and complaint e-mails. In an SMB these sit in survey tools, the ticket system, the shop or CRM, mailboxes and often Excel exports; paper comment cards are rare but exist in hospitality and retail.

Typical gaps: comments without score, segment or product reference; duplicates across platforms; mixed languages and slang; personal data and names in free text; no agreed category scheme (codeframe); scattered ownership of the sources.

Preparation steps: 1. Inventory the sources and their owners, and export them via API or connector (see PREP-005, GUIDE-003). 2. Merge into one table with ID, date, channel, score, product and language; remove duplicates (PREP-011). 3. Define a small codeframe of themes before classification (PREP-019) and label 100-200 comments as a test set with domain experts (PREP-018). 4. Mask names, e-mail addresses and order numbers before data leaves your systems (PREP-022).

Tools: the export functions of survey and ticket tools, Excel or Power Query, n8n for scheduled pulls.

Effort share: roughly 30-40 % of the project, mostly in consolidation and the codeframe, not in the model.

Implementation paths

Which path for whom

• SMB — A: mature feedback-analytics products with EU hosting and connectors exist at SMB prices, and review/NPS analysis is a standard process.
• Enterprise — B: feedback is strategic, volumes make seat or per-comment fees high, and results must flow into own CX data models and dashboards.
• Developer — B: cloud LLM and embedding APIs with a managed codeframe give the fastest custom build; self-hosting only if data demands it.
• Public sector — C: citizen feedback and procurement rules favour a sovereign, self-hosted or German-cloud stack with open models.

Path A — Buy (recommended for most SMBs)

• Enterprise CX/XM suites: Qualtrics XM (Text iQ), Medallia (Text Analytics and Ask Athena), InMoment, Forsta. These fit when the survey platform is already in place.
• Specialised feedback analytics: Caplena (Zurich, CH; GDPR and SOC 2 badges, connectors to Qualtrics, Medallia, Zendesk and Power BI) [5], Netigate Insights (formerly Lumoa; Netigate is Swedish, and Netigate states that all data stays in the EU) [7], Chattermill (UK), SentiSum (UK), Thematic, Keatext.
• Review management with sentiment built in: Trustpilot, TrustYou (DE, hospitality), Birdeye, Yext, ReviewTrackers. Ticket systems such as Zendesk and Freshdesk add sentiment to tickets.
• Choose A when: the volume is a few thousand to millions of comments a year from standard sources, and the team has no data scientists. Before buying, check EU hosting, connectors, German-language quality, codeframe editing and export of raw codes.

Path B — Build on cloud APIs

• Classic NLP APIs: Azure Language sentiment analysis and opinion mining (aspect-level, container option for on-prem). Microsoft is retiring these features on 31 March 2029 and directs new projects to Foundry models [3]. AWS Comprehend targeted sentiment supports English only [4], and Google Cloud Natural Language offers analyzeSentiment and entity sentiment. These APIs are cheap and fast but give generic polarity, not your codeframe.
• LLM-based (recommended for new builds): use Mistral (EU provider) or Claude, Gemini or GPT via EU data-residency endpoints (Azure OpenAI EU Data Zone, Vertex AI EU, Amazon Bedrock Frankfurt). Prompt them with the codeframe and few-shot examples, and require structured JSON output. Batch APIs cut cost for nightly runs.
• Pipeline: n8n / Power Automate or Python jobs pull from review APIs (Google Business Profile API, Trustpilot API) and survey exports. Results land in a warehouse (BigQuery/Snowflake/Postgres) and go to BI (Power BI, Metabase). Embeddings plus pgvector support "show me similar complaints".

Path C — Self-host (open source)

• Encoder classifiers (CPU feasible, HW1): oliverguhr/german-sentiment-bert (MIT licence, trained on 1.834 million German samples from reviews and social media; F1 micro 0.96 on HolidayCheck reviews, but only 0.75 on GermEval) [1]. Also cardiffnlp/twitter-xlm-roberta-base-sentiment for 8 languages [2], and SetFit for few-shot theme classifiers.
• Open LLMs for aspects and themes (HW2): Qwen3, Mistral Small, Gemma 3 or Llama-family models of about 8–30B. Serve them with vLLM or Ollama and use guided JSON decoding.
• Topic discovery: BERTopic (MIT licence, supports LLM topic labels) [8] with multilingual embeddings (e.g. multilingual-e5, BGE-M3). Store vectors in pgvector or Qdrant.
• PII: Microsoft Presidio before storage or prompting. Hardware: HW1 is enough for encoder-only pipelines; HW2 (one 24–32 GB GPU) is enough for an LLM processing tens of thousands of comments a day.

Path D — Custom training

Fine-tuning pays off at high volume, when a fixed codeframe of more than 30 themes must run cheaply and consistently, or with domain jargon (insurance claims, telecom tariffs, medical practices). In a 2024 evaluation across 13 SemEval datasets, LoRA-tuned 7–8B LLMs beat fully fine-tuned small models on ABSA. On the hardest subtask (aspect-sentiment quad prediction), fine-tuned LLaMA3-8B reached 57.97 F1, while GPT-4 with retrieved few-shot demonstrations reached 40.79 [9]. Typical data needs are 1,000–5,000 human-coded comments per codeframe. Analyst corrections from the review UI are the natural training set. You can also distil a large LLM's labels into a small model.

State of the art (2026)

• LLM-first coding: zero- and few-shot LLMs now handle polarity, aspects and multi-label theme coding in German and most EU languages without labelled data. Vendors have moved from rule and keyword taxonomies to LLM-generated codeframes and conversational "ask your feedback" agents (Caplena Insight Agent, Netigate Ask AI, Medallia) [5][6][7].
• Benchmarks: SemEval-2014–2016 ABSA tasks remain the reference. Research shows that LLMs rival fine-tuned models on simple polarity. Structured tasks such as triplet and quad extraction still favour fine-tuned models, and how few-shot examples are selected matters: hybrid BM25 + semantic retrieval beats random selection [9].
• Classic APIs are in decline. Microsoft's retirement notice for Azure Language sentiment shows the industry moving to general foundation models [3].
• Limits: irony and sarcasm, mixed sentiment, domain-specific polarity ("cheap" can be praise or criticism), and score/text mismatch (a 9/10 NPS with a complaint). Small groups produce spurious trends. LLMs can drift when the prompt or model version changes, which breaks time-series comparability, so pin model versions and re-validate on a gold set after every change. Review data is biased toward extremes and can contain fake reviews.

Shared resources & combinations

• Same text-classification stack (llm-small + text-classifier + embedding-text): LANG-012 ticket/email routing, LANG-014 moderation, LANG-015 entity extraction. One vLLM server plus one encoder-serving container (HW2) can serve all of them.
• Same feedback corpus, deeper analysis: LANG-016 topic & trend detection, LANG-019 qualitative coding, LANG-018 social listening (public social data), DATA-026 behavioural UX analytics.
• Upstream sources: SPEECH-002 call transcripts, CONV-001 chatbot logs, ANOM-010 fake-review filtering before analysis.
• Downstream actions: LANG-038 and LANG-002 for review replies, AGENT-020 for e-commerce customer replies, PRED-003 churn models (sentiment as a feature), and DATA-001 conversational BI over the coded results.
• Shared infrastructure: a vector DB (pgvector/Qdrant), a PII-redaction service (Presidio), workflow automation (n8n) and the BI layer.

Effort, cost & KPIs

• Path A: 1–4 weeks to connect sources and agree a codeframe. SaaS is priced per seat, per response volume or per source.
• Path B: 4–10 weeks for an MVP with two or three sources, a codeframe, a dashboard and alerts. LLM token cost per comment is low, and batch processing reduces it further.
• Path C: 2–3 months including hosting, monitoring and gold-set evaluation.
• KPIs: agreement with human coders (F1/kappa on a monthly gold sample), share of comments coded, time from feedback to detected issue, close-the-loop rate and time for detractors, and NPS/CSAT change on themes that were acted on.

People & change

Who and when: management sets the goal and budget; the department lead (CX, marketing) owns the codeframe and the follow-up process; IT connects the sources; the DPO is involved before go-live because comments contain personal data. Data owners of survey and ticket systems release the exports. End users (service and product staff) should see sample outputs early.

Works council: depends. Analysing customer feedback alone is rarely a §87(1) no. 6 BetrVG matter, but as soon as tickets or chats are analysed per agent, or employee surveys are included, objective suitability for performance monitoring is enough (BAG) and co-determination applies. Inform the works council anyway under §90(1) no. 3 and agree that no agent-level evaluation takes place.

Data protection: AVV with every vendor (Art. 28 GDPR); a DPIA is usually not needed for aggregated customer feedback, but check it when employee surveys or special categories appear (GUIDE-005).

AI literacy and communication: Art. 4 AI Act requires measures to support staff AI literacy; a short training on how the scores are made and where they err is enough (GUIDE-006). Start with a pilot in one channel (GUIDE-009), publish results openly.

Typical concerns: "the tool judges my work" - state that it analyses topics, not people. "Sarcasm is misread" - sample-check weekly; humans decide on actions (GUIDE-008).

Risks, regulation & compliance

• EU AI Act: typical customer-feedback analysis is minimal risk, and Art. 4 AI literacy applies. Text sentiment is not an "emotion recognition system" in the Act's sense, because that definition requires inference from biometric data (Art. 3(39), 3(34)) [11]. The Art. 5(1)(f) workplace ban therefore does not cover text analysis. It would apply to voice or face-based emotion inference on employees. Using feedback analysis to evaluate or monitor individual employees (e.g. scoring agents by customer comments) can fall under Annex III no. 4(b), which makes it high-risk; keep employee results aggregated. Art. 50 transparency applies only if a chatbot interacts with customers.
• GDPR: reviews and verbatims often contain names, order numbers and health details (e.g. in practice reviews). The legal basis is usually legitimate interest or survey consent. Apply data minimisation and redaction, sign a DPA with the vendor, and use EU hosting. Linking anonymous surveys back to customers needs an upfront transparency notice.
• Employee surveys: in Germany, §87(1) no. 6 BetrVG co-determination applies [10], so agree minimum group sizes (e.g. ≥ 5–10) and no-drill-down rules in a works agreement.
• Reviews: use platform APIs and respect their terms. Scraping can breach the terms and database rights.
• Risks: biased or drifting classifiers, over-trusting dashboards built on small samples, and misreading sarcasm. Mitigate with gold-set monitoring, confidence thresholds, displayed sample sizes and quotes as evidence.

Getting started

1. Inventory feedback sources, volumes and languages, and pick the two sources most tied to a business KPI (e.g. NPS verbatims and Google reviews).
2. Draft a codeframe of 15–30 themes with the business owners. Let an LLM propose themes from a sample of 1,000 comments, then curate them.
3. Hand-code 300–500 comments as a gold set and measure two or three tools or prompts against it, checking German quality specifically.
4. Define alert rules and a close-the-loop workflow for detractors and severe complaints.
5. Review the codeframe each quarter, and re-run the gold set after every model or prompt change.

Sources

1. Hugging Face — oliverguhr/german-sentiment-bert (model card, training data, F1 scores) — https://huggingface.co/oliverguhr/german-sentiment-bert
2. Hugging Face — cardiffnlp/twitter-xlm-roberta-base-sentiment — https://huggingface.co/cardiffnlp/twitter-xlm-roberta-base-sentiment
3. Microsoft Learn — Sentiment analysis and opinion mining in Azure Language (retirement notice 31 March 2029) — https://learn.microsoft.com/en-us/azure/ai-services/language-service/sentiment-opinion-mining/overview
4. AWS — Amazon Comprehend targeted sentiment — https://docs.aws.amazon.com/comprehend/latest/dg/how-targeted-sentiment.html
5. Caplena — AI feedback analytics platform (customer cases IKEA, DHL, Lufthansa) — https://caplena.com/
6. Medallia — Text Analytics — https://www.medallia.com/platform/text-analytics/
7. Netigate — Lumoa is now Netigate Insights — https://www.netigate.net/lumoa-is-netigate/
8. BERTopic — https://github.com/MaartenGr/BERTopic
9. Zhou et al. — A Comprehensive Evaluation of Large Language Models on Aspect-Based Sentiment Analysis (arXiv 2412.02279) — https://arxiv.org/html/2412.02279v1
10. Betriebsverfassungsgesetz §87 — https://www.gesetze-im-internet.de/betrvg/__87.html
11. EU AI Act Service Desk — Article 3 definitions — https://ai-act-service-desk.ec.europa.eu/en/ai-act/article-3
12. Regulation (EU) 2024/1689 (AI Act) — https://eur-lex.europa.eu/eli/reg/2024/1689/oj