Synthetic respondents & persona simulation for market research

Real fieldwork is slow and expensive: recruiting a panel, fielding a survey and analysing results takes weeks and costs a lot per complete, so many ideas are never tested at all. Large language models conditioned on demographic backstories, segment data or individual interview records can play "respondents" — answering questionnaires, reacting to concepts and sitting in simulated focus groups in minutes. Used well, this screens many variants cheaply and sharpens the few that go to real customers. Used badly, it produces confident, too-homogeneous answers that mirror the model's biases. The core discipline is calibration and validation against real human data.

Problem & value

• Today: concept, claim, message and price tests run through online panels (Toluna, Cint, Dynata, YouGov, GfK/NIQ) or agencies; qualitative work through recruited interviews and focus groups. Budgets limit how many variants can be tested; niche B2B audiences are hard to recruit.
• AI value: early-stage screening of many variants, questionnaire pre-testing (catch ambiguous wording before field), "boosting" small segments, exploring hypotheses between real waves, and querying past research conversationally.
• Evidence on fidelity (research): agents built from two-hour interviews with 1,052 US adults reproduced their answers on held-out General Social Survey items at 83% (interviews only) to 86% (interviews + surveys) of the participants' own two-week test-retest consistency, versus 74% for demographics-only agents [1]. A PyMC Labs/Colgate-Palmolive study on 57 personal-care concept surveys (9,300 human responses) reached about 90% of human test-retest reliability for purchase intent, with realistic response distributions (KS similarity > 0.85), when free-text answers were mapped to Likert scales via embeddings instead of asking for numbers directly [2].
• Evidence on limits: in a Political Analysis study, persona-prompted GPT-3.5 matched survey averages but showed far too little variance; ~48% of regression coefficients differed significantly from the real survey and ~32% of those had the wrong sign, and results drifted as the model was updated [3]. LLMs misalign with many demographic groups even when explicitly steered [4] and tend to flatten and essentialise identity groups [5].
• Vendor claims (not independently verified): e.g. Synthetic Users prices interviews at $2–60 each [8]; Lakmoos claims > 98% accuracy across 20 benchmark studies [9]; Aaru reports a mean total-variation distance of 7.62% across 2,993 questions [10]. Treat these as starting points for your own validation.

Variants

• Synthetic survey panels / AI consumer panels (covers merged DATA-019): segment-level "silicon samples" answering closed questions, calibrated against real reference samples and re-validated per wave.
• Persona simulation from panel and CRM data (covers merged REC-014): segment or customer "digital twins" grounded in first-party data for concept, message, pricing and questionnaire pre-tests; individual-level twins require explicit consent (see GDPR).
• Synthetic qualitative research: AI interviews and simulated focus groups for discovery, persona deep-dives and UX copy testing (feeds LANG-019 coding).
• Sample boosting / augmentation: imputing answers for under-represented cells or extra questions on top of an existing real study — only with transparent flagging.
• B2B: simulated buying committees (CIO, CFO, procurement) to stress-test value propositions and tender messaging.
• Public sector & policy: pre-testing citizen communication, forms and consultation questionnaires for comprehensibility; not a substitute for real public participation (LANG-026) or opinion polling.
• SMB / solo founders: quick reality checks of landing-page copy or product ideas with general-purpose chat assistants and persona prompts — useful for brainstorming only.

How it works

1. research brief + stimulus (concept, ad, price, questionnaire)
2. audience model: segments/quotas from census, panel or CRM data
3. persona construction (demographic backstory | segment profile | individual interview memory via RAG)
4. elicitation: LLM answers in free text / interview turns (multi-agent for focus groups)
5. response mapping (e.g. embedding similarity to Likert anchors, choice parsing)
6. aggregation + calibration (post-stratification, model ensembles, temperature/variance checks)
7. validation vs. held-out real sample (distribution distance, rank agreement, coefficient signs)
8. report with uncertainty + human researcher review
9. decide which variants go to real field

• Persona grounding is the main quality lever: demographics alone are weakest; rich individual records (interviews, past answers) are strongest [1].
• Elicitation method matters: direct numeric ratings from LLMs give unrealistic distributions; semantic similarity rating (SSR) and multi-model ensembles reduce this [2].
• Calibration layer (tabular ML, raking/post-stratification) corrects systematic offsets learned from paired synthetic/real studies.
• Human-in-the-loop: a researcher designs the study, judges plausibility, signs off any decision; go/no-go and launch decisions should rest on real-respondent data.

Data preparation

Synthetic respondents are only as good as the real data used to ground and validate them. Relevant sources: past survey results (Excel, SPSS exports, survey tools), CRM and shop data on customer segments, interview transcripts, support tickets and published market studies. These typically sit in departmental file shares, the survey tool and the CRM, often scattered across agencies.

Typical gaps: no common segment definitions, survey files with unlabelled variables, small or biased samples, interview notes only on paper or in personal folders, and unclear consent for reusing customer data. Preparation steps: (1) collect and list available studies and their sample sizes (see PREP-001); (2) restructure spreadsheets into tidy tables with a code book (see PREP-011); (3) define segments and a glossary of terms (see PREP-012); (4) build a validation set of real survey answers to compare against simulated ones (PREP-018); (5) pseudonymise any customer-level data (PREP-022) and check the legal basis for reuse (PREP-025).

Tools: Excel or Python/pandas for cleaning, the survey tool's export, a simple vector store for interview transcripts. Rough share of effort: 25-35 percent, mainly code book and validation set.

Implementation paths

Which path for whom

• smb: Path A — mature synthetic-panel and persona tools exist at SMB prices and the use is a standard market-research task.
• enterprise: Path B — research teams want personas grounded in their own survey and CRM data and need control over prompts and validation.
• developer: Path C — open-weight models allow reproducible simulation runs and cheap bulk sampling without per-token cost.
• public: Path B — on EU-hosted APIs for statistical offices or research bodies, where methods must be documented and no product is certified for it.

Path A — Buy (off-the-shelf)

• Synthetic audience / simulation platforms: Simile (individual-grounded simulations; named customers include CVS Health, Gallup and Deloitte) [6], Aaru [10], Electric Twin (UK, GDPR-ready, builds audiences from client data) [7], Synthetic Users (AI interviews, EU and US regional infrastructure, DPA) [8], Lakmoos (Czech Republic, EU) with custom private panels on client data and a fixed-price pilot [9].
• Incumbent research suites: major panel and research-software vendors (e.g. Qualtrics, Toluna, Ipsos, Kantar) have added synthetic panels/personas and "talk to your data" features on top of their human panels — useful because they can validate against their own real samples.
• Choose A when: marketing/insights teams want fast concept screening without ML staff. Ask every vendor for its validation methodology, per-category benchmarks, model-version pinning and EU data processing.

Path B — Build on cloud APIs

• Models: frontier LLMs with EU data residency — Mistral (Mistral Large/Medium, EU provider), Claude, GPT and Gemini via EU regions (AWS Bedrock Frankfurt, Azure OpenAI EU Data Zone, Vertex AI europe-west). Use several model families as an ensemble to dilute single-model bias.
• Embeddings for SSR-style Likert mapping and persona retrieval (e.g. Mistral Embed, OpenAI text-embedding-3, Cohere Embed); vector DB (pgvector, Qdrant) for individual memories.
• Orchestration: Python with TinyTroupe (Microsoft, MIT) for personas and simulated focus groups [11] or Concordia (Google DeepMind, Apache-2.0) for multi-agent social simulation [12]; pin model versions and log prompts/seeds for reproducibility.
• Choose B when: you own rich first-party research data and want a proprietary, validated simulator integrated with your survey platform.

Path C — Self-host (open source)

• Models: open-weight LLMs such as Qwen3, Llama 3.x/4, Mistral Small/Magistral, gpt-oss-120b or DeepSeek served with vLLM or SGLang; multiple families improve diversity.
• Stack: TinyTroupe or Concordia, pgvector/Qdrant, an embedding model (e.g. BGE-M3, multilingual-e5), and Python/R for weighting and validation statistics.
• Hardware: HW3 (one 80–141 GB GPU) for ~70–120B models that give noticeably better persona adherence; HW2 suffices for small-model prototyping. Running thousands of personas × questions is batch work — throughput, not latency, matters.
• Why: full control over model versions (a key reproducibility issue [3]) and no transfer of customer/panel data to third parties.

Path D — Custom training

Worth it for research agencies and large brands with long survey histories: fine-tune an open model on (respondent profile → answer) pairs from past studies so that it learns real answer distributions and variance, or train a calibration model on paired synthetic/real results. Needs tens of thousands of historical responses with consented, pseudonymised profiles, and a strictly held-out real test set per category.

State of the art (2026)

• From demographic prompts to grounded agents: "silicon samples" conditioned on survey backstories [13] have given way to agents grounded in individual interviews and behavioural records [1] and to commercial simulators trained on real outcome data [6][10].
• Better elicitation: SSR and related distribution-matching methods fix the "every LLM says 4 out of 5" problem for purchase-intent scales [2].
• Validation culture: credible vendors now report distribution distances or reliability ratios against real samples; benchmarks such as OpinionQA [4] are used to measure alignment with population groups.
• Persistent failure modes: too little variance and overconfident subgroup differences [3]; stereotyping and flattening of minorities and older people [4][5]; poor performance on truly novel categories, sensitive topics and future behaviour (stated ≠ revealed preference); drift when the underlying model changes; contamination (the model may "know" published survey results). Synthetic results are best for relative ranking of variants and hypothesis generation, weakest for absolute market sizing.

Shared resources & combinations

• Shared LLM + embedding + vector-DB stack with LANG-019 (qualitative coding of real and synthetic interviews), LANG-013 (feedback analysis — supplies grounding data), LANG-017 (market intelligence) and LANG-026 (consultation analysis).
• Natural pipeline: LANG-001 generates copy variants → LANG-027 screens them → DATA-027 A/B tests the finalists with real users → REC-004 segments feed persona definitions.
• DATA-005 synthetic tabular data shares privacy techniques and validation metrics; OPT-004 pricing can use simulated willingness-to-pay as an early prior only.
• EDU-010 role-play training reuses the same persona engine (TinyTroupe/Concordia-type).
• Hardware: one HW3 inference server running an open 70–120B model can serve this together with other batch LLM workloads in the LANG family.

Effort, cost & KPIs

• Path A: days to first results; pilot pricing in the low five figures is common (e.g. Lakmoos €10k one-month pilot [9]); enterprise licences often flat-rate with unlimited queries [7].
• Path B: 1–3 months for a validated simulator; cost dominated by researcher time for validation studies rather than tokens.
• Path C: 2–4 months plus GPU server; Path D: 3–6+ months with a data-science team.
• KPIs: agreement with held-out real samples (rank correlation of concepts, total-variation / KS distance of distributions, sign agreement of drivers), variance ratio synthetic/real, share of variants screened out before fieldwork, time and cost per insight, and — most important — hit rate of launch decisions later confirmed by real data.

People & change

Management and the research or marketing lead decide the question and how far simulated answers may influence decisions. Involve end users (analysts, product managers) early so they learn where the method is weak, and IT for accounts and data access. The DPO is needed if real customer data grounds the personas: check the legal basis for reuse and sign an AVV with each provider; a DPIA is usually not required for aggregated, pseudonymised segment data, but verify per case.

Works council: usually no co-determination, because the system analyses markets, not employee behaviour; if usage is logged per person (§87(1) no. 6 BetrVG applies on objective suitability), inform the council and agree a short rule. Inform it anyway about changed work processes (§90(1) no. 3).

AI literacy: support staff with a short session on bias, overconfidence and why synthetic answers are hypotheses, not evidence (AI Act Art. 4 as amended). Communicate that simulation prepares and narrows real research, it does not replace it. Pilot with one product question and run a parallel real survey to measure how close the results are. Typical concern is false certainty: address it with mandatory validation against real data, labelled outputs ("synthetic") and a human sign-off before decisions.

See GUIDE-008, GUIDE-009 and GUIDE-007.

Risks, regulation & compliance

• EU AI Act: minimal risk in typical use — internal research support is not in Annex III and outputs are not shown to the public as human content. AI literacy (Art. 4) applies. If synthetic quotes or "consumer" testimonials are published, Art. 50 transparency and consumer-protection law (UCPD) apply — presenting simulated answers as real customer opinions is misleading. Using persona simulation to support decisions about individuals (credit, hiring) would move into high-risk categories and is out of scope here.
• GDPR: segment personas built from aggregated or anonymised data involve little personal data; individual-level digital twins built from CRM, transcripts or panel profiles are profiling of personal data — need a legal basis (consent or carefully assessed legitimate interest), purpose-compatibility check (Art. 6(4)) when reusing data collected for other purposes, a DPIA for large-scale profiling, and DPAs with vendors. Respect deletion requests in persona memories.
• Research ethics & standards: ICC/ESOMAR principles require transparency about methods; label synthetic data clearly in every report and never blend it silently with real samples.
• Political use: simulating voter reactions is legal but sensitive; targeting and transparency rules for political advertising (Regulation (EU) 2024/900) still apply to resulting campaigns.
• Mitigations: pin model versions, ensemble models, validate per category, report uncertainty, keep a real-fieldwork gate before high-stakes decisions.

Getting started

1. Pick one decision type (e.g. early concept screening) where you have 3–5 past studies with real results.
2. Re-run those studies synthetically (vendor pilot or TinyTroupe + EU-region LLM) and measure rank agreement, distribution distance and variance vs. the real data.
3. Define acceptance thresholds; use synthetic results only where they pass, and label them.
4. Ground personas in your own first-party data after a GDPR check (aggregate first, individual twins only with consent).
5. Keep a real-respondent validation wave for every launch decision and feed it back into calibration.

Sources

1. Park et al. — LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals (arXiv 2411.10109) — https://arxiv.org/abs/2411.10109
2. Maier et al. — LLMs Reproduce Human Purchase Intent via Semantic Similarity Elicitation of Likert Ratings (arXiv 2510.08338) — https://arxiv.org/abs/2510.08338
3. Bisbee et al. — Synthetic Replacements for Human Survey Data? The Perils of Large Language Models, Political Analysis 32(4), 2024 — https://www.cambridge.org/core/journals/political-analysis/article/synthetic-replacements-for-human-survey-data-the-perils-of-large-language-models/B92267DC26195C7F36E63EA04A47D2FE
4. Santurkar et al. — Whose Opinions Do Language Models Reflect? (OpinionQA, arXiv 2303.17548) — https://arxiv.org/abs/2303.17548
5. Wang, Morgenstern, Dickerson — LLMs that replace human participants can harmfully misportray and flatten identity groups (arXiv 2402.01908) — https://arxiv.org/abs/2402.01908
6. Simile — product page — https://www.simile.com/
7. Electric Twin — product page — https://www.electrictwin.com/
8. Synthetic Users — product page — https://www.syntheticusers.com/
9. Lakmoos — product page — https://www.lakmoos.com/
10. Aaru — product page — https://aaru.com/
11. Microsoft TinyTroupe — https://github.com/microsoft/TinyTroupe
12. Google DeepMind Concordia — https://github.com/google-deepmind/concordia
13. Argyle et al. — Out of One, Many: Using Language Models to Simulate Human Samples (arXiv 2209.06899) — https://arxiv.org/abs/2209.06899
14. Regulation (EU) 2024/1689 (AI Act) — https://eur-lex.europa.eu/eli/reg/2024/1689/oj