Deep-research agent for market & desk research

Desk research (market sizing, competitor profiles, supplier scans, technology overviews) normally takes an analyst days of searching, reading and note-taking. A deep-research agent decomposes the question, runs many searches, reads the pages, reconciles conflicting sources and writes a cited report in minutes to an hour. It is a strong first-draft and breadth tool; the human still verifies key claims and owns the conclusion.

Problem & value

• Today: staff use search engines, paywalled databases and spreadsheets; or buy consultancy/analyst reports. Quality depends on who has time. Results are rarely reproducible or refreshed.
• AI value: breadth (many sources and languages), speed, a consistent report structure, and repeatability (the same brief can be re-run monthly). Typical use: pre-meeting briefings, supplier longlists, market entry scans, grant/trend scans, due-diligence pre-reads.
• Evidence on capability: OpenAI's BrowseComp benchmark of 1,266 hard web-lookup questions was created because simple fact benchmarks were saturated [1]; DeepResearch Bench provides 100 PhD-level tasks in 22 fields, scored on report quality (RACE) and citation grounding (FACT) [2]. These measure agents, not your business outcome - measure your own time saved (see KPIs). No vendor time-saving figure is quoted here because none was verified from a primary source.

Variants

• Market and competitor scan (overlap with LANG-017: that use case monitors competitors continuously from news feeds; this one produces an on-demand, question-driven report).
• Supplier and partner longlist for procurement (feeds AGENT-005).
• Technology / trend radar for R&D and innovation management.
• Public-sector desk research on funding, statistics, peer municipalities (see KNOW-017 for tenders).
• Hybrid web + internal: same agent also searches the company DMS or SharePoint (KNOW-002).
• Private use: buying decisions, travel, health-admin or study topics, with a consumer subscription.
• Out of scope: scientific reviews (KNOW-008), legal (KNOW-006), patents (KNOW-007), financial filings (KNOW-023) and person-related OSINT (KNOW-026) have their own sources and rules.

How it works

brief (question, scope, audience)
  -> clarifying questions + research plan (human can edit the plan)
  -> supervisor agent spawns parallel sub-researchers per sub-question
  -> search (web search API / internal index / MCP connectors) -> fetch -> parse (HTML, PDF)
  -> per-source notes with quote + URL, deduplication, source-quality scoring
  -> gap check ("what is still unknown?") -> further search rounds
  -> synthesis and report writing with inline citations
  -> verification pass (claim-to-source check, numbers recomputed) -> human review

• Core pattern: an LLM with tool use in a loop (plan, search, read, reflect), mostly a supervisor with parallel workers. Reasoning models are used for planning and synthesis, cheaper models for page summarisation.
• Open reference designs: STORM/Co-STORM (outline-first, multi-perspective questioning, human can join the discussion) [3]; LangChain Open Deep Research (LangGraph, four configurable model roles) [4]; GPT Researcher (planner + execution agents, also research over local documents) [5].
• Human-in-the-loop: approve the plan, whitelist/blacklist domains, check the 5-10 claims that drive the decision, and sign off before the report leaves the company.

Data preparation

Most input is the open web, so little classic data preparation is needed. What matters is the internal side: the question, the sources and the quality criteria. Where internal documents should be included, they sit in file shares, a DMS, SharePoint, CRM notes and mailboxes, often as scans or outdated versions. Typical gaps are vague research questions, no list of trusted sources, internal reports without metadata, and confidential files that must not reach an external tool.

Preparation steps: (1) write a standard research brief - question, scope, time frame, required sources, output format (PREP-016); (2) maintain a list of preferred and excluded sources and company terminology (PREP-015); (3) for internal sources, select a clean subset, OCR scans and remove duplicates (PREP-006, PREP-009); (4) set rules on which data may be entered into which tool (PREP-024); (5) collect two or three past reports as gold-standard examples and a check list for verification (PREP-018).

Tools: the research product's own file upload, Paperless-ngx for scans, a shared folder for approved sources. Effort share is low, about 10-20 percent; most time goes into prompt templates and verification routines.

Implementation paths

Which path for whom

• SMB: Path A - mature deep-research products exist at low per-seat prices and desk research is a standard process that needs no integration.
• Enterprise: Path B - analysts need reports grounded in internal sources and own quality rules, which a build on cloud APIs with a shared platform supports.
• Developer: Path C - open agent frameworks with local or open-weight models allow experiments and control over search sources and cost.
• Private: Path A - consumer research modes are good enough for study and personal projects.
• Public sector: Path C - sovereign or self-hosted setups avoid sending queries about procurement or policy topics to non-EU services.

Path A - Buy (recommended for most SMBs)

• Assistant built-ins: ChatGPT Deep Research (OpenAI), Gemini Deep Research (Google), Claude Research (Anthropic), Perplexity Deep Research, Microsoft 365 Copilot Researcher, Mistral Le Chat deep research (EU, France). Business tiers offer no-training-on-your-data terms and some EU data residency; verify per vendor.
• EU / German options: Mistral Le Chat (EU provider), Langdock (German, multi-model workspace with agents), Aleph Alpha-based stacks (check current product scope before buying).
• Specialised tools: AlphaSense, Hebbia (finance, see KNOW-023), Elicit/Consensus (science, see KNOW-008).
• Choose A when: research is occasional, topics are public, no confidential data enters the prompt, and speed of start matters. Always test with a brief where you already know the answer.

Path B - Build on cloud APIs

• Models: a frontier reasoning model for planning and writing (Claude, GPT, Gemini, Mistral Large via EU regions / EU providers) and a small, cheap model for page summaries. Providers also expose built-in web-search and fetch tools in their agent APIs.
• Search layer: Tavily, Exa, Brave Search API, Linkup (French) or Perplexity Search API [6]; Firecrawl or Jina Reader for page extraction. Check where queries are processed (US-based services receive your search terms).
• Orchestration: LangGraph (Open Deep Research as template) [4], or a plain supervisor loop with an agent SDK; MCP servers for internal sources and licensed databases (see AGENT-013).
• Choose B when: you need your own source whitelist, report templates, internal data, scheduled re-runs or integration into CRM/Notion/SharePoint. A first version takes days.

Path C - Self-host (open source)

• Orchestrators: GPT Researcher (Apache-2.0) [5], Open Deep Research (MIT, repository now archived) [4], STORM (MIT) [3].
• Models: Tongyi DeepResearch 30B-A3B (Apache-2.0, 30.5B total / ~3.3B active parameters, 128K context), trained specifically for agentic web search [7]; general open models (Qwen3, gpt-oss, Mistral, Llama families) served with vLLM or Ollama.
• Search and crawling: SearXNG (self-hosted metasearch), Crawl4AI, Playwright, plus an internal index (OpenSearch/Qdrant) for company documents.
• Hardware: the MoE agent model fits one 48-80 GB GPU (HW3) when quantised; smaller models run on HW2 but plan quality and long-context synthesis drop noticeably. Note that the web search itself still leaves the network; "on-prem" applies to the model and to internal documents only.

Path D - Custom training

Rarely worth it. Consider only for a vendor-scale product or a closed domain with a large trajectory corpus (plan/search/read traces with expert-rated reports): fine-tune a mid-size open model with supervised and reinforcement learning on search trajectories, as the Tongyi team did [7]. For companies, better ROI comes from prompt/template tuning, curated source lists and a reranker.

State of the art (2026)

• Productised agents from all major model providers run for several minutes to an hour, read dozens to hundreds of pages and return cited reports; Perplexity now also sells search, agent and embedding APIs separately [6], so the "search layer" has become a commodity building block.
• Benchmarks: BrowseComp (1,266 questions) measures persistent browsing [1]; DeepResearch Bench measures report quality and citation accuracy with an LLM judge [2]. In the LangChain project's own runs, scores varied with the model (RACE 0.4309 with gpt-4.1 defaults vs 0.4943 with GPT-5 as research model), at very different costs - model choice is a quality and cost lever [4]. Leaderboards change monthly; check the live one rather than a vendor claim.
• Techniques: supervisor/sub-agent parallelism, context compression between rounds, plan editing by the user, MCP connectors for private data, claim-level citation verification.
• Failure modes: (1) fabricated or mis-attributed citations - always click through; (2) reliance on SEO-spam and AI-generated pages as "sources"; (3) bias towards English and easily crawlable sites, paywalled and PDF-only data missed; (4) outdated figures presented as current; (5) confident synthesis over contradictory sources; (6) prompt injection hidden in web pages that tries to steer the agent or leak data; (7) market-size numbers copied from low-quality aggregators.

Shared resources & combinations

• Same building blocks (agent-framework + web-scraping + llm-reasoning + search/vector index): LANG-017 (competitive monitoring), KNOW-008 / KNOW-006 / KNOW-007 (specialised research with other sources), KNOW-023 (financial sources), KNOW-026 (OSINT), AGENT-005 (procurement agent consuming the supplier longlist), AGENT-007 (prospect research for sales outreach), KNOW-017 (tender discovery).
• Shared infrastructure: one self-hosted LLM server (HW3), the crawler/search gateway, the internal index from KNOW-001/KNOW-002 and the MCP connector layer can serve all of them.
• Downstream: KNOW-013 (turn finished reports into a knowledge graph), KNOW-018 (store notes in a personal knowledge base), report templates as slide decks/documents.

Effort, cost & KPIs

• Path A: hours to days; per-seat subscription, sometimes with usage caps on deep-research runs.
• Path B: days to ~4 weeks for a branded workflow; cost = tokens for dozens of page reads per report plus search API calls. Cost per report varies by an order of magnitude with model choice [4]; set a budget per run.
• Path C: 1-3 months incl. search gateway, evaluation and hosting; GPU server plus maintenance.
• KPIs: analyst hours per report, share of claims verified correct in spot checks (target set by you), citation accuracy (share of links that really support the claim), source diversity, cost per report, freshness of data, user adoption and rework rate.

People & change

Involve management to agree the purpose and which decisions may rest on AI-generated research, the department lead who owns the research process, and the end users - analysts, product managers, founders - who test results. IT clears accounts and tools; legal/compliance and the DPO set rules for confidential or personal data in prompts (see GUIDE-007 and GUIDE-005).

Works council: usually no. The tool supports content work and does not evaluate staff; however, if the vendor provides usage dashboards per person or the tool is rolled out as part of a larger suite that logs activity, §87(1) no. 6 BetrVG can apply, and §90(1) no. 3 asks for timely information on AI in work processes. Check the vendor's logging before roll-out; public bodies involve the Personalrat. Personal data is mainly an issue when researching named individuals or uploading customer files - keep these out of the first phase and conclude an AVV with the provider (GUIDE-010).

AI literacy: support staff with training on source checking, citation verification and hallucinated references (GUIDE-006). Communicate that the agent produces a first draft, not a finished analysis; every figure used in a decision must be traced to its source by a person. Start with a pilot group of 3-5 analysts on real questions and compare results with manual research (GUIDE-009). Typical concern: quality and plagiarism - address with a review checklist and source whitelist.

Risks, regulation & compliance

• EU AI Act: typically minimal risk; no Annex III category, AI literacy (Art. 4) applies. Art. 50 transparency (applicable from 2 August 2026) matters if AI-generated text is published to inform the public on matters of public interest without human editorial review; marking duties for generated content lie primarily with providers [8]. If the agent is used to profile individuals for HR, credit or law-enforcement decisions the classification changes - out of scope here (see KNOW-026).
• GDPR: briefs and web pages may contain personal data (managers, sole traders, reviewers). Avoid putting personal or confidential data into prompts of consumer tools; use business terms with a data processing agreement, EU residency where required, and minimise person-related research (purpose limitation, Art. 5-6 GDPR).
• Copyright and terms of use: respect robots.txt and licences when crawling; no wholesale republishing of paywalled content; EU text-and-data-mining exceptions (DSM Directive Art. 3/4) have opt-out reservations.
• Security: indirect prompt injection through web pages; give the agent read-only tools, no access to internal write actions, egress filtering, and separate internal-data and open-web agents.
• Quality and liability: label output as AI-assisted, keep the source list and retrieval date, verify numbers that drive decisions or investments.

Getting started

1. Pick 3 recurring research tasks and write the brief (question, scope, audience, output format, source rules).
2. Run them in two Path A tools and compare with a past human-made report; count wrong or unsupported claims.
3. Define a "source policy" (trusted domains, banned aggregator sites, languages) and a report template.
4. If results are good, wrap in Path B (LangGraph or a plain loop) with internal data via MCP, budget limits and a citation-check step; schedule monthly re-runs.
5. Train staff: verify top claims, never paste confidential data into consumer tools, document the review.

Sources

1. OpenAI / arXiv - BrowseComp: a benchmark for browsing agents (1,266 questions) - https://arxiv.org/abs/2504.12516
2. DeepResearch Bench (100 PhD-level tasks, RACE and FACT metrics) - https://github.com/Ayanami0730/deep_research_bench
3. Stanford OVAL - STORM and Co-STORM - https://github.com/stanford-oval/storm
4. LangChain - Open Deep Research (architecture, Deep Research Bench runs and cost) - https://github.com/langchain-ai/open_deep_research
5. GPT Researcher - https://github.com/assafelovic/gpt-researcher
6. Perplexity API documentation (Agent, Search, Embeddings APIs) - https://docs.perplexity.ai/
7. Alibaba Tongyi DeepResearch (30B-A3B, Apache-2.0) - https://github.com/Alibaba-NLP/DeepResearch
8. AI Act Article 50 transparency obligations (applies from 2 August 2026) - https://artificialintelligenceact.eu/article/50/
9. Regulation (EU) 2024/1689 (AI Act) - https://eur-lex.europa.eu/eli/reg/2024/1689/oj