Software users struggle to find features, configure complex settings and build reports, and support teams answer the same how-to questions again and again. An in-product copilot lives inside the application, knows which screen and object the user is on, answers with the user's own data, and executes tasks by calling the product's API under the user's permissions. The aim: faster onboarding, higher feature adoption and fewer support tickets. This is a core architecture for software vendors (ISVs), not a generic chatbot.
Problem & value
- Today: users read help centres, open tickets, or click through menus. Mature SaaS has hundreds of settings, filters and report builders; most users use a small part of them. Support and customer-success teams carry the load of how-to questions.
- Why distinct: an external FAQ bot (CONV-001) and document Q&A (KNOW-002) have no app state. The copilot needs context (page, selection, role), tool calls into the product and UI rendering inside the product.
- AI value: natural-language access to features ("show overdue invoices by region as a chart and schedule it weekly"), contextual onboarding, ticket deflection, and a product-differentiating capability. Vendors report deflection and adoption gains, but figures are self-reported and depend heavily on scope; measure your own baseline rather than assuming a number.
- Market: nearly every major business-software vendor now ships an embedded assistant (e.g. Microsoft Copilot in 365/Dynamics, Salesforce Agentforce, ServiceNow, SAP Joule [EU], Atlassian Rovo, Notion AI, Intercom Fin, HubSpot Breeze), and developer toolkits for embedding copilots raised significant funding in 2026 [3].
Variants
- Guide-only copilot: answers and walkthroughs, no write access. Lowest risk; typical first release.
- Action copilot: executes create/update operations after user confirmation.
- Agentic copilot: multi-step tasks (set up a project, migrate data) with plan preview.
- Generative UI: the assistant renders forms, tables and charts in the product's design system.
- Vertical: ERP/HR/finance (SAP Joule, DATEV, Personio-style assistants), CRM, analytics/BI (natural-language queries), low-code platforms, developer tools and admin consoles.
- Platform/MCP outward: the same API surface is exposed as an MCP server so external assistants can use it [2].
How it works
- user message + app context (route, selected entity, role, locale)
- context builder (what the user may see; tenant-scoped)
- retrieval (docs, help centre, release notes; hybrid search + reranker)
- LLM with tool definitions (product API / MCP tools, read vs write)
- policy layer (user's token, RBAC, rate limits, injection filters)
- tool execution under the USER's identity
- confirmation UI for writes
- result rendered in app UI
- audit log + feedback (thumbs, escalation to support)
- Identity propagation: the copilot calls the same services, validators and permission checks as the UI, with the end user's token, never a broad service account. MCP itself does not solve authorisation; the host application must [2].
- Tools: small, well-described, idempotent functions; separate read and write tools; dry-run or preview for destructive actions.
- Context window hygiene: pass only the data needed for the current screen; summarise long histories; keep tenant data strictly isolated.
- Human-in-the-loop: explicit confirmation for writes, deletions, payments, sending messages and permission changes; easy undo; handoff to a human agent with the conversation attached.
- UI protocol: frameworks such as CopilotKit with the AG-UI event protocol stream agent state and tool calls to the front end [3][4].
Data preparation
The copilot needs three kinds of content: help-centre articles, release notes and manuals (usually in a CMS, wiki, Confluence or PDFs), the product's API schemas (OpenAPI, MCP tool definitions) and the tenant data it reads at run time through the API. Typical gaps: outdated or contradictory articles, features documented only in support tickets or developer heads, missing plan/version labels, duplicate pages in several languages, and API descriptions that are too terse for a model to choose the right tool.
Preparation steps: curate one authoritative knowledge base and retire stale pages (PREP-013, PREP-017), add metadata for product version, plan and language (PREP-010), write tool descriptions with examples and clear read/write separation, and collect a golden set of real how-to questions with expected answers and API calls from support tickets (PREP-018, PREP-020). Check that the user's permissions are enforced in the API, not only in the UI (PREP-023), and pseudonymise ticket data before using it for evaluation (PREP-022). Tools: a docs crawler or CMS export, pgvector/Qdrant, and a staging tenant with synthetic data.
Rough share of effort: about 25-35 percent, mostly documentation clean-up and evaluation sets. Without it the copilot answers confidently from outdated text. See GUIDE-003 for a readiness check.
Implementation paths
Which path for whom
- smb: B, because the value lies in actions through your own product API and permission model, which off-the-shelf FAQ bots cannot do; use A only for a docs-only guide copilot.
- enterprise: B, because the copilot is part of a strategic product and needs tenant isolation, own evaluation and control over cost per conversation.
- developer: B, because tool calling, MCP and UI frameworks on EU-region APIs give the fastest route; move to C only for on-prem customers.
Path A — Buy (off-the-shelf)
- Embeddable assistant platforms: Intercom Fin, Zendesk AI agents, Ada, Kapa.ai (docs-grounded assistants), Algolia/Glean-style search with chat, Pendo and Appcues-style guides with AI. EU options: Mistral Le Chat Enterprise/agents, Aleph Alpha-based offerings, SAP Joule (for SAP customers), Parloa and Cognigy (NICE) for service contexts.
- Choose A when: you need docs-grounded how-to answers fast and no deep in-app actions. Check EU hosting, data-processing agreement, no training on your data, SSO and tenant isolation.
Path B — Build on cloud APIs (recommended for software vendors)
- Models: frontier models with tool calling and structured outputs (Claude, GPT, Gemini, Mistral Large/Medium) via EU regions (Azure OpenAI EU Data Zone, Google Vertex EU, AWS Bedrock Frankfurt/ Paris, Mistral La Plateforme as EU provider). Use a small, fast model for routing and a stronger one for planning.
- Orchestration: Vercel AI SDK, LangGraph, Mastra, Pydantic AI, or the vendors' agent SDKs; front end with CopilotKit, assistant-ui or custom React components [3][4].
- Integration: tools generated from your OpenAPI spec; optional MCP server for external clients [2]; retrieval with pgvector/OpenSearch/Qdrant over docs.
- Safety: prompt-injection filters, output validation against schemas, per-tool allow-lists [5].
Path C — Self-host (open source)
- Models: open-weight tool-calling models such as Qwen3, Mistral Small/Medium open releases, Llama 3.x, gpt-oss, served with vLLM or Ollama; embeddings such as BGE-M3 or multilingual-e5.
- Stack: LangGraph or Mastra, Qdrant/pgvector, Langfuse for tracing, Guardrails/LLM Guard for filters.
- Hardware: HW3 for a shared 30–120B tool-calling model under load; HW2 for a quantised ~30B model at pilot scale. Needed for on-prem or sovereignty-driven customers (public sector, banks).
Path D — Custom training
Fine-tune a small model for tool selection and your domain vocabulary only after logs from Path B/C show systematic tool-calling errors or cost pressure. Training data: real, consented and anonymised conversation traces with correct tool calls; evaluation sets matter more than volume.
State of the art (2026)
- Tool use is mature: strong models call dozens of tools reliably with structured outputs; MCP became the common way to expose product capabilities to assistants [2].
- Agent-UI protocols (AG-UI) and generative UI make assistants feel native rather than a chat popup [4].
- Evaluation: task-completion tests on a scripted staging tenant (golden conversations with expected API calls) are the practical standard; track tool-call accuracy and false-action rate.
- Limits and failure modes: wrong tool or parameter choice, hallucinated features that do not exist in the user's plan, stale docs, long multi-step tasks drifting, latency (multi-second tool loops) and cost per conversation. Prompt injection via user-generated content in the app (tickets, comments, imported files) is the main security risk: a copilot with write access can be steered into data exfiltration or unwanted actions [5].
Shared resources & combinations
- Retrieval stack (embedding-text, vector-db, reranker) shared with KNOW-002, KNOW-004 and CONV-001.
- Agent/tool layer shared with AGENT-002 (workflow automation) and AGENT-003 (computer use, a fallback when no API exists).
- Agent assist and escalation: CONV-007 for the human support side; CONV-004 for the internal IT variant.
Effort, cost & KPIs
- Path A: days to weeks, per-resolution or per-seat pricing.
- Path B: typically 2–6 months for a first action copilot (tool design, permission model, evaluation harness, UI); ongoing token cost scales with conversations and context size.
- Path C: 4–9 months incl. GPU operations.
- KPIs: task-completion rate, tool-call accuracy, false-action/undo rate, ticket deflection for how-to topics, feature adoption of targeted features, time-to-first-value for new users, CSAT, cost per conversation, escalation rate, injection-test pass rate.
People & change
Involve management early for scope, budget and AI policy; the department lead (product or support) owns the use case and the success metrics. IT and information security design identity propagation, tenant isolation and injection tests before the pilot (GUIDE-013). The data protection officer checks the legal basis, processor contracts and chat-log retention, and decides on a DPIA if the copilot reads broad tenant data (GUIDE-005). Customers should be told clearly that they talk to an AI (Art. 50 AI Act).
Works council: depends. If the copilot is customer-facing only, co-determination is usually not triggered; as soon as employees use it (support agents, internal admin copilot) and usage is logged, §87(1) no. 6 BetrVG applies because objective suitability for monitoring is enough, and §90(1) no. 3 requires early information about AI-supported work processes (GUIDE-004).
AI literacy: support staff and product teams need training on limits and failure modes; the Act asks for measures that support literacy, not a guaranteed level (GUIDE-006). Start with a beta cohort and AI champions from support, publish what the copilot may and may not do, and address the fear of job loss by positioning it as relief for repetitive how-to questions (GUIDE-008). Writes always need user confirmation, and a human handoff stays one click away.
Risks, regulation & compliance
- EU AI Act: usually limited risk: users must be told they interact with an AI system (Art. 50(1)) and AI-generated content must be marked as required. Becomes high risk only if the copilot is used inside Annex III use cases (e.g. HR decisions, credit scoring) offered in the product; then the provider's obligations apply. Art. 4 AI literacy applies to providers and deployers. Prohibited practices (Art. 5), such as manipulative techniques or emotion inference in the workplace, must not be built into engagement features [1].
- GDPR: the copilot processes personal data of users and of data subjects in the tenant. Needs a clear role split (the customer is controller, the vendor processor), DPA with the model provider, EU region or SCCs, no training on customer data by default, retention limits for chat logs, DPIA for broad data access, and Art. 22 care if actions have legal or similarly significant effects.
- Security: OWASP LLM Top 10 risks: prompt injection, excessive agency, sensitive information disclosure, insecure output handling [5]. Mitigate with least privilege, user-token execution, confirmations, output encoding, tenant isolation tests and red teaming.
Getting started
- Pick 10–20 top how-to ticket topics and 5 high-value in-app tasks; define success metrics.
- Ship a read-only, docs-grounded copilot first; log every question to find gaps.
- Define the tool layer from your OpenAPI spec; enforce user-token execution and RBAC; add confirmations.
- Build an evaluation harness with scripted conversations on a staging tenant, plus injection tests.
- Roll out to a beta cohort with feedback buttons and human escalation; review logs weekly.
Sources
- Regulation (EU) 2024/1689 (AI Act) — https://eur-lex.europa.eu/eli/reg/2024/1689/oj
- Model Context Protocol specification (incl. authorization) — https://modelcontextprotocol.io/specification
- TechCrunch — CopilotKit raises $27M to help devs deploy app-native AI agents (2026-05-05) — https://techcrunch.com/2026/05/05/copilotkit-raises-27m-to-help-devs-deploy-app-native-ai-agents/
- AG-UI protocol documentation — https://docs.ag-ui.com
- OWASP Top 10 for LLM Applications — https://genai.owasp.org/llm-top-10/
- Regulation (EU) 2016/679 (GDPR) — https://eur-lex.europa.eu/eli/reg/2016/679/oj