Product recommendations in e-commerce

Online shops carry thousands of products, and shoppers see only a few. Recommendation engines combine catalogue data and behaviour to rank what each visitor sees on product pages, in the cart, on the home page and in emails. Mature shops typically see a measurable lift in conversion and average order value; the main work is data quality, consent handling and honest A/B testing, not the algorithm.

Problem & value

• Today: static "bestseller" or manually curated lists, simple rules ("same category"), or the shop platform's default widget. Long-tail items are rarely discovered; new visitors see the same as everyone.
• AI value: relevance per visitor and context (cold-start handled with content embeddings), complementary items (accessories, refills), better use of stock and margin, and less manual merchandising.
• Benefit figures: vendor-reported uplifts vary widely and are not independent evidence. Treat them as hypotheses and measure your own lift with a holdout group (see KPIs). Amazon's item-to-item collaborative filtering paper remains the canonical industrial reference [1].

Variants

• SMB shop (Shopify, Shopware, WooCommerce): plugin or built-in widgets; a few placements.
• Marketplace / large catalogue: two-tower retrieval plus learned ranking, real-time features.
• B2B shop: reorder lists and account-level recommendations, see REC-010.
• Fashion: size/fit and visual similarity, see REC-012 and VIS-022.

How it works

1. events + catalogue + consent flags
2. feature/embedding build (item text+image embeddings, co-purchase statistics)
3. candidate retrieval (item-to-item, two-tower ANN search, popularity/trending)
4. ranking (gradient boosting or neural ranker; CTR/purchase probability)
5. re-ranking with business rules (stock, margin, diversity, exclusions, age-restricted goods)
6. placement API / email block
7. A/B test + holdout
8. feedback events

• Retrieval: matrix factorisation, item-to-item co-occurrence, sequence models (session-based, e.g. transformer-based next-item prediction) or text/image embeddings for cold-start items.
• Ranking: LightGBM/XGBoost on engineered features remains a strong baseline; deep rankers pay off at high traffic.
• LLM role (2026): generating item embeddings from descriptions, explaining a recommendation, conversational shopping assistants, and synthetic cold-start features. LLMs rarely replace the ranker itself at scale because of latency and cost.
• Human-in-the-loop: merchandisers set rules, pin or exclude items and review experiment results.

Data preparation

Data needed: product catalogue (titles, descriptions, images, attributes, stock), behavioural events (views, clicks, add-to-cart, orders) and the consent state per visitor. In an SMB these typically live in the shop system or ERP, web analytics, the newsletter tool and sometimes Excel price lists.

Typical gaps: thin or inconsistent product descriptions, missing attributes, duplicate variants, order data not linked to web sessions, events tracked without consent flag, no stable customer ID across devices, and little history for new shops (cold start).

Preparation steps: - Clean the catalogue: unique IDs, normalised attributes and categories (PREP-011, PREP-019), complete texts and images. - Export orders and events via API or connector (PREP-005); join them with a consent flag and pseudonymise IDs (PREP-022, PREP-025). - Define business rules (margin, stock, exclusions) and a KPI glossary (PREP-012). - Set up a clean A/B holdout from day one.

Tools: shop export or API, dbt or simple SQL/Python scripts, Snowplow or server-side tracking, a consent management platform.

Effort share: roughly 30-40 percent of the project; with a bought app much of it is catalogue and tracking clean-up. See also GUIDE-003.

Implementation paths

Which path for whom

• SMB: A. Mature, EU-hostable recommendation apps for Shopify, Shopware and similar shops exist at SMB prices, and the process is standard.
• Enterprise: B. Ranking is close to the core of the business, so custom rules, own event data and an own placement API on cloud APIs pay off and avoid per-session fees at high volume.
• Developer: C. Open-source recommenders (Gorse, implicit, RecBole) with pgvector or Qdrant run on CPU and give full control over data and ranking logic.

Path A — Buy (recommended for most SMBs)

• Personalisation platforms: Bloomreach, Algolia Recommend, Constructor, Dynamic Yield (Mastercard), Nosto, Klevu, Clerk.io (Denmark), Epoq (Germany), Prudsys (Germany), Emarsys/SAP, Raptor Services (Denmark), Relewise (Denmark). EU-based vendors are easier for GDPR contracts, but check sub-processors.
• Hyperscaler services: Google Vertex AI Search for commerce, Amazon Personalize (EU regions) [2][3].
• Shop-platform apps: Shopify and Shopware ecosystems offer recommendation apps.
• Choose A when: under a few hundred thousand sessions per day and no unusual ranking logic. Check consent-mode support, data location, export of your event data and A/B tooling.

Path B — Build on cloud APIs

• Amazon Personalize or Vertex AI Search for commerce as the model layer; your own event pipeline (Snowplow, Segment-like CDP, or server-side tracking), feature store and placement API.
• Text/image embeddings from a cloud embedding API (e.g. Mistral Embed, Cohere Embed, OpenAI, Gemini embeddings) stored in pgvector, OpenSearch or Qdrant for similar-item search.
• LLM layer for explanations or bundle copy via an EU-region endpoint; keep prompts free of personal data.

Path C — Self-host (open source)

• Libraries: implicit, LightFM, RecBole, TorchRec (Meta), TensorFlow Recommenders, NVIDIA Merlin (Transformers4Rec, HugeCTR) [4]; Gorse as a lightweight recommender service.
• Serving: Qdrant, Milvus or pgvector for ANN retrieval; LightGBM for ranking; Feast as feature store.
• Embeddings: open multilingual models such as BGE-M3, multilingual-e5 or Jina embeddings; SigLIP for images.
• Hardware: classic recommenders run on CPU; HW2 suffices for embedding and training a mid-size shop. Mid-size catalogues (under ~1M items) need no GPU in serving.

Path D — Custom training

Worth it for large catalogues and traffic (millions of events per day), when the lift of a custom sequence or two-tower model over a bought engine justifies a team. Needs months of clean event history; training data is your own logs with careful time-based splits to avoid leakage.

State of the art (2026)

• Two-stage architecture (retrieval then ranking) with ANN search is the industry standard; sequence-aware transformers and generative retrieval (semantic item IDs) are moving from research into production at the largest platforms [5].
• Offline metrics mislead: NDCG/recall gains often do not translate into online lift; popularity baselines are hard to beat on small data. Evaluate with online A/B tests and holdouts.
• LLM-based recommenders and conversational shopping assistants are being deployed, but they can hallucinate products or ignore stock; ground them on catalogue retrieval.
• Limits and failure modes: cold start, popularity bias and filter bubbles, feedback loops, recommending already-purchased or out-of-stock items, and sparse data for small shops where simple rules (same category, bestsellers, "bought together") perform nearly as well.

Shared resources & combinations

• Shared stack (embedding-text + vector-db): KNOW-005 semantic product search and VIS-022 visual search use the same item embeddings and index.
• Combine with: REC-006 (email blocks), REC-003 (next-best-action), PRED-020 (return prediction to avoid pushing high-return items), REC-012 (size/fit), REC-013 (sponsored ranking), AGENT-020 (shop operations agent). Event tracking and consent management are shared with all REC marketing cases.

Effort, cost & KPIs

• Path A: days to weeks; usually priced per sessions/requests or a share of revenue.
• Path B: 1-3 months for a pipeline, placements and experimentation; cloud cost is mostly event processing.
• Path C/D: 3-6+ months and an ML/data engineering team.
• KPIs: click-through rate on widgets, conversion rate, average order value, revenue per visitor (all from a randomised holdout), catalogue coverage, share of long-tail items shown, return rate of recommended items, latency (p95), and complaints or opt-out rate.

People & change

Involve: management (budget, goals) and the shop or marketing lead (process owner) from the start; IT for tracking, feeds and integration; the data protection officer early, because behavioural tracking and profiling need a legal basis and consent (GUIDE-005); data owners of catalogue and ERP for clean data.

Works council: usually no co-determination under §87(1) no. 6 BetrVG, because the system evaluates shoppers, not employees. It becomes relevant if staff dashboards log individual merchandisers' activity; then inform the council early (§90(1) no. 3 BetrVG, GUIDE-004).

Data protection: an AVV (Art. 28 GDPR) with every vendor; a DPIA (Art. 35 GDPR) is likely only for large-scale profiling.

AI literacy: the AI Act Art. 4 requires measures that support AI literacy; a short session for merchandisers and marketing on what the ranker does, its limits and how to set rules is enough (GUIDE-006).

Pilot and communication: start with one placement (product page or cart) and a holdout group, report uplift openly, and explain that the engine suggests while merchandisers keep control through pins, blocks and exclusion rules. Typical concerns are fear of losing curation control and filter bubbles; address them with editable rules and diversity checks. See GUIDE-009 and GUIDE-008.

Risks, regulation & compliance

• EU AI Act: typically minimal risk; no Annex III category. AI literacy (Art. 4) applies. Art. 5 prohibits manipulative or exploitative techniques that materially distort behaviour; avoid dark patterns, exploiting vulnerabilities (minors, financial distress) and hidden urgency signals [6].
• GDPR / ePrivacy: behavioural profiling needs a lawful basis; cookies or device access for tracking usually need consent (ePrivacy Directive Art. 5(3)). Contextual recommendations (current product, no profile) need far less. Honour opt-out, minimise retention, document legitimate interest assessments where relied on, and sign data processing agreements. Art. 22 GDPR is rarely triggered, as recommendations have no legal or similarly significant effect [7].
• Digital Services Act: online platforms and marketplaces must describe the main parameters of their recommender systems in the terms (Art. 27); very large platforms must offer a non-profiling option (Art. 38) [8].
• Consumer law: personalised pricing must be disclosed (Consumer Rights Directive); clearly label sponsored placements.
• Mitigations: consent-aware tracking, contextual fallback, diversity constraints, age-restricted item exclusions, audit log of rules, and bias checks across customer groups.

Getting started

1. Fix tracking first: server-side events, consent states, stable product IDs and a clean catalogue feed.
2. Start with simple placements (similar items on product pages, "bought together" in the cart) using a bought tool or built-in widget.
3. Run a randomised A/B test with a holdout for at least two business cycles; define the primary KPI before launch.
4. Add business rules (stock, margin, exclusions) and a contextual fallback for non-consenting visitors.
5. Only then evaluate custom models or embeddings-based cold-start for new items.

Sources

1. Linden, Smith, York — Amazon.com recommendations: item-to-item collaborative filtering (IEEE Internet Computing, 2003) — https://www.cs.umd.edu/~samir/498/Amazon-Recommendations.pdf
2. AWS — Amazon Personalize developer guide — https://docs.aws.amazon.com/personalize/latest/dg/what-is-personalize.html
3. Google Cloud — Vertex AI Search for commerce — https://cloud.google.com/solutions/retail-product-discovery
4. NVIDIA Merlin recommender framework — https://github.com/NVIDIA-Merlin/Merlin
5. Rajput et al. — Recommender Systems with Generative Retrieval (NeurIPS 2023) — https://arxiv.org/abs/2305.05065
6. Regulation (EU) 2024/1689 (AI Act), Art. 4 and 5 — https://eur-lex.europa.eu/eli/reg/2024/1689/oj
7. Regulation (EU) 2016/679 (GDPR) — https://eur-lex.europa.eu/eli/reg/2016/679/oj
8. Regulation (EU) 2022/2065 (Digital Services Act), Art. 27 and 38 — https://eur-lex.europa.eu/eli/reg/2022/2065/oj