Counterfeit & brand-infringement detection on online marketplaces

Fake and infringing goods are sold through marketplaces, social shops and standalone web shops faster than a legal team can search for them by hand. Brand-protection systems crawl these channels continuously, compare listing images, logos, text, prices and seller behaviour with the brand's own assets using multimodal embeddings, rank suspects by risk and submit takedown notices (including DSA notice-and-action) with a human approving the borderline cases. The same techniques run on the other side, inside marketplaces, to stop listings before they go live.

Problem & value

• Today: brand owners search marketplaces by keyword, rely on a few agencies, and file notices manually. Sellers re-list under new accounts, change titles, crop or mirror images and use look-alike logos, so manual work never catches up. Marketplaces must screen millions of new listings and respond to notices.
• AI value: image/logo similarity and text understanding find copies that keyword search misses, seller-level scoring finds repeat infringers, and automation of reporting shortens the time a listing stays live. Humans keep the legal judgement (is it fake, grey-market, parallel import or legitimate resale?).
• Evidence: commercial providers report scale rather than audited effect sizes. Red Points, for example, states on its website coverage of 5,000+ marketplaces and 1,300+ brands and publishes customer-reported removal counts (e.g. 638,653 listings removed for FILA) — vendor claims, not independent measurements [1]. Insist on a measured pilot with your own brand before trusting any rate.
• Context: the OECD/EUIPO work on trade in counterfeit goods and the EUIPO observatory are the standard references for market size; consult them for current figures rather than relying on vendor marketing.

Variants

• Brand owner (SMB to enterprise): monitor 5–50 marketplaces and social platforms for own brand, product designs and logos; typical in fashion, sportswear, cosmetics, electronics, spare parts, toys, food supplements.
• Spare parts / industrial (Mittelstand): counterfeit bearings, filters, tools — detection relies on part numbers, photos, serial/QR plausibility and price outliers; connects to KNOW-021.
• Marketplace operator: pre-publication screening of new listings, trusted-flagger intake, repeat-infringer detection and trader-traceability checks.
• Grey-market and unauthorised-reseller monitoring: MAP (minimum advertised price) and authorised-channel enforcement, not just fakes.
• Social commerce and ads: Instagram/TikTok/Telegram shops, sponsored ads for fake shops (overlaps with SEC-021 lookalike domains).
• Content and design infringement: stolen product photos and copied designs; for media/music piracy see ANOM-035; for trademark clearance before launch see KNOW-022.
• Private use: consumers checking a suspicious seller or product page ("is this shop/listing fake?") — mostly via browser safety tools; no dedicated stack needed.

How it works

1. brand data (logos, product photos, catalogue, authorised sellers, price lists)
2. crawler / marketplace APIs / reverse-image + keyword discovery (web-scraping)
3. listing normalisation (OCR on images, language detection, translation)
4. multimodal matching: image + logo embeddings (vector search), VLM check for logo/design/packaging details, text classifier for cues ("1:1", "AAA", "replica"), price & shipping-origin outliers
5. seller scoring: tabular model + graph analysis (shared images, addresses, payment/contact links)
6. risk ranking
7. analyst review screen with evidence side by side
8. action: notice-and-action report / brand-registry report / escalation to counsel, customs or test purchase
9. monitoring of outcome, re-listing detection, feedback labels to the models

• Discovery: keyword and reverse-image search, marketplace APIs where offered (many platforms expose brand-owner reporting tools), and compliant crawling; respect terms of use and robots/ToS.
• Matching: self-supervised or contrastive image encoders (DINOv2, SigLIP 2, OpenCLIP) give robust near-duplicate and "same-product" retrieval; a VLM then inspects fine detail (stitching, logo geometry, packaging text) and writes a short rationale. Final authenticity is rarely decidable from a listing photo alone — test purchases and lab or brand-expert checks remain the ground truth.
• Human-in-the-loop: high-confidence exact copies of protected images can be auto-reported under policy; anything about authenticity, parallel imports or first-sale/exhaustion goes to a person.

Data preparation

The raw material is brand data and listing data. Brand assets (logos, trademark registrations, product photos, SKU lists, authorised-reseller lists) usually sit in the DAM, ERP or product information system, on file shares and in agency archives. Listing data is crawled from marketplaces; past takedown cases live in legal files and e-mail.

Typical gaps: product photos in inconsistent sizes and formats; no reseller whitelist, so genuine sellers are flagged; trademark data only on paper or in PDF scans; past cases not recorded with outcomes; and no list of product features that distinguish originals from fakes.

Preparation steps:

1. Collect and clean a master set of product images and SKUs, and tag them (PREP-010).
2. Digitise trademark certificates and old case files with OCR (PREP-006).
3. Maintain a reseller whitelist as a structured table (PREP-011).
4. Harvest labels from past takedown results (PREP-020) and build a test set of real and fake listings, reviewed by product experts (PREP-018).
5. Gather gold-standard authenticity criteria and examples (PREP-016).

Tools: DAM or a clean shared drive, OCR (e.g. Paperless-ngx), a spreadsheet for whitelists. Rough share of effort: about 30 percent, mostly collecting and tagging brand assets.

Implementation paths

Which path for whom

• smb: A. Brand-protection services and marketplace brand registries are mature, affordable and handle crawling and takedown filing, which is a standard process.
• enterprise: B. Large brand owners integrate detection with their own product catalogues, legal case systems and customs processes, which a product rarely covers.
• developer: C. Open-source vision and embedding models (CLIP-style, vector database) on own hardware allow custom image matching without sending brand assets to third parties.

Path A — Buy (recommended for most brand owners)

• Brand-protection platforms: Red Points (Barcelona, Spain, EU-founded; marketplaces, websites, social, ads, automated takedown) [1]; also Corsearch, MarkMonitor, BrandShield, Incopro (UK), Ebay VeRO and Amazon Brand Registry / Project Zero as platform-native programmes for rights owners. Check the current vendor list during procurement — this market consolidates frequently.
• Platform-native tools: Amazon Brand Registry, eBay VeRO, Alibaba IP Protection Platform, Zalando/Otto partner programmes, Meta and TikTok IP reporting tools — free, but each is limited to its own channel.
• Choose A when: you have fewer than a few hundred SKUs at risk or no data-science team; ask for EU hosting, a data-processing agreement, per-marketplace coverage list, precision measured on your brand during a trial, and who bears takedown/abuse liability (see risks).

Path B — Build on cloud APIs

• Embeddings and VLMs: multimodal embedding APIs or frontier VLMs (Claude, Gemini, GPT, Mistral Pixtral-class) in EU regions to compare listing images with brand reference images and explain differences; Google Cloud Vision / Vertex AI Vision Warehouse or Azure AI Vision for logo detection and product search.
• Pipeline: scraping service or marketplace API → queue → embedding + vector DB → VLM review of top candidates → case management (Jira, ServiceNow, n8n) → notice templates.
• Choose B when: you need custom product logic (e.g. spare-part numbering) or want to embed detection in your own platform. Keep costs in check by filtering with cheap embeddings first, calling the VLM on the shortlist only.

Path C — Self-host (open source)

• Models: DINOv2 (Apache 2.0 code and weights) [3], SigLIP 2 (Apache 2.0, ViT-B to g, multilingual) [4], OpenCLIP model zoo [2]; object/logo detectors fine-tuned from YOLO-class or open-vocabulary detectors; open VLMs of the Qwen-VL family for fine-grained comparison; PaddleOCR for text in images.
• Search: Qdrant, Milvus, pgvector or OpenSearch k-NN for image/text vectors; Scrapy/Playwright for crawling.
• Caution on licences: the Meta Image Similarity Challenge 2021 code (a copy-detection benchmark baseline) is CC BY-NC 4.0, i.e. non-commercial [5] — use it for evaluation ideas, not in a commercial product without checking terms.
• Hardware: HW2 (one 24–32 GB GPU) is enough to embed millions of images per day and run a ~7–8B VLM on shortlists; CPU/vector DB scaling is usually the larger cost. Marketplace operators at billions of listings need HW3/HW4 or cloud.

Path D — Custom training

Worth it for marketplaces, or for brands with distinctive visual products where generic embeddings confuse look-alikes. Fine-tune an image encoder with metric learning on pairs "genuine vs. known counterfeit" from your own case history (hundreds to thousands of confirmed cases), and train a seller-risk model on labelled outcomes (gradient boosting; graph features for seller networks). Keep a held-out set of recent cases because counterfeiters adapt.

State of the art (2026)

• Embeddings: self-supervised and sigmoid-loss vision encoders (DINOv2, SigLIP 2) give strong zero-shot retrieval; copy-detection style fine-tuning improves robustness to crops, filters and overlays [3][4]. The public benchmark for this task is the Image Similarity Challenge (descriptor track and matching track) [5].
• VLM-assisted triage: VLMs explain why a listing looks suspicious (logo spacing, wrong packaging text, "brand + unusual price" patterns) and draft the notice; useful for triage, not proof of authenticity.
• Generative AI on both sides: counterfeiters use image generators and LLM-written listings to avoid duplicate-image detection and create many seller personas; defenders therefore shift toward seller-network, price, logistics and text-pattern signals instead of image identity alone (see ANOM-004, ANOM-015).
• Physical authentication (serialisation, QR/NFC, tamper labels, AI-based photo authentication of luxury goods) complements listing monitoring; listing monitoring cannot tell a perfect fake from a genuine item.
• Limits: false positives hit legitimate resellers, parallel importers and second-hand sellers (exhaustion of rights); text/image evidence can be circumvented; platforms differ in how fast they act; crawling is restricted by terms of service and anti-bot measures; coverage of closed apps (WhatsApp, Telegram) is weak.

Shared resources & combinations

• Same building blocks (web-scraping + vlm + embeddings + vector-db): VIS-022 (visual product search), VIS-035 (logo detection), SEC-021 (brand impersonation, fake shops, lookalike domains), KNOW-022 (trademark search and watch), ANOM-035 (content fingerprinting for piracy) — one crawler, one vector DB and one GPU (HW2) can serve all.
• Seller-side signals: ANOM-004 (fake accounts, bots), ANOM-010 (fake reviews — often used to boost counterfeit listings), ANOM-017 (forged certificates/documents, e.g. fake test reports), ANOM-011 (supply-chain risk, diversion in authorised channels).
• Workflow: case management and notice generation reuse the workflow-automation and agent-framework stack of AGENT-style back-office use cases; knowledge of legal templates can be served from a RAG index.

Effort, cost & KPIs

• Path A: days to a few weeks to onboard; typical pricing by number of brands, marketplaces or removals (get quotes).
• Path B: 2–4 months for a custom pipeline; costs dominated by crawling infrastructure and VLM calls on shortlists.
• Path C: 3–6 months incl. evaluation set and review UI; GPU server plus vector DB ops.
• KPIs: precision of reported listings (confirmed infringement / reported), recall on a test-purchase or red-team sample, time-to-takedown, share of listings removed, re-listing rate, repeat-infringer seller removals, false-positive complaints and counter-notices, estimated revenue/price-erosion recovery, cost per removal.

People & change

Who and when. Management and the brand or legal owner decide priorities and legal steps (notice-and-action, customs applications). The product or brand manager supplies authenticity criteria and reviews flagged listings. Data owners of image and SKU archives are involved early; IT sets up crawling and accounts. Suppliers and authorised resellers should be told about the whitelist so that they are not wrongly reported.

Works council. No in the typical case, as the tool analyses third-party listings, not employee behaviour. If reviewer activity is logged in a way that permits performance evaluation, §87(1) no. 6 BetrVG applies; §90(1) no. 3 requires timely information when AI changes work processes.

Data protection. Seller names and profiles are personal data when sole traders are involved. Keep data minimal, document the legitimate interest, conclude an AVV (Art. 28 GDPR) with the service provider and consult the DPO; a DPIA is usually not needed unless sellers are profiled at scale (GUIDE-005).

AI literacy and control. Show reviewers how match scores arise and where look-alike genuine products cause errors (GUIDE-006). A human always checks a listing before a notice is filed, since wrongful notices carry liability. Start with one product line as a pilot and measure precision (GUIDE-009, GUIDE-008).

Risks, regulation & compliance

• EU AI Act: minimal risk — it concerns products and sellers' commercial listings, not an Annex III area. If a system profiles natural persons (individual sellers) with decisions affecting their access to essential services, re-check classification; AI literacy (Art. 4) applies to operators. Do not use biometric analysis of sellers' faces.
• DSA (Regulation (EU) 2022/2065): marketplaces must offer a notice-and-action mechanism (Art. 16), give priority to notices from trusted flaggers (Art. 22), apply anti-misuse rules (Art. 23), and — for online marketplaces — trace traders (Art. 30) and design interfaces to enable trader compliance (Art. 31) [7]. Brand owners should file complete, substantiated notices; marketplaces must keep human oversight over contested decisions and a statement of reasons.
• IP law: trademark and design law, Enforcement Directive 2004/48/EC; the EU customs action regime lets rights holders file an application for action with customs. Abusive or mistaken takedowns can create liability (unfair competition, damages) — avoid fully automated notices on authenticity-dependent claims.
• GDPR: seller data (names, addresses, ratings) is personal data for sole traders; limit collection to what the case needs, record legal basis (legitimate interest, Art. 6(1)(f)) and retention, vendor DPA, EU hosting preferred.
• Scraping: check platform terms of service, database rights and the EU Data Act / unfair-competition angles; prefer official APIs and reporting programmes.
• Mitigations: human review of ambiguous cases, whitelist of authorised resellers, appeal handling, audit log of evidence and notices.

Getting started

1. Inventory what you must protect: top 20 SKUs, logos, trademark registrations (EUIPO / DPMA), authorised resellers, price floors.
2. Free first step: enrol in the platform programmes (Amazon Brand Registry, eBay VeRO, Alibaba IPP, Meta/TikTok IP tools) and log current notices to get a baseline.
3. Run a 4–8 week pilot with one Path A vendor (and/or a small Path B/C prototype) on 3 marketplaces; label every flagged listing to measure precision.
4. Define the decision rules: what is auto-reported, what needs legal review, what triggers test purchases or customs action.
5. Feed confirmed cases back as labels, review monthly, and add seller-network analysis once volume is high enough.

Sources

1. Red Points — brand protection product overview and stated figures (vendor claims) — https://www.redpoints.com/
2. OpenCLIP (mlfoundations) — open-source CLIP/SigLIP model zoo — https://github.com/mlfoundations/open_clip
3. Meta FAIR — DINOv2 (Apache 2.0 code and weights) — https://github.com/facebookresearch/dinov2
4. Google Research — SigLIP 2 (big_vision README) — https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md
5. Meta — Image Similarity Challenge 2021 (CC BY-NC 4.0) — https://github.com/facebookresearch/isc2021
6. Regulation (EU) 2024/1689 (AI Act) — https://eur-lex.europa.eu/eli/reg/2024/1689/oj
7. Regulation (EU) 2022/2065 (Digital Services Act) — https://eur-lex.europa.eu/eli/reg/2022/2065/oj