Returns & second-hand item condition grading (re-commerce)

Returned and pre-owned goods are graded by hand at a bench: staff look at an item, judge stains, wear and defects, and decide where it goes. This is slow, inconsistent between graders, and a bottleneck for resale. Vision-language and segmentation models identify the product, find and localise defects, map them to a written grading standard, and propose the route and the listing text, with a human checking borderline cases. The benefit is faster, more consistent grading and fewer items sent to low-value channels or destroyed.

Problem & value

• Today: returns arrive at a warehouse or a 3PL; a grader opens each parcel, inspects, photographs, grades and lists. Grades differ between people and sites, disputes with buyers ("not as described") follow, and anything uncertain defaults to the cheapest channel (outlet, liquidation, recycling).
• AI value: consistent application of a written rulebook, evidence-backed grades (defect crops attached), faster throughput per bench, automatic listing drafts, better disposition decisions (resell vs. refurbish vs. recycle) and fewer returns of second-hand items caused by unclear condition descriptions.
• Regulatory pull: the EU Ecodesign for Sustainable Products Regulation (ESPR) bans the destruction of unsold apparel, clothing accessories and footwear for large companies and requires disclosure of discarded unsold goods; the revised Waste Framework Directive introduces extended producer responsibility (EPR) for textiles. Both push operators towards resale, reuse and documented sorting [1][2].
• Vendor-reported gains vary by category; measure on your own return stream.

Variants

• Fashion/textiles: brand and category recognition, stains, pilling, holes, missing buttons, odour flags (cannot be seen; needs a human or sensor). Fine-grained grading is the hardest.
• Consumer electronics/phones/laptops: cosmetic grade from housing, screen and bezel photos; functional tests are separate (diagnostic software). Serial/IMEI read via OCR (see VIS-018).
• Furniture: scratches, water marks, structural damage; capture is harder.
• Marketplace self-grading (C2C): seller uploads photos; the model checks completeness, suggests condition class and flags mismatches between description and images.
• Private use: a household sorting wardrobe items for sale on a second-hand platform; platform photo assistants do a light version of this.

How it works

1. returned item arrives
2. barcode/RFID/ID scan
3. standardised photo station (fixed camera, light, 4-8 views, label, serial)
4. product identification (brand, model, category, size; visual search + OCR)
5. defect detection & segmentation (stain, tear, scratch, dent, missing part)
6. grade mapping (rulebook: defect type × size × location
7. A/B/C/reject)
8. disposition rules (grade + margin + demand + compliance
9. resale / refurbish / outlet / recycle)
10. listing draft (LLM) + price band
11. human review on low confidence, high-value or contested items
12. publish / route

• Identification: open-vocabulary detectors or VLMs read labels and match to the product catalogue; embedding search against catalogue images resolves look-alikes.
• Defect detection: instance segmentation or anomaly detection against a "good" reference; VLM captions explain defects in words. Measuring defect area in cm² requires a scale reference (calibration marker, fixed camera).
• Grade mapping: deliberately a deterministic rules layer on top of model outputs, so grades are explainable, auditable and tunable per channel or marketplace.
• Human-in-the-loop: items below a confidence threshold, above a value threshold, or flagged as authenticity-sensitive (luxury, branded electronics) go to a grader; the correction is stored as training data.

Data preparation

• Data needed: product master data and original prices (ERP/shop system), return reasons and customer claims (returns portal, mail), the grading rulebook (often only in the heads of senior graders or in Word/Excel files), and photos with the grader's final decision.
• Typical gaps: no written grade definitions per category, inconsistent grading between shifts, return photos taken ad hoc with varying light and background, missing EAN or serial numbers, duplicate article records, and no stored link between photo and final grade.
• Preparation steps: (1) write a clear rulebook with example photos per grade and defect type (PREP-019); (2) set up a standard photo station and capture protocol before collecting any data; (3) export master data and return history from ERP and shop (PREP-005); (4) harvest past grading decisions and resale outcomes as labels (PREP-020); (5) build a golden set of several hundred items graded by two senior staff (PREP-018).
• Tools: barcode scanner, fixed camera with a calibration marker, Label Studio or a simple spreadsheet for grader decisions, n8n for exports.
• Share of effort: roughly 30-40 percent, mostly rulebook definition and consistent capture; the model call itself is the small part. Start collecting photos with grades now, even before the AI is chosen.

Implementation paths

Which path for whom

• SMB: B - no mature, SMB-priced grading product fits mixed categories; a multimodal API plus a fixed photo station and a rules layer connects to your own shop and returns portal.
• Enterprise: B - grading rules and channel logic are a competitive asset and must integrate with WMS/ERP; at very high volumes or for authenticity-sensitive goods, Path C/D is worth evaluating.
• Developer: C - open-weight VLMs, detectors and SAM 2 give full control over rulebook, evidence crops and costs, and can be reused for other inspection use cases.

Path A — Buy (off-the-shelf)

• Re-commerce platforms: Optoro (US) for returns routing; EU refurbished marketplaces such as Refurbed and Back Market (EU) publish their own grading standards; fashion resale-ops providers and phone/laptop grading kiosks add computer-vision cosmetics grading.
• Choose A when volumes are modest, the category is mainstream and you prefer a managed intake-to-listing flow; verify EU data hosting, grade definitions you can configure, and API export of evidence images.

Path B — Build on cloud APIs (recommended default)

• Multimodal models (Claude, Gemini, GPT, Mistral Pixtral/Medium vision; EU-region endpoints via Azure OpenAI EU, Google Vertex EU, AWS Bedrock Frankfurt, or Mistral as EU provider) take the photo set plus the rulebook and return a structured JSON: product, defects (type, location, severity), proposed grade, confidence, rationale.
• Evidence crops via a segmentation model (SAM 2) so every defect is visible to the reviewer.
• Orchestration: returns portal/WMS webhooks → queue → model call → rules engine → ERP/listing API (Shopify, Zalando partner programme, Vinted Pro, eBay, Back Market seller API depending on channel). n8n or a small service is enough.
• Run a golden test set (hundreds of items graded by senior staff) to compare model vs. human before going live.

Path C — Self-host (open source)

• Models: Qwen-VL family or other open-weight VLMs for descriptions; DINOv2 / SigLIP embeddings for product matching; YOLO-class or RT-DETR detectors and SAM 2 for defect regions; PaddleOCR or Docling for labels and serial plates.
• Serving: vLLM or Ollama for the VLM; ONNX/TensorRT for detectors; photo station on an industrial PC.
• Hardware: HW2 (one 24-32 GB GPU) covers a single bench or several stations at warehouse throughput; HW1 edge devices suffice for detector-only stations.
• When: sensitive customer data, very high volumes, or need for offline stations in a warehouse.

Path D — Custom training

Worth it when grading is the business (large refurbishers, resale platforms). Needs thousands of images per category labelled with defect masks and final grades by expert graders, balanced across brands, lighting and defect types. Start from a pretrained backbone, fine-tune a defect segmenter and a grade classifier, and keep the rules layer. Grader corrections from production are the main data source.

State of the art (2026)

• General VLMs recognise most mainstream brands and categories and describe obvious damage well, which is enough for first-pass triage and listing drafts. Fine-grained decisions (is this faint pilling a B or a C? is a hairline screen scratch visible in use?) are inconsistent without calibrated capture and few-shot examples of the grading standard.
• Reliable results depend more on capture quality than on model choice: fixed lighting, neutral background, multi-angle and macro views, a scale reference.
• Segmentation foundation models (SAM family) make defect localisation cheap; open-vocabulary detection adds defect types without retraining.
• Limits and failure modes: non-visual defects (smell, function, hidden damage, counterfeit authenticity) are invisible; reflections on screens and glossy surfaces create false positives; dark or patterned fabrics hide stains; models may hallucinate defects or miss them, and over-confident automatic upgrading creates "not as described" returns. Treat the model as a consistent first grader, not the final authority on high-value items.

Shared resources & combinations

• Same stack as VIS-001 (visual inspection), VIS-023 (damage assessment for claims), VIS-026 (visual measurement): fixed-camera capture, segmentation, VLM descriptions.
• VIS-022 (visual search) and VIS-017 (product recognition) provide catalogue matching; VIS-018 (industrial OCR) reads labels and serial numbers; VIS-020 (waste sorting) handles the recycling end of the routing.
• DATA-021 (Digital Product Passport) is the future data sink for condition and repair history; PERS-018 (visual repair assistant) supports refurbishment.
• Shared VLM server (vlm + llm-general, HW2/HW3) and workflow engine across these use cases.

Effort, cost & KPIs

• Effort: Path A weeks; Path B 2-4 months incl. photo station, golden set, rules, integration; Path C/D 4-9 months. Cost is mostly capture labour plus small API cost.
• KPIs: grader-model agreement (kappa) per category, touchless share, items per bench-hour, share routed to best channel, "not as described" rate, days receipt-to-listing, share destroyed or landfilled.

People & change

• Involve early: management (scope, budget), the returns or warehouse lead as process owner, the graders and packers who work at the station, IT (shop/ERP integration), the data owners of product master data, and the data protection officer if photos or claims contain personal data. See GUIDE-001 and GUIDE-009 for choosing and piloting.
• Works council: depends. Grading quality per person or items per bench-hour are exactly the data that make a system "suited to monitor" under §87(1) no. 6 BetrVG (objective suitability suffices per BAG); §90(1) no. 3 requires timely information when work processes are planned with AI. Inform the council at concept stage and agree that grader-model comparisons are not used for individual performance review (GUIDE-004).
• Data protection: photos may show people, name tags or addresses; conclude an AVV with API vendors; a DPIA is normally not needed for item photos but check with the DPO (GUIDE-005).
• AI literacy: support staff with a short hands-on training on what the model sees, typical errors and how to override (AI Act Art. 4; GUIDE-006).
• Pilot and concerns: pilot with one category and 2-3 experienced graders as champions. Typical fear: "the machine replaces graders" - address by positioning the AI as a pre-grading assistant, keeping graders in control of low-confidence, high-value and contested items, and using their corrections to improve the system (GUIDE-008).

Risks, regulation & compliance

• EU AI Act: minimal risk; no Annex III use. Art. 4 AI literacy for staff. No Art. 5 issue unless photos are misused for biometric or emotion inference, which is out of scope. Chatbot-style buyer interaction needs Art. 50 transparency.
• GDPR: photos can capture people, faces, name tags, addresses on parcels, or personalised garments. Use a controlled station background, crop and blur incidental personal data, set retention limits and a processing agreement with API vendors; return and customer-claim data is personal data [3].
• Consumer law: accurate condition descriptions under the EU consumer rights framework and the Empowering Consumers Directive; the evidence images and audit trail protect against disputes.
• ESPR and waste law: document the disposition route; unsold-goods disclosure and the textile EPR regime need reliable data on what was resold, reused or recycled [1][2].
• Mitigations: published grading standard, human review for value and authenticity risk, drift checks on the golden set, bias checks across brands and colours, logging of model vs. final grade.

Getting started

1. Write down the grading rulebook with photo examples per grade and category; get graders to agree on it.
2. Build a standard photo station (fixed light, background, scale marker, 4-8 views) and collect 300-500 graded items.
3. Test two or three VLMs in Path B against this golden set; measure agreement per grade and per defect type.
4. Add the rules layer and confidence/value thresholds; pilot at one bench with human sign-off on every item.
5. Reduce review as agreement stabilises; track resale rate and "not as described" returns monthly.

Sources

1. Regulation (EU) 2024/1781 (Ecodesign for Sustainable Products Regulation) — https://eur-lex.europa.eu/eli/reg/2024/1781/oj
2. Directive (EU) 2025/1892 (revised Waste Framework Directive, textile EPR) — https://eur-lex.europa.eu/eli/dir/2025/1892/oj
3. Regulation (EU) 2016/679 (GDPR) — https://eur-lex.europa.eu/eli/reg/2016/679/oj
4. Regulation (EU) 2024/1689 (AI Act) — https://eur-lex.europa.eu/eli/reg/2024/1689/oj
5. SAM 2 — Segment Anything in Images and Videos — https://github.com/facebookresearch/sam2
6. DINOv2 — https://github.com/facebookresearch/dinov2
7. Qwen-VL — https://github.com/QwenLM/Qwen3-VL
8. circular.fashion — https://circular.fashion