Claims handlers and repairers traditionally need an expert visit or a stack of photos and a manual estimate before a motor or property claim can move. Computer vision and vision-language models read smartphone photos, locate damaged parts, grade severity and propose repair lines or a cost range within minutes. Insurers use it to settle simple claims faster and send complex ones to experts; the decision on payout stays with humans or with transparent, contestable rules.
Problem & value
- Today: a customer reports a loss, an adjuster or appraiser inspects (or reviews photos manually), a garage writes a quote, and the insurer checks it. Waiting times, inspection travel cost, inconsistent estimates and leakage (over- and under-payment) are the pain points. After catastrophe events (hail, flood, storm) capacity is the bottleneck.
- AI value: guided photo capture at first notice of loss (FNOL), instant triage, consistent damage grading, faster settlement for low-value claims, expert capacity focused on complex ones, plus a photo record for fraud checks and subrogation.
- Evidence: vendors publish case studies (cycle-time, touchless rates), but figures depend on portfolio and definitions; treat them as marketing until reproduced on your own claims. Use a shadow-mode pilot to get your own numbers.
Variants
- Motor insurance FNOL and appraisal: exterior damage, parts and repair-method suggestion, total-loss screening.
- Garage / body-shop estimating: photo-to-estimate inside the repair workflow, linked to parts catalogues and labour times.
- Fleet, leasing and rental: check-in/check-out condition reports, damage attribution between renters (drive-through scanners or phone capture); see VIS-037 for item condition grading in re-commerce.
- Property claims (home, commercial): roof, facade, water and storm damage from ground photos, drone images (VIS-003) or aerial imagery; contents damage via photo lists.
- Catastrophe response: area-level triage from satellite and aerial data, see VIS-058.
- Private use: a policyholder photographing damage for a claim or a used-car buyer documenting condition; typically via the insurer's or platform's app.
How it works
- guided capture (app/web link: angles, distance, lighting checks, VIN/plate photo)
- quality & integrity checks (blur, duplicates, reuse, metadata, edited/AI-generated image signals)
- vehicle/building identification and part or element segmentation
- damage detection & severity grading (scratch, dent, crack, missing part; water, roof, impact)
- repair logic: repair/replace, parts, labour (catalogue-linked), cost range; total-loss screen
- triage rules + confidence
- straight-through | adjuster review | expert/garage visit
- human decision, communication to customer, audit log
- Detection and segmentation: instance-segmentation models (YOLO- or Mask R-CNN/DETR-style, SAM-family) localise parts and damage; classifiers grade severity. Vision-language models add flexible description, plausibility checks against the customer's narrative and handling of unusual damage.
- Cost estimation: the reliable approach is to map detected damage to repair operations and parts via catalogue and labour-time data (e.g. Audatex/DAT/GT Motive-style systems) rather than letting a model guess a number.
- Photo integrity: since generative image editing makes fake damage photos cheap, capture inside the app (live camera, metadata, liveness of the scene) and manipulation detection are now standard parts of the pipeline; link to ANOM-003.
- Human-in-the-loop: low-confidence, high-value, injury-related, disputed or atypical claims go to an adjuster; the customer can always request human review; denials are never fully automated.
Data preparation
Data needed: damage photos with claim ID, vehicle or property data, repair estimates and invoices, and final settlement decisions. In an SMB (garage, fleet, property manager) this lives in the workshop or fleet software, the ERP, mail attachments, phone galleries and messenger chats; for insurers in the claims core system and DMS.
Typical gaps: inconsistent photo angles and quality, photos without claim reference, missing labels (which damage was actually repaired), duplicates, scanned handwritten reports, and photos showing people, licence plates or addresses.
Preparation steps: 1. Gather photos and claim records through exports or connectors (PREP-005). 2. Link photos to claim IDs and repair outcomes, harvesting labels from existing invoices (PREP-020). 3. Define damage categories before any classification (PREP-019). 4. Build a test set of reviewed claims with experts (PREP-018). 5. Blur plates and faces or pseudonymise exports (PREP-022). 6. Standardise photo capture with a guided app, so future images are usable (PREP-008).
Effort: around 30 to 40 percent when training or tuning; much lower with a bought product that brings its own models, where the main work is standard photo capture and clean claim data.
Implementation paths
Which path for whom
- Enterprise: Path B, because claims handling is the core process of an insurer, the volume makes seat costs high and the photo-AI has to integrate with the own claims system.
- SMB: Path A, because garages and fleets get mature, EU-available estimating tools with photo capture at SMB prices, and the process is standard.
- Developer: Path B, since vision-language APIs on EU regions allow fast prototypes for integrators; self-hosting only pays off at high volume.
Path A — Buy (recommended for garages and fleets; insurers often combine it with Path B)
- Specialised claims AI vendors: Tractable (UK/US; motor and property), CCC Intelligent Solutions (US), Solera/Audatex (estimating with AI photo capture), Ravin AI (vehicle inspection), Motionscloud (Munich, Germany; claims photo flows), DEKRA digital appraisal (EU). Property: Cape Analytics (Moody's), Nearmap, Verisk.
- Claims core systems (Guidewire, Duck Creek, Sapiens, Solera) increasingly ship or partner for photo-AI modules.
- Check: EU data residency, per-claim pricing, accuracy on your damage mix (compare with your adjusters), integration into FNOL and the estimating tool, explainability for disputes.
- Choose A when: you want proven workflows and vendor-maintained models, and volume does not justify an ML team.
Path B — Build on cloud APIs
- Frontier multimodal APIs (Claude, Gemini, GPT, Mistral Pixtral/Mistral Medium vision) with structured JSON output for damage description, plausibility checks and photo-quality feedback; cloud vision services (Azure AI Vision / Custom Vision, Google Vertex AI, AWS Rekognition Custom Labels, Frankfurt/EU regions) for trained detectors.
- Orchestrate with the claims platform (event-driven), capture SDK in your app, rules engine for triage. General VLMs are good for description and narrative consistency but weak at precise part-level localisation and costing; combine with a trained detector and a catalogue-based estimator.
Path C — Self-host (open source)
- Models: Ultralytics YOLO (AGPL, check licence), RT-DETR, Mask2Former, SAM 2 / SAM 3 for segmentation, open VLMs such as Qwen-VL family or InternVL for description, via vLLM.
- Data: public benchmarks such as CarDD (car damage detection) and COCO-based car-damage sets for bootstrapping; real value comes from your own labelled claims photos.
- Hardware: HW2 is sufficient for inference of detectors plus a 7-8B VLM at claims-volume; training detectors fits one GPU as well.
Path D — Custom training
Worth it when you have a large claims archive with photos, final repair invoices and settled amounts: fine-tune detection/segmentation for your market's vehicles, and train cost-regression models on actual invoices. Needs thousands of annotated claims, label consistency rules (damage taxonomy), and monitoring for drift (new models, seasons, camera types). Repair invoices are a natural, noisy label source.
State of the art (2026)
- Mature for common exterior motor damage: part-level detection, severity grading and triage are productised; leading vendors report production use at many insurers. Straight-through settlement is limited to low-value, clear-cut claims.
- Weak spots: hidden structural, underbody and ADAS/sensor damage (calibration), EV battery damage, reflections, dirt and poor lighting, old damage versus new, paint-repair judgements, and unusual vehicles. Property damage (roof, water) is harder than vehicles because causality and extent are not visible in a few photos.
- Trends: guided capture with real-time feedback, VLMs for narrative consistency, 3D/video scans (photogrammetry, drive-through gates) for condition reports, generative-AI image tampering as a rising fraud vector, and agentic claims workflows (AGENT-014) that call the damage model as one tool.
- Limits: models report plausible but wrong cost; accuracy varies by group of vehicles and lighting; always show the evidence (annotated photo) and a confidence value.
Shared resources & combinations
- Same stack as VIS-001 (defect detection), VIS-037 (item condition grading), VIS-003 (drone inspection), VIS-058 (disaster damage mapping): detector/segmenter training pipeline, annotation tooling, GPU inference server (HW2).
- Combines with: AGENT-014 (end-to-end claims handling), ANOM-003 (fraud detection incl. photo manipulation), PRED-037 (severity and reserving prediction), DOC-003 (claim forms and repair invoices), VIS-030 (blurring faces and plates in photos before sharing).
Effort, cost & KPIs
- Path A: weeks to a few months (integration into FNOL and claims system); per-claim or subscription fees.
- Path B: 3-6 months for a production pipeline including capture UX and estimating integration.
- Path C/D: 6-12 months incl. data labelling; ongoing MLOps.
- Cost drivers: labelling, estimating-catalogue licences, integration, model monitoring.
- KPIs: straight-through rate, cycle time FNOL-to-settlement, estimate deviation versus final invoice, leakage, appraiser-override rate, photo re-request rate, customer satisfaction (NPS), complaint/dispute rate, fraud hit rate.
People & change
Who and when: management and the claims or workshop lead decide scope and liability limits; legal and compliance check duties towards policyholders; IT integrates; the DPO is involved from the start because photos show persons, plates and homes; the works council is informed before the pilot. Involve adjusters and workshop staff as end users in testing, and plan customer communication, since customers are affected by the result.
Works council: depends. If the system logs and compares adjuster throughput, or measures their deviations from AI estimates, §87(1) no. 6 BetrVG applies (objective suitability for monitoring is enough); §90(1) no. 3 requires early information about AI-supported processes, and §80(3) allows an AI expert. Where companies have an elected works council, agree on a works agreement (GUIDE-004).
Data protection: a DPIA is likely when claim photos and decisions affect many people at scale, plus an AVV with every processor (GUIDE-005).
AI literacy: support staff with training on error patterns, such as reflections and old damage (Art. 4 AI Act, GUIDE-006).
Pilot and concerns: run a pilot on simple, low-value claims (GUIDE-009). Staff fear de-skilling and customers fear unfair rejection: the AI only proposes, and a human decides on every payout or rejection (GUIDE-008).
Risks, regulation & compliance
- EU AI Act: damage assessment of objects is generally minimal risk. Annex III high-risk applies to AI used for risk assessment and pricing in life and health insurance (Annex III, point 5(c)) and for creditworthiness; a pure damage-and-repair-cost tool for motor or property claims is not listed, but context matters: if outputs feed decisions on persons (e.g. automatic claim denial, fraud scoring of individuals, health-related claims), re-assess. Photo-manipulation detection and fraud scores should be assessed separately. AI literacy (Art. 4) applies. Nothing here involves Art. 5 prohibited practices, provided no emotion recognition or biometric categorisation is added.
- GDPR: photos contain personal data (people, plates, houses, locations, EXIF GPS); lawful basis is contract/claim handling; data minimisation (blur bystanders, strip unneeded metadata); Art. 22 limits solely automated decisions with significant effect, so provide human review and the ability to contest; retention per claim law; processor agreements and EU hosting.
- Sector: insurance supervision (Solvency II governance, EIOPA opinion on AI governance and risk management), IDD fair treatment, national rules on repair-cost assessment and total-loss.
- Risks and mitigations: biased or wrong estimates (shadow mode, sampling audits), customer trust (transparent explanations, human fallback), fraud with edited or AI-generated photos (in-app capture, forensics), vendor lock-in (own the labelled data), model drift (quarterly accuracy reviews).
Getting started
- Define the scope: claim types, value limits, and which decisions are automated versus advisory.
- Collect 500-1,000 historical claims with photos, estimates and final invoices as a test set.
- Run 2-3 Path A vendors in shadow mode and compare with appraisers on this set.
- Design guided capture and the customer's right to human review; agree the DPIA with the data protection officer.
- Launch for low-value claims first, monitor deviation and overrides monthly.
Sources
- Regulation (EU) 2024/1689 (AI Act), incl. Annex III — https://eur-lex.europa.eu/eli/reg/2024/1689/oj
- EIOPA — Opinion on AI governance and risk management — https://www.eiopa.europa.eu/publications/opinion-artificial-intelligence-governance-and-risk-management_en
- Tractable — AI for claims — https://tractable.ai
- CarDD: A New Dataset for Vision-based Car Damage Detection (arXiv 2211.00945) — https://arxiv.org/abs/2211.00945
- Segment Anything (SAM) project — https://github.com/facebookresearch/sam2
- Ultralytics YOLO documentation — https://docs.ultralytics.com
- GDPR Art. 22 — https://gdpr-info.eu/art-22-gdpr/
- Azure AI Vision documentation — https://learn.microsoft.com/azure/ai-services/computer-vision/