Image editing & retouching (background removal, upscaling, inpainting)

Cutting out products, removing distractions, fixing blemishes, enlarging low-resolution files and adapting images to many formats used to be hours of manual work in an image editor. Segmentation, inpainting and super-resolution models now do this in seconds, and instruction-based editors change a photo from a sentence. The business benefit is fast, consistent image preparation at scale; the compliance work is about labelling AI edits and not deceiving viewers.

Problem & value

• Today: shops, agencies, estate agents and communications teams prepare thousands of images: clipping masks, retouching, resizing, cleaning backgrounds, repairing low-resolution supplier or archive images. Outsourced retouching is billed per image and has turnaround times of days.
• AI value: one-click or batch processing, consistent marketplace-ready results (e.g. white or transparent backgrounds), generative fill that extends a crop to another aspect ratio, and upscaling of legacy material. Humans move from pixel work to review and art direction.
• Evidence: vendors publish feature descriptions rather than independent productivity studies; treat claimed time savings as vendor claims and measure on your own images [1][2].

Variants

• E-commerce / marketplaces: batch background removal, shadow generation, size and aspect normalisation per channel (see GEN-001 for full AI product photography).
• Marketing and agencies: generative expand/fill to reformat key visuals for banners (GEN-004).
• Photographers: culling, denoise, upscaling, skin and object retouching inside Lightroom/Photoshop.
• Real estate: removing clutter and cars (disclose edits; virtual staging is GEN-018).
• Archive / publishers: upscaling and cleaning scans; restoration is GEN-017.
• Private use: removing photo-bombers, backgrounds for profile or sticker images, enlarging prints.
• News and public bodies: only minimal technical corrections; generative alteration of documentary photos is generally an editorial integrity breach.

How it works

1. upload / watch folder / API call
2. analysis (segmentation, saliency, VLM scene description, face/text detection)
3. operation: matting | inpainting (mask + prompt) | outpainting | super-resolution | denoise
4. quality checks (edge halos, artefacts, identity drift, text corruption)
5. human review (sample or all, brand-critical images always)
6. export with metadata (Content Credentials / IPTC "digital source type") + DAM upload

• Segmentation / matting: promptable models such as Segment Anything (SAM 2) and dedicated matting models (e.g. BiRefNet, RMBG family) produce alpha masks including hair and glass.
• Inpainting / outpainting: diffusion or flow-matching models fill the masked region conditioned on the surrounding pixels and an optional prompt; object removal uses the same mechanism.
• Instruction editing: models such as FLUX.1 Kontext or Qwen-Image-Edit change an image from a text instruction while keeping the rest consistent.
• Upscaling: GAN or diffusion super-resolution (Real-ESRGAN, SwinIR, SUPIR-style models). Generative upscalers invent detail; this matters for evidence, documents and faces.
• Human-in-the-loop: mask correction, approval of generated fill, final sign-off on people images.

Data preparation

Needed data are the original photos (JPEG, PNG, RAW), brand guidelines with target sizes per channel and, for shops, article numbers to link results. Originals typically live on photographers' drives, in a DAM or shop backend, on file shares and in e-mail attachments from suppliers. Typical gaps are no single master copy, edited and original files mixed, missing metadata and licence information, inconsistent file names, low resolution, and portraits that contain personal data.

Preparation steps: keep untouched originals in a separate, read-only location, adopt a naming scheme linked to the article number (PREP-009, PREP-010), document target formats per channel, collect a small set of approved before/after examples as quality reference (PREP-016), and note licence and consent status per image (PREP-025). Batch tools such as ExifTool or a DAM handle metadata; scanned prints can be digitised first (PREP-006).

Data preparation is a small share, around 10 to 15 percent, because the models need no training data. Most time goes into a tidy folder structure and rules for originals.

Implementation paths

Which path for whom

• SMB: A, because background removal, upscaling and retouching are commodity features in affordable and EU-hostable tools.
• Enterprise: B, because large image volumes make per-seat tools costly and an automated API pipeline into the DAM or shop scales better.
• Developer: C, because open segmentation, inpainting and upscaling models run well on a single GPU and fit custom batch pipelines.
• Private: C, because local apps such as Upscayl or LM Studio-style tools keep personal photos on the own device and are free.

Path A — Buy (off-the-shelf)

• Creative suites: Adobe Photoshop / Lightroom / Firefly (Generative Fill, Expand, Remove, Upscale; Content Credentials on generative exports [2]), Affinity (Canva), Canva Magic Edit, Luminar Neo, Topaz Photo / Gigapixel, DxO PureRAW.
• Background-removal services: remove.bg (Kaleido), PhotoRoom (France, EU), Pixelcut, Clipdrop (Jasper), Cleanup.pictures, Claid.ai; bulk plugins for Shopify, WooCommerce and marketplaces.
• EU options: PhotoRoom (FR), Pixelz and retouching service providers in the EU; check hosting region, DPA and whether uploads are used for training.
• Choose A when: volume is moderate, no strict data-residency rule, and staff are not developers.

Path B — Build on cloud APIs

• Image-editing APIs: Black Forest Labs FLUX Kontext / FLUX Fill (EU-founded provider, API with regional endpoints), OpenAI image editing (gpt-image series), Google Gemini image editing ("Nano Banana" family) via Vertex AI with EU locations [3], Stability AI edit endpoints (remove background, erase, inpaint, outpaint, upscale), Adobe Firefly Services APIs [1], Photoroom API, Clipdrop API, Amazon Nova Canvas on Bedrock (inpainting, outpainting, background removal).
• Orchestration: n8n / Make / serverless functions triggered by DAM or PIM uploads; queue plus retry; store original and edited versions together with the operation log.
• Choose B when: you need pipelines (PIM → edit → channel export), custom rules or integration.

Path C — Self-host (open source)

• Models: SAM 2 and BiRefNet for masks [4], FLUX.1 Fill [dev] and Kontext [dev] (non-commercial licence for dev weights, check terms), Stable Diffusion inpainting variants, Qwen-Image-Edit (Apache-2.0), Real-ESRGAN and SwinIR for upscaling, LaMa for fast object removal.
• Tooling: ComfyUI or diffusers pipelines, rembg for batch cut-outs, Krita/GIMP plugins, Immich or other photo managers for private use; exposed as an internal API.
• Hardware: matting and LaMa run on HW1/CPU; diffusion editing is comfortable on HW2 (24-32 GB VRAM); large batch workloads on HW3.
• Choose C when: images are confidential (unreleased products, people, medical, ID) or volume makes API cost dominant.

Path D — Custom training

Rarely needed. A LoRA or small fine-tune can lock a house style, product category or lighting for consistent results; a few dozen to a few hundred curated images suffice. Train a custom matting model only for unusual material (e.g. transparent packaging, jewellery) with hand-labelled masks.

State of the art (2026)

• Instruction-based editing has replaced manual masks for many tasks: a single prompt changes colour, object or text in a photo while preserving identity and layout; the open-weight Qwen-Image-Edit and FLUX Kontext [dev] line brought this to self-hosting.
• Promptable segmentation (SAM family) makes click-to-select masks near-instant, including in video.
• Matting quality on hair, fur and semi-transparent objects is good on clean photos.
• Limits and failure modes: generated fill that does not match lighting or perspective; repeated patterns and "smeared" textures; corrupted text, logos and hands; identity drift on faces after repeated edits; hallucinated detail in upscaling; reduced quality at high resolution (many models work internally at about 1 MP and tile or rescale). Colour management (ICC) and RAW workflows are often lost.
• Provenance: C2PA Content Credentials are attached by Adobe, Google, OpenAI and others, but metadata can be stripped by re-saving or social platforms, so labelling cannot rely on it alone [2][5].

Shared resources & combinations

• Same stack (image-gen + segmentation + vlm): GEN-001 product imagery, GEN-004 layout, GEN-010 virtual try-on, GEN-017 restoration, GEN-018 virtual staging, GEN-026 headshots; one ComfyUI/diffusers server (HW2/HW3) serves all.
• Downstream: ANOM-015 (marking and provenance under Art. 50), SEC-011 (deepfake detection) for incoming material, DAM/PIM tagging with VLMs.
• Segmentation models are shared with computer-vision inspection use cases.

Effort, cost & KPIs

• Path A: hours to days; subscription or per-image pricing, typically cents per image for API-based services.
• Path B: 2-6 weeks for an integrated pipeline; costs scale with image count and resolution.
• Path C: 1-2 months; one GPU workstation plus maintenance; models are free but licences differ.
• KPIs: minutes per image, share of images accepted without rework, cost per image, turnaround time, marketplace rejection rate, share of edited images labelled correctly.

People & change

Involve the marketing or e-commerce lead as process owner, the people who edit images today, and IT for accounts or the batch pipeline; management decides which tools are approved (GUIDE-007). Legal or the DPO joins when portraits or customer images are processed: check the AVV with cloud vendors (Art. 28 GDPR) and consider a DPIA only for large-scale processing of faces. Label AI-edited images where Art. 50 AI Act applies, and use Content Credentials.

Works council: usually no, as the tool edits images and does not assess staff. Per-user usage logs in cloud tools can still make it relevant under §87(1) no. 6 BetrVG, and §90(1) no. 3 requires early information when work processes change; where unclear, brief the council. AI literacy: support staff with a short practical session on what the tools can and cannot do, copyright, and when retouching becomes misleading (Art. 4 AI Act, GUIDE-006). Typical concern: retouchers fear for their jobs; shift them to quality control and creative tasks, run a pilot with one product group, and keep a human check before publication, especially for product images where a changed colour or shape could mislead customers (GUIDE-008, GUIDE-009).

Risks, regulation & compliance

• EU AI Act: typically limited risk. Art. 50(2) requires providers of generative systems to mark synthetic output in machine-readable form; Art. 50(4) requires deployers to disclose deepfakes (image content that resembles real persons, objects, places and would falsely appear authentic), with a lighter duty for evidently artistic or creative works. Purely assistive or standard edits (cropping, denoise, colour correction) are largely outside the marking duty. Not prohibited unless used for Art. 5 practices such as manipulative or deceptive techniques causing significant harm. Art. 4 AI literacy applies [6].
• GDPR: photos of identifiable people are personal data (faces; biometric only when processed for unique identification). Need legal basis, DPA with the vendor, no training on uploads, EU region or self-hosting for sensitive images. Editing a person's image can also infringe personality rights (in Germany the right to one's own image, KUG) and must not misrepresent them.
• Copyright and licensing: check commercial-use terms of generated fill and of open-weight licences; editing third-party photos requires rights; Adobe-style "commercially safe" training claims are vendor claims.
• Misleading imagery: consumer-protection and advertising law (UWG) forbid deceptive product images (e.g. retouching away real defects, fake before/after); estate listings must disclose significant edits; journalism and evidence require unaltered originals.
• Mitigations: keep originals, log operations, attach Content Credentials, review people images, maintain a policy listing permitted and forbidden edits.

Getting started

1. Define the 3 most frequent tasks (e.g. cut-out, reframing, upscaling) and collect 50 representative images, including hard ones.
2. Test 2-3 Path A tools on these images; score accept-without-rework rate and edge quality.
3. Write an edit policy: permitted retouching, labelling rules, no deceptive edits, people-image review.
4. For volume, automate with an API or ComfyUI pipeline connected to your DAM/PIM and keep originals.
5. Enable Content Credentials / IPTC AI metadata on export and check that your CMS does not strip them.

Sources

1. Adobe — Firefly product description (Generative Fill/Expand/Remove, APIs) — https://helpx.adobe.com/legal/product-descriptions/adobe-firefly.html
2. Adobe — Firefly FAQ (Content Credentials on exports) — https://helpx.adobe.com/firefly/web/get-started/learn-the-basics/adobe-firefly-faq.html
3. Google Cloud — Vertex AI generative AI image editing documentation — https://cloud.google.com/vertex-ai/generative-ai/docs/image/overview
4. Segment Anything 2 (Meta) — https://github.com/facebookresearch/sam2
5. C2PA — Content Credentials technical specification — https://c2pa.org/specifications/
6. Regulation (EU) 2024/1689 (AI Act), Art. 50 — https://eur-lex.europa.eu/eli/reg/2024/1689/oj
7. Black Forest Labs — FLUX.1 Kontext documentation — https://docs.bfl.ai/
8. Stability AI — Image editing API — https://platform.stability.ai/docs/api-reference