Research integrity screening: image manipulation, paper-mill & duplicate-submission detection

Publishers, universities and funders receive far more manuscripts and proposals than humans can forensically examine. AI screening finds reused or altered microscopy and blot images, textual and structural fingerprints of paper mills, and the same manuscript submitted to several journals. It does not decide misconduct: it produces evidence-backed flags that editors and research-integrity officers (RIOs) verify under a due-process procedure.

Problem & value

• Today: peer reviewers rarely spot duplicated image panels, and manual forensic checks do not scale. Problems are often found only after publication by volunteer "sleuths", followed by corrections or retractions. Paper mills (commercial fabrication and authorship-selling services) and AI-generated text and images increase screening pressure.
• AI value: pre-publication screening at submission, systematic back-screening of institutional output, and consistent triage so human experts spend time on the few high-risk cases.
• Evidence of effect (qualified): a pilot at the American Society for Microbiology (ASM) with Imagetwin reports that pre-publication image screening significantly reduces staff time spent on image issues, while noting that earlier AI-based duplication tools had limited success [3]. Vendors publish database sizes and adoption claims (e.g. Imagetwin: 160M+ published figures; Proofig: comparison against 155M+ PubMed Open Source images); these are marketing figures and not independently validated [1][2]. No error rates are quoted here because no peer-reviewed head-to-head benchmark was verified for this review.

Variants

• Publisher submission screening: automatic checks at submission inside the editorial system.
• Institutional pre-submission and back-screening: university RIO offices check outgoing papers and investigate allegations (e.g. Pitt rolled out ProofigAI for researchers [4]; Stanford offers Proofig to affiliates [5]).
• Funder / grant screening: duplicate-proposal and double-funding checks, text-reuse and fabricated-reference checks.
• Post-publication sleuthing and monitoring: corpus-wide scans for tortured phrases, irrelevant references or recycled figures, feeding retraction and correction workflows.
• Duplicate-submission detection: consortium approach across publishers (cross-publisher hub, hashing of manuscripts).
• Preprint servers and conference / thesis screening: same pipeline, lower stakes per item, higher volume.
• Private use: not applicable; for student plagiarism and exam integrity see EDU-008.

How it works

1. manuscript intake (PDF/DOCX + figures + metadata)
2. figure extraction & panel segmentation (ocr-docparse, image-classification)
3. image forensics: within-paper and cross-corpus duplicate/overlap search (rotated, flipped, scaled, cropped, partial), splicing/cloning cues, AI-generation indicators
4. text signals: tortured phrases, template similarity, irrelevant/fabricated references, LLM-generated-text indicators (text-classifier, embeddings)
5. metadata & network signals: author/e-mail/affiliation anomalies, reviewer-suggestion rings (graph-ml)
6. cross-submission fingerprint match (hashes/embeddings against partner corpora)
7. risk score + highlighted evidence
8. human triage: editor / research-integrity officer requests raw data, contacts authors
9. decision under COPE-style process (correction, rejection, retraction, institutional inquiry)

• Image similarity: classical computer vision (keypoint matching, perceptual hashing) plus learned embeddings that are invariant to rotation, scale and flipping; early research already showed manipulation-invariant similarity for duplicate scientific images [6]. Findings must be verified by a human looking at the highlighted overlap, since false positives arise from legitimately similar controls, standard blot layouts or reused schematics with permission.
• Text signals: statistical detectors and fine-tuned classifiers; an LLM can summarise evidence but must not be the sole judge.
• Human-in-the-loop (mandatory): every flag is a hypothesis. Only editors/RIOs, following a documented procedure with right of reply for authors, may act.

Data preparation

Input is mostly manuscripts and grant applications (PDF/DOCX with embedded figures), raw image files, submission metadata from the editorial system (submission portal, reviewer database, ORCID records) and reference corpora of earlier publications. In a university or society publisher this typically lives in the submission system, file shares and mail attachments; older material exists only as scans.

Typical gaps: figures embedded as low-resolution flattened images, scanned legacy PDFs without usable text layer, missing raw image files, inconsistent author and affiliation spellings, duplicate author records, and unclear access rights to submitted-but-rejected manuscripts.

Preparation steps: extract figures and text per manuscript with a document parser (see PREP-006 for scans and OCR, PREP-007 for format conversion), normalise author, e-mail and affiliation data (PREP-011), define the case categories before screening (PREP-019) and agree what may be stored and for how long (PREP-025). Build an evaluation set of confirmed cases with integrity officers (PREP-018) and capture reviewer feedback on false positives (PREP-021). Pseudonymise exports used for tuning (PREP-022). See also DOC-014 for document parsing.

Rough share of effort: about 40 to 50 percent, mainly corpus building, metadata cleaning and legal clearance of reuse. Product vendors carry the reference corpora; your part is clean intake and a documented procedure.

Implementation paths

Which path for whom

• Enterprise (publishers, large funders): Path A, because mature integrity-screening products with cross-publisher corpora exist and a private corpus cannot be rebuilt in-house.
• Public (universities, research funders): Path B, combining cloud vision/text APIs on EU regions with the institution's own case workflow keeps control over procedures and evidence.
• Developer: Path B, since image-similarity and text-signal components are available as APIs and open libraries, while the cross-corpus reference data is the hard part.

Path A — Buy (recommended)

• Image integrity: Proofig (AI image-integrity checks; reuse, duplication, partial overlap, public image comparison; adopted by journal families) [1]; Imagetwin (Austrian vendor; duplication, manipulation and AI-generated-image detection; API integration into submission systems, used by large publishers) [2]; ReviewerZero (listed in library guides) [7]. Vendor claims need pilot validation.
• Publisher-side ecosystems: the STM Integrity Hub (duplicate-submission and paper-mill checks across publishers), Clear Skies, Signals and the Problematic Paper Screener (tortured-phrase detection) are established in this field; these are named from domain knowledge and their current feature sets should be confirmed with the vendors before purchase.
• Text overlap (adjacent): Turnitin / iThenticate / Crossref Similarity Check for plagiarism, not image fraud.
• Choose A when: you are a publisher, society or university needing quick, defensible tooling and editorial-system connectors (ScholarOne, Editorial Manager, OJS plugins). Check EU hosting, data-processing agreement, treatment of unpublished manuscripts (confidentiality, no training on uploads).

Path B — Build on cloud APIs

• Use multimodal VLMs (Claude, Gemini, GPT, Mistral) to read figure legends, check consistency between legend and image (e.g. "n=3 lanes" vs panel), and write evidence summaries. VLMs are not reliable pixel-level forensic detectors; pair them with deterministic image-matching code.
• Orchestrate with n8n or custom services; embedding APIs for text similarity; EU regions or EU providers (Mistral) for unpublished content.
• Choose B when: you add a "reasoning / explanation" layer on top of Path A or open-source forensics.

Path C — Self-host (open source)

• Image forensics libraries (OpenCV feature matching, perceptual hashes, DINOv2-style embeddings, CLIP-class models), FAISS or Qdrant/pgvector for a corpus index of open-access figures (PubMed Central open subset, Europe PMC).
• Text signals: scikit-learn or fine-tuned encoder classifiers, open LLMs via vLLM for legend/reference consistency.
• Hardware: HW2 (one 24-32 GB GPU) for embedding and open VLMs on moderate volume; HW1 can suffice for hashing-only pipelines. Unpublished manuscripts never leave the premises, which is the main reason to choose C for funders and confidential review.
• Ops effort: corpus ingestion and index maintenance dominate; maintaining forensic quality needs an expert.

Path D — Custom training

Worthwhile for large publishers or national funders: train detectors for specific image types (Western blots, microscopy, flow cytometry) and paper-mill text templates using confirmed retraction cases as positives. Needs hundreds to thousands of verified cases and a validated hold-out set; label quality is limited by the number of adjudicated cases.

State of the art (2026)

• Image duplication/reuse screening is now routine at many large publishers, supported by commercial tools with large reference corpora [1][2][3].
• Generative AI raises the bar: AI-generated or AI-edited figures (blots, microscopy, histology) lack traditional duplication traces, so vendors added generation-indicator detectors [2]. Such detectors are probabilistic and degrade under compression and format conversion; treat results as indicators only.
• Paper-mill detection combines text signals, metadata/network analysis and cross-publisher information sharing. Mills adapt quickly, so detectors need continuous retraining and sharing of confirmed cases.
• Limits and failure modes: false positives on legitimate reuse (shared controls, schematics, previously published figures with permission); false negatives for new manipulation types and for unpublished raw data; bias against non-native English writers when text-style detectors are used; reference corpora only cover open or licensed images. Results are not proof of misconduct.

Shared resources & combinations

• Image forensics shares embeddings, vector indexes and image-classification stacks with ANOM-017 (document tampering) and ANOM-014 (identity-document forgery); the AI-generated-content detectors overlap with ANOM-015.
• Text/paper-mill signals share methods with ANOM-010 (fake reviews: coordinated, template-like text) and graph analytics for collusion rings.
• Domain tooling: SCI-008 (microscopy & scientific image analysis) provides the image-analysis building blocks; KNOW-008 and DOC-014 provide literature parsing and reference verification.
• Policy sibling: EDU-008 for student exam and thesis integrity (different legal regime).
• Hardware: a single HW2 server with a vector DB and VLM serves all of these; the vector index is the shared asset.

Effort, cost & KPIs

• Path A: weeks for a pilot, a few months to integrate into editorial systems; licence pricing is vendor-specific (verify per quote).
• Path B/C: 3-9 months incl. corpus indexing and validation; a forensic expert plus an engineer.
• Path D: 6-18 months; depends on labelled case availability.
• KPIs: share of manuscripts screened, flags per 1,000 manuscripts, precision of flags after human review, time per case, cases leading to correction/retraction, author appeal outcomes, turnaround impact on honest authors.

People & change

Start with management and the research-integrity officer or editor-in-chief, who own the procedure and the decisions. Involve the department lead for editorial operations, IT for the submission-system integration, and the DPO early: author names, e-mails and affiliations are personal data, and a DPIA is likely because manuscripts and individuals are systematically evaluated, with a possible reputational impact. Agree AVV (Art. 28 GDPR) with every vendor and check where figures are processed.

Works council: depends. Screening submissions of external authors is not employee monitoring, but if staff or the institution's own researchers are screened, or editorial workflow logs staff actions, co-determination under §87(1) no. 6 BetrVG may apply and §90(1) no. 3 requires timely information. In public institutions the staff council (Personalrat) is the counterpart.

Concerns: false accusations and reputational harm. Address them with a written process: every flag is only a hypothesis, a human reviews the highlighted evidence, authors have a right of reply, and no automatic rejection. Pilot with one journal or one faculty. Support AI literacy (Art. 4 AI Act) of editors with short trainings on tool limits and false-positive patterns. See GUIDE-005, GUIDE-008 and GUIDE-010.

Risks, regulation & compliance

• EU AI Act: screening of manuscripts is not listed in Annex III and is typically minimal risk. It may become high-risk if used to make decisions about students in education (Annex III no. 3, e.g. evaluating learning outcomes, detecting prohibited behaviour during tests) or in employment contexts (Annex III no. 4) when applied to staff. Therefore depends; AI literacy (Art. 4) applies [8].
• GDPR: manuscripts, author identities, allegations and misconduct findings are personal data (Art. 6, possibly Art. 10-like sensitivity for allegations); require data-processing agreements, retention limits, purpose limitation, and fair procedure. Automated decision-making with legal effect (Art. 22) must be avoided: always a human decision [9].
• Confidentiality: unpublished manuscripts are confidential; vendors must not train on or retain them.
• Defamation / due process: an unverified flag shared outside the process can harm reputations; follow COPE guidance and DFG / institutional good-scientific-practice rules, notify authors and allow replies.
• Mitigations: two-person review, retain evidence, document tool versions and thresholds, do not rely on score alone, pilot with known cases.

Getting started

1. Define the process: who receives flags (editor, RIO), what is escalated, how authors reply.
2. Pilot Path A tools (e.g. Proofig and Imagetwin) with 100-200 manuscripts incl. known retracted cases to measure precision and workload.
3. Review contracts for EU hosting, no-training clauses and retention of unpublished content.
4. Add text/metadata signals and a duplicate-submission check as a second layer.
5. Track precision, time per case and author appeals quarterly; adjust thresholds.

Sources

1. Proofig — image integrity & image plagiarism checker — https://www.proofig.com/
2. Imagetwin — research image plagiarism checker — https://imagetwin.ai/image-plagiarism-detection/
3. ASM incorporates Imagetwin to address image duplication (PMC) — https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12505991/
4. University Times (Pitt) — rollout of ProofigAI — https://www.utimes.pitt.edu/news/research-protections
5. Stanford Lane Medical Library — Research Integrity Tools — https://laneguides.stanford.edu/research-integrity/tools
6. Image Forensics: Detecting duplication of scientific images with manipulation-invariant image similarity — https://arxiv.org/pdf/1802.06515
7. Texas Tech University Libraries — AI detection tools guide (lists ReviewerZero) — https://guides.library.ttu.edu/artificialintelligencetools/detection
8. Regulation (EU) 2024/1689 (AI Act) — https://eur-lex.europa.eu/eli/reg/2024/1689/oj
9. Regulation (EU) 2016/679 (GDPR) — https://eur-lex.europa.eu/eli/reg/2016/679/oj