Software supply-chain security: malicious package & model detection

Modern software is mostly third-party code, and attackers publish typosquatted, hijacked or backdoored packages and poisoned model files that run code the moment they are installed or loaded. Classic scanners only match known CVEs. AI-assisted behavioural analysis inspects what a package or model actually does, blocks suspicious versions at the repository gate and records SBOM and provenance evidence that the EU Cyber Resilience Act (CRA) and NIS2 increasingly demand.

Problem & value

• Today: developers install hundreds of transitive dependencies per project. Defence relies on CVE databases (which only know vulnerabilities, not malicious intent), maintainer reputation and manual review that does not scale. Malicious releases are usually live for hours or days before registries remove them, and install scripts (postinstall, setup.py) execute with the developer's or CI's privileges, where secrets and cloud tokens are present.
• Model artefacts are a second channel: many model formats (notably Python pickle) can execute code on load, so downloading a model from a public hub is comparable to running an unknown binary [4][5].
• AI value: LLMs and ML classifiers read code semantically (obfuscation, exfiltration, unexpected network calls, dropper patterns), compare a new release against its history, and summarise findings for triage, which shrinks the window between publication and block. Evidence for compliance is a by-product.
• Why now (EU): the CRA requires manufacturers of products with digital elements to exercise due diligence on third-party components, including open source, and to keep an SBOM; vulnerability and incident reporting duties start in September 2026, with the main obligations following in December 2027 [1]. NIS2 explicitly lists supply-chain security among risk-management measures [2].

Variants

• SMB / software house: a registry-aware scanner on pull requests plus a policy to pin and review new dependencies.
• Enterprise / platform team: a repository firewall in front of Artifactory/Nexus that quarantines new packages and models, with an internal approved-package catalogue.
• ML/AI teams: model-hub gatekeeping, conversion to safe formats (safetensors), ML-BOM and dataset provenance; links to SEC-013 and SEC-014.
• Manufacturers (CRA): SBOM, VEX and attestation pipeline per product release.
• Public sector: curated mirrors of approved open source, in line with national procurement and open-source security guidance.
• Private / hobby developers: free scanners in the IDE and package-manager install hooks.

How it works

1. developer / CI requests dependency or model
2. proxy / firewall intercepts (npm, PyPI, Maven, container, model hub)
3. metadata signals (age, maintainer change, name similarity, download anomalies)
4. static + sandboxed dynamic analysis (install script, network, filesystem, process behaviour)
5. ML/LLM scoring (obfuscation, exfiltration, capability diff vs previous release)
6. model-file scan (unsafe pickle opcodes/imports, format conversion, hash and signature check)
7. policy engine: allow | warn | quarantine | block
8. SBOM / VEX / attestation written to evidence store
9. human review of quarantined items (security + developer)

• Behavioural classifiers: trained on labelled malicious and benign releases; features include new capabilities (shell, network, env access) introduced between versions.
• LLM triage: explains why a diff is suspicious, drafts the report and reduces false positives; it must treat package content as untrusted input, since code and README text can carry prompt injection (see SEC-010).
• Human-in-the-loop: quarantine and exception decisions stay with a named owner; auto-block applies only to high-confidence verdicts, with a fast override path to avoid blocking releases unnecessarily.

Data preparation

The inputs are dependency manifests and lock files (package.json, requirements.txt, pom.xml), build and CI configuration, the package registries in use, and for AI projects also the model files pulled from Hugging Face. In an SMB these sit in Git repositories, a CI server, and sometimes only on a developer's laptop.

Typical gaps: no complete list of repositories and owners, unpinned versions or missing lock files, builds that pull directly from the internet, no SBOM, and forgotten legacy projects without any maintainer.

Preparation steps: 1. Inventory all repositories, languages and owners (PREP-001, PREP-004). 2. Commit lock files and pin versions; route installs through one proxy or mirror. 3. Generate a first SBOM per project (Syft, CycloneDX) and store it with the build. 4. Mark internal packages to prevent dependency-confusion attacks. 5. Record accepted exceptions with owner and expiry.

Tools: Git platform, CI, Syft, Dependency-Track. Rough share of effort: 10-20 percent, mostly repository clean-up; see GUIDE-003.

Implementation paths

Which path for whom

• SMB: Path A — registry-proxy and dependency-scanning services (Snyk, Socket, GitHub/GitLab built-ins, JFrog Xray) are mature, cheap per developer and plug into existing pipelines; the process is standard.
• Enterprise: Path A — buy a commercial software-composition-analysis and artifact-firewall platform that already holds threat intelligence on malicious packages; add Path C components such as Dependency-Track for SBOM management where needed.
• Developer: Path C — OSV-Scanner, Syft/Grype, GuardDog, Dependency-Track and a self-hosted proxy (Nexus, Artifactory OSS alternatives) run on a small server and are the usual tool chain.
• Public sector: Path C — self-hosted scanning and an internal package mirror keep source code and dependency lists on sovereign infrastructure and avoid procurement hurdles of foreign SaaS.

Path A — Buy (off-the-shelf)

• Package behaviour / repository firewalls: Socket (npm/PyPI/Maven/Go, PR bot, firewall), Sonatype Repository Firewall and Lifecycle, JFrog Xray and Curation (with model scanning for Hugging Face), Snyk, Mend, Checkmarx, Phylum (now part of Veracode), ReversingLabs Spectra Assure (software and model supply chain), Cloudsmith, GitHub Dependabot with malware advisories.
• Model artefacts: Protect AI Guardian (now Palo Alto Networks), HiddenLayer model scanner, JFrog ML model scanning, and the scanning that Hugging Face itself runs on uploaded repositories [5].
• EU hosting: most vendors are US-based; check data residency and DPA terms (they mostly process public package metadata).
• Choose A when: you want protection in days and have no dedicated supply-chain team.

Path B — Build on cloud APIs

• Feeds: OSV.dev and the OpenSSF malicious-packages repository, GitHub Advisory Database, deps.dev, registry APIs (npm, PyPI) for metadata and publish events [3].
• LLM triage: frontier models via EU regions or EU providers (e.g. Mistral) to summarise diffs and deobfuscate snippets, strict no-execute policy and structured outputs.
• Orchestration: CI plugin or proxy hook; verdicts into Slack/Jira; SBOM via Syft or cdxgen into Dependency-Track.

Path C — Self-host (open source)

• Detection: OpenSSF Package Analysis (sandboxed dynamic analysis), GuardDog (Datadog, heuristics plus Semgrep), OSV-Scanner, Trivy, Grype; model scanning with ModelScan, picklescan, and Fickling (Trail of Bits).
• Provenance: Sigstore (cosign), SLSA verification, in-toto, OpenSSF Scorecard.
• Gate and SBOM: Nexus/Artifactory or devpi/Verdaccio mirror, Syft, cdxgen, OWASP Dependency-Track.
• LLM layer: open code models (Qwen-Coder family, DeepSeek-Coder, Codestral-class) served with vLLM or Ollama for diff triage. HW2 (one 24 GB GPU) suffices; sandbox analysis needs isolated VMs (e.g. gVisor, Firecracker) without production network access.

Path D — Custom training

Worthwhile only for vendors or large organisations with proprietary registries: train classifiers on labelled malicious/benign releases (public datasets from OpenSSF and academic corpora plus internal incidents), using capability-diff features. Needs thousands of labelled samples and ongoing retraining because attackers adapt.

State of the art (2026)

• Behaviour beats signatures: vendors report catching new malicious releases from behaviour and capability changes before any CVE or advisory exists; no independent benchmark with agreed metrics exists, so treat vendor detection claims as unverified and pilot on your own history.
• Registry-side hardening: npm and PyPI introduced trusted publishing (OIDC), provenance attestations and 2FA requirements, but account takeover and install-script abuse remain common.
• Model supply chain: a shift from pickle to safetensors, plus hub-side scanning, reduces but does not remove risk; research has shown scanners can be evaded (e.g. via unusual pickle constructs), so loading untrusted models in isolation remains necessary [5].
• Attacks on AI tooling: AI coding assistants may hallucinate package names that attackers then register (slopsquatting); allow-listing and firewall checks catch this.
• Limits: false positives on legitimately obfuscated or native-binary packages, delayed-activation payloads, sandbox evasion, targeted attacks on specific build environments, and LLM triage errors. Detection complements, never replaces, pinning, lockfiles, reproducible builds and least-privilege CI.

Shared resources & combinations

• SEC-004 (own-code and CVE scanning) shares SBOM generation, SCA and the CI gate; SEC-003 malware analysis shares sandbox and classifier assets; SEC-008 consumes advisories and campaign intelligence.
• SEC-009 reuses SBOM and attestation evidence for NIS2/DORA/CRA; SEC-013 and SEC-014 cover model inventory and adversarial testing; SEC-010 protects the LLM triage layer; SEC-012 feeds design-review decisions on dependency choice.
• Shared stack: llm-code on HW2, sandbox cluster, artefact repository.

Effort, cost & KPIs

• Path A: days to weeks; pricing per developer or per repository; most effort is policy tuning.
• Path B: 1-3 months for proxy integration, SBOM pipeline and triage workflow.
• Path C: 2-4 months; modest GPU plus sandbox infrastructure and ongoing rule upkeep.
• KPIs: time from malicious publication to block, share of dependencies covered by the gate, false-positive rate and developer-blocking time, number of quarantined versions, SBOM coverage per release, share of models in safe formats with verified hashes.

People & change

Management and the head of development decide on the policy "what may enter the build"; IT security and IT run the proxy and scanners; legal and compliance track the Cyber Resilience Act and NIS2 evidence duties; developers are the daily users and decide whether the tool is accepted. Involve developers from the pilot onwards, because a gate that blocks builds without explanation is bypassed quickly.

Works council: depends. The scanner mainly checks packages, but logs of who installed or requested which dependency are individually attributable, so §87(1) no. 6 BetrVG can apply in companies with a works council. Inform it early (§90(1) no. 3) and agree that logs serve security, not performance review (GUIDE-004). Personal data are limited to developer names in logs; a DPIA is normally not needed, but the DPO should confirm and an AVV is required for SaaS vendors (GUIDE-005, GUIDE-010).

Start in warn-only mode on two or three active repositories (GUIDE-009), then enforce blocks for clearly malicious packages. Typical concerns: slower builds and false positives; address them with a fast exception process and visible metrics. Support AI literacy (Art. 4 AI Act) by explaining that the verdicts are probabilistic and by training developers on typosquatting and model-file risks (GUIDE-006, GUIDE-013). Humans own the exceptions and the final release decision.

Risks, regulation & compliance

• EU AI Act: minimal risk (no Annex III use; AI literacy duty under Art. 4). Not a prohibited practice. If used in critical infrastructure safety components, reassess.
• CRA (Regulation (EU) 2024/2847): SBOM, due diligence on components, vulnerability handling and reporting of actively exploited vulnerabilities; this use case produces the evidence but does not itself make a product compliant [1].
• NIS2 / DORA: supply-chain risk management duties; link to SEC-009 [2].
• GDPR: mainly package metadata; maintainer names or e-mails and developer telemetry (IDE/CI logs) can be personal data, so use a DPA, data minimisation and EU hosting where possible.
• Operational risks: over-blocking stops releases (use tiered policy and override), sandbox escape (isolate strictly), prompt injection through package content, vendor lock-in and over-trust in vendor claims.
• Licensing: the same pipeline can flag licence conflicts; do not treat verdicts as legal advice.

Getting started

1. Generate SBOMs for your top repositories (Syft or cdxgen) and load them into Dependency-Track.
2. Pilot one behavioural scanner (e.g. Socket or OpenSSF/GuardDog) in report-only mode on pull requests.
3. Put an internal mirror or firewall in front of npm/PyPI and add a short quarantine window for new versions.
4. Restrict model downloads to an approved list, prefer safetensors, verify hashes and load untrusted models only in isolation.
5. Define owners, override rules and CRA/NIS2 evidence retention, then review KPIs monthly.

Sources

1. European Commission - Cyber Resilience Act - https://digital-strategy.ec.europa.eu/en/policies/cyber-resilience-act
2. Directive (EU) 2022/2555 (NIS2), Art. 21 - https://eur-lex.europa.eu/eli/dir/2022/2555/oj
3. OpenSSF - malicious-packages and Package Analysis projects - https://github.com/ossf/package-analysis
4. OWASP - Software Component Verification Standard / CycloneDX (ML-BOM) - https://cyclonedx.org/capabilities/mlbom/
5. Hugging Face - Pickle scanning documentation - https://huggingface.co/docs/hub/security-pickle
6. Socket - documentation - https://docs.socket.dev
7. Datadog GuardDog - https://github.com/DataDog/guarddog
8. SLSA framework - https://slsa.dev
9. Regulation (EU) 2024/1689 (AI Act) - https://eur-lex.europa.eu/eli/reg/2024/1689/oj