Supply Chain
Vulnerabilities across training data, models, adapters, conversion pipelines, and deployment platforms — where third-party pre-trained models, datasets, and model artifacts can be tampered with, poisoned, or silently replaced.
What it is
LLM supply chains are exposed to vulnerabilities that affect the integrity of training data, models, adapters, conversion pipelines, and deployment platforms, producing biased output, breaches, or failures. Where traditional software risk focuses on code and dependencies, ML extends it to third-party pre-trained models, datasets, and model artifacts that can be manipulated through tampering, poisoning, or malicious replacement.
Building models depends on third-party weights, datasets, and reusable adapters (LoRA and other PEFT techniques) shared on public hubs. The 2026 entry treats model artifacts, provenance, and conversion/merge workflows as first-class attack surfaces, and on-device models widen the surface to firmware and manufacturing.
Some risks here overlap with LLM05 Data and Model Poisoning; this entry focuses on the supply-chain aspect. Supply-chain risks specific to agentic applications — MCP servers and tool registries — are owned by ASI04 in the OWASP Top 10 for Agentic Applications, and MITRE ATLAS catalogs the techniques under AML.T0010.
Common examples
- Vulnerable or outdated components: deprecated packages, serving frameworks, or models; coding assistants also hallucinate plausible package names that attackers pre-register (“slopsquatting”).
- Licensing risk: diverse software and dataset licences create legal and compliance exposure if unmanaged.
- Tampered pre-trained models: backdoors survive migration away from pickle — they can live in a computational graph and persist in “safe” formats like ONNX, or exploit parser memory-corruption bugs.
- Weak provenance and unsigned artifacts: Model Cards document but do not prove origin; unsigned models, adapters, and checkpoints can be swapped at the promotion boundary, especially when pipelines resolve a mutable “latest” tag.
- Vulnerable adapters and conversion/merge/quantization workflows: a malicious LoRA adapter or a hijacked conversion service injects behaviour; weights can be crafted so the full-precision model is benign while the quantized artifact misbehaves.
- On-device supply chain: compromised manufacturing, firmware exploits, and re-packaged apps with tampered models.
- Unclear terms and privacy policies: model-operator T&Cs can route application data into training and later exposure.
Prevention & mitigation
- Vet sources and suppliers. Review T&Cs and privacy policies, use trusted suppliers only, and re-assess on any change to their security posture.
- Apply classic component hygiene. Vulnerability scanning, patching to maintained versions, and verifying that AI-suggested dependencies actually exist and are the intended package.
- Red-team third-party models. Evaluate candidate models against the in-scope use cases, and continue in production with anomaly detection and adversarial-robustness testing.
- Maintain a signed inventory. Extend SBOMs to models, adapters, and datasets (AIBOM / ML-BOM, e.g. CycloneDX), and track licences in the same inventory.
- Use verifiable sources and signing. Cryptographic model signing backed by a transparency log binds an artifact to a signer identity — but signing proves integrity and origin, not safety, so pair it with immutable references, provenance policy, release gates, and behavioural evaluation.
- Treat conversion and merge as high-risk. Strictly monitor collaborative model-development environments and the promotion points where formats are transformed.
- Harden the edge. Encrypt on-device models with integrity checks, use attestation APIs, and reject unrecognised firmware and untrusted device states.
Example attack scenarios
- A compromised dependency reaches a development or inference environment, as in the 2022 PyTorch torchtriton attack that shadowed the legitimate nightly dependency and exfiltrated data.
- An attacker publishes a tampered model under a trusted-looking name (the PoisonGPT proof of concept), spreading misinformation while passing standard benchmarks.
- A supplier's LoRA adapter is subtly altered and later merged into a deployed model, providing a covert entry point.
- An attacker hijacks a model conversion or merge service to inject behaviour into an otherwise public model.
- Model namespace reuse: an author deletes their account, an attacker re-registers the freed name, and pipelines resolving the model by name alone pull the attacker's version, leading to remote code execution.
- A scanner-and-safe-loader bypass: corrupted pickle streams or a graph-level backdoor in a “safe” format defeat both the malware scanner and the safe-load flag.
- A compromised build pipeline signs a backdoored artifact with the organisation's own infrastructure, so it passes downstream provenance checks (as in the Ultralytics attack).
References
- OWASP Top 10 for LLM Applications 2026 — LLM04:2026 Supply Chain: genai.owasp.org
- OWASP CycloneDX ML-BOM & OWASP AIBOM project
- OpenSSF Model Signing project and Sigstore; SLSA release gates
- MITRE ATLAS — AML.T0010 AI Supply Chain Compromise
- OWASP A06:2021 Vulnerable and Outdated Components
How Neural Industries helps. We build the AI bill of materials for your models, adapters, and datasets, verify provenance and signing, and stress-test the promotion and conversion points where tampered artifacts slip into production.
Request an assessment →