AI Supply Chain Security: Models, Datasets and Safe Serialization

Modern AI applications are assembled from parts you did not build: pre-trained models, datasets, tokenizers, libraries, plugins and hosted services. Each is a dependency that can be tampered with, mislabeled or simply malicious. The OWASP Top 10 for LLM Applications lists supply chain risk as its own category. This guide explains the main threats and a practical set of controls.

What is in the AI supply chain

• Model weights downloaded from public hubs or vendors
• Datasets used for training, fine-tuning or evaluation
• Libraries and frameworks (training, inference, orchestration)
• Plugins, tools and servers that agents connect to (MCP security)
• Container images and base environments
• Third-party APIs and hosted models

Risk 1: Model files can execute code

Many model files are saved using Python's pickle format or formats built on it. Python's own documentation warns that unpickling data from an untrusted source can execute arbitrary code, because a pickle can instruct the interpreter to call any function during loading. A model file from an unknown uploader can therefore compromise the machine that loads it, before any "inference" happens.

Mitigations:

• Prefer safe formats such as safetensors, which stores only tensor data and cannot carry executable code.
• If you must load pickle-based files, do it only for sources you trust, inside an isolated sandbox with no credentials and restricted network access.
• Scan model files with tools that detect dangerous pickle operations, as a supplement to, not a replacement for, trust decisions.
• Some frameworks offer loading modes that restrict what can be deserialized; enable them.

Risk 2: Tampered or impersonating models

Attackers upload models with names resembling popular ones, or modify a legitimate model to behave maliciously under specific triggers (a "backdoor") while performing normally otherwise. These are hard to detect by testing alone.

Mitigations:

• Download from the official publisher's account or repository, and double-check the exact name.
• Pin to a specific revision or commit hash, not a moving tag such as "latest."
• Verify checksums or signatures where the publisher provides them.
• Mirror approved models into your own registry and let applications pull only from there.
• Review the model card, license and community reports before adopting.

Risk 3: Poisoned data

Training or fine-tuning on data that an attacker can influence (scraped web content, user-submitted examples, a public dataset) can plant behaviors or biases. The same applies to documents added to a retrieval index (RAG security).

Mitigations:

• Track the provenance of each dataset: source, date, license, collection method.
• Restrict and log who can add to training sets and indexes.
• Sample and inspect data; use deduplication and anomaly checks.
• Evaluate models on held-out tests, including checks for unexpected triggers.

Risk 4: Vulnerable or malicious libraries

AI projects pull in large dependency trees, and newly popular tools are targets for typosquatting and compromised releases. Apply ordinary software supply chain practice: lock versions, scan for known vulnerabilities, review new dependencies, and avoid installing packages suggested by a model without verifying they exist (package hallucination).

Risk 5: Unsafe code in model repositories

Some model repositories include custom code that runs on load (for example options that "trust remote code"). Treat that as installing software from the uploader. Read it, or avoid the option.

Build an AI bill of materials

An AI BOM is an inventory of the components of each AI system, analogous to a software bill of materials and to a cryptographic inventory. For each system record:

An inventory lets you answer quickly when a vulnerability or license problem is announced: "Which of our systems use this?"

Practical checklist

1. Approved sources only; mirror into an internal registry.
2. Safetensors or other non-executable formats by default.
3. Pinned versions with hash verification.
4. Sandbox for any untrusted model loading.
5. Dependency scanning in CI for AI projects too.
6. Provenance records for training data and indexed documents.
7. An AI BOM kept up to date.
8. Review of licenses and acceptable-use terms before deployment.

Treat a model as you would an executable downloaded from the internet: know where it came from, verify it, and run it with the least access it needs.