AI/ML Supply Chain Risk Mapping
Overview
Supply chain compromise in AI/ML involves adversarial actions targeting the supply chain of AI/ML systems. This includes manipulation of software, data, or model dependencies, leading to vulnerabilities in downstream applications.
Supply Chain Compromise - Model
AI and machine learning systems often rely on open-source models, which are downloaded and fine-tuned using private datasets. These models require executing saved code during loading, creating risks of compromise through traditional malware or adversarial techniques that manipulate the model’s behavior. This dependency on external sources makes AI/ML systems vulnerable to supply chain attacks.
Supply chain compromise in AI/ML Models are extensively mapped across multiple frameworks, including MITRE ATLAS (AML.T0010) for model-specific risks, MITRE ATT&CK (T1195.002) for software supply chain manipulation, and the OWASP Top 10 for LLMs (LLM03:2025) for vulnerabilities in pre-trained models. It also aligns with the Map and Govern functions of the NIST AI Risk Management Framework (AI RMF), emphasizing the identification, management, and oversight of AI supply chain risks to ensure system trust. For detail refer information below table.
Framework | File Format | Deserialization | Backdoor | Runtime | Vulnerability ID |
|---|---|---|---|---|---|
Tensorflow | .pb | ✅ | | | AIS-TF-D-01: Protobuf Deserialization |
Tensorflow | .pb | | ✅ | | AIS-TF-B-01: TensorFlow PB Backdoor |
Tensorflow | .h5 | ✅ | | | AIS-TF-D-02: TensorFlow/Keras H5 Deserialization |
Tensorflow | .h5 | | ✅ | | AIS-TF-B-02: TensorFlow/Keras H5 Backdoor with malicious layers |
Checkpoint(TF,PT) | .ckpt | ✅ | | | AIS-CP-D-01: TensorFlow/PyTorch Checkpoint (saved intermediate model) Deserialization |
Keras | .keras | ✅ | | | AIS-KR-D-01: Keras Deserialization |
Keras | .keras | | ✅ | | AIS-TF-B-02: TensorFlow/Keras H5 Backdoor with malicious layers |
Keras | .h5 | ✅ | | | AIS-TF-D-02: TensorFlow/Keras H5 Deserialization |
Keras | .h5 | | ✅ | | AIS-TF-B-02: TensorFlow/Keras H5 Backdoor with malicious layers |
PyTorch | .pt | ✅ | | | AIS-PT-D-01: Pickle Serialization in PyTorch Models |
PyTorch | .pth | ✅ | | | AIS-PT-D-01: Pickle Serialization in PyTorch Models |
PyTorch | .bin | ✅ | | | AIS-PT-D-02: Serialization in PyTorch Models |
ONNX | .onnx | | ✅ | | AIS-ON-B-01: ONNX Architecture Backdoor |
ONNX | .onnx | | | ✅ | AIS-ON-R-01: Corrupted or Manipulated File Format |
GGUF | .gguf | | | ✅ | AIS-GU-R-01: GGUF Runtime Threat |
Scikit-Learn | .pkl | ✅ | | | AIS-PK-D-01: Pickle Serialization |
Misc | .zip | ✅ | | | AIS-MI-D-01: Zip File Trojan or file corruption |
SafeTensors | .safetensors | ✅ | | | AIS-ST-D-01: Improper file Format |
SafeTensors | .safetensors | ✅ | | | AIS-ST-D-02: File shards – path traversal Error |
MITRE ATLAS Mapping: Model Compromise
The vulnerabilities highlighted in the table above align with the concerns outlined in the ML Supply Chain Compromise technique under the MITRE ATLAS framework (AML.T0010). Specifically, risks associated with compromised models are covered under the sub-technique ML Supply Chain Compromise: Model (AML.T0010.003). These compromises typically involve malicious modifications to pre-trained models, unauthorized changes in model weights, or backdoor injections in serialized formats.
MITRE ATT&CK Mapping: AI/ML Supply Chain Compromise
The vulnerabilities identified in AI/ML systems align with the Supply Chain Compromise technique in the MITRE ATT&CK framework (T1195). This technique encompasses risks associated with the manipulation of products or delivery mechanisms prior to their receipt by the final consumer, aiming to compromise data or systems.
Specifically, AI/ML systems are susceptible to:
- Compromise of Software Supply Chain (T1195.002): Attackers might tamper with the software supply chain by modifying application source code, manipulating update mechanisms, or replacing legitimate software with malicious versions. In the context of AI/ML, this could involve distributing altered pre-trained models or corrupted datasets.
OWASP Top 10 for LLM: Supply Chain Vulnerabilities
The vulnerabilities identified in AI/ML systems correspond to the Supply Chain risk outlined in the OWASP Top 10 for Large Language Models (LLMs) (LLM03:2025 Supply Chain). This risk emphasizes the susceptibility of LLM supply chains to various vulnerabilities that can compromise the integrity of training data, models, and deployment platforms. Key concern include:
- Vulnerable Pre-Trained Models: Utilizing third-party models that may contain hidden biases, backdoors, or malicious features due to tampering or poisoning attacks.
Supply Chain Compromise - Software
Supply chain compromise in software occurs when adversaries target software components, including dependencies, configurations, or distribution mechanisms. These compromises aim to inject malicious elements into the development or runtime environment, potentially leading to unauthorized access, data breaches, or system failures.
Supply chain compromise in AI/ML software is extensively mapped across multiple frameworks, including MITRE ATLAS (AML.T0010) for software-specific risks, MITRE ATT&CK (T1195.001) for manipulation of software supply chains, the OWASP Top 10 for LLMs (LLM03:2025) and the OWASP Top 10 (A06:2021) for risks associated with outdated and vulnerable components. Additionally, these risks align with the Map and Govern functions of the NIST AI Risk Management Framework (AI RMF), which emphasize proactive identification, management, and governance of software supply chain vulnerabilities to maintain trust and security in AI systems.
Framework | File Format | Security | Vulnerability ID |
|---|---|---|---|
Jupyter Notebook | .ipynb | ✅ | AIS-PY-S-01: Compromised Components, libraries |
Python | .py | ✅ | AIS-PY-S-01: Compromised components, libraries |
Misc | requirement.txt | ✅ | AIS-PY-S-01: Compromised components, libraries |
MITRE ATLAS Mapping: ML Software Compromise
The vulnerabilities highlighted in the table above align with the concerns outlined in the ML Supply Chain Compromise technique under the MITRE ATLAS framework (AML.T0010). Specifically, risks associated with compromised libraries, components and other artefacts are covered under the sub-technique ML Supply Chain Compromise: ML Software (AML.T0010.001). These compromises typically involve usage of malicious, deprecated libraries and components.
MITRE ATT&CK Mapping: AI/ML Supply Chain Compromise
The vulnerabilities identified in AI/ML systems align with the Supply Chain Compromise technique in the MITRE ATT&CK framework (T1195). This technique encompasses risks associated with the manipulation of products or delivery mechanisms prior to their receipt by the final consumer, aiming to compromise data or systems.
Specifically, AI/ML systems are susceptible to:
- Compromise of Software Dependencies and Development Tools (T1195.001): Adversaries may inject malicious code into software dependencies or development tools commonly used in AI/ML pipelines. This can lead to the execution of unauthorized code within AI models or data processing workflows.
OWASP Top 10 for LLM: Supply Chain Vulnerabilities
The vulnerabilities identified in AI/ML systems correspond to the Supply Chain risk outlined in the OWASP Top 10 for Large Language Models (LLMs) (LLM03:2025 Supply Chain). This risk emphasizes the susceptibility of LLM supply chains to various vulnerabilities that can compromise the integrity of training data, models, and deployment platforms. Key concern include:
- Outdated or Deprecated Components: Relying on components that are no longer maintained, leading to potential security issues.
- Manipulated Components: Relying on components that are no longer maintained, leading to potential security issues.
OWASP Top 10 Classic: Vulnerable and Outdated Components
The vulnerabilities identified in AI/ML systems correspond to the Vulnerable and Outdated Components risk outlined in the OWASP Top 10 (A06:2021 - Vulnerable and Outdated Components). This risk highlights the dangers posed by relying on software libraries, frameworks, and components that are outdated, unsupported, or contain known vulnerabilities. Key concerns include:
- Outdated or Deprecated Components: Using outdated or deprecated software components that lack security patches or updates, leaving systems exposed to exploitation. This is particularly relevant for machine learning systems, where dependencies often include pre-trained models and data processing libraries that may not be actively maintained.
- Manipulated Components: Employing components that have been tampered with, whether through malicious intent or accidental corruption, which can introduce security vulnerabilities and compromise system integrity.
Vulnerability ID Nomenclature:
Format (AIS-XX-Y-NN) | Description |
|---|---|
AIS | AIShield AI Spectra SAST Assessment Tag |
XX | Framework Tag: - TF: TensorFlow - CP: Checkpoint - KR: Keras - PT: PyTorch - ON: ONNX - GU: GGUF - MI: Misc - ST: SafeTensors - PY: Python file(.py) or notebook (.ipynb) |
Y | Risk Tag: - D: Deserialization - B: Backdoor - R: Runtime - S: Security |
NN | Unique Number |
Risk Overview
Deserialization Risks:
Occurs when unverified data is used to rebuild objects. Attackers may exploit these to introduce malicious code, compromising system integrity.
Backdoor Risks:
Hidden pathways allow attackers to manipulate model behavior through specific triggers. These covert exploits remain undetected during normal operations.
Runtime Risks:
Activated during model inference or task execution, runtime risks involve malicious code execution, leading to unauthorized access or manipulation.
Security Risks:
Risks due to security related aspects embedded in files or code inadvertently used.