AI/ML Supply Chain

18 min

Overview

AI supply chain attacks occur when attackers modify or replace machine learning libraries, models, or associated data used by systems. Such vulnerabilities can lead to unauthorized system access or behavior manipulation.

To start working with the APIs, visit the Supply Chain Attacks Guide.

Key Features

Report Generation

Produces detailed reports classifying risks as Low, Medium, High, or Critical.

Repository Integration

Seamlessly integrates with GitHub, Huggingface, and AWS S3 for automated scanning of repositories and detecting vulnerabilities.

Model Format Support

Supported frameworks and file formats include:

Framework	File Format	Deserialization Risks	Backdoor Risks	Runtime Risks
TensorFlow	.pb	✅	✅
Tensorflow	.h5	✅	✅	✅
TensorFlow/PyTorch Checkpoint	.ckpt	✅
Keras	.keras	✅	✅
Keras	.h5	✅	✅
PyTorch	.pt, .pth, .bin	✅
ONNX	.onnx			✅
Scikit-Learn	.pkl	✅
GGUF	.gguf			✅
SafeTensor	.safetensor	✅
Misc	.zip	✅

Additional File Formats

Framework	File Format	Detections
Jupyter Notebook	.ipynb	Hardcoded secrets,Passwords PII, Tokens(API, Web, other)
Python	.py	Hardcoded secrets,Passwords PII, Tokens(API, Web, other)

AI Software Bill of Materials (SBOM)

File Format	Detections
Requirements File (Autodiscovered)	Libraries, Unsafe Library Flags
Jupyter Notebook (Autodiscovered)	Libraries, Unsafe Library Flags

Risk Analysis

1. Deserialization Risks

Occurs when unverified data is used to rebuild objects. Attackers may exploit these to introduce malicious code, compromising system integrity.

Activation: Serialization attacks exploit the process of saving and loading machine learning models, specifically targeting vulnerabilities in the serialization and deserialization mechanisms. These attacks often involve malicious payloads embedded within serialized model files.
Purpose: The primary goal of serialization attacks is to gain unauthorized access, execute arbitrary code, or manipulate the system in unintended ways. Attackers leverage the trust developers place in model files and frameworks, embedding harmful code that executes during deserialization to compromise environments, exfiltrate sensitive data, or alter system functionality.
Detection: Serialization attack detection involves examining serialized files for suspicious code patterns and loading models in isolated environments, such as sandboxes, to monitor for unexpected behaviors or executions during deserialization.

2. Backdoor Risks

Hidden pathways allow attackers to manipulate model behavior through specific triggers. These covert exploits remain undetected during normal operations.

Activation: Backdoor threats involve hidden pathways or triggers embedded in the model’s architecture that activate only when a specific input or condition is provided.
Purpose: Backdoors are designed to manipulate model outputs for specific scenarios, enabling attackers to produce targeted malicious outputs without disrupting normal operations.
Detection: Backdoor risks are harder to detect as they appear dormant in normal use but can be identified by analyzing model architecture for unusual pathways or using specialized tools like Netron for visual inspection and security scanners to detect presence of unusual code.

3. Runtime Risks

Activated during model inference or task execution, runtime risks involve malicious code execution, leading to unauthorized access or manipulation.

Activation: These risks involve malicious code that executes during the model’s inference or runtime. The threat typically resides in the model files, and the malicious code is triggered as the model processes input data.
Purpose: The aim is to compromise the system at runtime, such as gaining unauthorized access, stealing data, or altering the model’s behavior dynamically.
Detection: Runtime risks often exploit code execution features in formats like TensorFlow’s SavedModel or Keras’ custom objects.

Benefits

Real-Time Scanning: Quickly identifies vulnerabilities in AI/ML models and notebooks.
Comprehensive Framework Support: Compatible with diverse model frameworks.
Dynamic Risk Identification: Adapts to evolving security threats.
Thorough Assessments: Provides a full spectrum of vulnerability analysis.
Standards Compliance: Aligns with OWASP, MITRE, and CWE standards.
Scalability: Automated workflows ensure efficient scaling.
Seamless Integration: Effortless compatibility with popular AI/ML platforms.
Detailed Reports: Helps prioritize vulnerabilities and allocate resources.
Competitive Advantage: Showcases commitment to security, appealing to stakeholders and clients.

Parameters

Parameter	Data Type	Description	Remarks
repo_type	String	The type of repository to scan	(e.g., github, gitlab, bitbucket, huggingface, s3_bucket, azure_blob, gcp_storage ).
repo_url	String	URL of the repository to be scanned.	Formats accepted - Hugginface - https://huggingface.co/<<username>>/<<reponame>> Github - https://github.com/<<username>>/<<reponame.git>>
branch_name	String	Name of the branch in the repository to be scanned	(e.g., main , dev).
depth	Integer	Number of recent commits to scan from the specified branch	e.g., depth: 10 scans the latest 10 commits.
username	String	Username used for authenticating access to private repositories when required.	(e.g., GitHub, GitLab, Bitbucket, huggingface)
pat	String	Personal Access Token used for authenticating access to private repositories	(e.g., GitHub, GitLab, Bitbucket, huggingface) The Personal Access Token must have sufficient permissions to read repository contents, metadata, and history—typically including repo (for GitHub), api (for GitLab), or equivalent scopes for other platforms.
model_id	String	Required if repo_type is file; identifier returned by the model ID generation API, used to locate the uploaded file for scanning.	Obtainable during <POST> Generate Model ID
aws_access_key_id	String	AWS access key ID used to authenticate and access S3 buckets when repo_type is s3_bucket.	It pairs with the aws_secret_access_key to ensure secure access and authorization.
aws_secret_access_key	String	AWS secret access key paired with aws_access_key_id for authenticating access to S3 resources.	It works with the aws_access_key_id to securely validate access permissions.
region	String	The AWS region where the S3 bucket is hosted	e.g., us-east-1, ap-south-1
bucket_name	String	Required if repo_type is s3_bucket or gcp_storage ; specifies the name of the cloud storage bucket to be scanned	if not provided, all accessible buckets will be auto-detected and scanned (requires list and read permissions)
azure_connection_string	String	Connection string used to authenticate and access Azure Blob Storage when repo_type is azure_blob.	must have permissions to list containers and read blobs (e.g., via Storage Blob Data Reader or Contributor roles).
container_name	String	Name of the Azure Blob Storage container to be scanned; required when repo_type is azure_blob.	if not provided, all accessible containers will be auto-detected and scanned (requires list and read permissions).
service_account_json_file	File	File containing the JSON key for a GCP service account; required for authentication when repo_type is gcp_storage.	must have permissions like storage.buckets.list, storage.viewer, storage.objectViewer for proper scanning access.

Sample Artifact

Refer to the Vulnerability Report for detailed insights.

Appendix

Glossary

Deserialization Risks: Vulnerabilities arising during object reconstruction from untrusted data or files.
Backdoor Risks: Undetected pathways that allow behavior manipulation.
Runtime Risks: Threats triggered during inference or file execution.

For further queries, contact Support.