---
title: Image Classification
slug: 6DEI-image-classification
docTags: 
createdAt: 2023-05-23T05:17:57.000Z
---

The below input parameters are for different attack types. To start working with the APIs view the [Image Classification](docId:7pfTV3D26cUJsz6vMQloO). 

## File upload format

- **Data**: The processed data, ready to be passed to the model for prediction, should be saved in a folder.

[Download sample data](https://aisdocs.blob.core.windows.net/reference/upload/Image/ImageClassification/mnist_data.zip)

- **Label**: A CSV file should be created with two columns: "image" and "label." The first column should contain the image name, and the second column should contain the label. The label should be in integer format. Check sample label file attached.

[Download sample label](https://aisdocs.blob.core.windows.net/reference/upload/Image/ImageClassification/mnist_label.zip)

- **Model**: The model should be saved in either .h5 or TensorFlow format with full architecture. Full architecture is needed when loading the model to the platofrm for assessment either in encrypted or unencrypted. This can be ignored when model is hosted as an API.

[Download sample model](https://aisdocs.blob.core.windows.net/reference/upload/Image/ImageClassification/mnist_model.zip)

:::hint{type="warning"}
**Note**:

1.  All files uploaded should be in zipped format. The above files are sample data for the MNIST use case.
2. ** Prerequisite: **Only 2-5 % of data is needed. The data should be representative and balanced  across all classes. 
   1. **Model Extraction**: requires \~(450-900) samples or \~(50-100) samples per class. 
   2. **Model Evasion**:  requires \~(810-1620) samples or \~(90-180) samples per class.
3. For poisoning, check poisoning section to get sample data, label and models. 
:::

## Common parameters

The below table parameters are common for all attact types such as Extraction, Evasion, and Poisoning. 

:::hint{type="info"}
To see the additional parameter specific for each attact type, such as Extraction and Evasion, refer to the below sections. 
:::

| Parameter                      | Data type | Description                                                                                                    | Remark                                                                                                                                                        |
| ------------------------------ | --------- | -------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| model\_Id                      | String    | Model\_id received during model registration.<br />We need to provide this model ID in query parameter in URL. | You have to do model registration only once for a model to perform model analysis. This will help you track the no of api call made, and it's success metric. |
| **Request Body (Json format)** |           |                                                                                                                |                                                                                                                                                               |
| normalize\_data                | String    | Model trained on Normalized data.                                                                              | if model is trained on normalized data, then set this parameter as "yes" else "no".                                                                           |
| input\_dimensions              | String    | Provide input dimension of the image                                                                           | the parameter should be string in the format "(height, weight, channel)" For example 28\*28\*1 for MNIST                                                      |
| number\_of\_classes            | String    | Number of prediction classes.                                                                                  | the parameter should be string. Example MNIST : 10 (Range >0 & \<=200)                                                                                        |
| model\_framework               | String    | Original model is built with tensorflow framework.                                                             | curretly supported framework are: tensorflow, scikit-learn, keras. (Option:\[tensorflow])                                                                     |



:::ExpandableHeading
## Extraction parameters

| Parameter                      | Data type | Description                                                                                                                                                                                      | Remark                                                                                                                                                                                                                                                                                    |
| ------------------------------ | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Request Body (Json format)** |           |                                                                                                                                                                                                  |                                                                                                                                                                                                                                                                                           |
| attack\_type                   | String    | You can select the attack type either Blackbox or Greybox.                                                                                                                                       | **Blackbox**: for performing model analysis, no information about model or data will be used.                                              **Greybox**: information about data will be leverage for creation of attack data              **Note:&#x20;**&#x6F;nly 2-5 % of data is needed |
| number\_of\_attack\_queries    | String    | Number of attack queries that model will be<br />subjected to.                                                                                                                                   | generally Heigher the number of attack queries, better would be the analysis. And it would take more time to process. (Range:  >0 & \<=400000)                                                                                                                                            |
| vulnerability\_threshold       | String    | Threshold percent of stolen model accuracy<br />at which defense model should be generated.                                                                                                      | Threshold percent of stolen model accuracy<br />at which defense model should be generated<br />(Range :  0.0 - 1)                                                                                                                                                                        |
| model\_api\_details            | String    | If use\_model\_api is Yes, then provide API<br />details of hosted model as encrypted JSON<br />string is mandatory                                                                              | provide this only if use\_model\_api is "yes".                                                                                                                                                                                                                                            |
| use\_model\_api                | String    | Use model API to train your model instead of uploading the model as a zip file.                                                                                                                  | when this parameter is yes, you don't have to upload model as zip. You can pass api url along with other verification credential in json file.                                                                                                                                            |
| defense\_bestonly <br />       | String    | Choose to train your model until it achieves the best results or above 95% accuracy.                                                                                                             | when selected **"yes"**, it will train N number of model and select best model. Ofcourse this will take longer time. If **"no"**, then once defense model accuracy reached above 95% It will stop                                                                                         |
| encryption\_strategy           | Int       | Choose a encryption strategy for you model. if model is uploaded directly as a<br />zip pick 0, 1 if model is encryted as .pyc and<br />uploaded as a zip. Ignore if use\_model\_api is<br />Yes | select 0: pass tensorflow model as it is, select 1: pass encrypted model. It could be .pyc file                                                                                                                                                                                           |
:::

:::ExpandableHeading
## Evasion Parameters

| Parameter                      | Data type | Description                                                                                                                                                                       | Remark                                                                                                                                                                                             |
| ------------------------------ | --------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Request Body (Json format)** |           |                                                                                                                                                                                   |                                                                                                                                                                                                    |
| model\_api\_details            | String    | If use\_model\_api is Yes, then provide API details of hosted model as encrypted JSON string is mandatory                                                                         | provide this only if use\_model\_api is "yes".                                                                                                                                                     |
| use\_model\_api                | String    | Use model API to train your model instead of uploading the model as a zip.                                                                                                        | when this parameter is yes, you don't have to upload model as zip. You can pass api url along with other verification credential in json file.                                                     |
| defense\_bestonly              | String    | Choose to train your model until it achieves the best results or above 95% accuracy.                                                                                              | when selected **"yes"**, it will train N number of model and select best model. Ofcourse this will take longer time. If **"no"**, then once defense model accuracy reached above 95% It will stop  |
| encryption\_strategy           | Int       | Choose a encryption strategy for you model. if model is uploaded directly as a zip pick 0, 1 if model is encryted as .pyc and uploaded as a zip. Ignore if use\_model\_api is Yes | select 0: pass tensorflow model as it is, select 1: pass encrypted model. It could be .pyc file                                                                                                    |
:::

:::ExpandableHeading
## Drift

- &#x20;**Reference data** : The reference images or the clean images should be saved in a folder.

[Download sample reference data](https://aisdocs.blob.core.windows.net/reference/upload/Image/ImageClassification/DataDrift/ReferenceData/Reference_Data.zip)

- **Reference label** : A CSV file should be created with two columns: "image" and "label." The first column should contain the image name, and the second column should contain the label. The label should be in integer format. Check sample label file attached.

[Download sample reference label](https://aisdocs.blob.core.windows.net/reference/upload/Image/ImageClassification/DataDrift/ReferenceData/Reference_Label.zip)

- **Test data** :  Dataset under test that might contain drifted images.

[Download sample test data](https://aisdocs.blob.core.windows.net/reference/upload/Image/ImageClassification/DataDrift/ReferenceData/Test_Data.zip)

- **Test label** : A csv file containing corresponding labels to the universal data, two&#x20;
  columns: ‘image’ and ‘label’. The ‘image’ column contains the image name including the extension, and the second column should contain the label. The label should be in integer format.

[Download sample test label](https://aisdocs.blob.core.windows.net/reference/upload/Image/ImageClassification/DataDrift/ReferenceData/Test_Label.zip)
:::

:::ExpandableHeading
## Outlier

- Date :  Data in a ZIP format needs to be checked for the presence of outliers.

[Download sample data](https://aisdocs.blob.core.windows.net/reference/upload/Image/ImageClassification/OutlierDetection/ReferenceData/data.zip)
:::

:::ExpandableHeading
## Poisoning&#x20;

- Data Poisoning
  - **Universal Dataset** : Data containing potential poisoning data that needs to
    be tested.
  - [Download sample universal dataset](https://aisdocs.blob.core.windows.net/reference/upload/Image/ImageClassification/DataPoisoning/ReferenceData/universal_dataset.zip) &#x20;
  - **Universal Label** : A csv file containing corresponding labels to the
    universal data, two columns: ‘image’ and ‘label’. The ‘image’ column contains the imagename including the extension, and the second column should contain the label. The label should be in integer format.
  - [Download sample universal label](https://aisdocs.blob.core.windows.net/reference/upload/Image/ImageClassification/DataPoisoning/ReferenceData/universal_label.zip)
  - **Data** : The processed data, ready to be passed to the model for prediction, should be saved in a folder.
  - [Download sample data](https://aisdocs.blob.core.windows.net/reference/upload/Image/ImageClassification/DataPoisoning/ReferenceData/data.zip)
  - **Label** : A CSV file should be created with two columns: "image" and "label." The first column should contain the image name, and the second column should contain the label. The label should be in integer format. Check sample label file attached.
  - [Download sample label ](https://aisdocs.blob.core.windows.net/reference/upload/Image/ImageClassification/DataPoisoning/ReferenceData/label.zip)
  - **Model** : The model should be saved in either .h5 or TensorFlow format with full architecture. Full architecture is needed when loading the model to the platform for assessment.
  - [Download sample model](https://aisdocs.blob.core.windows.net/reference/upload/Image/ImageClassification/DataPoisoning/ReferenceData/model.zip)

:::

:::ExpandableHeading
## Experimentation with values

To improve the accuracy, you can experiment with the following values for your attack input parameters. In our example we have used an MNIST dataset in our model and the below table reflects the parameters suitable for it. For more information, please refer to the [reference implementation](https://github.com/bosch-aisecurity-aishield/Reference-Implementations/tree/main/Product_Taskpair_wise/Image_Classification/Extraction).

| Task Pair/Analysis Type | Type of Attack Strategy  | No of queries  | Outcome                               |
| ----------------------- | ------------------------ | -------------- | ------------------------------------- |
| Extraction              | Blackbox                 | 60000          | Stolen model accuracy between 85%-90% |
| Extraction              | Greybox                  | 20000          | Stolen model accuracy above 90%       |
| Evasion                 | N/A                      | N/A            | Evasion report                        |
| Poisoning               | N/A                      | N/A            | Model is poisoned or not.             |
:::

:::hint{type="info"}
To access all sample artifacts, please visit [Artifacts](docId\:iJNEOCXoStabvvrsq11fa).&#x20;

- For specific artifact details,  refer&#x20;
  - Vulnerability Report : [Vulnerability Report](docId\:hL0uT2MWlCBkt8F97fr-W)    &#x20;
  - Sample Attacks : [Sample Attacks](docId:4G1mjM5lQjfm8t5WBVwpr)
  - Defense Report: [Defense Report](docId\:VtzlTtpja2VSF2j0STLSQ)                   &#x20;
  - Defense Model: [Defense Model](docId\:xSbxmZXW4vv14-8NmBF8M)
:::

:::hint{type="info"}
**Note**: For Image classification, supported attack types are - Extraction, Evasion and Poisoning, Drift and Outlier.
:::