WhyLabs

whylabs is a platform that helps organizations manage and monitor their machine learning models by automating data pipeline management and providing real time monitoring and insights, whylabs makes it easier for teams to build, test, and deploy models with confidence with whylabs, organizations can streamline their mlops pipeline and ensure the security and compliance of their models follow the steps below to integrate ai shield with whylabs to learn more, see the reference implementation on github for whylabs requirements to integrate ai shield with whylabs, you'll need the following a conda or python environment docker (install from https //docs docker com/desktop/windows/install/ https //docs docker com/desktop/windows/install/ ) steps follow these steps to integrate ai shield with whylabs clone the iris code from github sign up for the whylabs website and note the api key edit the env file with the respective api key and organization id the api key is valid for one year create a conda environment using conda env create f environment yml this will create the environment and install all the packages listed in the environment yml file however, you can also create the environment manually using conda create n whylabs python=3 7 13 numpy==1 21 5 and install the necessary packages based on requirements build the dockerfile without vpn and proxy off using docker build f dockerfile build arg python version=3 7 t whylabs flask this will create an image/container named whylabs flask, which has all the packages present in the requirement txt file you need to do this build every time you want to create the container run the docker container using docker run rm p 5000 5000 whylabs flask access the whylogs by hitting the api endpoint you can do this through swagger ui or a python post request you can see the logs in " https //hub whylabsapp com/models https //hub whylabsapp com/models " first, create a model id on the whylabs website, and then enter that id in the env file make changes to views py by adding the product defense prenet and logging changes to whylabs (about mnist) baseline for mnist 0 to 9 original model output if the prediction is non integer, generate an alert in whylabs