Seldon Tutorial: Boost Your ML Model’s Maturity with Seldon Core

Where does Seldon Core figure in MLOps?

A Machine Learning model goes through various stages to reach its deployment. Once in production, the model needs further monitoring since its predictions drive decisions and real-time actions. The model’s maturity also needs to be guarded against data drifts and unforeseen issues. There is, therefore, a need for Machine learning deployment tools that ensure the long-term, stable functioning of the published models. Tools such as Seldon Core have provided streamlined approaches to deploy and monitor ML models. Additionally, it offers interfaces to different languages, platforms, and further testing needs.

So what is Seldon Core? It is an open-source tool that converts ML models or language wrappers for Python, R, and Java into production REST/GRPC micro-services. It holds your deployed model on the three major pillars to streamline its management.

‍

The three major functions provided by Seldon Core ensure streamlined serving, monitoring, and explanation of an ML model. Model serving through Seldon Core ensures that various strategies like canaries, multi-armed bandits, and A/B testing orchestrate the published model. It also protects the deployed model against data drift and unforeseen changes in external factors through continuous monitoring and mitigation. The explainability of an ML model is very important for trust in the decision-making process. This is more important in some strictly governed industries where compliance is driven through model governance. Seldon Core works with efficient explainers like Alibi to enable black-box, white-box, local and global explanations. The containerized ecosystem facilitated by Seldon Core can take the maturity of your ML model through a streamlined journey aided by reliable Mlops tools.

 Recommended Reading: Prominence of Seldon in Model Development

Rolling up your sleeves

The detailed information about Seldon Core can be found at Seldon Core documentation (Seldon Core docs), and there are a plethora of community-documented notebooks. This Seldon core tutorial will gently make you confident in its setup and use.

Since Seldon Core works with Kubernetes containers, you need to ensure the following requirements on your system:

• A clone of the Seldon Core Git repository
• Authenticated and running kubectl
• An active Kubernetes cluster. You can set up the cluster by following commands:

     $> kubectl create namespace seldon

     $> kubectl config set-context $(kubectl config current-context) --namespace=seldon

• A Helm client that is a package manager for Kubernetes. Kustomize can also be used as an alternative installer method
• Seldon core Python package to install Seldon Core

The namespace ‘seldon-system’ is preferred to deploy Seldon Core. It can be created by:

$>kubectl create namespace seldon-system

The Seldon Core package can be installed in this newly created namespace. If using Helm, please run the command:

$> helm install seldon-core seldon-core-operator \

--repo https://storage.googleapis.com/seldon-charts \

--set usageMetrics.enabled=true \

--set istio.enabled=true \

--namespace seldon-system

Further installation options and instructions for alternative installers can be found in Seldon docs.

While it is advised to check if all of the required services are running, the rollout should also be finished by executing:

$> kubectl rollout status deploy/seldon-controller-manager -n seldon-system

Into the playground

With the requisites in place, we can now see how to use Seldon Core.

The various functions provided by Seldon Core include but are not limited to

• Deployment of a trained model pickle file as a prepackaged server instance. The model could be Scikit-learn or Tensorflow models, XGBoost binary, or an A/B test, among others
• Deployment of models after being converted to other formats like ONNX that allows interaction with the served model through APIs
• Support specialized frameworks such as OpenVino optimized models or NVIDIA TensorRT
• Wrap and deploy projects written in other languages such as C++
• Build reusable ML pipelines using Seldon and Kubeflow
• Wrap models built on platforms such as H2O, Triton, or Kafka topic, deploy as micro-services in Seldon Core through Minikube
• Deploy models to cloud-hosted infrastructures such as AWS, Azure, Alibaba Cloud, and so on.
• Record model metrics and deploy explainers like Alibi for model observability
• Canary rollouts using Istio or Ambassador

Let us now see how a trained Scikit-learn model can be deployed using Seldon Core.

The pickle file of the model will be published using the SKLearn server package that is shipped with Seldon. List of the steps involved:

• To serialize the model for the SKLearn server package, the model should be saved as model.joblib. Please ensure that the versions of the underlying frameworks of the model and inference server match
• Now specify SKLEARN_SERVER as the artifact in the config file. A sample config file can be found at ../servers/sklearnserver/samples/iris.yaml

‍

• The changes can be deployed to the Kubernetes cluster by running

$> kubectl apply -f ../servers/sklearnserver/samples/iris.yaml

$> kubectl rollout status deploy/$(kubectl get deploy -l seldon-deployment-id=sklearn -o jsonpath='{.items[0].metadata.name}')

• If you wish to change the inference method, please modify the method field in the parameters and update it to other acceptable values like ‘predict’ or ‘decision_function’

‍

• The REST request to this SKLearn server will look like this

X=!curl -s -d '{"data": {"ndarray":[[1.0, 2.0, 5.0, 6.0]]}}' \

    -X POST http://localhost:8003/seldon/seldon/sklearn/api/v1.0/predictions \

    -H "Content-Type: application/json"

d=json.loads(X[0])

print(d)

• In the Python script, you can simply instantiate a Seldon client and send requests

‍

• After you are done, it is suggested to run the delete command to clean up

$> kubectl delete -f ../servers/sklearnserver/samples/iris.yaml

‍

Wrapping up

In this blog, we gently introduced you to the required setup and many functionalities offered by Seldon Core. We hope you take the knowledge shared here to bring further maturity to your organization’s ML model lifecycle. You can know more about how Seldon figures in model serving. Also, there are many sample notebooks to familiarize you with using Seldon Core for specific requirements and infrastructure. Thank you for reading.

‍