Hands-On-Guide To Machine Learning Model Deployment Using Flask

After studying the right way to construct totally different predictive fashions now it’s time to grasp the right way to use them in real-time to make predictions. You can all the time test your mannequin skill to generalize once you deploy it in manufacturing. For instance, in case you have constructed a predictive mannequin that predicts whether or not a buyer will default or not then you’ll notice how good it’s to your mannequin to foretell the identical once you deploy it in real-time and begin predicting for brand new coming knowledge. 

Model Deployment might be outlined because the mannequin that’s stored in a manufacturing setting or a server the place it takes enter from the consumer and offers output in real-time. Suppose it’s a must to construct a mannequin that predicts whether or not to approve a mortgage for a buyer or not. The mannequin is educated on options like wage, dependents, mortgage quantity and several other different options then in real-time the mannequin will be capable of make predictions once you give enter of those fields to the mannequin. You have to present entries of options on which the mannequin is educated then solely it will be capable of make predictions.

The article demonstrates the right way to deploy a mannequin in real-time utilizing Flask and Rest API by way of which we’d be capable of make predictions for the incoming knowledge. We will construct a classification mannequin for classifying wine and can deploy it to make real-time class predictions.

What you’ll study from this text? 

  1. Working of mannequin deployment.
  2. Different modes of mannequin deployment.
  3. Model serialization and pickling 
  4. Real time prediction

1. Working of Model Deployment

After now we have constructed your predictive mannequin we deploy it in manufacturing. We give a report of options on which the mannequin is educated after which it provides prediction as an output. The mannequin is positioned on a server the place it might obtain a number of requests for predictions. After you’ve got deployed the mannequin it’s able to taking inputs and giving output to as many alternative requests made to it for predictions. 

Steps from constructing a mannequin to deploying it:- 


We first perceive the info and do exploratory knowledge evaluation on it. After that, we do function engineering and have choice. Once we’re carried out with that we begin doing mannequin constructing adopted by mannequin tuning. Then we consider the mannequin and test its efficiency. Once the whole lot is completed and the mannequin will get approval for deployment we then deploy it in real-time and computes prediction in real-time.

Machine Learning Model Deployment

2. Different modes of Model Deployment

There are primarily two totally different fashions of mannequin deployment which are Batch Mode and Real-time Mode. Let us perceive what every mode in mannequin deployment means. Consider a case the place a financial institution has deployed a mannequin that offers a prediction of mortgage approval for the purchasers. That mannequin runs a number of instances in a day say the timings to this are fastened. So, all of the incoming knowledge for the client is stored in ready and every time the mannequin runs the predictions are computed. Once it has computed the prediction once more the requests it receives are stored on maintain for subsequent time. This known as Batch Mode the place predictions are made by the mannequin in numerous batches. 

Consider a second case the place a mannequin is constructed to foretell the credit score rating of a buyer. A buyer fills out all of the required fields which are requested as a function to compute prediction and as quickly because the buyer hits the submit button on the identical second he receives his credit score rating. This known as Real-Time Mode the place real-time predictions are made. This mannequin depends on the infrastructure that’s able to dealing with the load and in a position to do processing and prediction in seconds.

3. Model serialization and pickling

To deploy the mannequin in manufacturing we first want to avoid wasting our mannequin. When we construct the mannequin on our native system and make predictions until that point the mannequin provides prediction however as quickly as we shut the python file the whole lot will get destroyed. So, it turns into necessary to avoid wasting the mannequin to keep away from doing all of the steps once more. This known as Pickling or Serialization in python. Model saving is necessary for each the modes of mannequin deployment. To save your mannequin we are going to make use of a pickle library that lets you save and cargo your mannequin. 

Classification mannequin

First, we have to import all of the required libraries. We will instantly use an information set that’s already current within the sklearn library for constructing the mannequin. Use the under code to import the libraries and cargo the info.  

import sklearn
import sklearn.datasets
import sklearn.ensemble
import sklearn.model_selection
import pickle
import os
wine = sklearn.datasets.load_iris()

After loading the dataset now we have break up the info set into coaching and testing units into the ratio of 80:20 respectively. After that, now we have initialized our mannequin that might be a random forest classifier. Use the under code to do the identical.

coaching, testing, training_labels, testing_labels = sklearn.model_selection.train_test_split(iris.knowledge, iris.goal, train_size=0.80)
rfcl = sklearn.ensemble.RandomForestClassifier(n_estimators=500)
rfcl.match(coaching, traning_labels)

After this, we are going to save the mannequin now we have constructed utilizing the pickle library. We first specified the trail the place we need to save our mannequin after which given the mannequin a file title. Then we saved it utilizing the dump command and loaded it. Use the under code to avoid wasting the mannequin.

pickle.dump(rfcl, open(filename, 'wb'))
load_model = pickle.load(open(filename, 'rb'))
consequence = load_model.rating(testing, testing_labels)

4. Real-time Prediction

Let us perceive the right way to make a real-time prediction by exposing the mannequin to an API utilizing the Flask framework. We will run the mannequin on the server as a relaxation API that may seize all of the incoming requests for prediction and can submit the output for each request.

We will now shortly code for the server script. We will first import all of the required libraries. If we don’t have a flask package deal put in we first want to put in it earlier than executing the under code.

Serve-side Script 

import numpy as np
from flask import Flask, request, jsonify
import pickle
import os

We will then outline the trail the place the mannequin is positioned and cargo it. After loading the mannequin we are going to outline the flask utility and URL for accessing the flask utility ends with /api. We have outlined the strategy to be Post. There are two issues we are able to do for communication over the net that will get and posts. We will use submit strategies to obtain the request and submit again.  Use the under code to the identical.

filename = ‘iris_model’
load_model = pickle.load(open(filename, 'rb'))
app = Flask(__name__)

After defining the flask utility we now outline the operate that might be used for getting the info from the consumer after which making predictions. First, we request for the info from the consumer that’s saved within the predic_request that holds totally different options on which the mannequin was educated. Then we make predictions utilizing the mannequin we loaded earlier than and retailer the prediction within the pred variable and finally return the worth of pred variable. 

def predict()
knowledge = request.get_json(pressure=True)
prediction = load_model.predict(predict_request)
pred = prediction[0]
return jsonify(int(pred))

We then outlined the primary operate that initiates the place the app will run by definition of the port. Use the under code to the identical.

if __name__ == '__main__':
    from werkzeug.serving import run_simple
    app.run(port=9000, debug=True)
model deploement

Now allow us to code the client-side python script. We have first imported the library that’s required after which the URL the place the server script is working. After that, now we have created a dictionary that takes the values from the consumer which is nothing however the options and saved in an information variable. After that knowledge is distributed to the server on the desired URL. The server then makes the prediction and sends it again to the shopper as JSON that’s printed on the shopper finish. You can run server and shopper python scripts to compute predictions. 

Client-side Script

import requests
import json

First, we have to run the server script after which shopper script to test the predictions. There might be a request made by the server to which the shopper will reply by sending the info for prediction. That knowledge might be despatched to the server and prediction might be made that may once more be despatched to the client-side.


I conclude the article by stating that now you’ve got a good concept of the mannequin deployment and make predictions in real-time. The three foremost necessary issues to be stored are the pickle file of the mannequin, server finish script, and shopper finish script. Also, you’ll be able to compute predictions on totally different machines through the use of the hyperlink the place the server is working. There are many different platforms the place you’ll be able to deploy your fashions like AWS, Microsoft Azure, and so forth. You may also discover them and estimate your mannequin efficiency. You can try this text “My First Kaggle Problem with CNN Model – To Count Fingers And Distinguish Between Left And Right Hand?” and save this constructed mannequin, deploy it and begin predicting.

Provide your feedback under


If you liked this story, do be a part of our Telegram Community.

Also, you’ll be able to write for us and be one of many 500+ specialists who’ve contributed tales at AIM. Share your nominations here.


Please enter your comment!
Please enter your name here