Navigating through the tech landscape: A step by step guide to publish your ML app on Gradio, Streamlit and Flask

Disclaimer: Parts of this blog is written by ChatGPT. ChatGPT can make mistakes, consider checking important information

Introduction

In the dynamic realm of machine learning, transforming a groundbreaking model from lines of code into a user-friendly application can be a daunting task. However, the ability to showcase your models in a practical and accessible way is crucial for sharing your work with the world. In this comprehensive guide, I'll walk you through the step-by-step process of publishing your machine learning application using three popular frameworks: Gradio, Streamlit, and Flask.

Whether you're a seasoned data scientist looking to deploy your latest model or a beginner eager to share your ML project with a wider audience, this guide aims to demystify the deployment process. Let's explore the unique features of each framework, helping you make informed decisions based on your project requirements and personal preferences.

From creating interactive user interfaces to handling model inference seamlessly, Gradio, Streamlit, and Flask offer distinct approaches to deploying machine learning applications. Throughout this journey, we'll cover essential concepts, highlight key differences between the frameworks, and provide practical tips to ensure a successful deployment.

Embark on this exploration with us as we delve into the intricacies of turning your machine learning models into real-world applications. By the end of this guide, you'll have the knowledge and confidence to navigate the deployment landscape, choosing the framework that best suits your project and enabling you to share your AI creations with the wider community. Let's get started!

UPDATES:

1. I received a lot of questions regarding the input shape.
Since I have implemented a CNN model, the input shape to my model is (None, 28, 28, 1), a grayscale image of size 28 pixels across 28 pixels.

For people who are only using an MLP, the input shape would be a flattened array of the MNIST input shape, which would be (None, 784, 1).

Please keep this detail in mind when passing the uploaded image as the input image to the model when running it on any application.

2. Regarding the app.py

I realize I did not explicitly mention what part of the code must be in the app.py file. All of your dependencies, that are the import statements and everything in the main script, must be included in the app.py

3. Regarding the test images

Download the jpeg images of the MNIST dataset from this link. Credit to Alexander Y. for creating this data set. You can follow him on Kaggle here.

4. Regarding problems with running all three applications in the same instance

No, you will not run into any problems provided you follow the instructions in this blog. The ports that streamlit and flask run on are different, 8501 and 5000 to be specific.

Gradio

Gradio is an open-source Python library that simplifies the process of creating user interfaces for machine learning models. It provides a high-level interface for building interactive and customizable interfaces around your machine learning models without requiring extensive knowledge of web development.

Key features of Gradio include:

Simple Integration: Gradio aims to make machine learning model deployment accessible to a broader audience by providing a straightforward interface. Users can quickly wrap their models with Gradio and create interactive interfaces with minimal code.
Wide Range of Input and Output Types: Gradio supports various input types such as text, images, and audio, making it versatile for different types of models. It also allows for diverse output representations.
Live Updates: Gradio provides live updates, allowing users to see changes to the interface in real-time as they modify the code. This facilitates an iterative and interactive development process.
Compatibility with Popular Frameworks: Gradio is designed to work seamlessly with popular machine learning frameworks like TensorFlow, PyTorch, and scikit-learn, making it adaptable to different modeling needs.
Deployment Options: Gradio allows for both local deployment and cloud deployment, providing flexibility in how and where your machine learning application is hosted.

The idea behind Gradio is simple: it takes a function, to which you can supply input and it gives you the output of the function in whatever form you need.

Now let's look at a step-by-step guide on how to create a gradio application

We will be using the hadnwritten digits MNIST classification as a problem to create a model and create an app using gradio.

Please note, that here I am assuming that a model is already created and I am going to start the tutorial from saving the model to your local machine. All the steps mentioned here can be run on Google Colab. Let's begin!

Step 1: Save the model to your local machine

Once you have created the model, let's say the model object is called model, save your model with the line

model.save('MNIST_99.h5')

Feel free to replace 'MNIST_99.h5' with the name of the model of your choice.

After saving the model, you can find it in the tab to your left, download the model and save it to in your local machine, because files on Colab are temproary, and only exist until the instance is running.

Step 2: Open a new notebook and upload your model

Open a new instance of Colab, and upload your model there. Depending on the size of your model and your internet upload speed it should take about 5 minutes.

Step 3: Install Gradio on your instance and run the following code

Install the gradio package using pip:

!pip install gradio

The following lines of code let's you create a simple gradio application that takes a 28 x 28 image input, which is the size of the images in the handwritten digits data set, and gives a simple text output on what the model ascertains the output of the image to be.

First, import all the necessary libraries

import tensorflow as tf

import cv2

import numpy as np

import gradio as gr

Now create the function:

#importing model

def model(up_img):

cnn = tf.keras.models.load_model("/content/MNIST_99.h5")

#import image

gray_image = cv2.cvtColor(up_img, cv2.COLOR_BGR2GRAY)

#preprocess

gray_image = np.expand_dims(gray_image, axis = -1)

gray_image = gray_image.reshape((-1, 28, 28, 1))

#predict

prediction = cnn.predict(gray_image)

#converting the float values to int values for y_test

prediction = np.argmax(prediction, axis = 1)

return "The prediction for your input is {}".format(int(prediction))

A few notes here:

Replace 'MNIST_99.h5' with the name of your model
The model that I created takes a 28 x 28 image as input and then flattens the image and passes it through an MLP.
Depending on how your model takes input, either as a 28 x 28 array or of size 784 x 1, change the shape from gray_image.reshape((-1, 28, 28, 1)) to gray_image.reshape((-1, 28 * 28, 1)) accordingly.
I have given a simple statement as the output for the application, feel free to change it to whatever you like.

Lastly, launch the Gradio Interface

demo = gr.Interface(fn = model, inputs = 'image', outputs = 'text')

demo.launch(debug = True)

Step 4: Rejoice!

This should launch your Gradio application within the Colab interface and also provide a link, that you can share.

The issue with the link is that the console must be active and running for the link to work, and also it expires in 72 hours. Fortunately, gradio offers an option to deploy your application to Huggingface Spaces.

And that's it, you have built an application that is able to classify the handwritten digits MNIST data set using gradio.

You can develop the code further to incorporate other elements such as examples, caching and using blocks to create a more detailed interface using the gradio documentation.

Streamlit

In the ever-evolving landscape of data science and visualization, Streamlit has emerged as a powerful ally for Python developers and enthusiasts seeking simplicity without compromising sophistication. This lightweight yet robust framework offers a streamlined path to transform data scripts into interactive web applications with unparalleled ease.

The key features of Streamlit include:

Simplicity and Ease of Use: Streamlit is designed to be beginner-friendly, allowing users to create interactive web applications with minimal code. The framework follows a straightforward syntax, making it accessible to those with basic Python knowledge.
Rapid Prototyping: Streamlit enables rapid prototyping of data applications, allowing users to quickly iterate and visualize their ideas. Users can make instant updates to their apps and see the changes in real-time without needing to restart the application.
Widgets for Interactivity: Streamlit provides a variety of widgets (such as sliders, buttons, and text inputs) that can be easily added to create interactive elements in the web app. These widgets make it simple to interact with and manipulate data in real-time.
Data Visualization: Streamlit seamlessly integrates with popular Python plotting libraries like Matplotlib, Plotly, and Altair, allowing users to create a wide range of data visualizations.
Integration with Machine Learning: Streamlit supports the integration of machine learning models, enabling users to showcase and interact with their models directly in the web application. This makes it an excellent tool for creating dashboards and interfaces for machine learning projects.

Just as we did before, the application we are trying to build will be a handwritten digits MNIST classification application, and I am now assuming that you have the model downloaded and ready to rock and roll! Please refer to Step 1 of the Gradio section to learn how to save your model and download it to your local machine on Colab.

Unlike Gradio, Streamlit does not directly run inline. We will have to expose the development site that Streamlit generates to a public page and access that page for our application development. No worries, I will run through each step in as much detail as possible.

So let's take a look at a step-by-step guide on how to create your first machine learning application using Streamlit.

Let's begin!

Step 1: Upload the model to your Colab instance

Open a new instance of Colab, and upload your model there. Depending on the size of your model and your internet upload speed it should take about 5 minutes.

Step 2: Install the Streamlit package and run the following code

Install the streamlit package using the following command

!pip install streamlit

From here on, there are a few changes that we have to make to run Streamlit (and for Flask as well in the next section) applications.

The first thing is that, Streamlit, unlike Gradio, does not create a public URL that you can share across the internet. It creates a link that can only be used within the same network.

The problem is that since Colab is essentially a virtual machine, the link that Streamlit generates can be opened only within THAT virtual machine's network, and you don't have access to that network.

So the solution is to expose the URL to the public using a local tunnel, making the URL accessible to any system anywhere, akin to the link you receive on Gradio.

We need to install a JavaScript package called localtunnel to achieve this. Along with this, we will have to create an app.py file to pass to streamlit run command which is just a single line of code.

Follow along and run these following lines of code:

Note: Run everything from the following line to the end of the main script in the same cell to create the app.py file

First, import the necessary libraries

# %%writefile app.py

import streamlit as st

import numpy as np

from PIL import Image

import tensorflow as tf

Note: I have commented the writefile command at the beginning. It is recommended the you run your code and debug if any errors, and then create the app.py file

Load the model

# Load your trained model

model = tf.keras.models.load_model("MNIST_99.h5")

Write the main function

def main():

st.title("MNIST Handwritten Digits Streamlit Interface") #title of the Streamlit webpage

# Allow users to upload their own image

uploaded_file = st.file_uploader("Upload Your Own Image", type=["jpg", "jpeg", "png"])

if uploaded_file is not None: #uploaded file exists

# Display the uploaded image

uploaded_image = Image.open(uploaded_file)

st.image(uploaded_image, caption="Uploaded Image", use_column_width=True)

#preprocess and pass the input to your model

#similar to Gradio

raw_image = np.array(uploaded_image)

reshaped_image = np.reshape(raw_image, (-1, 28, 28, 1))

prediction_raw = model.predict(reshaped_image)

predicted_label = np.argmax(prediction_raw)

# Display the predicted label

st.write("Prediction:", predicted_label)

if __name__ == "__main__":

main()

A few notes on code above:

1. Feel free to change all the outputs texts to your liking!

2. You can play around with Streamlit to also take input from a virtual sketchpad

3. Debug the code and create the app.py after that

Step 3: Install the JavaScript packages, expose the link through a tunnel and obtain a URL

Now for the interesting part. We are going to use the node package manager to install the localtunnel package, a tool that we can use to expose development server URLs to the internet.

Run the following lines of code:

!npm install -g localtunnel

!streamlit run /content/app.py &>/content/logs.txt &

!lt --port 8501 & curl ipv4.icanhazip.com

You should see an output similar to this, with a major change being the public URL generated. The generated URL is randomized each time the cell is run and is often a funny combination of three words and does not make sense!

Copy the EndPoint IP displaced in the third line from the bottom as shown above in the picture. This is the public IP of the virtual machine your Colab Notebook is running on. We need this as a sort of a password to access the public URL, which you will see once you click on the link given.

Click on the link to go to the Friendly Reminder page

Paste the link here and click to Submit.

Step 4: Aaaaannnndddd scene!

That's it! You can now view your application built on Streamlit and develop your application further.

Also remember, just like Gradio these links only function when the console is running and active, which is impossible to maintain on Google Colab since it's meant for interactive use only.

Once you have built the application you can deploy it on Spaces or GCP or AWS, as you please.

Now on to the final framework, Flask!

Flask

Flask is a micro web framework written in Python. It is classified as a microframework because it does not require particular tools or libraries. It has no database abstraction layer, form validation, or any other components where pre-existing third-party libraries provide common functions.

Flask is easy to learn and use, and it is a good choice for small to medium-sized web applications. It is also a good choice for developers who want to have more control over the development process.

Here are some of the features of Flask:

It is easy to learn and use.
It is a good choice for small to medium-sized web applications.
It is a good choice for developers who want to have more control over the development process.
It is lightweight and extensible.
It has a large and active community.

Flask is a micro web framework tool, which means we will have to create a proper HTML file to showcase our input and output.

Similar to Streamlit, Flask also needs it's development URL to be exposed to the internet and we will follow the same localtunnel method to achieve that. Although, traditionally, people prefer using ngrok to expose the development sites.

Same deal again, MNIST classification, model is downloaded and ready to be uploaded to Colab.

Let's get started!

Step 1: Create the HTML template and upload it to templates on Colab

Create index file

Enter the following lines of code in a notepad on your local machine and save the file as a 'index.html', changing the file format from text file to All files while doing so.

<!DOCTYPE html>

<head>

<title>MNIST Digit Prediction</title>

</head>

<body>

<h1>MNIST Digit Prediction</h1>

<button type="button" onclick="predict()">Predict</button>

</form>

function predict() {

// Get the selected file

const fileInput = document.getElementById('file-input');

const file = fileInput.files[0];

// Create FormData object

const formData = new FormData();

formData.append('file', file);

// Send the image to the Flask server

fetch('/predict', {

method: 'POST',

body: formData

})

.then(response => response.json())

.then(data => {

// Decode base64 image data

const imageData = 'data:image/png;base64,' + data.image.data;

// Display the image

const resultDiv = document.getElementById('result');

resultDiv.innerHTML = `

<p>Prediction: ${data.prediction}</p>

})

.catch(error => console.error('Error:', error));

}

</script>

</body>

</html>

Remember to change the file format to all files, before saving.

Create a new folder named templates and upload the index.html

Once you have connected to a runtime, create a new folder called 'templates' and upload the index.html file to that folder.

Ignore the rest of the files and folders in the above image for now!

Step 2: Upload the model and run the following lines of code

Upload the model as usual to your environment and install flask and run the following lines of code to create a simple Flask code that takes an image input, run the image through the model you have built and outputs the prediction of your model.

First install the flask library and import all dependencies

!pip install flask

Note: Run everything from the following line to the end of the main script in the same cell to create the app.py file

%%writefile app.py

from flask import Flask, render_template, request, jsonify

import numpy as np

from PIL import Image

import base64

import os

from io import BytesIO

from tensorflow.keras.models import load_model

Commenting the writefile command here so we can debug and then write the file.

Now render the template, get the image file from the user, run it through the model and show the output

app = Flask(__name__)

model = load_model('/content/MNIST_99.h5') # Replace with your model path

@app.route('/')

def home():

return render_template('index.html')

@app.route('/predict', methods=['POST'])

def predict():

# Get the image file from the request

image_file = request.files['file']

# Process the image

img = Image.open(image_file).convert('L') # Convert to grayscale

img = img.resize((28, 28)) # Resize to MNIST input size

img_array = np.array(img).reshape(1, 28, 28, 1) # Normalize

# Make prediction

prediction = model.predict(img_array)

predicted_digit = np.argmax(prediction)

# Convert the PIL Image to base64-encoded bytes

img_bytes = BytesIO()

img.save(img_bytes, format='PNG')

img_base64 = base64.b64encode(img_bytes.getvalue()).decode('utf-8')

# Create a response with the base64-encoded image and prediction

response = {

'prediction': int(predicted_digit),

'image': {

'format': 'png',

'data': img_base64

}

return jsonify(response)

if __name__ == '__main__':

app.run(debug=True)

Once you have run the code above successfully, write the file to app.py

Step 3: Expose the development server URL to the internet using localtunnel

Run the following lines of code to get a shareable link that is connected to the internet. The default port that Flask runs on is 5000, so before running the following code, make sure that the ports are not already engaged.

!npm install -g localtunnel

!flask run &>/content/logs.txt &

!lt --port 5000 & curl ipv4.icanhazip.com

The Flask application is now running in the background using the port 5000 and you have now created a localtunnel to access that port.

If you are in the same instance as the one you ran Streamlit code on, your Public IP will remain the same. Otherwise copy the IP shown at the bottom of the results and enter it in the following page where it asks for Endpoint IP.

Step 4: Test out your application

Once you have run everything above successfully, you should see something like this, which is the application you are trying to build.

IMPORTANT NOTE:

If you are facing issues where the local tunnel page is not showing your page but is stuck on loading, check if the application is actually running.

Follow these steps to do so:

1. Go back to Colab and stop running the cell that contains the !flask run command

2. In a different cell run !lsof -i :5000

3. Ideally you should see something of this sort, that indicates that the port 5000 is active:

COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME flask 3492 root 3u IPv4 94186 0t0 TCP localhost:5000 (LISTEN)

4. In case you don't see any process check your !flask run command and try again

So that's it, you have now learnt how to use different methods to test out your application while it is in the development phase and once you are done building the application, consider deploying it on AWS or GCP.

Search This Blog

Analyze

Navigating through the tech landscape: A step by step guide to publish your ML app on Gradio, Streamlit and Flask

Introduction

I realize I did not explicitly mention what part of the code must be in the app.py file. All of your dependencies, that are the import statements and everything in the main script, must be included in the app.py

Gradio

Step 1: Save the model to your local machine

Step 2: Open a new notebook and upload your model

Step 3: Install Gradio on your instance and run the following code

Step 4: Rejoice!

Streamlit

Step 1: Upload the model to your Colab instance

Step 2: Install the Streamlit package and run the following code

Step 3: Install the JavaScript packages, expose the link through a tunnel and obtain a URL

Step 4: Aaaaannnndddd scene!

Flask

Step 1: Create the HTML template and upload it to templates on Colab

Step 2: Upload the model and run the following lines of code

Step 3: Expose the development server URL to the internet using localtunnel

Step 4: Test out your application

Comments

Post a Comment