In this blog, we will demonstrate how to deploy multiple pre-trained models on a single Alibaba cloud ECS instance. We will use pre-trained models available on the Hugging face and Gradio to interact with the models. This approach provides a low-code deployment for ML models that can be useful for prototyping and proof of concept. For the purpose of demonstration, we take two examples for deployment: one related to Natural Language Processing (NLP) and other related to Computer Vision. Before we proceed, let us talk about Gradio and Hugging face first.
Gradio is a Python library that can be used to build, customize, and share web-based demos for any machine learning model. This allows us to interact with the model using a web browser.
Hugging Face is an AI research organization that provides access to open-source tools and libraries for Natural Language Processing (NLP) and computer vision tasks. One of its most popular creations is the "Transformers" library, which provides pre-trained models for a wide range of applications, such as text classification, language translation, question-answering, and text generation, image classification etc.
By using Hugging face transformers and Gradio together, it is possible to make AI models more accessible and usable for a broader audience, from developers and researchers to end-users who may not have technical expertise in machine learning.
The pre-trained models used from hugging face are:
It is pertinent to mention that these two models are taken as example. The readers can try other models from Hugging face in less or more similar manner.
Let us discuss the details of using ECS instance for deployment in a step by step manner.
1. Spin up Alibaba Cloud ECS instance: The specification of the instance depends on the ML model. For our examples, I am using the following specifications:
Fig-1: ECS configuration
2. Preparing Setup (needed one time)
Once the ECS instance starts running, log into the instance. Using command prompt:
apt install python3-venv -y
python3 -m venv ai
source ai/bin/activate
pip install transformers
pip install gradio
pip install torch
pip install sentencepiece
pip install sacremoses
3. Python code for using pre-trained models
a) NLP Model: create a python file “nlp.py” by typing nano nlp.py at the terminal. Once the editor opens, copy and paste the following code:
from transformers import pipeline
import gradio as gr
pipe = pipeline("text-classification", model="distilbert-base-uncased-finetuned-sst-2-english")
demo = gr.Interface.from_pipeline(pipe,title="Text Classification using Alibaba Cloud ECS")
demo.launch(server_name="0.0.0.0", server_port=8000)
Fig-2 shows the code inside the nano editor. Save the file by pressing Ctrl+x, selecting “Y” and pressing enter.
Fig-2: Code for Sentiment Analysis
b) Image Classification Model
Create another python file using nano classify.py
and modify the code by changing the model type, name, title and port number, as shown in Fig-3. Exit and save the file.
Fig-3: Code for Image Classification
4. Run the models
To run these models, we need to open two terminals on Alibaba cloud ECS instance, as given in Fig-4. Make sure that the python virtual environment is activated. Execute the following at Alibaba Cloud ECS CLI on:
Fig-4: Running both models on the same ECS instances using two terminals
5. Interact with the models
To use the deployed models, open a browser on your PC and use the public IP address of your ECS instances with port number, separated by colon(:), as shown below in Fig-5 and Fig-6:
Instead of using ECS public IP address, I am using a domain name. You can optionally do so if, you have a domain name by associating it with the public IP of your ECS instance.
Fig-5: NLP: Sentiment Analysis model
Fig-6: Vision Transformer: Image Classification model
Alibaba Cloud Community - September 6, 2024
Regional Content Hub - July 4, 2024
Regional Content Hub - July 1, 2024
Farruh - March 20, 2024
Regional Content Hub - June 27, 2024
Regional Content Hub - July 1, 2024
A platform that provides enterprise-level data modeling services based on machine learning algorithms to quickly meet your needs for data-driven operations.
Learn MoreThis technology can be used to predict the spread of COVID-19 and help decision makers evaluate the impact of various prevention and control measures on the development of the epidemic.
Learn MoreOffline SDKs for visual production, such as image segmentation, video segmentation, and character recognition, based on deep learning technologies developed by Alibaba Cloud.
Learn MoreAccelerate AI-driven business and AI model training and inference with Alibaba Cloud GPU technology
Learn MoreMore Posts by JwdShah
kinder August 29, 2024 at 5:07 pm
Gradio is really powerful. I used it in combination with Automatio (web scraper) to get data from publicly available sources, and then feed it into Gradio. It was super easy since Gradio provides great UI to manage models easily.