Community

Blog Events Webinars Tutorials Forum
  1. Blog
  2. Events
  3. Webinars
  4. Tutorials
  5. Forum
Create Account
×
Community Blog Deploying Pre-trained Models on Alibaba Cloud ECS Using Hugging Face Transformers and Gradio

Deploying Pre-trained Models on Alibaba Cloud ECS Using Hugging Face Transformers and Gradio

This blog demonstrates how to deploy multiple pre-trained models on a single Alibaba cloud ECS instance using Hugging Face Transformers and Gradio

In this blog, we will demonstrate how to deploy multiple pre-trained models on a single Alibaba cloud ECS instance. We will use pre-trained models available on the Hugging face and Gradio to interact with the models. This approach provides a low-code deployment for ML models that can be useful for prototyping and proof of concept. For the purpose of demonstration, we take two examples for deployment: one related to Natural Language Processing (NLP) and other related to Computer Vision. Before we proceed, let us talk about Gradio and Hugging face first.

Gradio is a Python library that can be used to build, customize, and share web-based demos for any machine learning model. This allows us to interact with the model using a web browser.

Hugging Face is an AI research organization that provides access to open-source tools and libraries for Natural Language Processing (NLP) and computer vision tasks. One of its most popular creations is the "Transformers" library, which provides pre-trained models for a wide range of applications, such as text classification, language translation, question-answering, and text generation, image classification etc.

By using Hugging face transformers and Gradio together, it is possible to make AI models more accessible and usable for a broader audience, from developers and researchers to end-users who may not have technical expertise in machine learning.

The pre-trained models used from hugging face are:

  1. For NLP, Text Classification (Sentiment Analysis): DistilBERT Model
  2. For Computer Vision, Image Classification: Vision Transformer (ViT) model

It is pertinent to mention that these two models are taken as example. The readers can try other models from Hugging face in less or more similar manner.

Let us discuss the details of using ECS instance for deployment in a step by step manner.

1.  Spin up Alibaba Cloud ECS instance: The specification of the instance depends on the ML model. For our examples, I am using the following specifications:

1


2
Fig-1: ECS configuration

2.  Preparing Setup (needed one time)

Once the ECS instance starts running, log into the instance. Using command prompt:

  1. Update the packages list by executing: apt update
  2. Create virtual environment (I am using “ai” as the name of environment):
apt install python3-venv -y
python3 -m venv ai
  1. Activate the environment by using: source ai/bin/activate
  2. Install the following necessary python libraries:
pip install transformers
pip install gradio
pip install torch
pip install sentencepiece
pip install sacremoses

3.  Python code for using pre-trained models

a) NLP Model: create a python file “nlp.py” by typing nano nlp.py at the terminal. Once the editor opens, copy and paste the following code:

from transformers import pipeline
import gradio as gr
pipe = pipeline("text-classification", model="distilbert-base-uncased-finetuned-sst-2-english")
demo = gr.Interface.from_pipeline(pipe,title="Text Classification using Alibaba Cloud ECS")
demo.launch(server_name="0.0.0.0", server_port=8000)

Fig-2 shows the code inside the nano editor. Save the file by pressing Ctrl+x, selecting “Y” and pressing enter.

3
Fig-2: Code for Sentiment Analysis

b) Image Classification Model

Create another python file using nano classify.py and modify the code by changing the model type, name, title and port number, as shown in Fig-3. Exit and save the file.

4
Fig-3: Code for Image Classification

4.  Run the models

To run these models, we need to open two terminals on Alibaba cloud ECS instance, as given in Fig-4. Make sure that the python virtual environment is activated. Execute the following at Alibaba Cloud ECS CLI on:

  • Terminal one: python run nlp.py
  • Second Terminal: python run classify.py

5
Fig-4: Running both models on the same ECS instances using two terminals

5.  Interact with the models

To use the deployed models, open a browser on your PC and use the public IP address of your ECS instances with port number, separated by colon(:), as shown below in Fig-5 and Fig-6:

Instead of using ECS public IP address, I am using a domain name. You can optionally do so if, you have a domain name by associating it with the public IP of your ECS instance.

6
Fig-5: NLP: Sentiment Analysis model


Open a second browser, and change the port number (9000) to interact with the computer vision model. Upload an image and see the classification results.

7
Fig-6: Vision Transformer: Image Classification model


Try different images and see the responses of the model.
AI Artificial Intelligence Machine Learning Computer Vision Tutorials Model Training Generative AI Gradio Transformers
1 2 0
Share on

Read next post:

Deploying Alibaba Cloud Large Language Model (Tongy Qianwen) with Graphical and Command Line Interfaces

JwdShah

8 posts | 4 followers

You may also like

Comments

kinder August 29, 2024 at 5:07 pm

Gradio is really powerful. I used it in combination with Automatio (web scraper) to get data from publicly available sources, and then feed it into Gradio. It was super easy since Gradio provides great UI to manage models easily.

JwdShah

8 posts | 4 followers

Related Products

More Posts by JwdShah

See All