Build Your Own ChatGPT: Deploying OLLAMA and Open-WebUI with Qwen on Alibaba Cloud ECS

Introduction

In this blog post, we will explore how to set up and run Alibaba Cloud Open Source Qwen 0.5 model using Ollama and Open WebUI on an ECS instance. Once completed, we will be able to use the capabilities of the Qwen model through a user-friendly web interface. Here is the video:

Step 1: Choosing the Right Alibaba Cloud ECS Instance

For this simple demo, I am using ecs.g7.large (2 Cores, 8 GB RAM). However, when selecting an ECS instance, consider these key factors for AI workloads:

• CPU/GPU: Look for instances with higher vCPU counts and use GPU for efficient model computations.

• Memory: Models like Qwen can be memory-intensive; aim for more than what I am using here.

• Storage: SSD-based storage improves read/write speeds crucial for model loading.

• Bandwidth: Ensure sufficient bandwidth for smooth interactions via Open-WebUI.

Step 2: Spinning Up the ECS Instance

Follow these steps to create your instance:

Login to Alibaba Cloud Console.
Navigate to the ECS console and click "Create Instance".
Select the chosen configuration, a suitable region, and Ubuntu Server (Ubuntu 22.04 64 bit UEFI Edition) as the OS.
Configure Security Group network settings to assign a public IP and set security group rules to allow traffic on TCP ports 22 (SSH), 11434 (OLLAMA), and 3000 (Open-WebUI).

Step 3: Accessing Your ECS Instance

Use SSH to connect:

ssh root@<your-instance-public-ip>

Step 4: Installing Docker and Docker Compose

Update and install Docker:

apt update
apt install docker.io
apt install docker-compose

Step 5: Deploying OLLAMA & Open-WebUI Containers

Create the docker-compose.yml file using nano or vi editor:

version: '3.8'

services:
  ollama:
    image: ollama/ollama
    container_name: ollama
    volumes:
      - ollama:/root/.ollama
    ports:
      - "11434:11434"
    restart: always

  open-webui:
    image: ghcr.io/open-webui/open-webui:ollama
    container_name: open-webui
    volumes:
      - ollama:/root/.ollama
      - open-webui:/app/backend/data
    ports:
      - "3000:8080"
    restart: always

volumes:
  ollama:
  open-webui:

Save the above content into a file named docker-compose.yml in your desired directory and services using:

docker-compose up -d

To access the Ollama Container and pull the Qwen 0.5 model, first enter the Ollama container by typing the following:

docker exec -it ollama /bin/bash

Once Inside the container, execute the following command to pull the Qwen 0.5 model:

ollama pull qwen:0.5

Once done, make sure that the Qwen model is available, use ollama list.

Step 6: Interacting with Your AI Platform

• ECS Public Access: With the public IP and security group rules in place, anyone can access your Open-WebUI by navigating to http://<your-instance-public-ip>:3000. Use sign up by providing Name, Email and Password as shown below:

• Model Usage: After creating account, interact with the Alibaba Cloud Qwen:0.5b model:

Now we can provide different prompts to the model to get response:

Here is the output of the model:

Conclusion

You have now successfully set up Ollama and Open WebUI on your Alibaba Cloud ECS instance, pulled the Qwen 0.5 model, and accessed a web interface for easy interaction. This setup not only simplifies model deployment but also enhances user experience with an intuitive interface. Enjoy experimenting with the Qwen model and exploring its capabilities!

Lastly, It is pertinent to mention that if you need more accurate response, then better use Alibaba Cloud LLMs like Qwen 4b etc. available at https://ollama.com/library/qwen which may need high specifications of hardware for a better response.

If you have any questions or run into any issues, feel free to leave a comment below.

Community

Build Your Own ChatGPT: Deploying OLLAMA and Open-WebUI with Qwen on Alibaba Cloud ECS

Introduction

Step 1: Choosing the Right Alibaba Cloud ECS Instance

Step 2: Spinning Up the ECS Instance

Step 3: Accessing Your ECS Instance

Step 4: Installing Docker and Docker Compose

Step 5: Deploying OLLAMA & Open-WebUI Containers

Step 6: Interacting with Your AI Platform

Conclusion

Read previous post:

JwdShah

You may also like

Comments

JwdShah

Related Products

Alibaba Cloud for Generative AI

Offline Visual Intelligence Software Packages

AI Acceleration Solution

Tongyi Qianwen (Qwen)