Platform For AI:Quickly deploy Qwen in EAS

Perform model inference on the web UI

1. Find the service that you want to view and click View Web App in the Service Type column.

2. Perform model inference on the web UI.

Perform model inference by using online debugging

1. Click Online Debugging in the Actions column of the service that you want to view. The Online Debugging tab appears.

2. In the Body section, specify the request in the JSON format and click Send Request. The response is returned in the Debugging Information section on the right side.

   Note

   In this example, the debugging information is in the list format. The input field is the input content, and the history field is the history dialogue. The body is a list that contains two sections. The first section is the question, and the second section is the answer to the question.

   1. You can start the inference by entering a request without the history field. Example:

      {"input": "Where is the provincial capital of Zhejiang?"}

      The service returns the result that contains the history field. Example:

      Status Code: 200
      Content-Type: application/json
      Date: Mon, 14 Aug 2023 12:01:45 GMT
      Server: envoy
      Vary: Accept-Encoding
      X-Envoy-Upstream-Service-Time: 511
      Body: {"response":"The provincial capital of Zhejiang is Hangzhou. ","history":[["Where is the provincial capital of Zhejiang?","The provincial capital of Zhejiang is Hangzhou."]]} 

   2. You can include the history field in the following request to perform a continuous conversation. Example:

      {"input": "What about Jiangsu?", "history": [["Where is the provincial capital of Zhejiang?","The provincial capital of Zhejiang is Hangzhou."]]}

      The service returns the result. Example:

      Status Code: 200
      Content-Type: application/json
      Date: Mon, 14 Aug 2023 12:01:23 GMT
      Server: envoy
      Vary: Accept-Encoding
      X-Envoy-Upstream-Service-Time: 522
      Body: {"response":"The provincial capital of Jiangsu is Nanjing.","history":[["Where is the provincial capital of Zhejiang?","The provincial capital of Zhejiang is Hangzhou."],[ "What about Jiangsu?","The provincial capital of Jiangsu is Nanjing."]]} 

Perform model inference by using APIs

You can call the service by calling API operations.

1. On the service details page, click View Endpoint Information. In the Invocation Method dialog box, obtain the values of the Public Endpoint and Token parameters.

   

2. Call the service based on the information that you obtained in the terminal. Example:

   curl -d '{"input": "What about Jiangsu?", "history": [["Where is the provincial capital of Zhejiang?", "The provincial capital of Zhejiang is Hangzhou."]]}' -H "Authorization: xxx" http://xxxx.com

   The service returns the result. Example:

   {"response":"The provincial capital of Jiangsu is Nanjing.","history":[["Where is the provincial capital of Zhejiang?","The provincial capital of Zhejiang is Hangzhou."],["What about Jiangsu?","The provincial capital of Jiangsu is Nanjing."]]}

Send an HTTP request to the service based on your business requirements. For more information about debugging, refer to the SDK that is provided by PAI in the Deploy inference services topic. Sample Python code:

import requests
import json

data = {"input": "Who are you?"}
response = requests.post(url='http://qwen-demo.16623xxxxx.cn-hangzhou.pai-eas.aliyuncs.com/',
              headers={"Authorization": "yourtoken"},
              data=json.dumps(data))

print(response.text)

data = {"input": "What can you do?", "history": json.load (response.text)["history"]}


response = requests.post(url='http://qwen-demo.16623xxxxx.cn-hangzhou.pai-eas.aliyuncs.com/',
              headers={"Authorization": "yourtoken"},
              data=json.dumps(data))

print(response.text)

Perform model inference in streaming mode

1. On the service details page, click View Endpoint Information. In the Invocation Method dialog box, obtain the values of the Public Endpoint and Token parameters.

   

2. In the terminal, run the following Python code to send a streaming request based on the information that you obtained.

   #encoding=utf-8
   from websockets.sync.client import connect
   import os
   import platform
   
   def clear_screen():
       if platform.system() == "Windows":
           os.system("cls")
       else:
           os.system("clear")
   
   
   def print_history(history):
       print("Welcome to the Qwen-7B model. Start the conversation by entering a content. Press clear to clear the conversation history and stop to terminate the program.")
       for pair in history:
           print(f"\nUser: {pair[0]}\nQwen-7B: {pair[1]}")
   
   
   def main():
       history, response = [], ''
       clear_screen()
       print_history(history)
       with connect("<service_url>", additional_headers={"Authorization": "<token>"}) as websocket:
   
           while True:
               query = input("\nUser: ")
               if query.strip() == "stop":
                   break
               websocket.send(query)
               while True:
                   msg = websocket.recv()
                   
                   if msg == '<EOS>':
                       break
                   clear_screen()
                   print_history(history)
                   print(f"\nUser: {query}")
                   print("\nQwen-7B: ", end="")
                   print(msg)
                   response = msg
                   
               history.append((query, response))
   
   
   if __name__ == "__main__":
       main()

   • Replace <service_url> with the endpoint that you obtained in Step 1 and replace http in the endpoint with ws.

   • Replace <token> with the service token that you obtained in Step 1.