Overview
This topic describes how to customize a retrieval-augmented generation (RAG) application that is created in Model Studio with user-defined functions. For more information about the RAG application, see Best practices for building an official documentation assistant based on RAG.
Based on a previous constructed RAG application, you can configure relevant APIs and achieve the following features:
Obtain the API key of your account.
Obtain model training information.
Obtain the bills of your account.
Features applied
The parameter generation capability of Assistant API.
The function calling capability of Assistant API.
The RAG capability of the application.
Prerequisites
Alibaba Cloud Model Studio is activated. For more information, see Activate Alibaba Cloud Model Studio.
An API key is obtained. For more information, see Obtain an API key.
An SDK of the latest version is installed. For more information, see Install the SDK.
An application is created and the application ID is obtained. For more information, see App center and Obtain an app ID and a workspace.
Replace YOUR_DASHCOPE_API_KEY with your API key.
export DASHSCOPE_API_KEY=YOUR_DASHSCOPE_API_KEYdef call_rag_app(prompt='How do I obtain an API key in Model Studio? '):
# prompt=What capabilities are provided by Model Studio for mixed training?
# How do I obtain an API key in Model Studio?
response = Application.call(app_id='your_app_id',
prompt=prompt)
if response.status_code != HTTPStatus.OK:
print('request_id=%s, code=%s, message=%s\n' % (response.request_id, response.status_code, response.message))
answer = response.status_code
else:
print('request_id=%s\n output=%s\n usage=%s\n' % (response.request_id, response.output, response.usage))
answer = response.output["text"]
return answerUser-defined functions
You can prepare private APIs or functions that can be used to process local requests.
This best practice uses mock functions to simulate actual situations.
def get_api_key(user_id):
return "sk_" + user_id+' You have incurred a charge of RMB 8.88'
#
def get_code(user_id):
return 'https://www.alibabacloud.com/help/en/model-studio/developer-reference/sdk-example'
def get_model_cost(user_id):
answer =""
model_name=['qwen-max','qwen-turbo','qwen_plus']
for model in model_name:
answer+='\nmodel_name: '+ model+ '\ncost '+str(len(model))+' \n token usage'+str(999)
return str(answer)
def get_train_model(user_id):
answer =""
model_list=['qwen-max-sft-v1']
for model in model_list:
answer+='\nmodel_name: '+ model+ '\ntrain_acc '+str(86.44)+' \n training time'+'21 hours 40 mintues'
return str(answer)
def get_application(user_id):
# mocked api key
return str([
{'app_id':'123', 'app name': 'Smart assistant'},
{'app_id': '124', 'app name': 'Travel assistant'},
])
Implement function calling by using Assistant API
The following sample code shows how to call user-defined functions based on documentation retrieval:
call_rag_app: calls the created application.
send_message: passes a message to the assistant.
user_config: configures specific user information in external or environment variables.
# coding: utf8
from http import HTTPStatus
from dashscope import Application, Assistants, Messages, Runs, Threads
from bailian_function_utils import *
import json
import os
import dashscope
dashscope.base_http_api_url = 'https://dashscope-intl.aliyuncs.com/api/v1'
# -*- encoding: utf-8 -*-
def create_assistant():
# create assistant with information
assistant = Assistants.create(
model="qwen-max",
name='smart helper',
description='A smart assistant, which can call plug-ins based on user queries. ',
instructions='You are only tasked with processing information and cannot perform secondary interactions with users. First, summarize the user input. Then, enhance the user query by calling plug-ins. '
"Follow these instructions to call the appropriate plug-ins in order:"
'1. If the query involves model training, call the plug-in for obtaining model training information. Return all the results queried about model training without asking the user for specific model details. '
'2. If the query involves an assistant API, call the plug-in for querying best practices. Return a link to the best practice code of the specific assistant API without needing user interaction. '
'3. If the query involves an API key without mentioning the preceding scenarios, return the specific API key. '
'Do not ask the user additional questions. Instead, proactively initiate any potential calls. ',
tools=[
{
'type': 'function',
'function': {
'name': 'Query best practices',
'description': 'This function is used to return a link to the best practice code of the specific assistant API.',
'parameters': {
'type': 'object',
'properties': {
'user_id': {
'type': 'str',
'description': 'User ID'
},
},
'required': ['user_id']},
},
},
{
'type': 'function',
'function': {
'name': 'Obtain model training information',
'description': 'This function is used to obtain the information about the models deployed in Model Studio that are trained by using additional data, including the specific information about the Supervised Fine-Tuning (SFT) model.',
'parameters': {
'type': 'object',
'properties': {
'user_id': {
'type': 'str',
'description': 'User ID'
},
},
'required': ['user_id']
}
}
},
{
'type': 'function',
'function': {
'name': 'Obtain app creation information',
'description': 'This function is used to query app-related information. To be specific, this function returns the number of applications created in the user path and the application IDs. ',
'parameters': {
'type': 'object',
'properties': {
'user_id': {
'type': 'str',
'description': 'User ID'
},
},
'required': ['user_id']
}
}
},
{
'type': 'function',
'function': {
'name': 'Query the model usage',
'description': 'This universal function is used to query the usage of ultra-large language models, including Qwen models. ',
'parameters': {
'type': 'object',
'properties': {
'user_id': {
'type': 'str',
'description': 'User ID'
},
},
'required': ['user_id']
}
}
},
{
'type': 'function',
'function': {
'name': 'Query an API key',
'description': 'This function is used to return the specific API key.',
'parameters': {
'type': 'object',
'properties': {
'user_id': {
'type': 'str',
'description': 'User ID'
},
'required': ['user_id']
}
}
},
},
],
)
return assistant
function_mapper = {
"Query an API key": get_api_key,
"Obtain model training information": get_train_model,
"Obtain application creation information": get_application,
"Query best practices": get_code,
}
def send_message(assistant, message=''):
print(f"Query: {message}")
# create a thread.
thread = Threads.create()
print(thread)
# create a message.
message = Messages.create(thread.id, content=message)
print(message)
run = Runs.create(thread.id, assistant_id=assistant.id)
print(run)
# # get run statue
# run_status = Runs.get(run.id, thread_id=thread.id)
# print(run_status)
# wait for run completed or requires_action
run_status = Runs.wait(run.id, thread_id=thread.id)
print('Before the plug-ins are called:')
print(run_status)
if run_status.status == 'failed':
print('run failed:')
print(run_status.last_error)
# if prompt input tool result, submit tool result.
if run_status.required_action:
f = run_status.required_action.submit_tool_outputs.tool_calls[0].function
func_name = f['name']
param = json.loads(f['arguments'])
print(f)
if func_name in function_mapper:
output = function_mapper[func_name](**param)
else:
output = ""
tool_outputs = [{
'output':
output
}]
run = Runs.submit_tool_outputs(run.id,
thread_id=thread.id,
tool_outputs=tool_outputs)
# should wait for run completed
run_status = Runs.wait(run.id, thread_id=thread.id)
print(run_status)
# verify_status_code(run_status)
run_status = Runs.get(run.id, thread_id=thread.id)
# print(run_status)
# verify_status_code(run_status)
# get the thread messages.
msgs = Messages.list(thread.id)
# print(msgs)
# print(json.dumps(msgs, default=lambda o: o.__dict__, sort_keys=True, indent=4))
print("Results:")
for message in msgs['data'][::-1]:
print("content: ", message['content'][0]['text']['value'])
print("\n")
if __name__ == '__main__':
assistant = create_assistant()
# answer=call_rag_app(prompt='Model training? ') # example 1
answer=call_rag_app(prompt='How do I create an assistant API? ') # example 2
# 'How do I use an assistant API?'#output["text"]#call_rag_app(How do I create an assistant API? )
user_config = os.environ["user_config"]
send_message(assistant=assistant, message=user_config + answer)
Evaluate assistant with RAG
Test case 1
Input: "How do I create an assistant API?"
answer=call_rag_app(prompt='How do I create an assistant API? ')
Response:
{'name': 'Query best practices', 'arguments': '{"user_id": "user_123456"}', 'output': None}
Results:
content: The current user is user_123456. Creating an assistant API involves several key steps. The following brief process is summarized based on the provided documentation:
1. **Prepare the development environment**:
- Make sure that your system supports Python 3.8 or later.
- Run the `pip install dashscope>=1.17.0` command to install the DashScope library.
- Run the `export DASHSCOPE_API_KEY=YOUR_DASHSCOPE_API_KEY` command to specify your API key. Replace YOUR_DASHSCOPE_API_KEY with your actual API key.
2. **Create an assistant**:
- Run the `from http import HTTPStatus import dashscope` command to import the necessary library.
- Use the `dashscope.Assistants.create` method to create an assistant. You need to specify the model name, such as `qwen-max`, assistant name, description, and instructions. You can also specify tools or functions to be used, such as text-to-image conversion plug-in, search engine plug-in, or custom functions. Example:
```python
assistant = dashscope.Assistants.create(
model='qwen-max',
name='My helper',
description='A tool helper.',
instructions='You are a helpful assistant. When asked a question, use tools wherever possible.',
tools=[
{'type': 'type' }, # Image generation plug-in
{'type': 'quark_search' }, # Search engine plug-in
{
'type': 'function',
'function': {
'name': 'big_add',
'description': 'Add two numbers',
'parameters': {
'type': 'object',
'properties': {
'left': {'type': 'integer', 'description': 'The left operand'},
'right': {'type': 'integer', 'description': 'The right operand'}
},
'required': ['left', 'right']
}
}
}
]
)
```
- After the assistant is created, check the value of `assistant.status_code`. If the value is `HTTPStatus.OK`, the creation is successful. Then, obtain the assistant ID.
3. **Create a thread**:
- To interact with the assistant, you need to create a thread. A thread is a container that carries messages. Each message represents a user query or a reply from the assistant.
```python
thread = dashscope.Threads.create(messages=[{"role": "user", "content": "Tell me something about AI."}])
```
- Similarly, you need to check the value of `thread.status_code` to ensure that the thread is created. Then, record the thread ID for future use.
After the preceding steps are complete, you have created an assistant and a thread for the first conversation with the assistant. Then, you can send a message to the thread to obtain a reply from the assistant.
content: For more information about the preceding steps, see [SDK examples](https://www.alibabacloud.com/help/en/model-studio/developer-reference/sdk-example). Refer to the specific guidelines and code samples in the document to ensure smooth creation of an assistant API.
Test case 2
Input: "Model training?"
# answer=call_rag_app(prompt='Model training? ')
Response:
{'name': 'Obtain model training information', 'arguments': '{"user_id": "user_123456"}', 'output': None}
Results:
content: The current user is user_123456. Model training is a key section in machine learning and deep learning. Model training uses specific datasets to tune the internal parameters of models so that the models can make accurate predictions or decisions based on input data. To train a model in Model Studio, perform the following steps:
1. **Prepare a training dataset**:
- Prepare training data that meets the requirements. The data reflects the knowledge or skills that you want the model to learn. The following file formats are supported: JSONL, XLS, and XLSX. The dataset must contain at least 40 data records. You can upload your data on the Training Data page. Download the template from Model Studio and enter your data in the specified format, if required.
2. **Select a training method**:
- Model Studio provides various training methods, including full-parameter training, efficient training, and Reinforcement Learning from Human Feedback (RLHF) training. Full-parameter training updates all model parameters. This method is suitable for new capability learning and global optimization, but the training takes an extended period of time. Efficient training accelerates training by using matrix decomposition and is suitable for local optimization. RLHF training uses a reward mechanism to optimize models under a reinforcement learning framework, and is suitable for specific performance improvements.
3. **Configure training parameters**:
- You can train new models in the Model Tuning module. After you select a foundation model, you can configure the training dataset, validation data splitting method, mixed training ratio, and hyperparameters. The mixed training ratio is optional. Mixed training allows you to mix your own training data with the data of the Qianwen foundation model based on the specified ratio to enhance model generalizability.
4. **Start training**:
- Submit a training task after you preview and confirm the training configuration. The training task is queued up for execution. During this period, you can view the training status and related metrics of the training task in the list of managed models. You are charged for the training process based on the number of tokens in the training dataset and the number of training epochs. The unit price is CNY 0.1 per 1,000 tokens.
5. **Monitor and evaluate the training**:
- During the training process, you can monitor the training progress and performance metrics in Model Studio. After the training is complete, the system prompts you to test and evaluate the model before you determine whether to deploy the model.
6. **Be aware of system upgrades**:
- A system upgrade is planned for Model Studio during March 15, 2024 to March 16, 2024. Training tasks may be affected during this period. We recommend that you do not arrange training tasks during this period.
Make sure that you understand and agree to relevant billing policies before you start training. You can select appropriate training policies and parameters based on the model tuning requirements to achieve the optimal training outcomes.
content: user_123456, you have trained a model in Model Studio, with the following details:
- **Model name**: qwen-max-sft-v1
- **Training accuracy**: 86.44%
- **Training duration**: 21 hours and 40 minutes
The model has been trained and is ready for deployment or further evaluation. Plan subsequent steps based on your business requirements. Note: A system upgrade is planned for Model Studio during March 15, 2024 to March 16, 2024. Do not arrange new training tasks during this period.
Test case 3
Input: "The bill of my API Key in Model Studio?"
answer = call_rag_app(prompt='The bill of my API Key in Model Studio?')
Response:
{'name': 'apikey query', 'arguments': '{"user_id": "user_123456"}', 'output': None}
Results:
content: The current user is user_123456. In Model Studio, the use of API Key is directly associated with your bill, which is calculated based on the model and uit price <ref>[4][5]</ref>. When you use API or SDK to call services in Model Studio, make sure you know about the authorization scope of your API Key, which decides what resources you can access and whether your calls incur fees <ref>[1]</ref>. The documentations do not mention about your bill, but they mention that the models are billed based on different unit prices and have throttling rules <ref>[4][5]</ref>. If you want to view or manage the billing of your API Key, you need to visit the billing management page of Alibaba Cloud or Model Studio <ref>[4][5]</ref>.
content: Your API Key is `sk_user_123456`. Till now, you have incurred a charge of RMB 8.88. For more information about the billing details, visit the billing management page of Alibaba Cloud or Model Studio.