Assistant API is a development tool designed to facilitate the rapid creation of agent applications. Compared with the console, Assistant API offers a flexible coding approach for deeper business integration. With built-in session management and a plug-in mechanism, it streamlines the development process. This topic describes what is Assistant API and how it works.
What is assistant?
An assistant, or agent, is an AI-powered application that integrates a large language model (LLMs) with a suite of tools. It can respond to user queries, execute code, generate images, perform web searches, and call business interfaces or local functions.
An assistant supports dialog management and can save or modify conversation history. You can regenerate answers if you do not like them or change the prompt and ask again.
Although the assistants are similar to the agent applications in Model Studio, you cannot manage them in the same way. The assistants are exclusively managed through Assistant API, but the agent applications are managed through the console.
How assistant works
Upon receiving user input, the assistant determines whether to call a tool using its built-in LLM.
If one or more tools are required, the model selects and calls the appropriate tools. The output of the tools is combined with the user input to generate a response by the model again.
If no tool is needed, the response is generated directly from the input.
Supported models
Model | Name |
Qwen-Turbo | qwen-turbo |
Qwen-Plus | qwen-plus |
Qwen-Max | qwen-max |
Qwen2-57B-A14B-Instruct | qwen2-57b-a14b-instruct |
Qwen2-72B-Instruct | qwen2-72b-instruct |
For model compatibility of Assistant API, the actual results prevail. For more information about the models, see List of models.
Supported tools
Tools | Identifier | Purpose |
Python code interpreter | code_interpreter | Runs Python code in tasks such as mathematical calculation, data analysis and visualization, and task processing. |
Image generation | text_to_image | Text-to-image generation. |
Custom plug-ins | ${plugin_id} | Integrates custom interfaces to expand business capabilities. |
Function calling | function | Executes specific functions on on-premises devices, independent of external network services. |
For more information, see Plug-in management.
Core components of Assistant API
The following table describes the core components of Assistant API:
Object | Feature |
Assistant | The AI dialog assistant that integrates the LLM with a variety of tools. |
Thread | Manages historical sessions, maintaining the continuity of interactions between users and the assistant. |
Message | Records the messages from the user and the assistant. |
Run | Processes interaction by inputting context information into the assistant. |
The Assistant API employs a thread mechanism to ensure messages are processed sequentially, preserving the coherence of the dialog. The process involves:
Create a message instance: Users create a message instance using the
Message.create()method, assigning it to a specific thread to maintain context association.Start the runtime environment: Initialize the runtime environment for the assistant with the
Run.create()function.Wait for results: Obtain results with the
wait()function, ensuring synchronization and response accuracy.
Internally, the Assistant API implements a thread-based ordered message queue, automatically sorting messages by their creation time. This guarantees that messages are received and processed in the order they were sent, ensuring response accuracy.
Getting started
For more information about how to use Assistant API, see:
Assistant API references: These topics describe the components of Assistant API to assist you in resolving development challenges.