How to perform a Q&A test - OpenSearch - Alibaba Cloud Documentation Center

The console provides a built-in Q&A Test page for users to perform Q&A tests. When you enter a question, the model matches it with corresponding results to provide an answer. After you purchase and configure an instance, you can configure different parameters to test Q&A effects for different scenarios and requirements. Then, you can select the optimal parameter configuration based on the results. This topic describes how to perform a Q&A test in the console and explains the customizable parameters.

Prerequisites

An OpenSearch-LLM Intelligent Edition instance is created. For more information, see Create an instance.
Data configuration is completed. For more information, see Data configuration.

Procedure

The following procedure uses a video file as an example to demonstrate the complete process from uploading a video to automatic parsing by the knowledge base, and then performing a Q&A test based on the video content to return relevant results.

Log on to the OpenSearch console and select LLM-Based Conversational Search Edition. In the left-side navigation pane, click Instance Management. Find the instance that you want to manage and click Manage in the Actions column. On the Instance Details page, click Configuration Center, and then click Data Configuration. Click File Import, select the file that you want to upload, and then click Upload File to import the file to the knowledge base.
Wait for the file to be uploaded successfully. When the data query status shows as completed, click Q&A Test in the left-side navigation pane to ask questions to the model.
After you enter the Q&A test page, click Model Configuration in the upper-right corner. You can configure Q&A parameters, Prompt parameters, Document retrieval parameters, Reference image parameters, Query understanding parameters, Manual intervention parameters, and Other parameters based on your search requirements. Then, enter your question in the dialog box and click Send..
View the Q&A test results, which will be returned based on the content in your uploaded knowledge base.

Parameters

Q&A parameters
Parameter	Type	Required	Default value	Description
options.chat.model	String	Yes	opensearch-qwen	The LLM that is used for the Q&A test. The supported context length and the maximum number of input and output tokens vary based on the LLM:
Prompt	String	No	Default prompt template	The prompt template that is used for the Q&A test. For information about the prompt templates that are supported, see Manage prompts
question.session	Boolean	No	true	false: disables the multi-round conversation feature. true: enables the multi-round conversation feature. The system returns results based on the recent n rounds of conversations. session: specifies the source of the conversation. The system returns results based on the context of the conversations with the same source.
options.chat.enable_deep_search	Boolean	No	false	Specifies whether to enable deep search. true: enables deep search. Multi-round inference is required to return comprehensive data results. This increases the time and computing resources consumed by a conversation. false: disables deep search.
options.retrieve.web_search.enable	Boolean	No	false	Specifies whether to enable the Internet search feature. true: enables the Internet search feature. In this case, the system returns answers based on data searched over the Internet. This increases the time and computing resources consumed by a conversation. false: disables the Internet search feature.
options.chat.stream	Boolean	No	true	Specifies whether to enable HTTP chunked transfer encoding. true (default) false

Prompt parameters
Parameter	Type	Required	Description
options.chat.prompt_config.attitude	String	No	The tone of the conversation. This parameter is included in the built-in prompt template. Default value: normal. normal (default) polite patience
options.chat.prompt_config.rule	String	No	The detail level of the conversation. Default value: detailed. detailed stepbystep
options.chat.prompt_config.noanswer	String	No	The information returned if the system fails to find an answer to the question. Default value: sorry. sorry: Sorry, I cannot answer your question based on known information. uncertain: I don't know.
options.chat.prompt_config.language	String	No	The language of the answer. Default value: Chinese Chinese English Thai Korean
options.chat.prompt_config.role	Boolean	No	Specifies whether to enable a custom role to answer the question.
options.chat.prompt_config.role_name	String	No	The name of the custom role. Example: AI Assistant.
options.chat.prompt_config.out_format	String	No	The format of the answer. Default value: text. text table list markdown

Document retrieval parameters
Parameter	Type	Required	Description
options.retrieve.doc.filter	String	No	The filter that is used to filter documents in the knowledge base based on a specific field during document retrieval. By default, this parameter is left empty. For examples of how to use filters, see Filter parameters. The following fields are supported: table: a table. raw_pk: the primary key of a document. category: the category of a document. score: the score of a document. timestamp: the timestamp of a document. Example: "filter" : "raw_pk=\"123\"" # Obtains data from the documents whose primary key is 123. "filter" : "category=\"value1\"" # Obtains data from the documents whose category is value1. "filter" : "category=\"value1\" OR category=\"value2\"" # Obtains data from the documents whose category is value1 or value2. "filter" : "score>1.0" # Obtains data from the documents whose score is greater than 1.0. "filter" : "timestamp>1356969600" # Obtains data from the documents whose timestamp is greater than January 1, 2013.
options.retrieve.doc.top_n	Integer	No	The number of documents to be retrieved. Valid values: (0, 50]. Default value: 5.
options.retrieve.doc.sf	Float	No	The threshold of the vector score for document retrieval. If the sparse vector model is disabled, the parameter value ranges from 0 to 2.0 and the default value is 1.3. The smaller the value, the higher the document relevance but the fewer the retrieved documents. Conversely, less relevant documents may be retrieved. If the sparse vector model is enabled, the default value is 0.35. The larger the value, the higher the document relevance but the fewer the retrieved documents. Conversely, less relevant documents may be retrieved.
options.retrieve.doc.dense_weight	Float		The weight of the dense vector during document retrieval if the sparse vector model is enabled. Valid values: (0.0, 1.0). Default value: 0.7.
options.retrieve.doc.formula	String	No	The formula based on which the retrieved documents are sorted. Note For information about the syntax, see Fine sort functions. Algorithm relevance and geographical location relevance are not supported.
options.retrieve.doc.operator	String	No	The operator between terms obtained after text segmentation during document retrieval. This parameter takes effect only if the sparse vector model is disabled. AND: The documents that match all the terms are retrieved. Default value: AND. OR: The documents that match at least one of the terms are retrieved.

Reference image parameters
Parameter	Type	Required	Default value	Description
options.retrieve.image.sf	Float	No	1	The threshold of the vector score for document retrieval. If the sparse vector model is disabled, the parameter value ranges from 0 to 2.0 and the default value is 1.0. The smaller the value, the higher the document relevance but the fewer the retrieved documents. Conversely, less relevant documents may be retrieved. If the sparse vector model is enabled, the default value is 0.5. The larger the value, the higher the document relevance but the fewer the retrieved documents. Conversely, less relevant documents may be retrieved.
options.retrieve.image.dense_weight	Float	No	0.7	The weight of the dense vector during image retrieval if the sparse vector model is enabled. Valid values: (0.0, 1.0). Default value: 0.7.

Query understanding parameters
Parameter	Type	Required	Valid range	Description
options.retrieve.qp.query_extend	Boolean	No	-	Specifies whether to extend queries. The extended queries are used to retrieve document segments in OpenSearch. Default value: false. false (default): does not extend queries. true: extends queries. An additional interaction with the LLM is performed. This slows down the system response. Do not extend queries for applications that require fast response.
options.retrieve.qp.query_extend_num	Integer	No	(0, +∞)	The maximum number of queries to be extended if the query extension feature is enabled. Default value: 5.

Manual intervention parameters
Parameter	Type	Required	Description
options.retrieve.entry.sf	Float	No	The threshold of the vector score for manual intervention. Valid values: [0, 2.0]. Default value: 0.3. The smaller the value, the higher the document relevance but the fewer the retrieved documents. Conversely, less relevant documents may be retrieved.

Deep search parameters
Parameter	Type	Required	Description
options.chat.agent.think_process	Boolean	No	Specifies whether to display the thinking process.
options.chat.agent.max_think_round	Integer	No	The number of thinking rounds (maximum 20).
options.chat.agent.language	String	No	The language for thinking process and answer. AUTO: choose Chinese or English by query. CN: Chinese. EN: English.

Other parameters
Parameter	Type	Required	Description
options.retrieve.return_hits	Boolean	No	Specifies whether to return document retrieval results. If you set this parameter to true, the search_hits parameter is returned in the response.
options.chat.history_max	Integer	No	The maximum number of rounds of conversations based on which the system returns results. Maximum value: 20. Default value: 1.
options.chat.link	Boolean	No	Specifies whether to return the URL of the reference source. To be specific, this parameter specifies whether the reference source is included in the content generated by the model. Valid values: true false (default) Sample response if you set this parameter to true: You can resize the disk of an Elastic Compute Service (ECS) instance online or offline[^1^]. If you use the online resizing method, you can resize the disk without the need to restart the instance. If you use the offline resizing method, you must restart the instance[^1^]. To resize a disk, perform the following steps: Log on to the ECS console, find the disk that you want to resize, click Resize in the Actions column, and then select a resizing method based on your business requirements[^1^]. If you need to resize partitions and file systems, you can obtain relevant information by using the CLI or in the console[^2^]. After an ECS disk is resized, you cannot reduce the capacity. We recommend that you implement reasonable capacity planning[^3^]. `[^`Number`^]` indicates the ordinal number of the retrieved document in the reference of the returned results. For example, `[^1^]` indicates the first document in the reference.
options.chat.rich_text_strategy	String	No	The processing method of rich text. If this parameter does not exist or is left empty, rich text is not enabled, and the default processing method is used: inside_response: The rich text tag in the answer is directly restored to the original text in the Markdown format. Note that a table is directly inserted into the Markdown file in the HTML format. extend_response: The actual content of each rich text tag in the answer is returned by rich_text_ref. A picture is returned as a URL, a table is returned in the HTML format, and code is returned in the text format. For more information, see Rich text.
options.retrieve.graph	Boolean	No	Specifies whether to perform query association and retrieval based on graph relationships. This parameter takes effect only if GraphRAG is enabled in data configurations.
options.chat.enable_llm_knowledge	Boolean	No	Specifies whether to use an LLM to return an answer if no search results are obtained. true false

Note

You can perform Q&A tests by calling API operations or using OpenSearch SDKs.

Prerequisites

Procedure

Parameters

Q&A parameters

Prompt parameters

Document retrieval parameters

Reference image parameters

Query understanding parameters

Manual intervention parameters

Deep search parameters

Other parameters