Perform a Q&A test - OpenSearch - Alibaba Cloud Documentation Center

The OpenSearch console provides the Q&A Test page for you to perform Q&A tests. After you enter a question on this page, the large language model (LLM) generates an answer for the question. After you purchase and configure an OpenSearch LLM-Based Conversational Search Edition instance, you can configure different parameters to test Q&A effects for different scenarios and requirements. Then, you can select the optimal parameter configuration based on the effects. This topic describes how to perform a Q&A test in the OpenSearch console. This topic also describes the parameters that can be configured for a Q&A test.

Prerequisites

An OpenSearch LLM-Based Conversational Search Edition instance is created. For more information, see Create an instance.
Data is imported to the OpenSearch LLM-Based Conversational Search Edition instance. For more information, see Import data.

Procedure

Log on to the OpenSearch console and select OpenSearch LLM-Based Conversational Search Edition in the upper-left corner. On the page that appears, click Instance Management in the left-side navigation pane, find the instance that you want to manage, and then click Q&A Test in the Actions column.
On the page that appears, configure Q&A parameters, prompt parameters, document retrieval parameters, reference image parameters, query understanding parameters, manual intervention parameters, and other parameters based on your requirements. Then, enter a question in the dialog box and click Send.
View the Q&A test result.

Parameters

Q&A parameters
Parameter	Type	Required	Default value	Description
options.chat.model	String	Yes	opensearch-qwen	The LLM that is used for the Q&A test. The supported context length and the maximum number of input and output tokens vary based on the LLM.
Prompt	String	No	Default prompt template	The prompt template that is used for the Q&A test. For information about the prompt templates that are supported, see Manage prompts.
question.session	Boolean	No	true	false: disables the multi-round conversation feature. true: enables the multi-round conversation feature. The system returns results based on the recent n rounds of conversations. session: specifies the source of the conversation. The system returns results based on the context of the conversations with the same source.
options.chat.stream	Boolean	No	true	Specifies whether to enable HTTP chunked transfer encoding. true (default) false

Prompt parameters
Parameter	Type	Required	Description
options.chat.prompt_config.attitude	String	No	The tone of the conversation. This parameter is included in the built-in prompt template. Default value: normal. Valid values: normal polite patience
options.chat.prompt_config.rule	String	No	The detail level of the conversation. Default value: detailed. Valid values: detailed stepbystep
options.chat.prompt_config.noanswer	String	No	The information returned if the system fails to find an answer to the question. Default value: sorry. Valid values: sorry: Sorry, I cannot answer your question based on known information. uncertain: I don't know.
options.chat.prompt_config.language	String	No	The language of the answer. Default value: Chinese. Valid values: Chinese English Thai Korean
options.chat.prompt_config.role	Boolean	No	Specifies whether to enable a custom role to answer the question.
options.chat.prompt_config.role_name	String	No	The name of the custom role. Example: AI assistant.
options.chat.prompt_config.out_format	String	No	The format of the answer. Default value: text. Valid values: text table list markdown

Document retrieval parameters
Parameter	Type	Required	Description
options.retrieve.doc.filter	String	No	The filter that is used to filter documents in the knowledge base based on a specific field during document retrieval. By default, this parameter is left empty. For more information, see the "filter" section of the Extended parameters topic. The following fields are supported: table: a table. raw_pk: the primary key of a document. category: the category of a document. score: the score of a document. timestamp: the timestamp of a document. Example: "filter" : "raw_pk=\"123\"" # Obtains data from the documents whose primary key is 123. "filter" : "category=\"value1\"" # Obtains data from the documents whose category is value1. "filter" : "category=\"value1\" OR category=\"value2\"" # Obtains data from the documents whose category is value1 or value2. "filter" : "score>1.0" # Obtains data from the documents whose score is greater than 1.0. "filter" : "timestamp>1356969600" # Obtains data from the documents whose timestamp is greater than 1356969600.
options.retrieve.doc.top_n	Integer	No	The number of documents to be retrieved. Valid values: (0, 50]. Default value: 5.
options.retrieve.doc.sf	Float	No	The threshold of the vector score for document retrieval. If the sparse vector model is disabled, the parameter value ranges from 0 to 2.0 and the default value is 1.3. The smaller the value, the higher the document relevance but the fewer the retrieved documents. Conversely, less relevant documents may be retrieved. If the sparse vector model is enabled, the default value is 0.35. The larger the value, the higher the document relevance but the fewer the retrieved documents. Conversely, less relevant documents may be retrieved.
options.retrieve.doc.dense_weight	Float		The weight of the dense vector during document retrieval if the sparse vector model is enabled. Valid values: (0.0, 1.0). Default value: 0.7.
options.retrieve.doc.formula	String	No	The formula based on which the retrieved documents are sorted. Note For information about the syntax, see Fine sort functions. Algorithm relevance and geographical location relevance are not supported.
options.retrieve.doc.operator	String	No	The operator between terms obtained after text segmentation during document retrieval. This parameter takes effect only if the sparse vector model is disabled. AND (default): The documents that match all the terms are retrieved. OR: The documents that match at least one of the terms are retrieved.

Reference image parameters
Parameter	Type	Required	Default value	Description
options.retrieve.image.sf	Float	No	1	The threshold of the vector score for document retrieval. If the sparse vector model is disabled, the parameter value ranges from 0 to 2.0 and the default value is 1.0. The smaller the value, the higher the document relevance but the fewer the retrieved documents. Conversely, less relevant documents may be retrieved. If the sparse vector model is enabled, the default value is 0.5. The larger the value, the higher the document relevance but the fewer the retrieved documents. Conversely, less relevant documents may be retrieved.
options.retrieve.image.dense_weight	Float	No	0.7	The weight of the dense vector during image retrieval if the sparse vector model is enabled. Valid values: (0.0, 1.0). Default value: 0.7.

Query understanding parameters
Parameter	Type	Required	Valid range	Description
options.retrieve.qp.query_extend	Boolean	No	-	Specifies whether to extend queries. The extended queries are used to retrieve document segments in OpenSearch. Default value: false. false: does not extend queries. true: extends queries. An additional interaction with the LLM is performed. This slows down the system response. Do not extend queries for applications that require fast response.
options.retrieve.qp.query_extend_num	Integer	No	(0, +∞)	The maximum number of queries to be extended if the query extension feature is enabled. Default value: 5.

Manual intervention parameters
Parameter	Type	Required	Description
options.retrieve.entry.sf	Float	No	The threshold of the vector score for manual intervention. Valid values: [0, 2.0]. Default value: 0.3. The smaller the value, the higher the document relevance but the fewer the retrieved documents. Conversely, less relevant documents may be retrieved.

Other parameters
Parameter	Type	Required	Description
options.retrieve.return_hits	Boolean	No	Specifies whether to return document retrieval results. If you set this parameter to true, the search_hits parameter is returned in the response.
options.chat.history_max	Integer	No	The maximum number of rounds of conversations based on which the system returns results. You can specify up to 20 rounds. Default value: 1.
options.chat.link	Boolean	No	Specifies whether to return the URL of the reference source. To be specific, this parameter specifies whether the reference source is included in the content generated by the model. Valid values: true false (default) Sample response if you set this parameter to true: You can resize the disk of an Elastic Compute Service (ECS) instance online or offline[^1^]. If you use the online resizing method, you can resize the disk without the need to restart the instance. If you use the offline resizing method, you must restart the instance[^1^]. To resize a disk, perform the following steps: Log on to the ECS console, find the disk that you want to resize, click Resize in the Actions column, and then select a resizing method based on your business requirements[^1^]. If you need to resize partitions and file systems, you can obtain relevant information by using the CLI or in the console[^2^]. After an ECS disk is resized, you cannot reduce the capacity. We recommend that you implement reasonable capacity planning[^3^]. `[^`Number`^]` indicates the ordinal number of the retrieved document in the reference of the returned results. For example, `[^1^]` indicates the first document in the reference.
options.chat.rich_text_strategy	String	No	The processing method of rich text. If this parameter does not exist or is left empty, rich text is not enabled, and the default processing method is used. inside_response: The rich text tag in the answer is directly restored to the original text in the Markdown format. Note that a table is directly inserted into the Markdown file in the HTML format. extend_response: The actual content of each rich text tag in the answer is returned by rich_text_ref. A picture is returned as a URL, a table is returned in the HTML format, and code is returned in the text format. For more information, see Rich text.

Note

You can perform Q&A tests by calling API operations or using OpenSearch SDKs.