Description
OpenSearch provides five categories of statistical metrics for drop-down suggestions: core metrics, traffic metrics, click metrics, guided search metrics, and query analysis metrics.
Core metrics
Metric | Definition | Calculation method | Condition | Description |
Drop-down prompt PV | The number of successful drop-down suggestion requests. Failed requests are not counted for this metric. | Counts each drop-down suggestion request for which a success response is received. | Calculates the number of successful drop-down suggestion requests on the current day. | |
Drop-down prompt UV | The number of users who have sent drop-down suggestion requests and received results. Failed requests are not counted for this metric. | Counts each user only once regardless of how many drop-down suggestion requests the user has sent. | The user_id field must be specified in drop-down suggestion requests when users send requests by using OpenSearch SDK or calling the API operation. | Calculates the number of users who have sent drop-down suggestion requests on the current day. |
No recall result rate | The ratio of drop-down suggestion requests with empty results to all drop-down suggestion requests. | Divides the number of drop-down suggestion requests with empty results by Drop-down prompt PV. | Calculates the ratio of drop-down suggestion requests with empty results to all drop-down suggestion requests to assess the effect and content diversity of drop-down suggestion retrieval. | |
Drop-down prompt request error times | The number of times when a drop-down suggestion request is sent but an error response is received due to reasons such as throttling. | Counts each failed request. | Indicates whether you need to adjust drop-down suggestions, such as adding more drop-down suggestions to the existing ones. | |
Drop-down prompt PV-CTR | The click-through rate (CTR) of drop-down suggestions. | Divides Drop-down prompt IPV by Drop-down prompt PV. | The raw_query and from_request_id fields must be specified in search requests when users send requests by using OpenSearch SDK or calling the API operation. | Calculates the CTR of drop-down suggestions to assess the effects of drop-down suggestion retrieval and sorting. |
Drop-down prompt UV-CTR | The CTR of users who have sent drop-down suggestion requests and received results. | Divides Drop-down prompt IPV_UV by Drop-down prompt UV. | The raw_query and from_request_id fields must be specified in search requests when users send requests by using OpenSearch SDK or calling the API operation, and the user_id field must be specified in drop-down suggestion requests. | Calculates the CTR of users to assess the effects of drop-down suggestion retrieval and sorting. |
Drop-down prompt TOP3 PV-CTR | The CTR of the top three drop-down suggestions. | Divides the number of item page views (IPVs) on the top three drop-down suggestions by Drop-down prompt PV. | The raw_query and from_request_id fields must be specified in search requests when users send requests by using OpenSearch SDK or calling the API operation. | Calculates the CTR of the top three drop-down suggestions to assess the effect of drop-down suggestion sorting. |
Guided search PV ratio | The ratio of searches that users perform by clicking drop-down suggestions, which are called drop-down guided searches, to all searches. | Divides Guided search PV by Search PV. | The raw_query and from_request_id fields must be specified in search requests when users send requests by using OpenSearch SDK or calling the API operation. | Calculates the ratio of drop-down guided searches to all searches to assess the effects of drop-down suggestion retrieval and sorting. |
Guided search PV-CTR | The CTR of drop-down guided searches. | Divides the number of IPVs of drop-down guided searches by Guided search PV. | The raw_query and from_request_id fields must be specified in search requests when users send requests by using OpenSearch SDK or calling the API operation. The data collection feature must be enabled and the behavioral data about user clicks must be uploaded. | Assesses the retrieval and sorting effects of drop-down guided searches. |
Traffic metrics
Metric | Definition | Calculation method | Condition | Description |
Drop-down prompt PV | The number of successful drop-down suggestion requests. Failed requests due to reasons such as throttling are not counted for this metric. | Counts each drop-down suggestion request for which a success response is received. | Calculates the number of successful drop-down suggestion requests on the current day. | |
Drop-down prompt UV | The number of users who have sent drop-down suggestion requests and received results. Failed requests due to reasons such as throttling are not counted for this metric. | Counts each user only once regardless of how many drop-down suggestion requests the user has sent. | The user_id field must be specified in drop-down suggestion requests when users send requests by using OpenSearch SDK or calling the API operation. | Calculates the number of users who have sent drop-down suggestion requests on the current day. |
No recall result rate | The ratio of drop-down suggestion requests with empty results to all drop-down suggestion requests. | Divides the number of drop-down suggestion requests with empty results by Drop-down prompt PV. | Calculates the ratio of drop-down suggestion requests with empty results to all drop-down suggestion requests to assess the effect and content diversity of drop-down suggestion retrieval. | |
Less recall result rate | The ratio of drop-down suggestion requests whose results contain less than the expected number of drop-down suggestions to all drop-down suggestion requests. The expected number of drop-down suggestions is specified by the hits field. | Divides the number of drop-down suggestion requests whose results contain less than the expected number of drop-down suggestions by Drop-down prompt PV. | Calculates the ratio of drop-down suggestion requests whose results contain only a few items to all drop-down suggestion requests to assess the effect and content diversity of drop-down suggestion retrieval. | |
Drop-down prompt request error times | The number of times when a drop-down suggestion request is sent but an error response is received due to reasons such as throttling. | Counts each failed request. | Indicates whether you need to adjust drop-down suggestions, such as adding more drop-down suggestions to the existing ones. | |
Drop-down prompt request error rate | The ratio of failed drop-down suggestion requests to all drop-down suggestion requests. The requests failed due to reasons such as throttling. | Divides Drop-down prompt request error times by the sum of Drop-down prompt PV and Drop-down prompt request error times. | Indicates whether you need to adjust drop-down suggestions, such as adding more drop-down suggestions to the existing ones. |
Click metrics
Metric | Definition | Calculation method | Condition | Description |
Drop-down prompt IPV | The number of clicks on drop-down suggestions. | Counts each click on a drop-down suggestion. | The raw_query and from_request_id fields must be specified in search requests when users send requests by using OpenSearch SDK or calling the API operation. | Calculates the number of clicks on drop-down suggestions. |
Drop-down prompt IPV_UV | The number of users who have clicked drop-down suggestions. | Counts each user only once regardless of how many times the user has clicked drop-down suggestions. | The raw_query and from_request_id fields must be specified in search requests when users send requests by using OpenSearch SDK or calling the API operation, and the user_id field must be specified in drop-down suggestion requests. | Calculates the number of users who have clicked drop-down suggestions. |
Drop-down prompt PV-CTR | The CTR of drop-down suggestions. | Divides Drop-down prompt IPV by Drop-down prompt PV. | The raw_query and from_request_id fields must be specified in search requests when users send requests by using OpenSearch SDK or calling the API operation. | Calculates the CTR of drop-down suggestions to assess the effects of drop-down suggestion retrieval and sorting. |
Drop-down prompt UV-CTR | The CTR of users who have sent drop-down suggestion requests and received results. | Divides Drop-down prompt IPV_UV by Drop-down prompt UV. | The raw_query and from_request_id fields must be specified in search requests when users send requests by using OpenSearch SDK or calling the API operation, and the user_id field must be specified in drop-down suggestion requests. | Calculates the CTR of users to assess the effects of drop-down suggestion retrieval and sorting. |
Drop-down prompt TOP1 PV-CTR | The CTR of the top one drop-down suggestion. | Divides the number of clicks on the top one drop-down suggestion by Drop-down prompt PV. | The raw_query and from_request_id fields must be specified in search requests when users send requests by using OpenSearch SDK or calling the API operation. | Calculates the CTR of the top one drop-down suggestion to assess the effect of drop-down suggestion sorting. |
Drop-down prompt TOP3 PV-CTR | The CTR of the top three drop-down suggestions. | Divides the number of IPVs on the top three drop-down suggestions by Drop-down prompt PV. | The raw_query and from_request_id fields must be specified in search requests when users send requests by using OpenSearch SDK or calling the API operation. | Calculates the CTR of the top three drop-down suggestions to assess the effect of drop-down suggestion sorting. |
Drop-down prompt TOP5 PV-CTR | The CTR of the top five drop-down suggestions. | Divides the number of clicks on the top five drop-down suggestions by Drop-down prompt PV. | The raw_query and from_request_id fields must be specified in search requests when users send requests by using OpenSearch SDK or calling the API operation. | Calculates the CTR of the top five drop-down suggestions to assess the effect of drop-down suggestion sorting. |
Guided search metrics
Metric | Definition | Calculation method | Condition | Description |
Guided search PV | The number of drop-down guided searches. Search requests include paging requests. | Counts each search request that is sent to perform this query. | The raw_query and from_request_id fields must be specified in search requests when users send requests by using OpenSearch SDK or calling the API operation. | Calculates the number of drop-down guided searches to assess the effects of drop-down suggestion retrieval and sorting. |
Guided search PV ratio | The ratio of searches that users perform by clicking drop-down suggestions, which are called drop-down guided searches, to all searches. | Divides Guided search PV by Search PV. | The raw_query and from_request_id fields must be specified in search requests when users send requests by using OpenSearch SDK or calling the API operation. | Calculates the ratio of drop-down guided searches to all searches to assess the effects of drop-down suggestion retrieval and sorting. |
Guided search UV | The number of users who have clicked drop-down suggestions for searches. | Counts each user only once regardless of how many times the user has clicked drop-down suggestions for searches. | The user_id, raw_query, and from_request_id fields must be specified in search requests when users send requests by using OpenSearch SDK or calling the API operation. | Calculates the number of users who have clicked drop-down suggestions to assess the effects of drop-down suggestion retrieval and sorting. |
Guided search UV ratio | The ratio of users who have clicked drop-down suggestions to all users who have sent drop-down suggestion requests and received results. | Divides Guided search UV by Search UV. | The user_id, raw_query, and from_request_id fields must be specified in search requests when users send requests by using OpenSearch SDK or calling the API operation. | Calculates the ratio of users who have clicked drop-down suggestions to all users who have sent drop-down suggestion requests and received results to assess the effects of drop-down suggestion retrieval and sorting. |
Guided search no result rate | The ratio of drop-down guided searches with empty results to all drop-down guided searches. | Divides the number of drop-down guided searches with empty results by Guided search PV. | The raw_query and from_request_id fields must be specified in search requests when users send requests by using OpenSearch SDK or calling the API operation. | Calculates the ratio of drop-down guided searches with empty results to all drop-down guided searches to assess the effect and content diversity of drop-down suggestion retrieval. |
Guided search PV-CTR | The CTR of drop-down guided searches. | Divides the number of IPVs of drop-down guided searches by Guided search PV. | The raw_query and from_request_id fields must be specified in search requests when users send requests by using OpenSearch SDK or calling the API operation. The data collection feature must be enabled and the behavioral data about user clicks must be uploaded. | Assesses the retrieval and sorting effects of drop-down guided searches. |
Guided search has clicks on the percentage of PV | The ratio of drop-down guided searches whose results are clicked by users to all drop-down guided searches. | Divides the number of drop-down guided searches whose results are clicked by users by Guided search PV | The raw_query and from_request_id fields must be specified in search requests when users send requests by using OpenSearch SDK or calling the API operation. The data collection feature must be enabled and the behavioral data about user clicks must be uploaded. | Calculates the ratio of drop-down guided searches whose results are clicked by users to all drop-down guided searches to assess the retrieval and sorting effects of both drop-down suggestions and drop-down guided searches. |
Guided search for GMV | The gross merchandise volume (GMV) of purchases that users make based on drop-down suggestions. | Aggregates the transaction amounts guided by drop-down suggestions. | The data collection feature must be enabled and the behavioral data about user purchases must be uploaded. The value of the bhv_type field must be buy in the data that is uploaded. | Assesses the effects of drop-down suggestion retrieval, sorting, and drop-down suggestions on guiding users to make purchases. |
Guide search favorite conversion rate | The ratio of times when users add items from the results of drop-down guided searches to favorites to all drop-down guided searches. | Divides the number of times when users add items from the results of drop-down guided searches to favorites by Guided search PV. | The data collection feature must be enabled and the behavioral data about adding items to favorites must be uploaded. The value of the bhv_type field must be collect in the data that is uploaded. | Assesses the effects of drop-down suggestion retrieval, sorting, and drop-down suggestions on guiding users to add items to favorites. |
Guided search to shopping cart conversion rate | The ratio of times when users add items from the results of drop-down guided searches to the shopping cart to all drop-down guided searches. | Divides the number of times when users add items from the results of drop-down guided searches to the shopping cart by Guided search PV. | The data collection feature must be enabled and the behavioral data about adding items to the shopping cart, bookshelf, or playlist must be uploaded. The value of the bhv_type field must be cart in the data that is uploaded. | Assesses the effects of drop-down suggestion retrieval, sorting, and drop-down suggestions on guiding users to add items to the shopping cart. |
Guided search like conversion rate | The ratio of times when users like items in the results of drop-down guided searches to all drop-down guided searches. | Divides the number of times when users like items in the results of drop-down guided searches by Guided search PV. | The data collection feature must be enabled and the behavioral data about liking items must be uploaded. The value of the bhv_type field must be like in the data that is uploaded. | Assesses the effects of drop-down suggestion retrieval, sorting, and drop-down suggestions on guiding users to like items. |
Guided search review conversion rate | The ratio of times when users post comments on items in the results of drop-down guided searches to all drop-down guided searches. | Divides the number of times when users post comments on items in the results of drop-down guided searches by Guided search PV. | The data collection feature must be enabled and the behavioral data about posting comments must be uploaded. The value of the bhv_type field must be comment in the data that is uploaded. | Assesses the effects of drop-down suggestion retrieval, sorting, and drop-down suggestions on guiding users to make purchases. |
Query analysis metrics
Category | Metric | Definition | Calculation method | Condition | Description |
Queries with no drop-down suggestions | Query | One of the top 100 most frequently performed queries whose drop-down suggestion request receives empty results. | Calculates the frequency and the number of drop-down suggestions of the query, and compares the values against those of other queries. | Shows a query with no drop-down suggestions to assess the effect and content diversity of drop-down suggestion retrieval. | |
Drop-down prompt PV | The number of successful search requests that are sent to perform this query. | Counts each drop-down suggestion request for which a success response is received. | Calculates the number of successful search requests that are sent to perform this query to assess the severity of the issue in which no drop-down suggestions are returned for this query. | ||
Queries with only a few drop-down suggestions | Query | One of the top 100 most frequently performed queries whose number of drop-down suggestions is less than the expected number. The expected number of drop-down suggestions is specified by the hits field. | Calculates the frequency and the number of drop-down suggestions of the query, and compares the values against those of other queries and the values of the hits field. | Shows a query whose number of drop-down suggestions is less than the expected number to assess the effect and content diversity of drop-down suggestion retrieval. | |
Drop-down prompt PV | The number of successful search requests that are sent to perform this query. | Counts each drop-down suggestion request for which a success response is received. | Calculates the number of successful search requests that are sent to perform this query to assess the severity of the issue in which only a few drop-down suggestions are returned for this query. |
Usage notes
When the drop-down suggestion report displays metrics for a day, the report displays the metrics for the day before the current day by default. You can set the date picker control to another day as needed. After you upload behavioral data, the drop-down suggestion report based on the uploaded data is available the next day.
The drop-down suggestion report provides line charts to display the trends of metrics in the last seven days. You can set the date picker control to another time period as needed. You can view the line chart of a metric by clicking the corresponding icon.
To view a line chart, perform the following steps: In the left-side navigation pane, choose Report Statistics > Drop-down prompt report. Select an application from the Application Name drop-down list and a model from the Model name drop-down list. Click a metric category tab and select a metric from the Indicator change trend drop-down list. Then, set the date picker control to a time period as needed.
