OpenSearch:Rough sort functions

Rough sort is the process of selecting the top N high-quality documents from all documents that are retrieved. Then, the top N high-quality documents are scored and sorted in the fine sort process. This way, users can obtain the documents that best match their requirements. Rough sort affects the search performance, whereas fine sort affects the ultimate sort results. Therefore, you can use key factors of fine sort to roughly sort documents in an efficient and simple manner. You can use sort expressions to roughly and finely sort documents. This topic describes the feature functions used for rough sort.

1. Syntax: static_bm25()

2. Parameters: none

3. Return value: The return value is of the FLOAT type. The valid values range from 0 to 1.

4. Scenario 1: You can use the static_bm25() function in a rough sort expression to calculate text scores.

5. Usage notes:

   • By default, static_bm25() takes effect if you use the default rough sort expression.

1. Syntax: exact_match_boost()

2. Parameters: none

3. Return value: The return value is of the INT type. The valid values range from 0 to 99.

4. Scenario: The query clause is query=default:'开放搜索'^60 OR default:'opensearch'^50. You want to sort results based on the boost weight that is specified in the query clause for a matching term. For example, Document A contains the term 开放搜索, and Document B contains the term opensearch. In this case, Document A ranks higher than Document B. The rough sort expression is exact_match_boost().

5. Usage notes:

   • The field that you reference in the parameter of this function must be configured as an index field.

   • The default boost weight of the term for which no boost weight is specified in the query clause is 99.

   • If the exact_match_boost() function is used in the rough sort expression for an exclusive application, you can specify the sum or max field.

1. Syntax: timeliness(pubtime), timeliness

2. pubtime: the field whose timeliness is to be evaluated. The field value must be of the INT type in the unit of seconds.

3. Return value: The return value is of the FLOAT type. The valid values range from 0 to 1. A greater value indicates better timeliness. If the value of the field is later than the current time, 0 is returned.

4. Scenario: You can use the timeliness(create_timestamp) function in a rough sort expression to calculate the timeliness score of the create_timestamp field.

5. Usage notes:

   • The pubtime field must be configured as an attribute field.

1. Syntax: timeliness_ms(pubtime), timeliness_ms

2. pubtime: the field whose timeliness is to be evaluated. The field value must be of the INT type in the unit of milliseconds.

3. Return value: The return value is of the FLOAT type. The valid values range from 0 to 1. A greater value indicates better timeliness. If the value of the field is later than the current time, 0 is returned.

4. Scenario: You can use the timeliness_ms(create_timestamp) function in a rough sort expression to calculate the timeliness score of the create_timestamp field.

5. Usage notes:

   • The pubtime field must be configured as an attribute field.

1. Scenario overview: The relevance of a document is calculated from different dimensions. The scores that are calculated from different dimensions may be in different value ranges. For example, a web page can have millions of clicks while the text relevance score of the web page is a value from 0 to 1. You cannot compare such values in different value ranges. The normalize function can normalize the scores in different value ranges to scores in the same value range. This way, you can use the normalized scores for further calculation. The normalize function supports three normalization methods: linear normalization, log normalization, and arctangent normalization. The function automatically selects a normalization method based on the input parameters. If only the value parameter is set, the normalize function uses the arctangent function for normalization. If both the value and max parameters are set, the normalize function uses the logarithmic function for normalization. If all the value, max, and min parameters are set, the normalize function uses the linear function for normalization.

2. Syntax: normalize(value, max, min), normalize

3. value: the field in a document or the expression for which you want to normalize the field value or the return value. The field value or return value must be of the DOUBLE type. max: the maximum value of the value range after normalization. This parameter is optional. The maximum value must be of the DOUBLE type. min: the minimum value of the value range after normalization. This parameter is optional. The minimum value must be of the DOUBLE type.

4. Return value: The return value is of the DOUBLE type. The valid values range from 0 to 1.

5. Scenario 1: You want to normalize the value of the price field but do not know the value range of the price field. In this case, you can use the normalize function in the following format: normalize(price).

   Scenario 2: You want to normalize the value of the price field and know only the maximum value 100 in the value range of the price field. In this case, you can use the normalize function in the following format: normalize(price, 100)

   Scenario 3: You want to normalize the value of the price field and set the maximum value to 100 and minimum value to 1. In this case, you can use the normalize function in the following format: normalize(price, 100, 1)

   Scenario 4: You want to normalize the return value of the distance function to a value from 0 to 1. In this case, you can use the normalize function in the following format: normalize(distance(longitude_in_doc, latitude_in_doc, longtitude_in_query, latitude_in_query))

6. Usage notes:

   • The field referenced in the function must be set as an attribute field.

   • If the arctangent function is used for normalization and the field value or the return value of the specified expression is smaller than 0, the return value of the normalize function is 0.

   • If the logarithmic function is used for normalization, the value of the max parameter must be greater than 1.

   • If the linear function is used for normalization, the value of the max parameter must be greater than that of the min parameter.