OpenSearch:Intervention dictionaries for synonym configuration

Synonyms vary with business scenarios. Specific synonyms may not exist in the built-in dictionary for synonym configuration, or the built-in dictionary may contain invalid synonyms. To resolve the issue, OpenSearch allows you to customize synonyms.

After you create an intervention dictionary for synonym configuration, you can specify the intervention dictionary when you create or modify a query analysis rule. This way, you can intervene in synonym configuration.

OpenSearch provides a built-in dictionary to implement automatic synonym configuration. You can also create custom intervention dictionaries on the Dictionary Management page to intervene in synonym configuration. To intervene in synonym configuration, perform the following steps:

1. Create an intervene dictionary for synonym configuration. To create an intervention dictionary, log on to the OpenSearch console. In the left-side navigation pane, choose Search Algorithm Center > Retrieval Configuration. On the Basic Configuration page, click Dictionary Management in the left-side pane. On the Dictionary Management page, click Create. In the Create Query Analysis Dictionary panel, enter a name for the intervention dictionary, select a dictionary type, and then click Save. After the intervention dictionary is created, it appears in the dictionary list.

2. Add and manage intervention entries in the intervention dictionary. Find the created dictionary in the dictionary list and click Manage Entries in the Actions column to go to the Manage Entries page. On this page, add and manage intervention entries as needed.

You can intervene in synonym configuration in the following ways:

• Add a synonym and enable it for a search query. Then, when you conduct a search by using the search query, OpenSearch returns both documents that contain the search query and those that contain the synonym.

• Add a synonym and block it for a search query. Then, when you conduct a search by using the search query, OpenSearch does not return documents that contain the synonym.

  1. Use the intervention dictionary. After you add intervention entries to the intervention dictionary, you can associate the intervention dictionary with a query analysis rule on an application as needed.

  2. Test and publish the intervention dictionary. After the intervention dictionary is associated with the query analysis rule, we recommend that you perform a search test before you apply the rule to online environments. This ensures expected search performance.

• If one or more (up to five) consecutive semantic terms of the custom search query that you specify are not stop words and match the predefined search query in an intervention entry, the intervention entry takes effect.

• For example, a custom search query is specified as christian dior aj co-branding low-cut sneakers and the following intervention entries are added: christian dior -> dior, aj -> air jordan, and co-branding -> collaboration. In this case, the three intervention entries all take effect. As a result, OpenSearch returns documents that contain one of the eight expanded search queries, such as "dior aj co-branding low-cut sneakers", "dior air jordan co-branding low-cut sneakers", "dior aj collaboration low-cut sneakers", and "dior air jordan collaboration low-cut sneakers".

• Intervention entries for terms whose weight is higher in the custom search query after analysis take precedence. For example, the following intervention entries are added: sphagitis -> pharyngitis, quinsy, excessive internal heat, sore throat, throat inflammation, low fever -> fever, fervescence, high body temperature, and medicine -> traditional chinese medicine, western medicine, chinese patent drug. A custom search query is specified as "which medicine is best for sphagitis low fever". After analysis, the terms "sphagitis" and "medicine" have a weight of 7, whereas the term "low fever" has a weight of 4. As a result, the intervention entries that take effect are "sphagitis -> pharyngitis, quinsy, excessive internal heat, sore throat, throat inflammation" and "medicine -> traditional chinese medicine, western medicine, chinese patent drug".

• If multiple terms in the custom search query after analysis have the same weight, the intervention entry for the term that ranks first in the custom search query takes precedence. For example, the following intervention entries are added: apple mobile phone -> iphone and mobile phone charger -> power bank. A custom search query is specified as apple mobile phone charger. In this case, the intervention entry that takes effect is apple mobile phone -> iphone.

• For intervention entries whose predefined search queries share the same beginning, the intervention entry that has a longer predefined search query takes precedence. The length of a predefined search query indicates the number of semantic terms after analysis. For example, the following intervention entries are added: mobile phone charger -> power bank and mobile phone -> cellphone. A custom search query is specified as which mobile phone charger brand is good. In this case, the intervention entry that takes effect is mobile phone charger -> power bank.

• Human interventions take precedence. If a custom intervention entry conflicts with a built-in entry, only the custom intervention entry takes effect. A conflict indicates the situation in which the predefined search query in an intervention entry includes or is included in the semantic terms that are defined by OpenSearch for the search query. The following examples show the detailed information:

  • Example 1: The predefined search query in an intervention entry includes the semantic terms that are defined by OpenSearch for the search query. The built-in entry involved is "biscuit -> cookie", and a custom search query is specified as baked biscuit. In this case, the logic of the custom search query is expanded to "baked AND (biscuit OR cookie)". If a custom intervention entry baked biscuit -> baked cookie is added, the logic of the custom search query is expanded to "baked biscuit OR baked cookie". The built-in entry "biscuit -> cookie" is ignored.

  • Example 2: The predefined search query in an intervention entry is included in the semantic terms that are defined by OpenSearch for the search query. The built-in entry involved is apple mobile phone -> iphone, and a custom search query is specified as "apple mobile phone". In this case, the logic of the custom search query is expanded to "apple mobile phone OR iphone". If a custom intervention entry mobile phone -> cellphone is added, the logic of the custom search query is expanded to "apple AND (mobile phone OR cellphone)".

Note:

• The number of synonyms allowed is determined by the number of terms in the specified search query after analysis. The current maximum number of synonyms for a single search query is 36.

Scenario: You have created query analysis rules with synonym configuration capabilities for the OpenSearch application of your E-commerce shopping guide service. After you apply these rules to the online application, the returned search results are unsatisfactory. To resolve the issue, intervention in synonym configuration is implemented.

Problem description: After a user enters the search query "apple", the documents about the desired commodities are not retrieved. Such documents exist in the database, but they contain the synonym "apple inc." instead of the search query "apple".

Cause: In the built-in dictionary for synonym configuration, "apple inc." is not specified as a synonym of "apple".

Solution: Create an intervention dictionary for synonym configuration and add an intervention entry "apple -> apple inc." for the search query "apple". Then, associate the intervention dictionary with a query analysis rule on the online application.

Procedure:

1. Log on to the OpenSearch console. In the left-side navigation pane, choose Search Algorithm Center > Retrieval Configuration. On the Basic Configuration page, click Dictionary Management in the left-side pane. On the Dictionary Management page, click Create.

In the Create Query Analysis Dictionary panel, enter a name for the intervention dictionary and set the Dictionary Type parameter to Synonym.

2. Find the created intervention dictionary and click Manage Entries in the Actions column. On the page that appears, click Add Intervention Entry. In the Add Intervention Entries panel, enter apple in the Search Query field and apple inc. in the Add Synonym field, and click Save.

3. On the Query Analysis Rule Configuration page, associate the created intervention dictionary for synonym configuration with a query analysis rule. Do not apply the rule to the online application in this step.

4. Perform a search test. OpenSearch returns both documents that contain the synonym "apple inc." and those that contain the search query "apple". The search performance is as expected.

• You cannot change the name and type of an intervention dictionary after it is created.

• You must specify different search queries for different intervention entries. You can add, remove, and modify the synonyms to be enabled or blocked for added intervention entries.

• You can specify multiple synonyms to be enabled or blocked for a single search query. Separate the synonyms with semicolons (;).

• If an intervention entry remains in the Validating state, you can click Refresh to obtain the latest state.

• An intervention dictionary can be associated with multiple query analysis rules.

• OpenSearch uses the built-in dictionaries together with the intervention entries that you add. If you enable synonym configuration when you create a query analysis rule, the built-in dictionary for synonym configuration is automatically selected.

• If an intervention dictionary is associated with a query analysis rule, you cannot delete the dictionary regardless of whether the rule is applied to an online or offline application. You must first disassociate the dictionary from the rule.

• A maximum of 20 intervention dictionaries for synonym configuration can be created.

• You can specify only one search query when you add an intervention entry for synonym configuration. A maximum number of five synonyms can be enabled or blocked for a single search query.

• A maximum of 1,000 intervention entries can be added to an intervention dictionary for synonym configuration.

• An intervention entry takes effect if its predefined search query matches one of the terms in the custom search query after analysis. For example, the following intervention entry is added: "beijing -> capital of china". In this case, if a custom search query is specified as "beijing", OpenSearch returns both documents that contain "beijing" and those that contain "capital of china". If a custom search query is specified as "welcome to beijing", OpenSearch returns both documents that contain "welcome to beijing" and those that contain "welcome to capital of china".

• OpenSearch normalizes the content of intervention entries. All uppercase letters are converted into lowercase letters, and all full-width characters are converted into half-width characters.

• For more Information about the SDK, see PushInterventionDictionaryEntries.

• If you use the new OpenSearch console, you must enter code in the following JSON format when you add multiple intervention entries for synonym configuration at a time:

[{"cmd":"add","word":"hey","alias":["hei","hei2"],"antiAlias":["hi"]}]

If you use the old OpenSearch console, you must enter code in the following JSON format when you add multiple intervention entries for synonym configuration at a time:

[{"cmd":"add","word":"hey","alias":["hei","hei2"],"anti_alias":["hi"]}]