OpenSearch allows intervention in named entity recognition (NER) results and entity priorities. If NER results are not as expected, intervention can be implemented.
Concepts
Intervention in NER results
To intervene in NER results, you can create an intervention dictionary for NER and associate the intervention dictionary with a query analysis rule. Semantic entities are closely related to an analyzer. Therefore, when you create an intervention dictionary for NER, an analyzer must be used. An intervention dictionary can be associated with a query analysis rule only when the intervention dictionary uses the same analyzer as that used for indexes.
When you create an intervention dictionary for NER, you can configure a tailored model analyzer only for exclusive applications.
You can enable exact match or partial match for search queries specified in an intervention dictionary.
A maximum of 1,000 intervention entries can be added to an intervention dictionary for NER configuration.
The search query to be specified in an intervention entry must be normalized. Otherwise, the intervention entry can be added but cannot be used for intervention. The normalization process involves converting uppercase letters into lowercase letters and full-width characters into half-width characters.
When you associate an intervention dictionary for NER with a query analysis rule, you can specify whether to ignore spaces in search queries.
In some cases, the analysis results of specific search queries may vary. For example, after an analyzer is updated, the analysis results that are obtained after a search query is entered may be different from the analysis results that are obtained after an intervention entry is added. In such cases, the intervention entry is invalid. An error code 6612 is returned for the search query. When this error code is returned, modify this intervention entry in the OpenSearch console.
Intervention in entity priorities
For ease of use, OpenSearch provides default priority configurations for entities. You can modify the default configurations as required. The modifications to priority configurations may affect the rules for rewriting search queries.
When you configure entity priorities, make sure that both high and medium priorities are specified. Otherwise, the NER-based search query rewriting feature does not take effect.
Make sure that only one priority level is specified for each entity.
Overview
OpenSearch provides a built-in dictionary to implement NER. You can perform the following steps to intervene in NER:
Create an intervention dictionary for NER. To create an intervention dictionary, log on to the OpenSearch console. In the left-side navigation pane, choose Search Algorithm Center > Retrieval Configuration. On the Basic Configuration page, click Dictionary Management in the left-side pane. On the Dictionary Management page, click Create. In the Create Query Analysis Dictionary panel, enter a name for the intervention dictionary, select a dictionary type, and then click Save. After the intervention dictionary is created, it appears in the dictionary list.
Add and manage intervention entries in the intervention dictionary. Find the created dictionary in the dictionary list and click Manage Entries in the Actions column to go to the Manage Entries page. On this page, add and manage intervention entries as needed.
Use the intervention dictionary. After you add intervention entries to the intervention dictionary, you can associate the intervention dictionary with a query analysis rule on an application as needed.
Test and publish the intervention dictionary. After the intervention dictionary is associated with the query analysis rule, we recommend that you perform a search test before you apply the rule to online environments. This ensures expected search performance.
Example
Scenario: You have created query analysis rules with NER capabilities for the OpenSearch application of your E-commerce shopping guide service. After you apply these rules to the online application, the returned search results are unsatisfactory. To resolve the issue, intervention in NER is implemented.
Problem description: A user enters the following search query: Chanel Foundation. By default, the following NER results are returned: Chanel is classified as a regular term and Foundation is classified as a type of material. After the search query is rewritten, the following query clause is obtained: title:'Foundation' RANK title:'Chanel'. However, Chanel is not a regular term in business. Cause: Chanel is a special term in business. In this case, intervention in NER is required.
Solution: Create an intervention dictionary for NER and associate the intervention dictionary with a query analysis rule on the online application.
Procedure:
Log on to the OpenSearch console. In the left-side navigation pane, choose Search Algorithm Center > Retrieval Configuration. On the Basic Configuration page, click Dictionary Management in the left-side pane. On the Dictionary Management page, click Create.
In the Create Query Analysis Dictionary panel, enter a name for the intervention dictionary, set the Dictionary Type parameter to Entity Recognition, and then select an analyzer type. You can set the Analyzer Type parameter to Built-in Analyzer or Custom Analyzer based on the type of analyzer used by indexes specified in the query analysis rule that you want to use. Available analyzers are automatically matched and displayed in the Associated Analyzer drop-down list. Only analyzers for text from the E-commerce industry support NER.
If a tailored model analyzer is used for indexes specified in the query analysis rule, set the Analyzer Type parameter to Tailored Model Analyzer, select the name of your application from the Select Instance drop-down list, and then select the tailored model analyzer.
Find the created intervention dictionary and click Manage Entries in the Actions column. On the page that appears, click Add Intervention Entry. In the Add Intervention Entries panel, enter a search query in the Search Query field and modify the NER results.
The following figure shows the error NER results.
The following figure shows the NER results after intervention.
If the intervention dictionary for NER is created based on a tailored model analyzer, you must select the version of the analyzer used for the tailored retrieval model when you add an intervention entry.
The Intervention Type parameter can be set to Full Match or Partial Match. A value of Full Match indicates that intervention is implemented when the specified search query fully matches the intervention query. A value of Partial Match indicates that intervention is implemented when the specified search query contains the intervention query.
On the Query Analysis Rule Configuration page, associate the created intervention dictionary for NER with a query analysis rule. Do not apply the rule to the online application in this step.
You can specify whether to ignore spaces in search queries during intervention in NER. By default, spaces are not ignored. For example, the search query is "Dress Women", and the intervention entry is "DressWomen". If you choose to ignore spaces, intervention is implemented based on the intervention entry. If you choose not to ignore spaces, intervention is not implemented.
Perform a search test. In this example, the documents related to Chanel Foundation are retrieved.
