Getting started for common scenarios - OpenSearch - Alibaba Cloud Documentation Center

Purchase an instance

For more information, see Purchase an OpenSearch Vector Search Edition instance.

Configure an instance

On the details page of the purchased instance, you can view that the instance is in the Pending Configuration state. The system automatically deploys an empty instance based on the number and specifications of Query Result Searcher (QRS) workers and Searcher workers that you purchase. Before you can use the instance for search, perform the following steps: configure a table, add a data source, configure fields, configure an index schema, and then perform reindexing for the instance.

1. Configure the basic information of a table

In the left-side pane on the instance details page, click Table Management. On the Table Management page, click Add Table. In the Basic Table Information step of the Create wizard, configure the Table Name, Data Shards, Number of Resources for Data Updates, and Scenario Template parameters and click Next.

Parameters:

Table Name: the name of the table. You can customize the table name.
Data Shards: the number of data shards in the table. If you create multiple index tables in an OpenSearch instance, make sure that the index tables contain the same number of shards. Alternatively, make sure that at least one index table contains one shard and other index tables contain the same number of shards.
Number of Resources for Data Updates: the number of resources used for data updates. By default, OpenSearch provides a free quota of two resources for data updates for each data source in an OpenSearch Vector Search Edition instance. Each resource consists of 4 CPU cores and 8 GB of memory. You are charged for resources that exceed the free quota. For more information, see Billing overview of OpenSearch Vector Search Edition for the international site (alibabacloud.com).
Scenario Template: the template that is used to create the table. Valid values: Common Template, Vector: Image Search, and Vector: Semantic Search for Text.

2. Add a data source

In the Data Synchronization step, add a data source. You can add a MaxCompute data source or an API data source. In this example, MaxCompute + API is selected as Full Data Source. Configure the Project, AccessKey, AccessKey Secret, Table, and Partition Key parameters, set the Automatic Reindexing parameter to Yes or No, and then click Check. If the data source information passes the check, click Next.

For more information about MaxCompute data sources, see Create a table for a MaxCompute data source.
For more information about API data sources, see Create a table for an API data source.
For more information about Object Storage Service (OSS) data sources, see Create a table for an OSS data source.

3. Configure fields

OpenSearch provides relevant preset fields based on the scenario template that you select and automatically imports all fields from the data source to the field list.

In the Field Configuration step, configure fields. You must configure at least two fields: a primary key field and a vector field. You must define the vector field as a multi-value field of the FLOAT type.

If you need to build a vector index based on categories, you can add a category field between the primary key field and the vector field.

Note:

The primary key field and vector field are required. For the primary key field, you must set the Type parameter to INT or STRING and select the option in the Primary Key column. For the vector field, you must set the Type parameter to FLOAT and select the check box in the Vector Field column.
By default, the vector field is a multi-value field of the FLOAT type, and multiple values of the vector field are separated by HA3 delimiters (^]). This delimiter is encoded as \x1D in the UTF format. You can also enter a custom multi-value delimiter.
When you configure a vector index, you must specify the fields in the order of the primary key field, namespace field, and vector field. The namespace field is optional. The preceding figure shows an example.

4. Configure the index schema

Vector index

OpenSearch automatically creates indexes for the primary key field and vector field. The index names are the same as the field names. You need to only configure the vector index in the OpenSearch console.

You must separately configure parameters for the advanced configurations of the vector index. For more information, see Common configurations of vector indexes.

Note

The primary key field and vector field are required. The namespace field is optional and can be left empty.
You can configure only the three fixed fields for the Fields Contained parameter and cannot add fields.
The system automatically configures the parameters for a vector index. If you have no special requirements, click Next to complete the configuration.
Namespace field: If the engine version of the instance is vector_service_1.0.2 or earlier, the namespace field cannot be of the STRING type. If the engine version of the instance is vector_service_1.0.2 or later, no limit is imposed on the field type.

5. Confirm the creation

In the Confirm step, click Confirm.

6. View the change history

In the left-side pane on the instance details page, click Change History. On the Data Source Changes tab of the page that appears, you can view all finite-state machines (FSMs) related to the processes of creating a table, creating indexes, and performing reindexing for full data. After the search engine is built, you can run query tests in the instance.

7. Run query tests

Sample query

{
  "vector": [0.0019676427,0.005902928,0.021644069,0.21644068,0.12199384,0.043288138,0.007870571,0.0,0.08460863,0.041320495,0.043288138,0.035417568,0.011805856,0.055093993,0.12592913,0.017708784,0.021644069,0.0019676427,0.0,0.0,0.0019676427,0.078705706,0.1987319,0.041320495,0.039352853,0.0039352854,0.007870571,0.0039352854,0.0039352854,0.017708784,0.035417568,0.06886749,0.0019676427,0.0019676427,0.013773498,0.049191065,0.2125054,0.22824654,0.123961486,0.0039352854,0.0,0.0,0.021644069,0.14560555,0.078705706,0.1987319,0.22824654,0.005902928,0.064932205,0.0019676427,0.0019676427,0.021644069,0.027546996,0.035417568,0.22824654,0.22824654,0.1337997,0.023611711,0.009838213,0.007870571,0.0039352854,0.0039352854,0.017708784,0.20069954,0.033449925,0.005902928,0.019676426,0.035417568,0.015741142,0.029514639,0.13183205,0.123961486,0.029514639,0.0,0.027546996,0.22824654,0.15741141,0.0,0.0039352854,0.043288138,0.18889369,0.072802775,0.055093993,0.17315255,0.08460863,0.0019676427,0.007870571,0.035417568,0.22824654,0.10034977,0.009838213,0.021644069,0.062964566,0.027546996,0.015741142,0.04525578,0.086576276,0.033449925,0.023611711,0.017708784,0.0,0.0,0.03738521,0.072802775,0.16724962,0.035417568,0.031482283,0.20463483,0.043288138,0.011805856,0.0039352854,0.051158708,0.023611711,0.11412327,0.13183205,0.16134669,0.049191065,0.023611711,0.0039352854,0.0039352854,0.049191065,0.035417568,0.015741142,0.0039352854,0.03738521,0.08264099,0.094446845,0.021644069],
  "topK": 10,
  "includeVector": true
}

vector: the vector to be queried.
topK: the top K documents to be queried.
includeVector: specifies whether to return vector information in documents.

Sample results

For more information about the query syntax, see the "Syntax" section of this topic.

Syntax

Syntax for vector-based queries: Vector-based query
Syntax for primary key-based queries: Primary key-based query
Syntax for filter expressions: Filter expression

Use an SDK to perform vector-based queries

Use an SDK to perform vector-based queries or primary key-based queries. For more information, see Query data.
Use an SDK to upload or delete documents. For more information, see Update data.