You can call the CreateSearchIndex operation to create one or more search indexes for a data table. When you create a search index, you can add the fields that you want to query to the search index and configure advanced settings for the search index. For example, you can configure the routing key and presorting settings.
Prerequisites
An OTSClient instance is initialized. For more information, see Initialize an OTSClient instance.
A data table for which the max Versions parameter is set to 1 is created. In addition, the TimeToLive parameter of the data table must meet one of the following conditions: For more information, see Create a data table.
The TimeToAlive parameter of the data table is set to -1, which specifies that data in the data table never expires.
The TimeToAlive parameter of the data table is set to a value other than -1, and update operations on the data table are prohibited.
Usage notes
The data types of the fields in a search index must match the data types of the fields in the data table for which the search index is created. For more information, see Data type mappings.
To set the TimeToLive parameter of a search index to a value other than -1, make sure that the UpdateRow operation is prohibited on the data table for which the search index is created. The value of the TimeToAlive parameter for the search index must be less than or equal to the value of the TimeToAlive parameter for the data table. For more information, see Specify the TTL of a search index.
Parameters
When you create a search index, you must configure the TableName, IndexName, and IndexSchema parameters. You must also configure the FieldSchemas, IndexSetting, and IndexSort parameters in the IndexSchema parameter. The following table describes the preceding parameters.
Parameter | Description |
TableName | The name of the data table. |
IndexName | The name of the search index. |
FieldSchemas | The list of field schemas. In each field schema, configure the following parameters:
|
IndexSetting | The settings of the search index, including the settings of the RoutingFields parameter. RoutingFields (optional): the custom routing fields. You can specify specific primary key columns as routing fields. Tablestore distributes data that is written to a search index across different partitions based on the routing fields. The data whose routing field values are the same is distributed to the same partition. |
IndexSort | The presorting settings of the search index, including the settings of the Sorters parameter. If you leave the IndexSort parameter empty, field values are sorted by primary key. Note You can skip the presorting settings for search indexes that contain Nested fields. Sorters (required): the presorting method for the search index. Valid values: PrimaryKeySort and FieldSort. For more information, see Sorting and paging.
|
TimeToLive | (Optional) The retention period of the data in the search index. Default value: -1. If the retention period of data exceeds the value of the TimeToLive parameter, the data expires. Tablestore automatically deletes the expired data. The value of this parameter must be greater than or equal to 86400. A value of 86400 specifies one day. You can also set this parameter to -1, which specifies that data never expires. For information about how to use the time to live (TTL) feature for search indexes, see Specify the TTL of a search index. |
Examples
Create a search index by using the default configurations
The following sample code provides an example on how to create a search index by using the default configurations. In this example, the search index consists of the following fields: the col_keyword field of the Keyword type, the col_long field of the Long type, and the col_vector field of the Vector type.
func createSearchIndex(client *tablestore.TableStoreClient) {
request := &tablestore.CreateSearchIndexRequest{}
request.TableName = "<TABLE_NAME>"
request.IndexName = "<SEARCH_INDEX_NAME>"
request.IndexSchema = &tablestore.IndexSchema{
FieldSchemas: []*tablestore.FieldSchema{
{
FieldName: proto.String("col_keyword"),
FieldType: tablestore.FieldType_KEYWORD, // The String type.
Index: proto.Bool(true),
EnableSortAndAgg: proto.Bool(true),
},
{
FieldName: proto.String("col_long"),
FieldType: tablestore.FieldType_LONG, // The Long type.
Index: proto.Bool(true),
EnableSortAndAgg: proto.Bool(true),
},
{
FieldName: proto.String("col_vector"),
FieldType: tablestore.FieldType_VECTOR, // The Vector type.
Index: proto.Bool(true),
VectorOptions: &tablestore.VectorOptions{
VectorDataType: tablestore.VectorDataType_FLOAT_32.Enum(),
Dimension: proto.Int32(4), // Set the number of vector dimensions to 4 and the distance measurement algorithm for vectors to the dot product algorithm.
VectorMetricType: tablestore.VectorMetricType_DOT_PRODUCT.Enum(),
},
},
},
}
_, err := client.CreateSearchIndex(request)
if err != nil {
fmt.Println("Failed to create searchIndex with error:", err)
return
}
}Create a search index with the IndexSort parameter specified
The following sample code provides an example on how to create a search index with the IndexSort parameter specified. In this example, the search index consists of the col1 field of the Keyword type and the col2 field of the Long type.
func createSearchIndex_withIndexSort(client *tablestore.TableStoreClient){
request := &tablestore.CreateSearchIndexRequest{}
request.TableName = "<TABLE_NAME>" // Specify the name of the data table.
request.IndexName = "<SEARCH_INDEX_NAME>" // Specify the name of the search index.
schemas := []*tablestore.FieldSchema{}
field1 := &tablestore.FieldSchema{
FieldName: proto.String("col1"), // Specify the name of the field by calling the proto.String method. This method is used to request a string pointer.
FieldType: tablestore.FieldType_KEYWORD, // Specify the type of the field.
Index: proto.Bool(true), // Enable indexing for the field.
EnableSortAndAgg: proto.Bool(true), // Enable sorting and aggregation.
}
field2 := &tablestore.FieldSchema{
FieldName: proto.String("col2"),
FieldType: tablestore.FieldType_LONG,
Index: proto.Bool(true),
EnableSortAndAgg: proto.Bool(true),
}
schemas = append(schemas, field1, field2)
request.IndexSchema = &tablestore.IndexSchema{
FieldSchemas: schemas, // Specify the fields that are included in the search index.
IndexSort: &search.Sort{ // Specify the index presorting settings. Data is sorted based on the value of the col2 field in ascending order and then sorted based on the value of the col1 field in descending order.
Sorters: []search.Sorter{
&search.FieldSort{
FieldName: "col2",
Order: search.SortOrder_ASC.Enum(),
},
&search.FieldSort{
FieldName: "col1",
Order: search.SortOrder_DESC.Enum(),
},
},
},
}
resp, err := client.CreateSearchIndex(request) // Call a client to create the search index.
if err != nil {
fmt.Println("error :", err)
return
}
fmt.Println("CreateSearchIndex finished, requestId:", resp.ResponseInfo.RequestId)
}Create a search index with the TTL specified
Make sure that update operations on the data table are prohibited.
func createIndexWithTTL(client *tablestore.TableStoreClient) {
request := &tablestore.CreateSearchIndexRequest{}
request.TableName = "<TABLE_NAME>"
request.IndexName = "<SEARCH_INDEX_NAME>"
schemas := []*tablestore.FieldSchema{}
field1 := &tablestore.FieldSchema{
FieldName: proto.String("col1"), // Specify the name of the field by calling the proto.String method. This method is used to request a string pointer.
FieldType: tablestore.FieldType_KEYWORD, // Specify the type of the field.
Index: proto.Bool(true), // Enable indexing for the field.
EnableSortAndAgg: proto.Bool(true), // Enable sorting and aggregation.
}
field2 := &tablestore.FieldSchema{
FieldName: proto.String("col2"),
FieldType: tablestore.FieldType_LONG,
Index: proto.Bool(true),
EnableSortAndAgg: proto.Bool(true),
}
schemas = append(schemas, field1, field2)
request.IndexSchema = &tablestore.IndexSchema{
FieldSchemas: schemas, // Specify the fields that are included in the search index.
}
request.TimeToLive = proto.Int32(3600 * 24 * 7) // Set the TTL of the search index to seven days.
resp, err := client.CreateSearchIndex(request)
if err != nil {
fmt.Println("error :", err)
return
}
fmt.Println("createIndexWithTTL finished, requestId:", resp.ResponseInfo.RequestId)
}Create a search index with the highlight feature enabled
The following sample code provides an example on how to create a search index with the highlight feature enabled. In this example, the search index consists of the following fields: the col_keyword field of the Keyword type, the col_long field of the Long type, the col_text field of the Text type, and the col_nested field of the Nested type. The col_nested field consists of the following subfields: the level1_text subfield of the Text type and the level2_text subfield of the Text type. In this example, the highlight feature is enabled for the col_text field of the Text type, the level1_text subfield of the Text type, and the level2_text subfield of the Text type.
func createSearchIndexwithHighlighting(client *tablestore.TableStoreClient) {
request := &tablestore.CreateSearchIndexRequest{}
request.TableName = "<TABLE_NAME>"
request.IndexName = "<SEARCH_INDEX_NAME>"
request.IndexSchema = &tablestore.IndexSchema{
FieldSchemas: []*tablestore.FieldSchema{
{
FieldName: proto.String("col_keyword"),
FieldType: tablestore.FieldType_KEYWORD, // The String type.
Index: proto.Bool(true),
EnableSortAndAgg: proto.Bool(true),
},
{
FieldName: proto.String("col_long"),
FieldType: tablestore.FieldType_LONG, // The Long type.
Index: proto.Bool(true),
EnableSortAndAgg: proto.Bool(true),
},
{// Enable the highlight feature for non-Nested fields.
FieldName: proto.String("col_text"),
FieldType: tablestore.FieldType_TEXT, // The Text type.
Index: proto.Bool(true),
EnableSortAndAgg: proto.Bool(true),
EnableHighlighting: proto.Bool(true),
},
{// Enable the highlight feature for the subfield of the Nested field.
FieldName: proto.String("col_nested"),
FieldType: tablestore.FieldType_NESTED,
FieldSchemas: []*tablestore.FieldSchema{
{
FieldName: proto.String("level1_text"),
FieldType: tablestore.FieldType_TEXT,
Index: proto.Bool(true),
EnableHighlighting: proto.Bool(true),
},
{
FieldName: proto.String("level1_nested"),
FieldType: tablestore.FieldType_NESTED,
FieldSchemas: []*tablestore.FieldSchema{
{
FieldName: proto.String("level2_text"),
FieldType: tablestore.FieldType_TEXT,
Index: proto.Bool(true),
EnableHighlighting: proto.Bool(true),
},
},
},
},
},
},
}
_, err := client.CreateSearchIndex(request)
if err != nil {
fmt.Println("Failed to create searchIndex with error:", err)
return
}
}FAQ
References
After you create a search index, you can use the query methods provided by the search index to query data from multiple dimensions based on your business requirements. A search index usually provides the following query methods: term query, terms query, match all query, match query, match phrase query, prefix query, range query, wildcard query, geo query, Boolean query, KNN vector query, nested query, and exists query.
If you call the Search operation to query data, you can sort or paginate the rows that meet the query conditions by using the sorting and paging features. For more information, see Sorting and paging.
If you call the Search operation to query data, you can use the collapse (distinct) feature to collapse the result set based on a specific field. This way, data of the specified type appears only once in the query results. For more information, see Collapse (distinct).
You can specify the TTL for a search index to delete historical data in the search index or extend the retention period of data in the search index. For more information, see Specify the TTL of a search index.
If you want to analyze data in a table, you can call the Search operation to use the aggregation feature or the SQL query feature. For example, you can query the maximum and minimum values, the sum of the values, and the number of rows. For more information, see Aggregation and SQL query.
If you want to obtain all rows that meet the query conditions without the need to sort the rows, you can call the ParallelScan and ComputeSplits operations to use the parallel scan feature. For more information, see Perform a parallel scan.
You can dynamically modify the schema of a search index to add, update, or remove index fields in the search index. For more information, see Dynamically modify the schema of a search index.
You can call the ListSearchIndex operation to query all search indexes that are created for a data table. For more information, see List search indexes.
You can call the DescribeSearchIndex operation to query the description of a search index, such as the field information and search index configurations. For more information, see Query the description of a search index.
You can delete a search index that you no longer require. For more information, see Delete a search index.