All Products
Search
Document Center

Tablestore:KNN vector query

Last Updated:Jul 26, 2024

You can use the k-nearest neighbor (KNN) vector query feature to perform an approximate nearest neighbor search based on vectors. This way, you can find data items that have the highest similarity as the vector that you want to query in a large-scale dataset.

The KNN vector query feature is in invitational preview and disabled by default. To use the KNN vector query feature, submit a ticket.

Prerequisites

Usage notes

  • Tablestore SDK for Java V5.17.0 or later supports the KNN vector query feature. Make sure that the correct version of Tablestore SDK for Java is installed.

    Note

    For information about the version history of Tablestore SDK for Java, see Version history of Tablestore SDK for Java.

  • Limits are imposed on the number of vector fields and the number of dimensions for a vector field. For more information, see Search index limits.

  • The search index server has multiple partitions. Each partition of the search index server returns the top k nearest neighbors to the vector that you want to query. The top k nearest neighbors returned by the partitions are aggregated on the client node. If you want to use tokens to query all data by page, the total number of rows in the response is related to the number of partitions of the search index server.

Parameters

Parameter

Required

Description

fieldName

Yes

The name of the vector field.

topK

Yes

The top K query results that have the highest similarity as the vector that you want to query. For information about the maximum value of the topK parameter, see Search index limits.

Important
  • A greater value of K indicates higher recall rate, query latency, and costs.

  • If the value of the topK parameter is less than the value of the limit parameter in SearchQuery, the server automatically uses the value of the limit parameter as the value of the topK parameter.

float32QueryVector

Yes

The vector for which you want to query the similarity.

filter

No

The filter. You can use a combination of query conditions that are not KNN vector query conditions.

Examples

The following sample code provides an example on how to query the top 10 vectors in a table that have the highest similarity as the specified vector. In this example, the top 10 vectors must meet the following query conditions: the value of the Col_Keyword column is hangzhou and the value of the Col_Long column is less than 4.

private static void knnVectorQuery(SyncClient client) {
    SearchQuery searchQuery = new SearchQuery();
    KnnVectorQuery query = new KnnVectorQuery();
    query.setFieldName("Col_Vector");
    query.setTopK(10); // Return the top 10 vectors in the table that have the highest similarity as the specified vector. 
    query.setFloat32QueryVector(new float[]{0.1f, 0.2f, 0.3f, 0.4f});
    // Specify the query conditions for the top 10 vectors: the value of the Col_Keyword column is hangzhou and the value of the Col_Long column is less than 4. 
    query.setFilter(QueryBuilders.bool()
            .must(QueryBuilders.term("Col_Keyword", "hangzhou"))
            .must(QueryBuilders.range("Col_Long").lessThan(4))
    );
    searchQuery.setQuery(query);
    searchQuery.setLimit(10);
    // Sort the query results based on scores. 
    searchQuery.setSort(new Sort(Collections.singletonList(new ScoreSort())));
    SearchRequest searchRequest = new SearchRequest("<TABLE_NAME>", "<SEARCH_INDEX_NAME>", searchQuery);
    SearchRequest.ColumnsToGet columnsToGet = new SearchRequest.ColumnsToGet();
    columnsToGet.setColumns(Arrays.asList("Col_Keyword", "Col_Long"));
    searchRequest.setColumnsToGet(columnsToGet);
    // Call the Search operation. 
    SearchResponse resp = client.search(searchRequest);
    for (SearchHit hit : resp.getSearchHits()) {
        // Display the scores. 
        System.out.println(hit.getScore());
        // Display the data. 
        System.out.println(hit.getRow());
    }
}

References

  • When you use a search index to query data, you can use the following query methods: term query, terms query, match all query, match query, match phrase query, prefix query, range query, wildcard query, geo query, Boolean query, KNN vector query, nested query, and exists query. You can use the query methods provided by the search index to query data from multiple dimensions based on your business requirements.

    You can sort or paginate rows that meet the query conditions by using the sorting and paging features. For more information, see Sorting and paging.

    You can use the collapse (distinct) feature to collapse the result set based on a specific column. This way, data of the specified type appears only once in the query results. For more information, see Collapse (distinct).

  • If you want to analyze data in a data table, you can use the aggregation feature of the Search operation or execute SQL statements. For example, you can obtain the minimum and maximum values, sum, and total number of rows. For more information, see Aggregation and SQL query.

  • If you want to obtain all rows that meet the query conditions without the need to sort the rows, you can call the ParallelScan and ComputeSplits operations to use the parallel scan feature. For more information, see Parallel scan.