Determine whether a column exists or whether the value of a column is empty - Tablestore

Exists query is also called NULL query or NULL-value query. This query is used in sparse data to determine whether a column of a row exists. For example, you can query the rows in which the value of the address column is not empty.

Note

If you want to perform an exists query on a Nested column, you can use nested query.
If you want to check whether a column contains empty values, you must use ExistsQuery together with mustNotQueries of BoolQuery.
If one of the following conditions is met, the system considers that a column does not to exist. In this example, the city column is used.
- The type of the city column in the search index is a basic type such as keyword. If a row in which the city column does not exist in the data table, the search index considers that the city column does not exist.
- The type of the city column in the search index is a basic type such as keyword. If a row in which the value of the city column is an empty array in the data table ("city" = "[]"), the search index considers that the city column does not exist.

API operations

You can call the Search or ParallelScan operation and set the query type to ExistsQuery to perform an exists query.

Parameters

Parameter	Description
fieldName	The name of the column that you want to query.
query	The query type. Set this parameter to ExistsQuery.
getTotalCount	Specifies whether to return the total number of rows that meet the query conditions. The default value of this parameter is false, which specifies that the total number of rows that meet the query conditions is not returned. If you set this parameter to true, the query performance is compromised.
tableName	The name of the data table.
indexName	The name of the search index.
columnsToGet	Specifies whether to return all columns of each row that meets the query conditions. You can configure the returnAll and columns fields for this parameter. The default value of the returnAll field is false, which specifies that not all columns are returned. In this case, you can use the columns field to specify the columns that you want to return. If you do not specify the columns that you want to return, only the primary key columns are returned. If you set the returnAll field to true, all columns are returned.

Methods

You can perform an exists query in the Tablestore console or by using the Tablestore CLI or Tablestore SDKs. Before you perform an exists query, make sure that the following preparations are made:

You have an Alibaba Cloud account or a RAM user that has permissions to perform operations on Tablestore. For information about how to grant Tablestore operation permissions to a RAM user, see Use a RAM policy to grant permissions to a RAM user.
If you want to use Tablestore SDKs or the Tablestore CLI to perform a query, an AccessKey pair is created for your Alibaba Cloud account or RAM user. For more information, see Create an AccessKey pair.
A data table is created. For more information, see Operations on tables.
A search index is created for the data table. For more information, see Create a search index.
If you want to use Tablestore SDKs to perform a query, an OTSClient instance is initialized. For more information, see Initialize an OTSClient instance.
If you want to use the Tablestore CLI to perform a query, the Tablestore CLI is downloaded and started, and information about the instance that you want to access and the data table are configured. For more information, see Download the Tablestore CLI, Start the Tablestore CLI and configure access information, and Operations on data tables.

Use the Tablestore console

Go to the Indexes tab.
1. Log on to the Tablestore console.
2. In the top navigation bar, select a resource group and a region.
3. On the Overview page, click the name of the instance that you want to manage or click Manage Instance in the Actions column of the instance.
4. On the Tables tab of the Instance Details tab, click the name of the data table or click Indexes in the Actions column of the data table.
On the Indexes tab, find the search index that you want to use to query data and click Manage Data in the Actions column.
In the Search dialog box, specify query conditions.
1. By default, the system returns all attribute columns. To return specific attribute columns, turn off All Columns and specify the attribute columns that you want to return. Separate multiple attribute columns with commas (,).
  Note
  By default, the system returns all primary key columns of the data table.
2. Select the And, Or, or Not logical operator based on your business requirements.
  If you select the And logical operator, data that meets the query conditions is returned. If you select the Or operator and specify a single query condition, data that meets the query condition is returned. If you select the Or logical operator and specify multiple query conditions, data that meets one of the query conditions is returned. If you select the Not logical operator, data that does not meet the query conditions is returned.
3. Select an index field and click Add.
4. Set the Query Type parameter to ExistsQuery(ExistsQuery).
5. By default, the sorting feature is disabled. If you want to sort the query results based on specific fields, turn on Sort and specify the fields based on which you want to sort the query results and the sorting order.
6. By default, the aggregation feature is disabled. If you want to collect statistics on a specific field, turn on Collect Statistics, specify the field based on which you want to collect statistics, and then configure the information that is required to collect statistics.
Click OK.
Data that meets the query conditions is displayed in the specified order on the Indexes tab.

Use the Tablestore CLI

You can use the Tablestore CLI to run the search command to query data by using search indexes. For more information, see Search index.

Run the search command to use the search_index search index to query data and return all indexed columns of each row that meets the query conditions.
```
search -n search_index --return_all_indexed
```

Enter the query conditions as prompted:

{
    "Offset": -1,
    "Limit": 10,
    "Collapse": null,
    "Sort": null,
    "GetTotalCount": true,
    "Token": null,
    "Query": {
        "Name": "ExistsQuery",
        "Query": {
            "FieldName": "col_keyword"
        }
    }
}

Use Tablestore SDKs

You can use the following Tablestore SDKs to perform an exists query: Tablestore SDK for Java, Tablestore SDK for Go, Tablestore SDK for Python, Tablestore SDK for Node.js, Tablestore SDK for .NET, and Tablestore SDK for PHP. In this example, Tablestore SDK for Java is used.

The following sample code provides an example on how to query all rows in which the specified column is not empty.

public static void existQuery(SyncClient syncClient) {
    // Query whether the col_keyword field exists. The col_keyword field is not a Nested field. 
    {
        {
            // The following sample code provides an example on how to query whether a field of a non-Nested type exists. 
            SearchQuery searchQuery = new SearchQuery();
            ExistsQuery existQuery = new ExistsQuery(); // Set the query type to ExistsQuery. 
            existQuery.setFieldName("col_keyword");
            searchQuery.setQuery(existQuery);
            SearchRequest searchRequest = new SearchRequest("<TABLE_NAME>", "<SEARCH_INDEX_NAME>", searchQuery);
            SearchResponse resp = syncClient.search(searchRequest);
        }
        {
            // Construct a builder. 
            SearchResponse resp = syncClient.search(SearchRequest.newBuilder()
                    .indexName("<SEARCH_INDEX_NAME>")
                    .tableName("<TABLE_NAME>")
                    .searchQuery(
                            SearchQuery.newBuilder()
                                    .query(QueryBuilders.exists("col_keyword"))
                                    .limit(10)
                                    .build())
                    .build());
        }
    }
    // The Nested structure is [{"keyword":"a","long":123}]. 
    {
        // Query whether the col_nested parent field exists. The col_nested field is of the Nested type. 
        {
            SearchResponse resp = syncClient.search(SearchRequest.newBuilder()
                    .indexName("<SEARCH_INDEX_NAME>")
                    .tableName("<TABLE_NAME>")
                    .searchQuery(
                            SearchQuery.newBuilder()
                                    .query(QueryBuilders.nested()
                                            .scoreMode(ScoreMode.None)
                                            .path("col_nested")
                                            .query(QueryBuilders.exists("col_nested")))
                                    .limit(10)
                                    .getTotalCount(false)
                                    .build())
                    .build());
        }
        // Query whether the col_nested.keyword field exists. The col_nested.keyword field is of the Nested type. 
        {
            SearchResponse resp = syncClient.search(SearchRequest.newBuilder()
                    .indexName("<SEARCH_INDEX_NAME>")
                    .tableName("<TABLE_NAME>")
                    .searchQuery(
                            SearchQuery.newBuilder()
                                    .query(QueryBuilders.nested()
                                            .scoreMode(ScoreMode.None)
                                            .path("col_nested")
                                            .query(QueryBuilders.exists("col_nested.keyword")))
                                    .limit(10)
                                    .getTotalCount(false)
                                    .build())
                    .build());
        }
    }
}

Billing rules for

When you use a search index to query data, you are charged for the read throughput that is consumed. For more information, see Billable items of search indexes.

FAQ

References

When you use a search index to query data, you can use the following query methods: term query, terms query, match all query, match query, match phrase query, prefix query, range query, wildcard query, fuzzy query, Boolean query, geo query, nested query, KNN vector query, and exists query. You can select query methods based on your business requirements to query data from multiple dimensions.
You can sort or paginate rows that meet the query conditions by using the sorting and paging features. For more information, see Perform sorting and paging.
You can use the collapse (distinct) feature to collapse the result set based on a specific column. This way, data of the specified type appears only once in the query results. For more information, see Collapse (distinct).
If you want to analyze data in a data table, you can use the aggregation feature of the Search operation or execute SQL statements. For example, you can obtain the minimum and maximum values, sum, and total number of rows. For more information, see Aggregation and SQL query.
If you want to obtain all rows that meet the query conditions without the need to sort the rows, you can call the ParallelScan and ComputeSplits operations to use the parallel scan feature. For more information, see Parallel scan.