Elasticsearch:FAQ about open source Elasticsearch

Overview

• How do I configure the thread pool size for indexes?

• What do I do if OOM occurs?

• How do I manually manage a shard?

• What are the cache clearing policies for Elasticsearch?

• How do I reroute index shards?

• When I query an index, the statusCode: 500 error message is displayed. What do I do?

• How do I change the value of the auto_create_index parameter?

• How long is required to create a snapshot that will be stored in OSS?

• How do I specify the number of shards when I create an index?

• When I use the elasticsearch-repository-oss plug-in to migrate data from a self-managed Elasticsearch cluster, the following error message is displayed. What do I do?

• How do I change the time zone for data visualization in the Kibana console?

• What type of data can I perform Elasticsearch term queries on?

• What are the precautions for using aliases in Elasticsearch?

• What do I do if the too_many_buckets_exception error message is returned during a query?

• How do I delete multiple indexes at a time?

• Can I change the value of the script.painless.regex.enabled parameter?

• How do I change the mapping configurations, number of primary shards, and number of replica shards for an index in an Elasticsearch cluster?

• What do I do if I want to separately store the values of a field?

• What do I do if I want or I do not want a specific field to be aggregated?

How do I configure the thread pool size for indexes?

What do I do if OOM occurs?

curl -u elastic:<password> -XPOST "localhost:9200/<index_name>/_cache/clear?pretty"

How do I manually manage a shard?

Use the reroute API or Cerebro. For more information, see Cluster reroute API and Cerebro.

What are the cache clearing policies for Elasticsearch?

How do I reroute index shards?

curl -XPOST 'localhost:9200/_cluster/reroute' -d '{
    "commands" : [ {
        "move" :
            {
              "index" : "test", "shard" : 0,
              "from_node" : "node1", "to_node" : "node2"
            }
        },
        {
          "allocate" : {
              "index" : "test", "shard" : 1, "node" : "node3"
          }
        }
    ]
}'

When I query an index, the statusCode: 500 error message is displayed. What do I do?

How do I change the value of the auto_create_index parameter?

PUT /_cluster/settings
{
    "persistent" : {
        "action": {
          "auto_create_index": "false"
        }
    }
}

How long is required to create a snapshot that will be stored in OSS?

If the number of shards, memory usage, disk usage, and CPU utilization of your cluster are normal, about 30 minutes are required to create a snapshot for 80 GB of index data.

How do I specify the number of shards when I create an index?

You can divide the total data size by the data size of each shard to obtain the number of shards. We recommend that you limit the data size of each shard to 30 GB. If the data size of each shard exceeds 50 GB, query performance is severely affected.

When I use the elasticsearch-repository-oss plug-in to migrate data from a self-managed Elasticsearch cluster, the following error message is displayed. What do I do?

Error message: ERROR: This plugin was built with an older plugin structure. Contact the plugin author to remove the intermediate "elasticsearch" directory within the plugin zip.

Solution: Change the name of the ZIP plug-in package from elasticsearch to elasticsearch-repository-oss, and copy the package to the plugins directory.

How do I change the time zone for data visualization in the Kibana console?

What type of data can I perform Elasticsearch term queries on?

What are the precautions for using aliases in Elasticsearch?

The total number of shards for indexes that have the same alias must be less than 1,024.

What do I do if the too_many_buckets_exception error message is returned during a query?

Error message: "type": "too_many_buckets_exception", "reason": "Trying to create too many buckets. Must be less than or equal to: [10000] but was [10001].

Solution: You can change the value of the size parameter for bucket aggregations. For more information, see Limit the number of buckets that can be created in an aggregation. You can also resolve this issue based on the instructions provided in Increasing max_buckets for specific Visualizations.

How do I delete multiple indexes at a time?

PUT /_cluster/settings
{
  "persistent": {
     "action.destructive_requires_name": false
  }
}

Can I change the value of the script.painless.regex.enabled parameter?

In most cases, the default value false is used for the script.painless.regex.enabled parameter, and we recommend that you do not change the value. If you want to use a regular expression in a Painless script, you can set the script.painless.regex.enabled parameter to true in the elasticsearch.yml configuration file. Regular expressions consume a large amount of resources. In open source Elasticsearch, the value true is not recommended for the script.painless.regex.enabled parameter. Therefore, we recommend that you do not set the parameter to true if unnecessary.

How do I change the mapping configurations, number of primary shards, and number of replica shards for an index in an Elasticsearch cluster?

• If you want to modify the mapping configurations for an existing index, we recommend that you reindex data.

  Note

  For information about the field types that are supported in the mapping configurations, see Data field type.

• You cannot change the number of primary shards for an existing index. If the number of primary shards cannot meet your business requirements, you can call the reindex API to reindex data.

  Note

  To prevent adjustments after an index is created, we recommend that you plan the number of primary shards and replica shards before you create the index.

• If you want to change the number of replica shards for each primary shard in an existing index, you can run the following command:

  PUT test/_settings
  {
    "number_of_replicas": 0
  }

What do I do if I want to separately store the values of a field?

By default, Elasticsearch does not separately store values for fields, except the _source field. If you want to separately store values for a field, you can set the store property in the mappings configuration of the related index to true for the field.

The following code provides an example on how to separately store the values of the my_field field:

PUT / my_index {
	"mappings": {
		"properties": {
			"my_field": {
				"type": "text",
				"store": true
			}
		}
	}
}

What do I do if I want or I do not want a specific field to be aggregated?

Whether a field can be aggregated depends on the data type of the field and whether related field data (doc_values or fielddata) exists.

• By default, fields of the numeric, date, or keyword type can be aggregated based on doc_values.

  Note

  doc_values indicates the column-oriented storage model, which is optimized for sorting, aggregation, and script operations.

• By default, fields of the text type cannot be aggregated. If you need to aggregate such fields, you must enable fielddata in the mappings configuration.

  Note

  After fielddata is enabled, all text data is loaded into memory. This significantly increases memory usage.

  PUT /my_index{
    "mappings": {
      "properties": {
        "my_text_field": {
          "type": "text",
          "fielddata": true
        }
      }
    }
  }

• If you do not want a field to be aggregated, you can use one of the following methods to implement this. You can select a method based on your application logic.

  • Set the enabled property to false for the field.

  • Exclude the field from the related document.