Upgrade the configuration of an Elasticsearch cluster - Elasticsearch

If your Elasticsearch cluster cannot meet workload and performance requirements due to data volume increase or high memory usage, you can upgrade the configuration of the cluster. If the cluster load is not high and indexes in the cluster have replica shards, the cluster can still provide services during the upgrade. However, the performance of the cluster may be affected, and more time may be required for the upgrade.

Prerequisites

The following operations are performed:

Evaluate the specifications and storage capacity of your cluster. For more information, see Evaluate specifications and storage capacity.
In the Kibana console of your cluster, check whether your cluster stores indexes in the close state. If your cluster stores such indexes, you must open the indexes. Otherwise, the upgrade fails.
- Run the following command to view the statuses of indexes:
```
GET /_cat/indices?v
```
- Run the following command to open an index in the close state:
```
POST /<index_name>/_open
```
  Replace <index_name> with the name of the index in the close state.
In the Kibana console, run the following command to query the number of replica shards for indexes. The number of replica shards must be at least 1. Before you make a change to a multi-zone cluster, you must make sure that the number of replica shards for each index in the cluster is less than the number of zones in which the cluster is deployed. We recommend that you set the number of replica shards to 1. After the change is complete, you can manually increase the number of replica shards based on your business requirements. For more information about how to change the number of replica shards for indexes in a cluster, see Index Templates.
Note
If the number of replica shards is 0, the high availability mechanism cannot be implemented, and business stability cannot be ensured.
```
GET _cat/indices?v
```

Precautions

Item	Description
Impact on services	After you upgrade the configuration of your cluster, the system restarts the cluster to make the changes take effect. The time required for the restart depends on the specifications, data structure, and data volume of the cluster. In most cases, the restart requires a few hours. Therefore, we recommend that you upgrade the configuration of your cluster during off-peak hours. For more information, see Restart a cluster or node. Important In most cases, if the indexes of your cluster have replica shards and the load of your cluster is normal, your cluster can still provide services during a cluster configuration change. The following items indicate that the load of a cluster is normal: The CPU utilization of each node in the cluster is about 60%, the heap memory usage of each node in the cluster is about 50%, and the value of NodeLoad_1m for each node is less than the number of vCPUs for the node. If the indexes of your cluster do not have replica shards, the load of the cluster is high, and large amounts of data are written to or queried in your cluster, access to the cluster may time out during a cluster configuration upgrade or downgrade. We recommend that you configure an access retry mechanism for your client and increase the number of replica shards before you upgrade or downgrade the configuration of your cluster. This reduces the impact on your business. During the configuration upgrade of a cluster, data stored in the cluster is not lost. However, if the cluster is overloaded or is in an abnormal state, access to the cluster may time out. We recommend that you configure an access retry mechanism for your client and configure replica shards for indexes in the cluster. If your Elasticsearch cluster is in an abnormal state (indicated by the color yellow or red), you must select Forced Update when you upgrade the configuration of the cluster. This may affect services. If you set Update Mode to Blue-green Update when you upgrade the configuration of your Elasticsearch cluster, the IP addresses of the nodes in the cluster will change. If you have specified the IP addresses of the nodes in the configuration file of the cluster, you must update the IP addresses of the nodes in the configuration file at the earliest opportunity after the upgrade.
Changes in billing	After you submit a configuration upgrade order, your cluster is charged based on the new configuration. For more information about billing rules, see Pay-as-you-go and Subscription. Note During a configuration upgrade, you can check the price of your order on the Upgrade/Downgrade page in real time.
Version upgrades	You cannot upgrade the version of your cluster during a configuration upgrade. For more information, see Upgrade the version of a cluster. Note If you perform a version upgrade during a configuration upgrade, the system displays the "UpgradeVersionMustFromConsole" error message. Since May 2021, Alibaba Cloud Elasticsearch no longer provides Elasticsearch V7.4 clusters. Existing clusters can still be used. If your cluster is of V7.4, we recommend that you purchase a V7.10 cluster and migrate data from the original cluster to the V7.10 cluster.
Specification upgrades	You can upgrade only one type of node during each upgrade. The node types include data nodes, warm nodes, client nodes, dedicated master nodes, Kibana nodes, and elastic nodes. Note You can remove data nodes from your cluster to downgrade the configuration of the cluster. For more information about how to remove data nodes, see Scale in a cluster.
Updates of disk types	Disks with low storage performance can be updated to disks with high storage performance. The following types of disks are listed in ascending order of their storage performance: ultra disks, standard SSDs, and enhanced SSDs (ESSDs). You can update the disk types of nodes in your cluster based on your business requirements. Note You cannot update SSDs to ESSDs at performance level 0.
Resource insufficiency in the zone where your cluster resides	If the zone where your cluster resides has insufficient resources for a configuration upgrade, you can migrate the nodes in the zone to another before the upgrade. For more information, see Migrate nodes in a zone.
Shard allocation	After the configuration of a cluster is upgraded, the system evenly reallocates shards to nodes in the cluster. In some cases, loads on the nodes may be unbalanced. For example, inappropriate shard configurations may lead to unbalanced loads on the nodes. For more information about the analysis for and solutions to unbalanced loads on nodes in a cluster, see Unbalanced loads on a cluster.

Procedure

Log on to the Alibaba Cloud Elasticsearch console.
In the left-side navigation pane, click Elasticsearch Clusters.
In the top navigation bar, select the resource group to which your cluster belongs and the region where your cluster resides.
On the Elasticsearch Clusters page, find the cluster and click Upgrade Configuration in the Actions column.

On the Upgrade/Downgrade page, change the configuration of the cluster.

You can change node specifications, storage types, or storage space per node.

The Current Config section of the Upgrade/Downgrade page shows the current configuration of the cluster. You can use this as a reference during the upgrade.

Follow the instructions on the Upgrade/Downgrade page to upgrade the configuration of your cluster based on your business requirements. For more information about the parameters, see Parameters on the buy page. The following table describes only some of the parameters.

Parameter	Description
Data Node	The maximum storage space supported by a single data node varies based on the disk type of the node. You can check specific limits in the Elasticsearch console. Important If the value of Data Node Disk Type in the Current Config section is Ultra Disk and your cluster is in an abnormal state (indicated by the color yellow or red), you cannot increase the value of Data Node Storage Space to a value greater than 2,048. If you want to increase the value of Data Node Storage Space to a value greater than 2,048, you must make sure that your cluster is in a normal state before the upgrade. Alternatively, you can increase the number of data nodes during the upgrade. Data nodes with 1 vCPU and 2 GiB of memory are designed only for testing purposes. Do not use such data nodes in the production environment. The service level agreement (SLA) does not apply to such data nodes. Therefore, we recommend that you upgrade your data nodes with 1 vCPU and 2 GiB of memory.
Dedicated Master Node	You can purchase or upgrade dedicated master nodes.
Warm Node	You can purchase or upgrade warm nodes.
Client Node	You can purchase or upgrade client nodes.
Kibana Node	You can upgrade your Kibana node. Important When you purchase an Alibaba Cloud Elasticsearch cluster, the value of the Kibana Node parameter can be only Yes. Due to the impact on performance and stability, we recommend that you select the specifications of 2 vCPUs and 4 GiB of memory or higher for the Kibana node. The Kibana node with one vCPU and 2 GiB of memory is free of charge. However, we recommend that you use the Kibana node with one vCPU and 2 GiB of memory only for testing purposes.
Forced Update	If your cluster is in an abnormal state (indicated by the color red or yellow) and your services are severely affected, you must immediately upgrade the configuration of the cluster. In this case, we recommend that you select Forced Update. Important During a forced update, the system ignores the cluster status. This accelerates the update. However, the actual time required to recover the cluster depends on the data volume, workloads, and configurations of the cluster. After the forced update, the system restarts your cluster. During the restart, the services running on the cluster may be unstable. If you do not select Forced Update, the system uses the default mode to restart your cluster to make the changes take effect. For more information, see Restart a cluster or node.
Intelligent Update	The Intelligent Update feature is enabled by default. When the Intelligent Update feature is enabled, the system selects an appropriate method based on the modified items to update the configuration of your cluster. You can disable the Intelligent Update feature. After this feature is disabled, you can select an update mode based on your business requirements. The following modes are supported: Blue-green Update: The system adds the same number of nodes to your cluster, seamlessly migrates data from the original nodes to the new nodes, and removes the original nodes from the cluster. This process is seamless but time-consuming, and the IP addresses of nodes are changed. If you have specified the IP addresses of the nodes in the configuration file of the cluster, you must update the IP addresses of the nodes in the configuration file at the earliest opportunity after the upgrade. In-place Update: The system performs a rolling update for the nodes that you want to update. This process does not involve data migration, and the time that is required for this process is not affected by the data volume of your cluster. However, the performance of your cluster may be affected. For more information, see Cluster update methods. Note If you do not have special requirements for your business, we recommend that you enable the Intelligent Update feature.

Read the terms of service, select Elasticsearch Terms of Service, and then click Buy Now.
After you complete the payment, the system restarts the cluster to make the changes take effect.

FAQ

References

API operation for upgrading the configuration of a cluster: UpdateInstance
View change progress and change records
Configure cluster alerting