All Products
Search
Document Center

Elasticsearch:Perform a check before a version upgrade

Last Updated:Nov 14, 2024

Before you upgrade the version of an Alibaba Cloud Elasticsearch cluster, you must perform the required manual checks. We also recommend that you manually check the cluster status and configuration compatibility and modify the configurations based on your business requirements. This topic describes the required manual checks and the recommended cluster status and configuration compatibility checks before a version upgrade. It also provides the methods that can be used to modify incompatible configurations.

Background information

Perform manual checks

Before you upgrade the version of your cluster, you must manually perform the following operations:

  • Run the following command to check whether your cluster stores indexes in the close state:

    GET _cat/indices?v

    Check for indexes that are in the close state

    If your cluster stores indexes in the close state, run the following command to open the indexes:

    POST test/_open
  • If you want to update the kernel of your cluster, check whether a later kernel version is available. You can update the kernel of a cluster only if a later kernel version is available for the cluster.

    You can go to the Basic Information page of your cluster to check whether a later kernel version is available for the cluster. A later kernel version is available

  • Check whether a client is used to access the cluster. If a client is used, make sure that the version of the client is compatible with that of the cluster. If the two versions are incompatible, upgrade the version of the client. For more information about the version compatibility between clients and clusters, see Compatibility.

If you want to upgrade the version of a cluster from V5.X to V6.X, you must also perform the following operations:

  • Split each multi-type index stored in the V5.X cluster into multiple single-type indexes.

    Clusters of V6.X or later do not support multi-type indexes. If the V5.X cluster stores multi-type indexes, you can write data to the indexes after the cluster version is upgraded to V6.X. However, if you create multi-type indexes in V6.X, errors are reported. We recommend that you split each multi-type index into single-type indexes before the upgrade.

  • Run the following command to check whether the cross-cluster search feature is enabled for the cluster. If the feature is enabled for the cluster, the search.remote parameter is displayed in the return result, and the value of this parameter is not null.

    GET _cluster/settings

    If the feature is enabled for the cluster, run the following command to disable the feature. After you upgrade the version of the cluster, you can enable the feature again.

    PUT _cluster/settings
    {
      "persistent": {
        "search.remote.*": null
      },
      "transient": {
        "search.remote.*": null
      }
    }
    Important

    The search.remote parameter is used to configure the cross-cluster search feature in V5.X, whereas the cluster.remote parameter is used in V6.X.

Cluster status check

When you upgrade the version of your cluster, you must click Precheck to enable the system to check whether the cluster status and cluster load are normal. The version of your cluster can be upgraded only if the cluster is in a normal state and its load is normal. Before you upgrade the version of your cluster, you can also refer to the items listed in the following table to perform a manual check.

Check item

Normal state

Cluster status

The cluster is in the Normal state (indicated by the color green).

JVM heap memory usage

The JVM heap memory usage of the cluster is less than 75%.

Disk usage

The disk usage of nodes is less than the value of cluster.routing.allocation.disk.watermark.low.

Replica shards

  • All indexes are configured with replica shards.

  • Before you make a change to a multi-zone cluster, you must make sure that the number of replica shards of each index in the cluster is less than the number of zones in which the cluster is deployed. After the change is complete, you can manually increase the number of replica shards based on your business requirements. For more information about how to change the number of replica shards of indexes in a cluster, see Index Templates.

Snapshots

The system created snapshots for the cluster during the last hour.

Custom plug-ins

The cluster does not have custom plug-ins installed.

Elastic Compute Service (ECS) instances in the zone where the cluster resides

The zone where the cluster resides has sufficient ECS instances.

Note

When you upgrade the version of a cluster, the system adds nodes of a later version to the cluster. Then, the system migrates data stored on original nodes to the added nodes and removes the original nodes from the cluster. Make sure that the zone where the cluster resides has sufficient ECS instances before the upgrade.

YML configuration file

The cluster of a later version is compatible with the YML configuration file in an earlier version.

Configuration compatibility check

When you upgrade the version of a cluster to V6.X, the system automatically checks whether incompatible configurations exist between the versions. If you do not modify the incompatible configurations, your cluster services may be affected after the upgrade. To ensure that the cluster version can be upgraded, you can run the GET _cluster/settings and GET*/_settings?flat_settings=true command to check whether the configurations listed in the following table are configured for your cluster before you perform the upgrade.

Important

If the index template contains the configurations listed in the following table, the template cannot be used to create indexes after the version upgrade.

No.

Configuration level

Configuration category

Parameter

1

Cluster

Snapshot settings

cluster.routing.allocation.snapshot.relocation_enabled

2

Storage throttling settings

indices.store.throttle.type and indices.store.throttle.max_bytes_per_sec

3

Index

Similarity settings

index.similarity.base

4

Shadow replica settings

index.shared_filesystem and index.shadow_replicas

5

Index storage settings

index.store.type

6

Storage throttling settings

index.store.throttle.type and index.store.throttle.max_bytes_per_sec

7

include_in_all setting in the mapping configuration of an index

include_in_all

Note

The indexes that are created before an upgrade from V5.X to V6.X and have this parameter configured can still be used after the upgrade. The indexes that are created after the upgrade do not support this parameter.

8

Version settings for index creation

index.version.created

Note

This parameter specifies that indexes cannot be upgraded across major versions. For example, you cannot directly upgrade the indexes created in V5.X to V7.X. Before an upgrade from V5.X to V7.X, you must call the reindex API to migrate data in the indexes to a V7.X cluster. Then, delete the indexes from the V5.X cluster and upgrade the version of the V5.X cluster.

9

Index template

Similarity settings

index.similarity.base

10

Shadow replica settings

index.shared_filesystem and index.shadow_replicas

11

Index storage settings

index.store.type

12

Storage throttling settings

index.store.throttle.type and index.store.throttle.max_bytes_per_sec

13

include_in_all setting in the mapping configuration of the index template

include_in_all

14

_all setting in the mapping configuration of the index template

_all

15

Type settings in the mapping configuration of the index template

None

Note

Check whether the mapping configuration in the index template contains multiple type settings.

Note
  • The parameters listed in the preceding table are deprecated in V6.0 and later. For more information, see Breaking changes in 6.0.

  • The preceding check items are CRITICAL check items. If a CRITICAL check item is reported, the cluster fails the compatibility check, and the cluster version cannot be upgraded. The later version is incompatible with the configurations indicated by CRITICAL check items. You must refer to Modify incompatible configurations to modify the incompatible configurations and perform a check again. If a WARNING check item is reported, the cluster fails the compatibility check, but the cluster version can still be upgraded. The configurations indicated by WARNING check items are ignored after the upgrade.

Modify incompatible configurations

This section provides the methods that are used to modify incompatible configurations.

  • Cluster-level incompatible configurations

    For cluster-level incompatible configurations, you can disable the configurations.

    Configuration category

    Command to disable the configuration

    Snapshot settings

    PUT _cluster/settings
    {
      "persistent": {
        "cluster.routing.allocation.snapshot.relocation_enabled": null
      },
      "transient": {
        "cluster.routing.allocation.snapshot.relocation_enabled": null
      }
    }

    Storage throttling settings

    PUT _cluster/settings
    {
      "persistent": {
        "indices.store.throttle.type": null,
        "indices.store.throttle.max_bytes_per_sec": null
      },
      "transient": {
        "indices.store.throttle.type": null,
        "indices.store.throttle.max_bytes_per_sec": null
      }
    }
  • Index-level incompatible configurations

    For index-level incompatible configurations, you can disable the configurations.

    Configuration category

    Command to disable the configuration

    Additional information

    Similarity settings

    PUT test_index/_settings
    {
       "index.similarity.base.*": null
    }

    These configurations can be modified only after indexes are closed. You cannot read data from or write data to closed indexes. After the modifications, you can open the indexes. The following example shows how to open and close the test_index index:

    • Close the index

      POST test_index/_close
    • Open the index

      POST test_index/_open

    Shadow replica settings

    PUT test_index/_settings
    {
        "index.shared_filesystem": null,
        "index.shadow_replicas": null
    }

    Index storage settings

    PUT test_index/_settings
    {
       "index.store.type": null
    }

    Storage throttling settings

    PUT test_index/_settings
    {
      "settings": {
        "index.store.throttle.type": null,
        "index.store.throttle.max_bytes_per_sec": null
      }
    }

    None.

    Note

    Indexes that have the include_in_all parameter configured can still be used in the later version. You do not need to modify this parameter.

  • Index template-level incompatible configurations

    The following example shows how to modify configurations in the test_template index template:

    1. Run the GET _template/test_template command to query the test_template index template.

      The query result shows that the index template contains the following incompatible configurations: index storage settings, _all, and include_in_all.

      {
       "test_template": {
         "order": 0,
         "template": "test_*",
         "settings": {
           "index": {
             "store": {
               "throttle": {
                 "max_bytes_per_sec": "100m"
               }
             }
           }
         },
         "mappings": {
           "test_type": {
             "_all": {
               "enabled": true
             },
             "properties": {
               "test_field": {
                 "type": "text",
                 "include_in_all": true
               }
             }
           }
         },
         "aliases": {}
       }
      }
    2. Run the following command to delete the incompatible configurations and update the index template:

      PUT _template/test_template
      {
         "order": 0,
         "template": "test_*",
         "settings": {
         },
         "mappings": {
           "test_type": {
             "properties": {
               "test_field": {
                 "type": "text"
               }
             }
           }
         },
         "aliases": {}
      }