Uneven distribution of hot data on nodes - Elasticsearch

Elasticsearch evenly allocates shards to data nodes and uses hash maps to evenly route documents to shards. In this case, some nodes may store more hot data than others, which results in high loads on these nodes. To address this issue, you can restart your cluster or manually migrate shards to re-allocate shards. This topic describes how to manually migrate shards.

Problem description

A node with high loads stores a large number of shards that have the same attribute. For example, a node stores only primary shards.
A node with high loads stores more shards for business indexes than other nodes.

Precautions

Manual shard migration is only a temporary solution to the high loads of nodes. If some nodes are absent from the cluster for a short time and shards are re-allocated, this issue may occur again. Therefore, before you migrate shards, we recommend that you optimize shard allocation based on the instructions provided in Imbalanced loads on a cluster.
If the shards that store hot data are evenly allocated, but the cluster runs with high loads, we recommend that you upgrade the configuration of the cluster.

Solution

Disable shard allocation.
```
PUT /_cluster/settings
{
  "transient" : {
     "cluster.routing.allocation.enable" : "none"
  }
}
```
Important The preceding command is used only to temporarily disable shard allocation. After shards are migrated, you must enable shard allocation again. To enable shard allocation, run the PUT /_cluster/settings command with cluster.routing.allocation.enable set to all.
Manually migrate shards.
In this example, Shard 3 of the index_parkingorder_v1 index is migrated from the node whose IP address is 192.168.130.77 to the node whose IP address is 192.168.130.78.
```
POST /_cluster/reroute
{
 "commands" : [
  {
   "move" : {
    "index" : "index_parkingorder_v1", 
    "shard" : 3,
    "from_node" : "192.168.130.77", 
    "to_node" : "192.168.130.78"
   }
  }
 ]
}
```
Note
- When you migrate a shard of an index to a node, make sure that the node does not store shards that belong to the same index or have the same sequence number. For example, Replica Shard 3 of an index exists on Node A. In this case, Primary Shard 3 of the index cannot be migrated to Node A.
- For more information about how to manually migrate shards, see cluster-reroute.
Check the shard allocation status.
```
GET _cat/shards?v
```
If the command is successfully run, the result shown in the following figure is returned.

After the shard is migrated, enable shard allocation.

PUT /_cluster/settings
{
 "transient" : {
  "cluster.routing.allocation.enable" : "all"
 }
}