If you find that the configurations of your cluster do not meet your expectations after the cluster is created, you can refer to the methods provided in the following table to modify the configurations.
Warning
If you need to unsubscribe from or release your cluster, first back up your data. For more information about how to back up data, see Create manual snapshots and restore data from manual snapshots. After the unsubscription or release, your data stored on the cluster is deleted and cannot be restored.
| Configuration item | Solution |
|---|---|
| Billing method | If you purchased a pay-as-you-go cluster, you can switch the billing method of the cluster to subscription. For more information, see Change the billing method of a cluster from pay-as-you-go to subscription. |
| Version | This configuration item can be modified if one of the following conditions is met: - The version of the cluster that you purchased is V5.5.3, and you want to upgrade the version to V5.6.16. - The version of the cluster that you purchased is V5.6.16, and you want to upgrade the version to V6.3.2. - The version of the cluster that you purchased is V6.3.2, and you want to upgrade the version to V6.7.0. |
| For information about how to upgrade the version of a cluster, see Upgrade the version of a cluster. If your version upgrade does not meet the preceding conditions, we recommend that you unsubscribe from or release your cluster and purchase another cluster of the desired version. | |
| Region | You cannot modify this configuration item. We recommend that you unsubscribe from or release your cluster and purchase another cluster based on your business requirements. |
| Zone | You can migrate nodes to the desired zone. For more information, see Migrate nodes in a zone. Note Before you migrate nodes, you must make sure that your cluster is in the Active state. |
| Number of zones | You cannot modify this configuration item. We recommend that you unsubscribe from or release your cluster and purchase another cluster based on your business requirements. |
| Specifications | You can modify this configuration item. For more information, see Upgrade the configuration of a cluster. |
| Storage type | You can modify this configuration item. For more information, see Upgrade the configuration of a cluster. |
| Cloud disk encryption | You cannot modify this configuration item. We recommend that you unsubscribe from or release your cluster and purchase another cluster based on your business requirements. |
| Storage space per node | You can modify this configuration item. For more information, see Upgrade the configuration of a cluster. |
| Number of data nodes | You can modify this configuration item. For more information, see Upgrade the configuration of a cluster. |
| Network type, virtual private cloud (VPC), and vSwitch | You cannot modify these configuration items. We recommend that you unsubscribe from or release your cluster and purchase another cluster based on your business requirements. Note The network type of an Elasticsearch cluster can only be VPC. |
| Username | The default username is elastic. You cannot modify this configuration item. You can create a user in the Kibana console and grant the required permissions to the user. For more information, see Use the RBAC mechanism provided by Elasticsearch X-Pack to implement access control. |
| Password | You can modify this configuration item. For more information, see Reset the access password for an Elasticsearch cluster. |
For the configuration items that are not provided in the preceding table, check whether you can modify the items on the configuration upgrade or downgrade page. For more information, see Upgrade the configuration of a cluster and Downgrade the configuration of a cluster.
| Version on the buy page | Specific version |
|---|---|
| 8.13 | 8.13.4 |
| 8.9 | 8.9.1 |
| 8.5 | 8.5.1 |
| 7.16 | 7.16.2 |
| 7.10 | 7.10.0 |
| 7.7 | 7.7.1 |
| 6.8 | 6.8.6 |
| 6.7 | 6.7.0 |
| 6.3 | 6.3.2 |
| 5.6 | 5.6.16 |
| 5.5 | 5.5.3 |
If you have a self-managed Elasticsearch cluster, we recommend that you select a version that is nearest to the version of the cluster when you purchase an Alibaba Cloud Elasticsearch cluster. For example, you can select a version whose minor version is nearest to that of your self-managed Elasticsearch cluster. If you do not have a self-managed Elasticsearch cluster, we recommend that you select the latest version.
Check whether the RAM user that you use is granted the permissions to obtain the list of VPCs. For more information, see View the information about a RAM user. If the RAM user that you use is not granted the permissions, grant the permissions to the RAM user. For more information, see Create a custom policy.
The issue occurs because no vSwitches are available in the zone that you selected. To resolve this issue, go to the vSwitch page in the VPC console to check whether vSwitches are available in the selected zone. If no vSwitches are available in the selected zone, you must create a vSwitch. For more information, see Create a VPC with an IPv4 CIDR block.
No, the endpoint of the new cluster is different from that of the original cluster. After you purchase the new cluster, we recommend that you modify the client code and unsubscribe from or release the original cluster to avoid service interruptions.
Log on to the Alibaba Cloud Management Console. In the top navigation bar, choose Expenses > Orders. In the left-side navigation pane of the Expenses and Costs console, click Unsubscribe. On the Unsubscribe page, perform cluster unsubscription or order cancellation. For more information, see Methods for unsubscribing resources.
No, you cannot purchase an Elasticsearch cluster that has only one node. An Elasticsearch cluster must have a minimum of two data nodes. For more information, see Parameters on the buy page.
Take one of the following measures:
If the resources that you want to purchase are still unavailable after you take all of the preceding measures, try again later. Resources are dynamic. If resources are insufficient, Alibaba Cloud replenishes the resources at the earliest opportunity.
Data nodes with the specifications of 1 vCPU and 2 GiB of memory may affect the performance of Elasticsearch clusters. Alibaba Cloud Elasticsearch no longer provides data nodes with such specifications since May 2021. Existing data nodes with such specifications can still be used. Data nodes with the specifications of 1 vCPU and 2 GiB of memory are suitable only for online learning and are not suitable for production environments. The service level agreement (SLA) does not apply to clusters that contain such data nodes. Therefore, we recommend that you upgrade your data nodes with the specifications of 1 vCPU and 2 GiB of memory at the earliest opportunity. For more information, see Upgrade the configuration of a cluster.
A period of time is required before an Elasticsearch cluster can provide services after it is purchased. The period of time varies based on the specifications, data structure, and data volume of the cluster. In most cases, a few hours are required.
No, you do not need to purchase a Kibana node after you purchase an Elasticsearch cluster. When you purchase an Alibaba Cloud Elasticsearch cluster, the value of the Kibana Node parameter is fixed as Yes. You can select the specifications of the Kibana node based on your business requirements. For more information, see Create an Alibaba Cloud Elasticsearch cluster.
Important
Due to the impact on performance and stability, we recommend that you select the specifications of 2 vCPUs and 4 GiB of memory or higher for the Kibana node. The Kibana node with one vCPU and 2 GiB of memory is free of charge. However, we recommend that you use the Kibana node with one vCPU and 2 GiB of memory only for testing purposes.
Check whether the region that you selected in the top navigation bar of the Elasticsearch console is the region where your cluster resides. If the region that you selected is correct but you still cannot find your cluster, we recommend that you clear the browser cache or use an on-premises network.
You can use dedicated master nodes to perform operations on clusters, such as creating indexes, deleting indexes, tracking nodes, and allocating shards. The stability of dedicated master nodes is important to the health of clusters. We recommend that you purchase dedicated master nodes for your Elasticsearch cluster if you want to use the cluster in the following scenarios:
Client nodes are used to forward all query and write requests received by an Elasticsearch cluster to data nodes and merge the query results of data nodes. If you want to use your Elasticsearch cluster in scenarios in which aggregate queries are required, we recommend that you purchase client nodes for the cluster. We recommend that you purchase client nodes and data nodes with the same specifications for your Elasticsearch cluster based on a ratio of 1:5. For example, if you purchase two client nodes for your Elasticsearch cluster, we recommend that you purchase 10 data nodes. A minimum of two client nodes must be purchased. For information about how to evaluate specifications and storage capacity for an Elasticsearch cluster, see Evaluate specifications and storage capacity.
The default username is elastic. You can also create a custom user. For more information, see Use the RBAC mechanism provided by Elasticsearch X-Pack to implement access control.
You can upgrade the versions of clusters only from V5.5 to V5.6, from V5.6 to V6.3, or from V6.3 to V6.7.
If you want to perform upgrades between other versions or perform version downgrades, purchase an Elasticsearch cluster of the desired version. Then, migrate data from the original cluster to the new cluster and unsubscribe from or release the original cluster.
Note
You can apply for a free trial on an Elasticsearch V8.5 or V8.9 cluster. After you create such a cluster, you are not allowed to modify the cluster.
No, for security purposes, you are not allowed to log on to your Elasticsearch cluster over SSH. If you want to modify the configuration of your cluster, use the cluster configuration feature of Elasticsearch. For more information, see Overview of cluster configuration.
Yes, Logstash V6.7 is compatible with Elasticsearch V6.3. For more information, see Compatibility matrixes.
No, Elasticsearch cannot be used as a data source of Quick BI. You can use Kibana to analyze data and present analysis results.
Yes, Elasticsearch supports scoring plug-ins. When you create an index, Elasticsearch allows you to create a tokenizer. This way, when you search for data, Elasticsearch uses a scoring plug-in to sort search results by score. For more information, see Step 5: Search for data.
Yes, Elasticsearch supports Lightweight Directory Access Protocol (LDAP). If you want to use LDAP to authenticate the requests that are sent to your Elasticsearch cluster, you must deploy an on-premises Elasticsearch cluster of the same version and use this cluster to conduct an authentication test. If LDAP runs as expected, configure your cluster to support LDAP based on the configurations of the on-premises Elasticsearch cluster. For more information, see Use X-Pack to configure LDAP authentication.
Yes, Alibaba Cloud provides Elasticsearch SDK for Java. Different Elasticsearch versions use different SDKs. For more information, see Java API.
By default, Elasticsearch clusters use the kernel of the latest version. For more information about kernel versions, see AliES release notes. If your cluster does not use the kernel of the latest version, the A new kernel patch is available message appears on the Basic Information page of your cluster. You can click the message to view the current kernel version of your cluster.
If your Elasticsearch cluster is in an abnormal state (indicated by the color red or yellow) and you want to restart the cluster, you can forcefully restart the cluster. During the forced restart, your Elasticsearch service may be unstable, data may be lost, or data read and write operations may fail. Proceed with caution.
The vulnerability is fixed after your Elasticsearch cluster is restarted. For more information, see [[Vulnerability notice] RCE vulnerability in Apache Log4j 2](https://www.alibabacloud.com/help/en/es/product-overview/vulnerability-announcement-or-rce-vulnerability-in-apache-log4j-2#concept-2163543).
No, you do not need to upgrade the version of your Elasticsearch cluster. You need to only follow the instructions provided in Procedure to fix the vulnerability.
You can use one of the following methods to enable communication between Elasticsearch clusters that reside in different regions over an internal network:
You can migrate data to an Alibaba Cloud Elasticsearch cluster from another Alibaba Cloud Elasticsearch cluster, a self-managed Elasticsearch cluster, or a third-party Elasticsearch source. The data migration solution and tool vary based on the data migration scenario. For more information, see Select a data migration solution.
Before you enable HTTPS for an Elasticsearch cluster of a version other than the following versions, you must make sure that the cluster contains client nodes.
If an Elasticsearch cluster is of one of the preceding versions and contains client nodes, you can disable the client nodes for the cluster. If an Elasticsearch cluster is of a version other than the preceding versions and contains client nodes, you cannot release, unsubscribe from, or disable the client nodes for the cluster.
For security purposes, after you enable HTTPS for an Elasticsearch cluster, the system regularly maintains and updates the certificates on which the cluster depends. You cannot perform a rolling update for certificates installed on data nodes in Elasticsearch clusters of V7.10 or earlier. To reduce the impact of a node restart on online services during a certificate update, the system deploys the certificates on client nodes, which are used to forward requests. When you enable HTTPS for an Elasticsearch cluster that does not contain client nodes, the system displays a message to prompt you to purchase client nodes for the cluster. You must purchase client nodes for the cluster before you can enable HTTPS for the cluster. For more information, see Enable HTTPS.
Alibaba Cloud Elasticsearch allows you to allocate a maximum of 1,000 shards for indexes on a single data node in Elasticsearch V7.X clusters. The number of shards that can be allocated for indexes on a single data node is not limited for Elasticsearch clusters of other versions. You must configure shards for indexes on a single data node based on the specifications of the Elasticsearch cluster. For more information, see Evaluate specifications and storage capacity and Size your shards.
You can run the following command in which the max_shards_per_node parameter is configured to temporarily change the maximum number of shards that a single data node can store:
PUT /_cluster/settings
{
"transient": {
"cluster": {
"max_shards_per_node":10000
}
}
}Important
We recommend that you do not specify an excessively large value for the max_shards_per_node parameter. To prevent cluster stability issues caused by excessively high loads of an Elasticsearch cluster, we recommend that you increase the number of nodes in the cluster or reduce the number of shards in the cluster, and appropriately plan shards for the cluster.
By default, the X-Pack monitoring component collects monitoring data from an Elasticsearch cluster at intervals of 10 seconds and stores the data to the indexes whose names start with .monitoring- in the cluster. In Elasticsearch V6.X clusters, the .monitoring-es-6- and .monitoring-kibana-6- indexes are used to store monitoring data. These indexes are rolled over every day. The name of a .monitoring-es-6- index ends with the date when the monitoring data is stored.
A .monitoring-es-6- index stores information such as the cluster status, cluster statistics, node statistics, and index statistics. Such indexes consume a large amount of disk space. For more information, see Configure monitoring indexes.
Alibaba Cloud Elasticsearch uses the industry-standard AES-256 encryption algorithm and Key Management Service (KMS) to encrypt disks for Elasticsearch clusters. For more information, see Overview of disk encryption.
Only Elasticsearch V5.X clusters support port 9300 for TCP and port 9200 for HTTP and HTTPS. Elasticsearch clusters of other versions support only port 9200.
Note
You cannot use Transport Client to access Elasticsearch clusters of V6.0 or later over port 9300. You can access only Elasticsearch V5.X clusters over port 9300.
You can use Monstache to synchronize data from ApsaraDB for MongoDB to Alibaba Cloud Elasticsearch in real time. For more information, see Use Monstache to synchronize data from MongoDB to Alibaba Cloud Elasticsearch in real time.
When you restart an Elasticsearch cluster or node, the system displays the required time. The time is estimated based on the specifications, data structure, and data volume of the cluster or node. In most cases, a few hours are required to restart a cluster. For more information, see Restart a cluster or node.
No, the system does not restart an Elasticsearch cluster after you enable or disable the Public Network Access feature for the cluster. Only the status of the Public Network Access feature changes. This does not affect your cluster.
No, the system does not restart an Elasticsearch cluster after you reset the password used to access the cluster. After you reset the password for a cluster, the system only reloads the data of the cluster but does not restart the cluster. For more information, see Reset the access password for an Elasticsearch cluster.
Yes, the restart of the cluster is affected if the indexes in an Elasticsearch cluster have no replica shards. Services may be interrupted during the restart of the cluster. In most cases, if the load of a cluster is not high and the indexes in the cluster have replica shards, the cluster can still provide services during a restart. However, access timeouts may occur during a restart in some cases. For example, if a number of nodes in the cluster are forced to restart at the same time, the cluster is heavily loaded and is not accessible, the indexes in the cluster do not have replica shards, or large amounts of data are written or queried during a restart or forced restart, access timeouts may occur. In these cases, we recommend that you design a retry mechanism on your client first and restart the cluster during off-peak hours.
On the Basic Information page of your Elasticsearch cluster, click Restart in the upper-right corner. In the Restart dialog box, select Node Role for Object and select the type of node that you want to restart in the Node field. For more information, see Restart a cluster or node.
You can use one of the following methods to restart a single node:
We recommend that you view the details of the cluster change task in the Tasks dialog box in the Elasticsearch console. For more information, see View the progress of a cluster task. The restart of an Elasticsearch cluster whose version is not V7.16 requires a few hours. If the change progress remains unchanged for a long period of time, you can refer to the instructions described in the following table to troubleshoot the issue.
| Possible cause | Solution |
|---|---|
| An error occurs on a plug-in that is installed for the Elasticsearch cluster. As a result, nodes in the cluster cannot be restarted. | Delete the plug-in. |
| Shards cannot be allocated due to excessively high disk usage. Note |
Delete specific indexes or temporarily set the number of replica shards to 0 for the indexes. |
| Shards cannot be allocated due to invalid configurations of cluster parameters. | Run the GET /_cluster/allocation/explain?pretty command to query the reason why shards are not allocated to nodes and resolve the issue based on the actual situation. |
| The number of replica shards is greater than the number of nodes in the Elasticsearch cluster. | Adjust the number of replica shards. |
| An out-of-memory (OOM) error occurs due to the excessively small specifications of the Elasticsearch cluster. | Upgrade the configuration of the Elasticsearch cluster. |
You can view the cluster monitoring data to check the disk usage of each node in the Elasticsearch cluster. For more information, see Metrics and exception handling suggestions.
No, the nodes in an Elasticsearch cluster cannot be periodically restarted. If you want to enable periodical restart for the nodes in your Elasticsearch cluster, you can call the RestartInstance API operation. If you call this API operation, you must also configure a scheduled task and node information.
This issue is caused by unbalanced loads on the cluster. Unbalanced loads may be caused by several reasons, which include inappropriate shard settings, uneven segment sizes, unseparated hot and cold data, and persistent connections that are used for Service Load Balancer (SLB) instances and multi-zone architecture. Resolve the issue based on the actual scenario. For more information, see Unbalanced loads on a cluster.
Important
If the number of replica shards that you specify for an index is greater than the number of nodes minus 1, the cluster enters a state indicated by the color yellow.
Solutions
Run the GET _cat/indices?v command to query the distribution of shards for indexes and identify the index that is in a state indicated by the color yellow. Then, change the number of replica shards for the index to 0. After the cluster recovers to a normal state, change the number of replica shards for the index from 0 to the original setting.
After the number of replica shards is changed to 0, if nodes stop providing services due to errors, data stored on the nodes may be lost. Proceed with caution when you change the number of replica shards. After the cluster recovers to a normal state, change the number of replica shards from 0 to the original setting at the earliest opportunity. The recovery of the cluster requires approximately 1 minute.
PUT test/_settings
{
"index" : {
"number_of_replicas":"0"
}
}If an error occurs on the node on which primary shards are distributed, the cluster enters a state indicated by the color red. You can run the GET /_cat/indices?v command to query the distribution of shards for indexes and identify the index that is in a state indicated by the color red. Then, troubleshoot the issue based on the causes and solutions described in the following table.
| Cause | Solution |
|---|---|
| The resources of the cluster are insufficient due to unbalanced loads on nodes. | Change the total number of primary and replica shards to an integral multiple of the number of data nodes in the cluster to balance loads on nodes. For more information, see What do I do if shards are not evenly distributed on nodes in an Elasticsearch cluster? |
| The cluster stores invalid indexes. | Clear invalid indexes on a regular basis, such as monitoring indexes whose names start with .monitoring. For more information about how to configure monitoring indexes, see Configure monitoring indexes. |
| Shards are not allocated to nodes. | Run the GET /_cluster/allocation/explain?pretty command to query the reason why shards are not allocated to nodes and resolve the issue based on the actual situation. After the issue is resolved, run the POST /_cluster/reroute?retry_failed=true command to reallocate shards to nodes. |
| The cache occupies a large amount of resources. | Run the POST /<Index name>/_cache/clear?fielddata=true command to clear the cache. |
| A cluster update operation such as configuration upgrade is being performed on the cluster. | Pause the update operation and select Forced Update on the Upgrade/Downgrade page to forcefully update the cluster. For more information, see Upgrade the configuration of a cluster. |
| The resources of the cluster are insufficient because the cluster uses low specifications such as 1 vCPU and 2 GiB of memory or 2 vCPUs and 4 GiB of memory. | Upgrade the configuration of the cluster. For more information, see Upgrade the configuration of a cluster. Note |
| The disk usage exceeds 85%. | We recommend that you delete the historical data you no longer require or that you expand the capacity of disks. For more information, see High disk usage and read-only indexes. |
Troubleshoot the issue based on the causes and solutions described in the following table.
| Cause | Solution |
|---|---|
| The number of queries or write requests per second spikes. | Reduce the number of queries or write requests per second for the cluster, reduce the amount of data to write to the cluster in parallel, or scale out or up the cluster. We recommend that you perform stress testing in the production environment and select appropriate specifications. |
| The cache for indexes occupies a large amount of resources. | Run the POST /Index name/_cache/clear?fielddata=true command to clear the cache. |
| The cluster uses low specifications. | Upgrade the configuration of the cluster. For more information, see Upgrade the configuration of a cluster. |
| Loads on nodes in the cluster are unbalanced. | Change the total number of primary and replica shards to an integral multiple of the number of data nodes in the cluster to balance loads on nodes. For more information, see What do I do if shards are not evenly distributed on nodes in an Elasticsearch cluster? |
Run the DELETE /Index name command to delete invalid indexes. After the disk usage is lower than 75%, forcefully upgrade the configuration of disks in the Elasticsearch console. For more information, see Upgrade the configuration of a cluster. If the disk usage of a node is excessively high, you must optimize the configuration of shards. For more information, see What do I do if shards are not evenly distributed on nodes in an Elasticsearch cluster?
Note
To prevent the impact of high disk usage on Alibaba Cloud Elasticsearch, we recommend that you enable disk usage monitoring and alerting. You must view the alert notification in time and take appropriate measures in advance. For more information, see Metrics and exception handling suggestions. When the disk usage of a node exceeds different thresholds, impacts on the cluster are different. The following descriptions provide the related information in detail:
Troubleshoot the issue based on the causes and solutions described in the following table.
| Cause | Solution |
|---|---|
| The cache for the cluster occupies a large amount of memory. | If the cache for the cluster occupies a large amount of memory for a short period of time, run the POST /Index name/_cache/clear?fielddata=true command to clear the cache. If the cache for the cluster occupies a large amount of memory for a long period of time, upgrade the configuration of the cluster. For more information, see Upgrade the configuration of a cluster. The memory usage of the cluster may periodically increase but no alert is generated, which may be caused by business fluctuations or memory reclaim of the cluster. This is a normal phenomenon. |
| The read or write throughput of the cluster is high. | Stop the read or write operation, install a throttling plug-in, and then enable the throttling feature of the plug-in. For more information, see Use the aliyun-qos plug-in. |
| Invalid indexes occupy a large amount of memory. | Delete invalid indexes such as monitoring indexes whose names start with .monitoring to release resources. You can specify a retention duration for such indexes. For more information, see Configure monitoring indexes. |
| Shards are not evenly distributed on nodes, and loads on nodes are unbalanced. | Change the total number of primary and replica shards to an integral multiple of the number of data nodes in the cluster and make sure that shards are evenly distributed on nodes to balance loads on the nodes. For more information, see What do I do if shards are not evenly distributed on nodes in an Elasticsearch cluster? |
| Abnormal queries exist. For example, a user on the business side sends a query request that contains a string with numerous special characters. | Run the GET _cat/tasks?v command to obtain the ID of the time-consuming query task and run the GET _tasks?detailed=true&actions=*read/search* command to obtain the detailed query statement and save and analyze the statement. You can also call the task cancel API, restart the cluster, or restart only heavily loaded nodes in the cluster to quickly cancel the query. |
Appropriately plan shards and reallocate shards for nodes. Make sure that the total number of primary and replica shards is an integral multiple of the number of data nodes in the cluster. This ensures that data is evenly distributed on each data node and prevents heavy loads on a node due to uneven shard distribution. The following descriptions provide examples on how to allocate primary and replica shards for nodes:
Note
Uneven shard distribution leads to unbalanced cluster loads. You can use one of the following methods to check whether shards are evenly allocated for nodes in an Elasticsearch cluster:
GET _cat/shards?v command to query the shard information of the indexes in the cluster. If nodes with heavy loads have a large amount of shards, shards are not evenly distributed on all nodes.The error message indicates that a stack overflow error occurs because the amount of data written to the stack by using Lucene exceeds the upper limit. This issue is related to regular expression-based queries and fuzzy match. This issue is fixed in Elasticsearch V6.0 and later. We recommend that you upgrade the configuration of the cluster at the earliest opportunity or optimize the query statement that you use. For more information, see java.lang.StackOverflowError for the entire cluster.
Run the GET _nodes/stats/jvm?pretty command. By default, the Java Virtual Machine (JVM) heap memory of an Elasticsearch cluster is half of the memory of the cluster. You cannot change the size of the JVM heap memory of an Elasticsearch cluster.
Adjust the value of the thread_pool.write.queue_size parameter in the YML file of the cluster. For more information, see Configure the YML file. Before you adjust the size of the document write queue for the cluster, you can run the GET /_cat/thread_pool?v command to view the queue usage in the cluster.
Important
For Elasticsearch clusters whose versions are earlier than V6.0, use the thread_pool.index.queue_size parameter to configure the size of the document write queue.
You can use a range query of Elasticsearch to query the data of a specific period of time. For more information, see Range query.
If you want to export data of a specific period of time, you can use Logstash to filter data and obtain the data that you want to export. For more information, see Logstash configuration files.
Yes, the amount of data that can be written to an Elasticsearch cluster at a time by using a bulk write request is limited to 100 MB. If this limit is exceeded, you must adjust the amount of data that you write to the cluster at a time. The amount of data that is written to an Elasticsearch cluster at a time by using a bulk write request is calculated by using the following formula: Amount of data that is written to an Elasticsearch cluster at a time by using a bulk write request = Number of documents × Size of each document. You may not be able to precisely estimate the amount of data that is written at a time based only on the number of documents because the amount is also related to the size and complexity of each document. If each document stores a large amount of data, you can reduce the number of documents to write at a time. We recommend that you debug the amount of data to write at a time from 5 MB to 15 MB. By default, the amount of data that is written to an Elasticsearch cluster at a time by using a bulk write request cannot exceed 100 MB. For more information, see HTTP settings. For information about how to debug the amount of data, see Using and Sizing Bulk Requests in the documentation for open source Elasticsearch.
By default, Elasticsearch runs at Coordinated Universal Time (UTC). A difference exists between UTC and the actual time. Elasticsearch does not support time zone adjustment. You can convert time based on the difference. For example, UTC is 8 hours later than Beijing time, which indicates that the time of data stored in Elasticsearch is 8 hours later than the actual time. In this case, you can convert time for data by using one of the following methods:
"time" : "2022-07-15T12:58:17.136+0800".filter{ ruby{ code => "event.set('update_time', event.get('update_time').time.localtime + 8*60*60)" } }.If no result is returned for a query in an Elasticsearch cluster or a long period of time is required before results can be returned, the query is a slow query. You can view slow query logs in the console and locate the cause based on the instructions in Metrics and exception handling suggestions. The following table describes common causes and related solutions.
| Cause | Solution |
|---|---|
| Loads on nodes are unbalanced due to uneven shard distribution. | Change the total number of primary and replica shards to an integral multiple of the number of data nodes in the cluster to balance loads on nodes. For more information, see What do I do if shards are not evenly distributed on nodes in an Elasticsearch cluster? |
| The resources of the cluster are insufficient. | If queries that consume a large amount of resources are performed on the cluster, we recommend that you optimize the statements used for the queries or upgrade the configuration of the cluster. The queries can be aggregate queries, term queries, script queries, or fuzzy match. For information about how to upgrade the configuration of an Elasticsearch cluster, see Upgrade the configuration of a cluster. Note |
The query performance of an Elasticsearch cluster is related to the health status of the cluster. If the memory usage of an Elasticsearch cluster is lower than 80% and loads on nodes in the cluster are balanced, the cluster can provide high query performance.
The write throughput of the cluster is excessively high and circuit breaking is triggered for the cluster due to insufficient resources of the cluster.
Solution
If the operations in the following solutions cannot be performed, you must stop all query and write operations and forcefully restart the cluster. After the cluster recovers to a normal state, use one of the following solutions to resolve the issue.
1）Run the POST /Index name/_cache/clear?fielddata=true command to clear the cache for indexes. If the issue persists, go to the next step.
2）Run the GET /_cat/indices?v command to check whether shards are not evenly distributed on nodes in the cluster. For more information, see What do I do if shards are not evenly distributed on nodes in an Elasticsearch cluster? If the issue persists, go to the next step.
3）Reduce the concurrency for the write operations, delete invalid indexes to release resources, and reduce the use of the Kibana monitoring feature.
To disable the Kibana monitoring feature, run the following command:
PUT _cluster / settings {
"persistent": {
"xpack.monitoring.collection.enabled": false
}
}If the issue persists, go to the next step.
4）Upgrade the configuration of the cluster.
Yes, you can delete multiple indexes in an Elasticsearch cluster at a time. To perform this operation, you must set the Index Deletion parameter to Allow Wildcards in the YML file of the cluster and restart the cluster. After the cluster is restarted, the indexes are deleted at a time by using wildcards. For more information, see Configure the YML file.
Warning
Deleted indexes cannot be recovered. Before you configure the Index Deletion parameter, make sure that the setting of the parameter does not affect your business.
This is a known issue. To troubleshoot the issue, upgrade the kernel version of the cluster to V1.5.0 or later. For more information, see Upgrade the version of a cluster.
The index.max_result_window parameter is provided by Elasticsearch for paged queries that are performed by using the from and size parameters and specifies the maximum number of documents that can be returned for a paged query. The default value of the index.max_result_window parameter is 10000. If the maximum number of queried documents exceeds this value, the following error message is reported: Result window is too large, from + size must be less than or equal to: [10000].
In a search scenario in which deep paging is required, you may need to increase the value of the index.max_result_window parameter. You can run the following command to change the value of the index.max_result_window parameter. The parameter value in the following command is for reference only. If the Elasticsearch cluster is restarted after the command is successfully run, the setting of this parameter still takes effect.
PUT /my_index/_settings
{
"index": {
"max_result_window": 50000
}
}Important
If a large number of results are returned for the query, we recommend that you do not perform deep paging by using the from and size parameters. Otherwise, a large amount of CPU and memory resources are consumed. In a search scenario in which deep paging is required, we recommend that you use the scroll or search_after parameter.
This issue occurs because the index type that is used for the update is different from the original index type. An index in an Elasticsearch cluster can have only one type. If you want to update the data for an index in an Elasticsearch cluster, you must use an index type that is the same as the original index type.
Note
In open source Elasticsearch 7.0 and later, the value of the type field in the mapping configuration is fixed as _doc.
Log on to the Kibana console of the Elasticsearch cluster and run the following command. For information about how to log on to the Kibana console, see Log on to the Kibana console.
GET _search
{
"query": {
"match_all": {}
}
}You can also query all documents in an index on the Discover page of the Kibana console. Before you perform the query on the Discover page, you must create an index pattern. For information about how to use the Kibana console, see Kibana Guide.
Mastering Elasticsearch on Alibaba Cloud: Configuration and Troubleshooting Tips
Data Geek - September 10, 2024
Data Geek - September 3, 2024
Data Geek - August 27, 2024
Data Geek - August 7, 2024
Data Geek - April 30, 2024
Alibaba Cloud Indonesia - May 11, 2023
Alibaba Cloud Elasticsearch
Alibaba Cloud Elasticsearch helps users easy to build AI-powered search applications seamlessly integrated with large language models, and featuring for the enterprise: robust access control, security monitoring, and automatic updates.
Learn More
CloudBox
Fully managed, locally deployed Alibaba Cloud infrastructure and services with consistent user experience and management APIs with Alibaba Cloud public cloud.
Learn More
Alibaba Cloud Flow
An enterprise-level continuous delivery tool.
Learn MoreMore Posts by Data Geek
