As businesses grow, so does the volume of their data and the necessity for scalable, reliable, and efficient data storage and search solutions. Elasticsearch has long been the go-to open-source search and analytics engine for handling large volumes of data in near real-time. However, managing an Elasticsearch cluster on-premises or through an IaaS provider might not always align with the demands of flexibility and ease of use. This is where Alibaba Cloud Elasticsearch comes in, providing an easy escalator to the cloud without service interruption.
In this tutorial, we'll walk through the steps of using Alibaba Cloud's Object Storage Service (OSS) to migrate data from a self-managed Elasticsearch cluster to an Alibaba Cloud Elasticsearch cluster.
1）A self-managed Elasticsearch cluster.
2）An OSS bucket in the same region as your Elasticsearch cluster.
3）An Alibaba Cloud Elasticsearch cluster.
For the sake of illustration, let's assume we work with Elasticsearch 6.7.0. For more details on setting up these components, you can check out Alibaba Cloud Elasticsearch's official documentation here
Follow these steps to move your data over to the more robust and scalable Alibaba Cloud Elasticsearch service.
Ensure you have the self-managed Elasticsearch cluster ready, an OSS bucket created, and an Alibaba Cloud Elasticsearch cluster set up.
1- Deploy your self-managed cluster on Alibaba Cloud ECS for optimal compatibility. More details [here](https://www.alibabacloud.com/help/en/ecs/index.html).
2- Activate OSS and create a bucket in the appropriate region. [Activate OSS](https://www.alibabacloud.com/help/en/oss/).
3- Create your Alibaba Cloud Elasticsearch cluster within the same region as your OSS bucket. [Create a cluster](https://www.alibabacloud.com/en/product/elasticsearch).This plugin is required to integrate Elasticsearch with OSS and must be installed on all nodes of the self-managed cluster.
1$ wget https://github.com/aliyun/elasticsearch-repository-oss/releases/download/v6.7.0/elasticsearch-repository-oss-6.7.0.zip
2$ sudo unzip -d /usr/local/elasticsearch-6.7.0/plugins/elasticsearch-repository-oss elasticsearch-repository-oss-6.7.0.zip
3$ sudo /usr/local/elasticsearch-6.7.0/bin/elasticsearch-plugin install file:///usr/local/elasticsearch-repository-oss-6.7.0.zip
4$ cd /usr/local/elasticsearch-6.7.0
5$ ./bin/elasticsearch -dConnect to your cluster and execute the following command to establish a repository:
1$ sudo curl -H "Content-Type: application/json" -XPUT localhost:9200/_snapshot/my_backup -d'
2{
3 "type": "oss",
4 "settings": {
5 "endpoint": "http://oss-cn-hangzhou-internal.aliyuncs.com",
6 "access_key_id": "yourAccessKeyId",
7 "secret_access_key":"yourAccessKeySecret",
8 "bucket": "yourBucketName",
9 "compress": true
10 }
11}'Now, create a snapshot of the indexes you intend to migrate.
1$ curl -H "Content-Type: application/json" -XPUT localhost:9200/_snapshot/my_backup/snapshot_1?pretty -d'
2{
3 "indices": "index1,index2"
4}'Log into Kibana and set up an OSS snapshot repository similar to that for the self-managed cluster.
1- Kibana Console
2- Dev Tools1PUT _snapshot/my_backup
2{
3 "type": "oss",
4 "settings": {
5 "endpoint": "oss-cn-hangzhou-internal.aliyuncs.com",
6 "access_key_id": "yourAccessKeyId",
7 "secret_access_key": "yourAccessKeySecret",
8 "bucket": "yourBucketName",
9 "compress": true
10 }
11}1POST _snapshot/my_backup/snapshot_1/_restore
2{
3 "indices": "*,-.monitoring*,-.security_audit*",
4 "ignore_unavailable": "true"
5}View the indexes and search the data to ensure everything is as expected.
1GET /_cat/indices?v
2GET /index1/_search
1{
2 "took" : 2,
3 "timed_out" : false,
4 "_shards" : {
5 "total" : 5,
6 "successful" : 5,
7 "skipped" : 0,
8 "failed" : 0
9 },
10 "hits" : {
11 "total" : 1,
12 "max_score" : 1.0,
13 "hits" : [
14 {
15 "_index" : "index1",
16 "_type" : "doc",
17 "_id" : "1",
18 "_score" : 1.0,
19 "_source" : {
20 "productName" : "testpro",
21 "annual_rate" : "3.22%",
22 "describe" : "testpro description"
23 }
24 }
25 ]
26 }
27}Now that you've figured out how to transfer your data from a self-managed Elasticsearch cluster to Alibaba Cloud Elasticsearch, you can leverage the scalable storage, enhanced security, and a suite of tools provided by Alibaba Cloud.
For further insights or to clarify any doubts, Alibaba Cloud's extensive documentation and community forums provide a resource trove tailored for developers and enterprise solutions.
Ready to start your journey with Elasticsearch on Alibaba Cloud? Explore our tailored Cloud solutions and services to take the first step towards transforming your data into a visual masterpiece.
Embark on Your 30-Day Free Trial, and see for yourself the powerful features that Alibaba Cloud Elasticsearch has to offer! Click here to begin your cloud Elasticsearch adventure!
Bolstering Your Cluster Stability with Alibaba Cloud Elasticsearch Aliyun-qos Plugin
Migrate Your Elasticsearch Data to Alibaba Cloud Using Logstash
Data Geek - May 9, 2024
Data Geek - May 9, 2024
Data Geek - May 9, 2024
Data Geek - April 24, 2024
Data Geek - May 15, 2024
Data Geek - August 27, 2024
Alibaba Cloud Elasticsearch
Alibaba Cloud Elasticsearch helps users easy to build AI-powered search applications seamlessly integrated with large language models, and featuring for the enterprise: robust access control, security monitoring, and automatic updates.
Learn More
Cloud Migration Solution
Secure and easy solutions for moving you workloads to the cloud
Learn More
ISV Solutions for Cloud Migration
Alibaba Cloud offers Independent Software Vendors (ISVs) the optimal cloud migration solutions to ready your cloud business with the shortest path.
Learn More
Oracle Database Migration Solution
Migrate your legacy Oracle databases to Alibaba Cloud to save on long-term costs and take advantage of improved scalability, reliability, robust security, high performance, and cloud-native features.
Learn MoreMore Posts by Data Geek
