Manage Alibaba Cloud Elasticsearch with Terraform - Elasticsearch

Terraform lets you define infrastructure as code to create and manage Elasticsearch clusters in a repeatable, version-controlled way. This tutorial creates an Alibaba Cloud Elasticsearch cluster and its required network resources using a single Terraform configuration file.

Run the sample code with a few clicks in Terraform Explorer.

Prerequisites

A RAM user with minimum permissions. Use a RAM user that has only the required permissions to reduce the risk of leaking the AccessKey pair of your Alibaba Cloud account. For more information, see Create a RAM user and Grant permissions to a RAM user. Attach the following custom policy to the RAM user:

  {
    "Version": "1",
    "Statement": [
      {
        "Effect": "Allow",
        "Action": [
          "vpc:CreateVpc",
          "vpc:DeleteVpc",
          "vpc:CreateVSwitch",
          "vpc:DeleteVSwitch",
          "ecs:CreateSecurityGroup",
          "ecs:ModifySecurityGroupPolicy",
          "ecs:DescribeSecurityGroups",
          "ecs:ListTagResources",
          "ecs:DeleteSecurityGroup",
          "ecs:DescribeSecurityGroupAttribute"
        ],
        "Resource": "*"
      },
      {
        "Effect": "Allow",
        "Action": [
          "vpc:DescribeVpcAttribute",
          "vpc:DescribeRouteTableList",
          "vpc:DescribeVSwitchAttributes"
        ],
        "Resource": "*"
      },
      {
        "Effect": "Allow",
        "Action": [
          "elasticsearch:CreateInstance",
          "elasticsearch:DescribeInstance",
          "elasticsearch:ListAckClusters",
          "elasticsearch:UpdateDescription",
          "elasticsearch:ListInstance",
          "elasticsearch:ListAvailableEsInstanceIds"
        ],
        "Resource": "*"
      }
    ]
  }

A Terraform runtime environment set up using one of the following methods:
- Terraform Explorer (recommended): An online Terraform environment provided by Alibaba Cloud. No installation required. Log on to Terraform Explorer to get started.
- Cloud Shell: Terraform is preinstalled with identity credentials already configured. Run Terraform commands directly.
- On-premises machine: Install and configure Terraform locally. Suitable for environments with specific network requirements or custom development setups.

Important

This tutorial creates billable resources. Release or unsubscribe from the resources when they are no longer needed.

Resources created

Resource	Terraform resource type	Description
Virtual private cloud (VPC)	alicloud_vpc	Network isolation for the Elasticsearch cluster
Security group	alicloud_security_group	Network access control rules
vSwitch	alicloud_vswitch	Subnet within the VPC
Elasticsearch cluster	alicloud_elasticsearch_instance	Managed Alibaba Cloud Elasticsearch cluster (billable). See Billing overview.

Step 1: Write the Terraform configuration

Create a working directory and a configuration file named main.tf in the directory.

Copy the following code into the main.tf file. This configuration creates a VPC, a security group, a vSwitch, and an Elasticsearch cluster. The following table describes the Elasticsearch cluster parameters:

Important

Do not hardcode AccessKey credentials in the .tf file. Use environment variables (ALIBABA_CLOUD_ACCESS_KEY_ID and ALIBABA_CLOUD_ACCESS_KEY_SECRET) or configure credentials through the Terraform provider documentation. If you commit this file to version control, credentials in the file risk being leaked.

Parameter	Value	Description
`instance_charge_type`	`PostPaid`	Pay-as-you-go billing method.
`data_node_amount`	`2`	Number of data nodes in the cluster.
`data_node_spec`	`elasticsearch.sn2ne.large`	Instance type for data nodes.
`data_node_disk_size`	`20`	Disk size for each data node, in GB.
`data_node_disk_type`	`cloud_ssd`	Disk type for data nodes.
`data_node_disk_performance_level`	`PL1`	Performance level for cloud_ssd disks.
`version`	`6.7_with_X-Pack`	Elasticsearch version with X-Pack enabled.
`zone_count`	`1`	Number of zones for cluster deployment.
`master_node_spec`	`elasticsearch.sn2ne.large`	Instance type for dedicated master nodes.
`kibana_node_spec`	`elasticsearch.sn2ne.large`	Instance type for the Kibana node.

   # ------------------------------------
   # Variables
   # ------------------------------------
   variable "region" {
     default = "cn-qingdao"
   }

   variable "vpc_cidr_block" {
     default = "172.16.0.0/16"
   }

   variable "vsw_cidr_block" {
     default = "172.16.0.0/24"
   }

   variable "node_spec" {
     default = "elasticsearch.sn2ne.large"
   }

   # ------------------------------------
   # Provider and data sources
   # ------------------------------------
   provider "alicloud" {
     region = var.region
   }

   # Query available zones that support vSwitch creation and cloud_ssd disks
   data "alicloud_zones" "default" {
     available_resource_creation = "VSwitch"
     available_disk_category     = "cloud_ssd"
   }

   # Generate a random suffix to avoid resource name conflicts
   resource "random_integer" "default" {
     min = 10000
     max = 99999
   }

   # ------------------------------------
   # Network resources
   # ------------------------------------

   # Create a VPC
   resource "alicloud_vpc" "vpc" {
     vpc_name   = "vpc-test_${random_integer.default.result}"
     cidr_block = var.vpc_cidr_block
   }

   # Create a security group in the VPC
   resource "alicloud_security_group" "group" {
     name   = "test_${random_integer.default.result}"
     vpc_id = alicloud_vpc.vpc.id
   }

   # Create a vSwitch in the first available zone
   resource "alicloud_vswitch" "vswitch" {
     vpc_id       = alicloud_vpc.vpc.id
     cidr_block   = var.vsw_cidr_block
     zone_id      = data.alicloud_zones.default.zones[0].id
     vswitch_name = "vswitch-test-${random_integer.default.result}"
   }

   # ------------------------------------
   # Elasticsearch cluster
   # ------------------------------------
   resource "alicloud_elasticsearch_instance" "instance" {
     description          = "test_Instance"
     instance_charge_type = "PostPaid"                   # Pay-as-you-go billing
     data_node_amount     = "2"                          # Number of data nodes
     data_node_spec       = var.node_spec                # Data node instance type
     data_node_disk_size  = "20"                         # Data node disk size in GB
     data_node_disk_type  = "cloud_ssd"                  # Data node disk type
     vswitch_id           = alicloud_vswitch.vswitch.id  # Deploy in the created vSwitch
     password             = "es_password_01"             # Cluster access password
     version              = "6.7_with_X-Pack"            # Elasticsearch version with X-Pack
     master_node_spec     = var.node_spec                # Dedicated master node instance type
     zone_count           = "1"                          # Single-zone deployment
     master_node_disk_type = "cloud_ssd"                 # Master node disk type
     kibana_node_spec     = var.node_spec                # Kibana node instance type
     data_node_disk_performance_level = "PL1"            # SSD performance level

     tags = {
       Created = "TF",
       For     = "example",
     }
   }

Step 2: Initialize Terraform

Run the following command in the working directory to initialize the Terraform runtime environment:

terraform init

If the following output appears, Terraform is initialized:

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

Step 3: Create the Elasticsearch cluster

Run the following command to create all resources defined in the configuration:

terraform apply

When prompted, enter yes and press the Enter key. Cluster creation typically takes 10 to 30 minutes.

If the following output appears, all resources are created:

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes


Apply complete!  Resources: 5 added, 0 changed, 0 destroyed.

Step 4: Verify the result

Run the terraform show command

Run the following command in the working directory to view the details of the created resources:

terraform show

Log on to the Elasticsearch console

Log on to the Elasticsearch console and view the created cluster.

Step 5: Release resources

When the resources are no longer needed, run the following command to release all resources created by Terraform. For more information about the terraform destroy command, see Common commands.

terraform destroy

When prompted, enter yes and press the Enter key to confirm.

Complete sample code

Run the sample code with a few clicks in Terraform Explorer.

Sample code

variable "region" {
  default = "cn-qingdao"
}

data "alicloud_zones" "default" {
  available_resource_creation = "VSwitch"
  available_disk_category     = "cloud_ssd"
}

variable "vpc_cidr_block" {
  default = "172.16.0.0/16"
}

variable "vsw_cidr_block" {
  default = "172.16.0.0/24"
}

variable "node_spec" {
  default = "elasticsearch.sn2ne.large"
}
provider "alicloud" {
  region = var.region
}

resource "random_integer" "default" {
  min = 10000
  max = 99999
}

resource "alicloud_vpc" "vpc" {
  vpc_name   = "vpc-test_${random_integer.default.result}"
  cidr_block = var.vpc_cidr_block
}

resource "alicloud_security_group" "group" {
  name   = "test_${random_integer.default.result}"
  vpc_id = alicloud_vpc.vpc.id
}

resource "alicloud_vswitch" "vswitch" {
  vpc_id       = alicloud_vpc.vpc.id
  cidr_block   = var.vsw_cidr_block
  zone_id      = data.alicloud_zones.default.zones[0].id
  vswitch_name = "vswitch-test-${random_integer.default.result}"
}

resource "alicloud_elasticsearch_instance" "instance" {
  description           = "test_Instance"
  instance_charge_type  = "PostPaid"
  data_node_amount      = "2"
  data_node_spec        = var.node_spec
  data_node_disk_size   = "20"
  data_node_disk_type   = "cloud_ssd"
  vswitch_id            = alicloud_vswitch.vswitch.id
  password              = "es_password_01"
  version               = "6.7_with_X-Pack"
  master_node_spec      = var.node_spec
  zone_count            = "1"
  master_node_disk_type = "cloud_ssd"
  kibana_node_spec      = var.node_spec
  data_node_disk_performance_level = "PL1"
  tags = {
    Created = "TF",
    For     = "example",
  }
}

For more complete examples, visit the Terraform landing zone quickstarts on GitHub.