Create and manage a consumer group using Terraform - Terraform

Create, query, and delete consumer groups in ApsaraMQ for Kafka by using the alicloud_alikafka_consumer_group Terraform resource.

Note

Run the sample code in this topic in Terraform Explorer.

Before you begin

Before you begin, ensure that you have:

An Alibaba Cloud account or a Resource Access Management (RAM) user with an AccessKey pair. For security, use a RAM user instead of your Alibaba Cloud account. For more information, see Create a RAM user and Create an AccessKey pair

The following RAM permissions granted to the RAM user. For more information, see Grant permissions to RAM users

  {
      "Version": "1",
      "Statement": [
          {
              "Effect": "Allow",
              "Action": [
                  "vpc:DescribeVpcAttribute",
                  "vpc:DescribeRouteTableList",
                  "vpc:DescribeVSwitchAttributes",
                  "vpc:DeleteVpc",
                  "vpc:DeleteVSwitch",
                  "vpc:CreateVpc",
                  "vpc:CreateVSwitch"
              ],
              "Resource": "*"
          },
          {
              "Effect": "Allow",
              "Action": "bss:ModifyAgreementRecord",
              "Resource": "*"
          },
          {
              "Effect": "Allow",
              "Action": [
                  "bss:DescribeOrderList",
                  "bss:DescribeOrderDetail",
                  "bss:PayOrder",
                  "bss:CancelOrder"
              ],
              "Resource": "*"
          },
          {
              "Effect": "Allow",
              "Action": [
                  "ecs:CreateSecurityGroup",
                  "ecs:ModifySecurityGroupPolicy",
                  "ecs:DescribeSecurityGroups",
                  "ecs:ListTagResources",
                  "ecs:DeleteSecurityGroup",
                  "ecs:DescribeSecurityGroupAttribute",
                  "ecs:AuthorizeSecurityGroup",
                  "ecs:RevokeSecurityGroup"
              ],
              "Resource": "*"
          },
          {
              "Action": "alikafka:*",
              "Resource": "*",
              "Effect": "Allow"
          },
          {
              "Action": "ram:CreateServiceLinkedRole",
              "Resource": "*",
              "Effect": "Allow",
              "Condition": {
                  "StringEquals": {
                      "ram:ServiceName": [
                          "connector.alikafka.aliyuncs.com",
                          "instanceencryption.alikafka.aliyuncs.com",
                          "alikafka.aliyuncs.com",
                          "etl.alikafka.aliyuncs.com"
                      ]
                  }
              }
          }
      ]
  }

A Terraform runtime environment set up by using one of the following methods:
- Terraform Explorer: An online environment provided by Alibaba Cloud. No installation required.
- Cloud Shell: Terraform is preinstalled with identity credentials already configured.
- Local installation: Install and configure Terraform on your on-premises machine. Suitable for custom development environments or unstable network connections.

Required resources

Note

Some resources in this topic incur costs. Release or unsubscribe from resources that you no longer need.

This topic uses the following Terraform resources:

alicloud_vpc: Creates a virtual private cloud (VPC).
alicloud_vswitch: Creates a vSwitch in a VPC.
alicloud_security_group: Creates a security group.
alicloud_alikafka_instance: Creates an ApsaraMQ for Kafka instance.
alicloud_alikafka_consumer_group: Creates a consumer group.
alicloud_alikafka_consumer_groups: Queries consumer groups (data source).

Create a consumer group

This example creates a consumer group named tf-example in the China (Shenzhen) region. The consumer group name cannot be changed after creation.

Step 1: Write the configuration

Create a working directory and a configuration file named main.tf. Add the following code to create the prerequisite resources and the consumer group.

Prerequisite resources

variable "region" {
  default = "cn-shenzhen"
}

variable "instance_name" {
  default = "alikafkaInstanceName"
}

variable "zone_id" {
  default = "cn-shenzhen-f"
}

provider "alicloud" {
  region = var.region
}

# Create a VPC.
resource "alicloud_vpc" "default" {
  vpc_name   = "alicloud"
  cidr_block = "172.16.0.0/16"
}

# Create a vSwitch.
resource "alicloud_vswitch" "default" {
  vpc_id     = alicloud_vpc.default.id
  cidr_block = "172.16.192.0/20"
  zone_id    = var.zone_id
}

# Create a security group.
resource "alicloud_security_group" "default" {
  vpc_id = alicloud_vpc.default.id
}

# Create an ApsaraMQ for Kafka instance.
# Disk type: ultra disk (0). Disk size: 500 GB. Traffic specification: alikafka.hw.2xlarge.
resource "alicloud_alikafka_instance" "default" {
  name           = var.instance_name
  partition_num  = 50
  disk_type      = 0
  disk_size      = 500
  deploy_type    = 5
  io_max_spec    = "alikafka.hw.2xlarge"
  vswitch_id     = alicloud_vswitch.default.id
  security_group = alicloud_security_group.default.id
}

Consumer group resource

Add the following alicloud_alikafka_consumer_group resource block to main.tf:

variable "group_name" {
  default = "tf-example"
}

resource "alicloud_alikafka_consumer_group" "default" {
  consumer_id = var.group_name
  instance_id = alicloud_alikafka_instance.default.id
}

The following table describes the arguments for alicloud_alikafka_consumer_group:

Argument	Required	Description
`consumer_id`	Yes	The name of the consumer group. Cannot be changed after creation.
`instance_id`	Yes	The ID of the ApsaraMQ for Kafka instance.
`remark`	No	The description of the consumer group. Available since provider v1.268.0. Replaces the deprecated `description` argument.

Step 2: Initialize and apply

Initialize the Terraform working directory: Expected output:

   terraform init

   Initializing the backend...

   Initializing provider plugins...
   - Checking for available provider plugins...
   - Downloading plugin for provider "alicloud" (hashicorp/alicloud) 1.90.1...
   ...

   You may now begin working with Terraform. Try running "terraform plan" to see
   any changes that are required for your infrastructure. All Terraform commands
   should now work.

   If you ever set or change modules or backend configuration for Terraform,
   rerun this command to reinitialize your working directory. If you forget, other

Preview the changes:
```
   terraform plan
```

Create the resources: When prompted, enter yes and press Enter. Expected output:

   terraform apply

   alicloud_vpc.default: Refreshing state... [id=vpc-****]
   alicloud_security_group.default: Refreshing state... [id=sg-****]
   alicloud_vswitch.default: Refreshing state... [id=vsw-****]
   alicloud_alikafka_instance.default: Refreshing state... [id=alikafka_post-cn-****]

   Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the
   following symbols:
     + create

   Terraform will perform the following actions:

     # alicloud_alikafka_consumer_group.default will be created
     + resource "alicloud_alikafka_consumer_group" "default" {
         + consumer_id = "tf-example"
         + id          = (known after apply)
         + instance_id = "alikafka_post-cn-****"
       }

   Plan: 1 to add, 0 to change, 0 to destroy.

   Do you want to perform these actions?
     Terraform will perform the actions described above.
     Only 'yes' will be accepted to approve.

     Enter a value: yes

   alicloud_alikafka_consumer_group.default: Creating...
   alicloud_alikafka_consumer_group.default: Creation complete after 2s [id=alikafka_post-cn-****:tf-example]

   Apply complete!  Resources: 1 added, 0 changed, 0 destroyed.

Verify the result

Use `terraform show`

Run terraform show to view the consumer group details:

terraform show

Expected output:

# alicloud_alikafka_consumer_group.default:
resource "alicloud_alikafka_consumer_group" "default" {
    consumer_id = "tf-example"
    description = null
    id          = "alikafka_post-cn-****:tf-example"
    instance_id = "alikafka_post-cn-****"
}

Use the ApsaraMQ for Kafka console

Log on to the ApsaraMQ for Kafka console to view the consumer group.

Query consumer groups

Use the alicloud_alikafka_consumer_groups data source to query existing consumer groups.

Step 1: Add the data source

Add the following data source block to main.tf:

data "alicloud_alikafka_consumer_groups" "consumer_groups_ds" {
  instance_id = alicloud_alikafka_instance.default.id
}

The following table describes the arguments for alicloud_alikafka_consumer_groups:

Argument	Required	Description
`instance_id`	Yes	The ID of the ApsaraMQ for Kafka instance to query.

Step 2: Apply and verify

Preview the changes:
```
   terraform plan
```

Apply the configuration: When prompted, enter yes and press Enter. Expected output:

   terraform apply

   alicloud_vpc.default: Refreshing state... [id=vpc-****]
   alicloud_security_group.default: Refreshing state... [id=sg-****]
   alicloud_vswitch.default: Refreshing state... [id=vsw-****]
   alicloud_alikafka_instance.default: Refreshing state... [id=alikafka_post-cn-****]
   alicloud_alikafka_consumer_group.default: Refreshing state... [id=alikafka_post-cn-****:tf-example]
   data.alicloud_alikafka_consumer_groups.consumer_groups_ds: Reading...
   data.alicloud_alikafka_consumer_groups.consumer_groups_ds: Read complete after 1s [id=****]

   No changes. Your infrastructure matches the configuration.

   Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are
   needed.

   Apply complete!  Resources: 0 added, 0 changed, 0 destroyed.

Run terraform show to view the query results: Expected output:

   terraform show

   # data.alicloud_alikafka_consumer_groups.consumer_groups_ds:
   data "alicloud_alikafka_consumer_groups" "consumer_groups_ds" {
       groups      = [
           {
               consumer_id = "tf-example"
               id          = "alikafka_post-cn-****:tf-example"
               instance_id = "alikafka_post-cn-****"
               remark      = null
               tags        = {}
           },
       ]
       id          = "****"
       ids         = [
           "alikafka_post-cn-****:tf-example",
       ]
       instance_id = "alikafka_post-cn-****"
       names       = [
           "tf-example",
       ]
   }

Clean up resources

Release all resources created in this topic. For more information about terraform destroy, see Common commands.

terraform destroy

Complete sample code

Note

Run the sample code in Terraform Explorer.

Sample code

variable "region" {
  default = "cn-shenzhen"
}

variable "instance_name" {
  default = "alikafkaInstanceName"
}

variable "zone_id" {
  default = "cn-shenzhen-f"
}

variable "group_name" {
  default = "tf-example"
}

provider "alicloud" {
  region = var.region
}

# Create a VPC.
resource "alicloud_vpc" "default" {
  vpc_name   = "alicloud"
  cidr_block = "172.16.0.0/16"
}

# Create a vSwitch.
resource "alicloud_vswitch" "default" {
  vpc_id     = alicloud_vpc.default.id
  cidr_block = "172.16.192.0/20"
  zone_id    = var.zone_id
}

# Create a security group.
resource "alicloud_security_group" "default" {
  vpc_id = alicloud_vpc.default.id
}

# Create an ApsaraMQ for Kafka instance.
# Disk type: ultra disk (0). Disk size: 500 GB. Traffic specification: alikafka.hw.2xlarge.
resource "alicloud_alikafka_instance" "default" {
  name           = var.instance_name
  partition_num  = 50
  disk_type      = 0
  disk_size      = 500
  deploy_type    = 5
  io_max_spec    = "alikafka.hw.2xlarge"
  vswitch_id     = alicloud_vswitch.default.id
  security_group = alicloud_security_group.default.id
}

resource "alicloud_alikafka_consumer_group" "default" {
  consumer_id = var.group_name
  instance_id = alicloud_alikafka_instance.default.id
}

data "alicloud_alikafka_consumer_groups" "consumer_groups_ds" {
  instance_id = alicloud_alikafka_instance.default.id
  depends_on  = [alicloud_alikafka_consumer_group.default]
}

References

Manage consumer groups in the ApsaraMQ for Kafka console or by calling API operations. For more information, see Step 3: Create resources and CreateConsumerGroup.
For information about other arguments provided by the alicloud_alikafka_consumer_group resource, see alicloud_alikafka_consumer_group.

Before you begin

Required resources

Create a consumer group

Step 1: Write the configuration

Prerequisite resources

Consumer group resource

Step 2: Initialize and apply

Verify the result

Use terraform show

Use the ApsaraMQ for Kafka console

Query consumer groups

Step 1: Add the data source

Step 2: Apply and verify

Clean up resources

Complete sample code

Sample code

References

Use `terraform show`