How to configure horizontal pod autoscaling based on the metrics of ApsaraMQ for RabbitMQ - Container Service for Kubernetes

This topic describes how to configure horizontal pod autoscaling based on the metrics of ApsaraMQ for RabbitMQ and the event-driven autoscaler KEDA.

Prerequisites

The ack-keda component is deployed. For more information, see ACK KEDA.
An ApsaraMQ for RabbitMQ instance is created. For more information, see Step 2: Create resources.

Features

As the business of enterprises develops, the demand for messaging also increases. Message Queue for RabbitMQ is a messaging service that is developed based on highly available distributed storage. Message Queue for RabbitMQ supports Advanced Message Queuing Protocol (AMQP) 0-9-1 and features high concurrency, distributed architecture, and scalability. You can use Message Queue for RabbitMQ and KEDA to scale applications based on the queue length and messaging rate metrics. This helps you handle the fluctuating loads of your applications while improving resource utilization. Message Queue for RabbitMQ supports only AMQP for data transmission. Open source RabbitMQ supports multiple protocols for data transmission.

This topic provides an example on how to use ack-keda to convert the metrics of Message Queue for RabbitMQ to metrics supported by a Horizontal Pod Autoscaler (HPA).

Step 1: Create and deploy a Deployment

Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, click the name of the cluster that you want to manage and choose Workloads > Deployments in the left-side navigation pane.
On the Deployments page, click Create from YAML in the upper-right corner.

On the Create page, set Sample Template to Custom. Create a Deployment named sample-app based on the following content and click Create:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: sample-app
  namespace: default
  labels:
    app: sample-app
spec:
  replicas: 2
  selector:
    matchLabels:
      app: sample-app
  template:
    metadata:
      labels:
        app: sample-app
    spec:
      containers:
      - name: consumer
        image: consumer  # Replace with the actual image used by the consumer application of Message Queue for RabbitMQ. 
        resources:
          limits:
            cpu: "500m"

Step 2: Configure horizontal pod autoscaling based on the metrics of Message Queue for RabbitMQ

Log on to the ApsaraMQ for RabbitMQ console. In the left-side navigation pane, click Instances.
On the Instances page, click the name of the instance that you want to use to go to the Instance Details page. On the Endpoint Information tab, record the endpoint in the Endpoint column of Public Endpoint.
In the left-side navigation pane, click Static Accounts and record the username and password that are used to log on to the instance.
In the left-side navigation pane, click vhosts and record the name of the virtual host (vhost). Example: amq-test.
Run the following command to create a connection string:
```
echo -n "amqp://rabbitmq-username:rabbitmq-password@localhost:5672/vhost" | base64
```
Replace rabbitmq-username and rabbitmq-password with the username and password that you recorded, replace localhost with the public endpoint that you recorded, and replace vhost with the name of the vhost.

Create a Secret based on the following YAML content:

apiVersion: v1
kind: Secret
metadata:
  name: keda-rabbitmq-secret
data:
  host: YW1xWXpNmd4TVRBNEXN0 # The connection string that you created.

Use the following YAML content to create a TriggerAuthentication object:
```
apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
  name: keda-trigger-auth-rabbitmq-conn
  namespace: default
spec:
  secretTargetRef:
    - parameter: host
      name: keda-rabbitmq-secret # keda-rabbitmq-secret is the Secret that you created in the preceding step. 
      key: host
```
Run the following command to deploy the TriggerAuthentication object in the cluster:
```
kubectl apply -f secret.yaml
kubectl apply -f rabbitmq-trigger-auth.yaml
```
After the TriggerAuthentication object is deployed, you can use RabbitMQ triggers in KEDA. You can also reference the TriggerAuthentication object to connect to the Message Queue for RabbitMQ instance to query metrics.

Create a file named ScaledObject.yaml and copy the following content to the file:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: rabbitmq-scaledobject
  namespace: default
spec:
  scaleTargetRef:
    name: rabbitmq-deployment
  maxReplicaCount: 10
  minReplicaCount: 1
  triggers:
  - type: rabbitmq
    metadata:
      protocol: amqp
      queueName: amq-test
      mode: QueueLength
      value: "20"
      metricName: custom-testqueue 
    authenticationRef:
      name: keda-trigger-auth-rabbitmq-conn

Parameter	Description
scaleTargetRef	The object that you want to scale. In this example, the value is set to gsample-appg, which is the name of the Deployment that you created in Step 1: Create and deploy a Deployment.
maxReplicaCount	The maximum number of replicated pods.
minReplicaCount	The minimum number of replicated pods.
protocol	The protocol that is used by ack-keda and the ApsaraMQ for RabbitMQ instance to communicate. Valid values: auto, http, and amqp.
queueName	The name of the queue from which you want to read data.
value	The scale-out threshold.

Run the following command to create a ScaledObject:

// Deploy a ScaledObject. 
kubectl apply -f ScaledObject.yaml   

scaledobject.keda.sh/rabbitmq-scaledobject created

// Query the status of the ScaledObject. 
kubectl get ScaledObject

NAME                    SCALETARGETKIND      SCALETARGETNAME   MIN   MAX   TRIGGERS   AUTHENTICATION                    READY   ACTIVE   FALLBACK   AGE
rabbitmq-scaledobject   apps/v1.Deployment   sample-app        1     10    rabbitmq   keda-trigger-auth-rabbitmq-conn   True    False    False      17s

// Check whether the HPA is deployed to scale the application. 
kubectl get hpa

NAME                             REFERENCE               TARGETS      MINPODS   MAXPODS   REPLICAS   AGE
keda-hpa-rabbitmq-scaledobject   Deployment/sample-app   0/20 (avg)   1         10        2          2m35s

Step 3: Produce and consume data to trigger application scaling

The following code is used to produce and consume data:

)package main

import (
	"fmt"
	"log"
	"time"
	"github.com/streadway/amqp"
)

const (
	queueName = "queue-test"    // Replace with the name of the queue from which you want to read data. 
	numMsgs   = 10000
	pauseTime = 10 * time.Millisecond
  url = "amqp://Mjpt****:QT****@amqp-cn-zxux009.cn-beijing.amqp-0.net.mq.amqp.aliyuncs.com:5672/amq-test"   // The URL of the Message Queue for RabbitMQ instance. The URL must be in the amqp://guest:password@localhost:5672/vhost format.
)

func main() {
	conn, err := amqp.Dial(url)
	failOnError(err, "Failed to connect to RabbitMQ")
	defer conn.Close()

	ch, err := conn.Channel()
	failOnError(err, "Failed to open a channel")
	defer ch.Close()

	q, err := ch.QueueDeclare(
		queueName,
		true,
		false,
		false,
		false,
		nil,
	)
	failOnError(err, "Failed to declare a queue")
	go produce(ch, q)
	select {}
}

func produce(ch *amqp.Channel, q amqp.Queue) {
	for i := 0; i < numMsgs; i++ {
		msg := fmt.Sprintf("Message %d", i)
		err := ch.Publish(
			"",
			q.Name,
			false,
			false,
			amqp.Publishing{
				ContentType: "text/plain",
				Body:        []byte(msg),
			},
		)
		failOnError(err, "Failed to publish a message")
		log.Printf("Successed to publish a message: %s", msg)
		time.Sleep(pauseTime)
	}
}

func failOnError(err error, msg string) {
	if err != nil {
		log.Fatalf("%s: %s", msg, err)
	}
}

Run a producer program to produce data and then run the following command to query information about the HPA:
```
kubectl get hpa
```
Expected output
```
NAME                               REFERENCE               TARGETS           MINPODS   MAXPODS   REPLICAS   AGE
keda-hpa-rabbitmq-scaledobject   Deployment/sample-app   443000m/20 (avg)   1         10        10         9m15s
```
The output shows that the number of replicated pods for the sample-app application is scaled to the maximum value specified in KEDA.

Stop the producer program and run a consumer program. Then, run the following command:

kubectl get hpa -w

Expected output

NAME                               REFERENCE               TARGETS            MINPODS   MAXPODS   REPLICAS   AGE
keda-hpa-rabbitmq-scaledobject   Deployment/sample-app   443000m/20 (avg)   1         10        10         9m15s
keda-hpa-rabbitmq-scaledobject   Deployment/sample-app   235000m/20 (avg)   1         10        10         9m51s
keda-hpa-rabbitmq-scaledobject   Deployment/sample-app   0/20 (avg)         1         10        10         10m
keda-hpa-rabbitmq-scaledobject   Deployment/sample-app   0/20 (avg)         1         10        1          15m

The output shows that the number of replicated pods for the sample-app application is scaled to the minimum value specified in KEDA at a point in time after data consumption ends.

References

For more information about how to configure horizontal pod autoscaling based on the metrics of ApsaraMQ for RocketMQ and the event-driven autoscaler KEDA, see Horizontal pod autoscaling based on the metrics of ApsaraMQ for RocketMQ.