To use different load balancing features, you can add annotations to the YAML file of a Service to configure Classic Load Balancer (CLB) instances, listeners, and backend server groups.
Table of contents
Annotation usage notes
The values of annotations are case-sensitive.
Check the versions of the cloud controller manager (CCM) components supported by this feature before you use the annotations. For more information about how to update the CCM component, see Manage components. For more information about the release notes for the CCM component, see Cloud Controller Manager.
Starting from September 11, 2019, the keyword
alicloud
inannotations
is changed toalibaba-cloud
.Example:
Before the change:
service.beta.kubernetes.io/alicloud-loadbalancer-id
After the change:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id
The system still supports annotations that use the
alicloud
keyword. You do not need to modify the existing annotations.
Common operations to manage CLB instances
Create an Internet-facing CLB instance
apiVersion: v1
kind: Service
metadata:
name: nginx
namespace: default
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
run: nginx
type: LoadBalancer
Create an internal-facing CLB instance
Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type
Description | Default | Supported CCM version |
To create an internal-facing CLB instance, set the value to intranet. Valid values:
| internet | CCM 1.9.3 and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type: "intranet"
name: nginx
namespace: default
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
run: nginx
type: LoadBalancer
Create a CLB instance that uses the specified specifications
The following annotations are used.
Annotation | Description | Default | Supported CCM version |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-charge-type | The metering method of the CLB instance. Valid values:
| paybytraffic | CCM 1.9.3 and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-spec | The specifications of the CLB instance. For more information, see CreateLoadBalancer. You can use this annotation to create a CLB instance that uses the specified specifications or update the specifications of a CLB instance. Important If you use the CLB console to update the specifications of a CLB instance, the Cloud Controller Manager (CCM) may restore the CLB instance to the original specifications. Proceed with caution. You can use the CLB console to modify the specifications of only CLB instances that use the pay-by-specification metering method. | slb.s1.small | CCM 1.9.3 and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-instance-charge-type: "PayBySpec"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-spec: "slb.s1.small"
name: nginx
namespace: default
spec:
ports:
- port: 443
protocol: TCP
targetPort: 443
selector:
run: nginx
type: LoadBalancer
Use an existing CLB instance
Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id
Description | Default | Supported CCM version |
The ID of the CLB instance. You can use this annotation to specify an existing CLB instance.
| None | CCM 1.9.3.81-gca19cd4-aliyun and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id: "${YOUR_LOADBALANCER_ID}" # You can configure only one access control list (ACL).
name: nginx
namespace: default
spec:
ports:
- port: 443
protocol: TCP
targetPort: 443
selector:
run: nginx
type: LoadBalancer
Use an existing CLB instance and forcefully overwrite the listeners of the CLB instance
The following annotations are used. Use the following annotations to forcefully overwrite the listeners of a CLB instance. If a listener port conflict occurs, the original listener is deleted.
Annotation | Description | Default | Supported CCM version |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id | The ID of the CLB instance. You can use this annotation to specify an existing CLB instance.
| ||
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-force-override-listeners | Specifies whether to overwrite the listeners of an existing CLB instance. Valid values:
Important If you use an existing CLB instance and set | "false" | CCM 1.9.3.81-gca19cd4-aliyun and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id: "${YOUR_LOADBALANCER_ID}"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-force-override-listeners: "true"
name: nginx
namespace: default
spec:
ports:
- port: 443
protocol: TCP
targetPort: 443
selector:
run: nginx
type: LoadBalancer
Specify the primary zone and secondary zone when you create a CLB instance
The following annotations are used. After you specify the primary zone and secondary zone for a CLB instance, the zones cannot be changed.
CLB instances in some regions do not support the primary and secondary zones. The actual information on the CLB instance creation page shall prevail.
Annotation | Description | Default | Supported CCM version |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-master-zoneid | The ID of the zone for the primary backend server. | None | CCM 1.9.3.10-gfb99107-aliyun and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-slave-zoneid | The ID of the zone of the secondary backend server. | None | CCM 1.9.3.10-gfb99107-aliyun and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-master-zoneid: "cn-hangzhou-k"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-slave-zoneid: "cn-hangzhou-j"
name: nginx
namespace: default
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
run: nginx
type: LoadBalancer
Create a pay-by-bandwidth CLB instance
The following annotations are used. Both annotations are required.
Annotation | Description | Default | Supported CCM version |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-charge-type | The metering method of the CLB instance. Valid values:
| paybytraffic | CCM 1.9.3 and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-bandwidth | The maximum bandwidth of the CLB instance. This annotation applies only to Internet-facing CLB instances. For more information about the limits on metering methods for Internet-facing CLB instances, see Change the metering method of an Internet-facing CLB instance. | 50 | CCM 1.9.3.10-gfb99107-aliyun and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-charge-type: "paybybandwidth"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-bandwidth: "2"
name: nginx
namespace: default
spec:
ports:
- port: 443
protocol: TCP
targetPort: 443
selector:
run: nginx
type: LoadBalancer
Specify a vSwitch for a CLB instance
The following annotations are used. Both annotations are required.
Annotation | Description | Default | Supported CCM version |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type | To create an internal-facing CLB instance, set the value to intranet. Valid values:
| internet | CCM 1.9.3 and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-vswitch-id | The ID of the vSwitch in which the CLB instance resides. The vSwitch and the cluster must be deployed in the same virtual private cloud (VPC). When you use this annotation, you must set You can log on to the VPC console to query vSwitch IDs. | None | CCM 1.9.3 and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type: "intranet"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-vswitch-id: "${YOUR_VSWITCH_ID}"
name: nginx
namespace: default
spec:
ports:
- port: 443
protocol: TCP
targetPort: 443
selector:
run: nginx
type: LoadBalancer
Specify an IP address for an internal-facing CLB instance
The following annotations are used. These annotations are required.
Annotation | Description | Default | Supported CCM version |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type | To create an internal-facing CLB instance, set the value to intranet. Valid values:
| internet | CCM 1.9.3 and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-vswitch-id | The ID of the vSwitch in which the CLB instance resides. The vSwitch and the cluster must be deployed in the same VPC. When you use this annotation, you must set You can log on to the VPC console to query vSwitch IDs. | None | CCM 1.9.3 and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ip | The IP address of the internal-facing CLB instance.
| None | CCM 2.7.0 and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type: "intranet"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-vswitch-id: "${YOUR_VSWITCH_ID}"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ip: "192.168.x.x"
name: nginx
namespace: default
spec:
type: LoadBalancer
ports:
- port: 80
targetPort: 80
name: http
selector:
app: nginx
Add additional tags to a CLB instance
Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-additional-resource-tags
Description | Default | Supported CCM version |
The tags to be added. Separate multiple tags with commas (,), such as | None | CCM 1.9.3 and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-additional-resource-tags: "Key1=Value1,Key2=Value2"
name: nginx
namespace: default
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
run: nginx
type: LoadBalancer
Create an IPv6 CLB instance
Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ip-version
Description | Default | Supported CCM version |
The IP version of the CLB instance. The IP version cannot be changed after it is specified. To use this annotation, make sure that the kube-proxy mode of the cluster is set to IPVS. Valid values:
| ipv4 | CCM 1.9.3.220-g24b1885-aliyun and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ip-version: "ipv6"
name: nginx
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
app: nginx
type: LoadBalancer
Enable deletion protection for a CLB instance
Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-delete-protection
Description | Default | Supported CCM version |
Specify whether to enable deletion protection for the CLB instance. Valid values:
Important If you manually enable deletion protection in the CLB console for a CLB instance that is created for a LoadBalancer Service, you can run the | on | CCM 1.9.3.313-g748f81e-aliyun and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-delete-protection: "on"
name: nginx
spec:
externalTrafficPolicy: Local
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
app: nginx
type: LoadBalancer
Enable the configuration read-only mode for a CLB instance
Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-modification-protection
Description | Default | Supported CCM version |
Specify whether to enable configuration modification protection for the CLB instance. Valid values:
| ConsoleProtection | CCM 1.9.3.313-g748f81e-aliyun and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-modification-protection: "ConsoleProtection"
name: nginx
spec:
externalTrafficPolicy: Local
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
app: nginx
type: LoadBalancer
Specify the name of a CLB instance
Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-name
Description | Default | Supported CCM version |
The name of the CLB instance. It must be 2 to 128 characters in length, and can contain letters, digits, periods (.), underscores (_), and hyphens (-). It must start with a letter. | None | CCM 1.9.3.313-g748f81e-aliyun and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-name: "your-svc-name"
name: nginx
spec:
externalTrafficPolicy: Local
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
app: nginx
type: LoadBalancer
Specify a resource group to which a CLB instance belongs
Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-resource-group-id
Description | Default | Supported CCM version |
The ID of the resource group to which the CLB instance belongs. The resource group cannot be changed after it is specified. You can log on to the Resource Management console to query resource group IDs. | None | CCM 1.9.3.313-g748f81e-aliyun and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-resource-group-id: "rg-xxxx"
name: nginx
spec:
externalTrafficPolicy: Local
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
app: nginx
type: LoadBalancer
Specifies a hostname for the Service.
The following annotations are used.
Annotation | Description | Default | Supported CCM version |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port | Separate multiple values with commas (,), such as | None | CCM 1.9.3 and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-hostname | The hostname of the Service. The hostname must comply with the naming conventions of domain names. Take note of the following items:
| None | CCM 2.3.0 and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "http:80"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-hostname: "${your_service_hostname}"
name: nginx-svc
namespace: default
spec:
ports:
- name: http
port: 80
protocol: TCP
targetPort: 80
selector:
app: nginx
type: LoadBalancer
Expected output:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx-svc loadBalancer 47.100.XX.XX www.example.com 80:30248/TCP,443:32670/TCP 10s
Create a pay-by-LCU CLB instance
Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-instance-charge-type
Description | Default | Supported CCM version |
The metering method of the CLB instance. Valid values:
Important You cannot change the specification of a pay-by-LCU CLB instance. You cannot specify | PayBySpec | CCM 2.4.0 and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-instance-charge-type: "PayByCLCU"
name: nginx
namespace: default
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
run: nginx
type: LoadBalancer
Common operations to manage listeners
Specify the session persistence period for a TCP-based CLB instance
Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-persistence-timeout
Description | Default | Supported CCM version |
The session persistence period. This annotation applies only to TCP listeners. If a CLB instance has multiple TCP listeners, the configuration applies to all the TCP listeners. Unit: seconds. Valid values: 0 to 3600. Default value: 0. By default, session persistence is disabled. For more information, see CreateLoadBalancerTCPListener. | 0 | CCM 1.9.3 and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-persistence-timeout: "1800"
name: nginx
namespace: default
spec:
ports:
- port: 443
protocol: TCP
targetPort: 443
selector:
run: nginx
type: LoadBalancer
Enable session persistence for an HTTP or HTTPS listener by inserting a cookie
The following annotations are used. The following annotations are required if you insert a cookie to enable session persistence.
The following annotations apply only to HTTP-based and HTTPS-based CLB instances.
If a CLB instance has multiple HTTP or HTTPS listeners, the configuration applies to all the HTTP or HTTPS listeners.
Annotation | Description | Default | Supported CCM version |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session | Specify whether to enable session persistence. This annotation applies only to HTTP and HTTPS listeners. Valid values:
For more information, see CreateLoadBalancerHTTPListener and CreateLoadBalancerHTTPSListener. | off | CCM 1.9.3 and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session-type | The method that is used to process cookies. This annotation applies only to HTTP and HTTPS listeners. This annotation is required when service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session is set to on. Valid values:
For more information, see CreateLoadBalancerHTTPListener and CreateLoadBalancerHTTPSListener. | None | CCM 1.9.3 and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cookie-timeout | The timeout period of the cookie. This annotation is required when service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session is set to on and service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session-type is set to insert. Unit: seconds. Valid values: 1 to 86400. For more information, see CreateLoadBalancerHTTPListener and CreateLoadBalancerHTTPSListener. | None | CCM 1.9.3 and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port | Separate multiple values with commas (,), such as | None | CCM 1.9.3 and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session: "on"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session-type: "insert"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cookie-timeout: "1800"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "http:80"
name: nginx
namespace: default
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
run: nginx
type: LoadBalancer
Configure an ACL for a CLB instance
The following annotations are used. These annotations are required.
Before you use this annotation to create a CLB instance that is configured with access control, you need to create an ACL in the CLB console and record the ID of the ACL.
Annotation | Description | Default | Supported CCM version |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-status | Specify whether to enable access control. Valid values:
| off | CCM 1.9.3.164-g2105d2e-aliyun and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-id | The ID of the ACL that is associated with a listener. This annotation is required when | None | CCM 1.9.3.164-g2105d2e-aliyun and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-type | The type of the ACL. Valid values:
| None | CCM 1.9.3.164-g2105d2e-aliyun and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-status: "on"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-id: "${YOUR_ACL_ID}"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-type: "white"
name: nginx
namespace: default
spec:
ports:
- port: 443
protocol: TCP
targetPort: 443
selector:
run: nginx
type: LoadBalancer
Configure port forwarding for a CLB instance
Port forwarding allows a CLB instance to forward requests from an HTTP port to an HTTPS port.
The following annotations are used. These annotations are required.
Annotation | Description | Default | Supported CCM version |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port | Separate multiple values with commas (,), such as | None | CCM 1.9.3 and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id | The ID of a certificate for the CLB instance. You can log on to the CLB console and view certificate IDs on the Certificates page. Note For more information about how to create a certificate, see Use a certificate from Alibaba Cloud SSL Certificates Service. | None | CCM 1.9.3.164-g2105d2e-aliyun and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-forward-port | Forward requests from an HTTP port to an HTTPS port, such as | None | CCM 1.9.3.164-g2105d2e-aliyun and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "https:443,http:80"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${YOUR_CERT_ID}"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-forward-port: "80:443"
name: nginx
namespace: default
spec:
ports:
- name: https
port: 443
protocol: TCP
targetPort: 80
- name: http
port: 80
protocol: TCP
targetPort: 80
selector:
run: nginx
type: LoadBalancer
Set the scheduling algorithm for a CLB instance
Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-scheduler
Description | Default | Supported CCM version |
The scheduling algorithm. Valid values:
| rr | CCM 1.9.3 and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-scheduler: "wrr"
name: nginx
namespace: default
spec:
ports:
- port: 443
protocol: TCP
targetPort: 443
selector:
run: nginx
type: LoadBalancer
Create a UDP listener
apiVersion: v1
kind: Service
metadata:
name: nginx
namespace: default
spec:
ports:
- port: 80
protocol: UDP
targetPort: 80
selector:
run: nginx
type: LoadBalancer
Create an HTTP listener
Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port
Description | Default | Supported CCM version |
Separate multiple values with commas (,), such as | None | CCM 1.9.3 and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "http:80"
name: nginx
namespace: default
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
run: nginx
type: LoadBalancer
Create an HTTPS listener
The following annotations are used.
The HTTPS listeners of the CLB instance decrypt HTTPS requests into HTTP requests and forward the requests to pods on the backend servers.
Annotation | Description | Default | Supported CCM version |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port | Separate multiple values with commas (,), such as | None | CCM 1.9.3 and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id | The ID of a certificate for the CLB instance. You can log on to the CLB console and view certificate IDs on the Certificates page. Note For more information about how to create a certificate, see Use a certificate from Alibaba Cloud SSL Certificates Service. | None | CCM 1.9.3.164-g2105d2e-aliyun and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "https:443"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${YOUR_CERT_ID}"
name: nginx
namespace: default
spec:
ports:
- port: 443
protocol: TCP
targetPort: 80
selector:
run: nginx
type: LoadBalancer
Create a listener that has health checks enabled
Enable TCP health checks
The following annotations are used. These annotations are required.
By default, health checks are enabled for TCP ports.
Annotation | Description | Default | Supported CCM version |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-switch | Specify whether to enable health checks for a TCP or UDP listener. Valid values:
| on | CCM 2.6.0 and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type | The type of the health check. Valid values:
For more information, see CreateLoadBalancerTCPListener. | tcp | CCM 1.9.3 and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout | The timeout period of a health check. This annotation applies to TCP health checks. If a backend server, such as an ECS instance, does not return a health check response within the specified timeout period, the server fails the health check. Unit: seconds. Valid values: 1 to 300. If the value of | 5 | CCM 1.9.3 and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold | The number of consecutive successful health checks that must occur before an unhealthy backend server is declared healthy. Valid values: 2 to 10. For more information, see CreateLoadBalancerTCPListener. | 3 | CCM 1.9.3 and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold | The number of consecutive failed health checks that must occur before a healthy backend server is declared unhealthy. Valid values: 2 to 10. For more information, see CreateLoadBalancerTCPListener. | 3 | CCM 1.9.3 and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval | The interval at which health checks are performed. Unit: seconds. Valid values: 1 to 50. For more information, see CreateLoadBalancerTCPListener. | 2 | CCM 1.9.3 and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-switch: "on"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type: "tcp"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout: "8"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold: "4"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold: "4"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval: "3"
name: nginx
namespace: default
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
run: nginx
type: LoadBalancer
Enable UDP health checks
The following annotations are used. These annotations are required.
By default, health checks are enabled for UDP ports.
Annotation | Description | Default | Supported CCM version |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-switch | Specify whether to enable health checks for a TCP or UDP listener. Valid values:
| on | CCM 2.6.0 and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout | The timeout period of a health check. This annotation applies to TCP health checks. If a backend server, such as an ECS instance, does not return a health check response within the specified timeout period, the server fails the health check. Unit: seconds. Valid values: 1 to 300. If the value of | 5 | CCM 1.9.3 and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold | The number of consecutive successful health checks that must occur before an unhealthy backend server is declared healthy. Valid values: 2 to 10. For more information, see CreateLoadBalancerTCPListener. | 3 | CCM 1.9.3 and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold | The number of consecutive failed health checks that must occur before a healthy backend server is declared unhealthy. Valid values: 2 to 10. For more information, see CreateLoadBalancerTCPListener. | 3 | CCM 1.9.3 and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval | The interval at which health checks are performed. Unit: seconds. Valid values: 1 to 50. For more information, see CreateLoadBalancerTCPListener. | 2 | CCM 1.9.3 and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-switch: "on"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval: "5"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout: "10"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold: "3"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold: "3"
name: nginx
namespace: default
spec:
ports:
- port: 80
protocol: UDP
targetPort: 80
selector:
run: nginx
type: LoadBalancer
Disable health checks for a TCP or UDP listener
Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-switch
Description | Default | Supported CCM version |
Specify whether to enable health checks for a TCP or UDP listener. Valid values:
| on | CCM 2.6.0 and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-switch: "off" #Disable health checks.
name: nginx
namespace: default
spec:
ports:
- port: 443
protocol: TCP
targetPort: 443
selector:
run: nginx
type: LoadBalancer
Enable HTTP health checks
The following annotations are used.
Annotation | Description | Default | Supported CCM version |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-flag | Valid values:
| off | CCM 1.9.3 and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type | The type of the health check. Valid values:
For more information, see CreateLoadBalancerTCPListener. | tcp | CCM 1.9.3 and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-uri | The Uniform Resource Identifier (URI) that you want to use for health checks. This annotation is not required when the type of the health check is set to TCP. For more information, see CreateLoadBalancerTCPListener. | None | CCM 1.9.3 and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout | The timeout period of a health check. This annotation applies to TCP health checks. If a backend server, such as an ECS instance, does not return a health check response within the specified timeout period, the server fails the health check. Unit: seconds. Valid values: 1 to 300. If the value of | 5 | CCM 1.9.3 and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold | The number of consecutive successful health checks that must occur before an unhealthy backend server is declared healthy. Valid values: 2 to 10. For more information, see CreateLoadBalancerTCPListener. | 3 | CCM 1.9.3 and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold | The number of consecutive failed health checks that must occur before a healthy backend server is declared unhealthy. Valid values: 2 to 10. For more information, see CreateLoadBalancerTCPListener. | 3 | CCM 1.9.3 and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval | The interval at which health checks are performed. Unit: seconds. Valid values: 1 to 50. For more information, see CreateLoadBalancerTCPListener. | 2 | CCM 1.9.3 and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port | Separate multiple values with commas (,), such as | None | CCM 1.9.3 and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-method | The health check method used by HTTP listeners. Valid values:
| None | CCM 2.3.0 and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-flag: "on"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type: "http"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-uri: "/test/index.html"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold: "4"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold: "4"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout: "10"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval: "3"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "http:80"
# Specify the health check method. This annotation is optional.
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-method: "head"
name: nginx
namespace: default
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
run: nginx
type: LoadBalancer
Enable connection draining for a listener.
The following annotations are used. These annotations are required.
Only TCP and UDP are supported.
Annotation | Description | Default | Supported CCM version |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain | Specify whether to enable connection draining. Valid values:
| None | CCM 2.0.1 and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain-timeout | The timeout period of connection draining. Unit: seconds. Valid values: 10 to 900. | None | CCM 2.0.1 and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain: "on"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain-timeout: "30"
name: nginx
namespace: default
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
run: nginx
type: LoadBalancer
Use the X-Forwarded-Proto header to identify the listener protocol of a CLB instance.
The following annotations are used.
Only HTTP and HTTPS are supported.
Annotation | Description | Default | Supported CCM version |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port | Separate multiple values with commas (,), such as | None | CCM 1.9.3 and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-xforwardedfor-proto | Specifies whether to use the X-Forwarded-Proto header to obtain the listener protocol of a CLB instance. Valid values:
| off | CCM 2.1.0 and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "http:80"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-xforwardedfor-proto: "on"
name: nginx
namespace: default
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
run: nginx
type: LoadBalancer
Specify the timeout period of idle connections for a listener
The following annotations are used.
Only HTTP and HTTPS are supported.
Annotation | Description | Default | Supported CCM version |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port | Separate multiple values with commas (,), such as | None | CCM 1.9.3 and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-idle-timeout | Specify the timeout period of idle connections for a listener. Unit: seconds. Valid values: 1 to 60. | 15 | CCM 2.1.0 and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "http:80"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-idle-timeout: "30"
name: nginx
namespace: default
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
run: nginx
type: LoadBalancer
Disable HTTP/2 for a listener.
The following annotations are used.
Only HTTPS is supported.
Annotation | Description | Default | Supported CCM version |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port | Separate multiple values with commas (,), such as | None | CCM 1.9.3 and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id | The ID of a certificate for the CLB instance. You can log on to the CLB console and view certificate IDs on the Certificates page. Note For more information about how to create a certificate, see Use a certificate from Alibaba Cloud SSL Certificates Service. | None | CCM 1.9.3.164-g2105d2e-aliyun and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-http2-enabled | Specify whether to enable HTTP/2. Valid values:
| on | CCM 2.1.0 and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "https:443"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${YOUR_CERT_ID}"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-http2-enabled: "off"
name: nginx
namespace: default
spec:
ports:
- port: 443
protocol: TCP
targetPort: 80
selector:
run: nginx
type: LoadBalancer
Specify the timeout period of requests for a listener.
The following annotations are used.
Only HTTP and HTTPS are supported.
Annotation | Description | Default | Supported CCM version |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port | Separate multiple values with commas (,), such as | None | CCM 1.9.3 and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-request-timeout | The timeout period of requests. Unit: seconds. Valid values: 1 to 180. If the backend server fails to respond within the timeout period, the CLB instance stops waiting and returns an HTTP 504 error code to the client. | 60 | CCM 2.3.0 and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "http:80"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-request-timeout: "60"
name: nginx
namespace: default
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
run: nginx
type: LoadBalancer
Specify the timeout period of connections for a listener.
Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-established-timeout
Only TCP is supported.
Description | Default | Supported CCM version |
The timeout period of connections. Unit: seconds. Valid values: 10 to 900. For more information, see CreateLoadBalancerTCPListener. | None | CCM 2.3.0 and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-established-timeout: "60"
name: nginx
namespace: default
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
run: nginx
type: LoadBalancer
Configure security policies for a listener
The following annotations are used.
Only HTTPS is supported.
Annotation | Description | Default | Supported CCM version |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port | Separate multiple values with commas (,), such as | None | CCM 1.9.3 and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id | The ID of a certificate for the CLB instance. You can log on to the CLB console and view certificate IDs on the Certificates page. Note For more information about how to create a certificate, see Use a certificate from Alibaba Cloud SSL Certificates Service. | None | CCM 1.9.3.164-g2105d2e-aliyun and later |
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-tls-cipher-policy | The TLS security policy. Each security policy contains TLS protocol versions and cipher suites available for HTTPS. For more information, see CreateLoadBalancerHTTPSListener. Valid values:
| tls_cipher_policy_1_0 | CCM 2.4.0 and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "https:443,http:80"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${YOUR_CERT_ID}"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-tls-cipher-policy: "tls_cipher_policy_1_2"
name: nginx
namespace: default
spec:
ports:
- name: https
port: 443
protocol: TCP
targetPort: 443
- name: http
port: 80
protocol: TCP
targetPort: 80
selector:
run: nginx
type: LoadBalancer
Configure both TCP and UDP for a listener
Only clusters whose Kubernetes version is 1.24 and later support this feature. For more information about how to update a cluster, see Update the Kubernetes version of an ACK cluster.
apiVersion: v1
kind: Service
metadata:
name: nginx
namespace: default
spec:
ports:
- name: tcp
port: 80
protocol: TCP
targetPort: 80
- name: udp
port: 80
protocol: UDP
targetPort: 81
selector:
app: nginx
sessionAffinity: None
type: LoadBalancer
Configure Proxy Protocol for TCP and UDP listeners
Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-proxy-protocol
Description | Default | Supported CCM version |
Specify whether to enable Proxy Protocol for a TCP or UDP listener. After Proxy Protocol is configured, client IP addresses can be passed to backend servers. Valid values:
Important Enabling this feature may cause service interruptions. You must stop and update your applications if you to enable Proxy Protocol. Proceed with caution. | off | CCM 2.6.0 and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-proxy-protocol: "on"
name: nginx
namespace: default
spec:
ports:
- port: 443
protocol: TCP
targetPort: 443
selector:
run: nginx
type: LoadBalancer
Common operations to manage backend server groups
Add worker nodes that have specified labels as the backend servers of a CLB instance
Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-backend-label
Description | Default | Supported CCM version |
Specify the labels of worker nodes that you want to add as the backend servers of the CLB instance. Separate multiple labels with commas (,), such as | None | CCM 1.9.3 and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-backend-label: "failure-domain.beta.kubernetes.io/zone=ap-southeast-5a"
name: nginx
namespace: default
spec:
ports:
- port: 443
protocol: TCP
targetPort: 443
selector:
run: nginx
type: LoadBalancer
Add the nodes on which pods run as the backend servers of a CLB instance
Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-scheduler
By default, externalTrafficPolicy
is set to Cluster for a Service. In Cluster mode, all nodes in the cluster are added as the backend servers of the CLB instance. In Local mode, only nodes where the pods are deployed are added as the backend servers of the CLB instance.
Description | Default | Supported CCM version |
The scheduling algorithm. Valid values:
| rr | CCM 1.9.3 and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-scheduler: "wrr"
name: nginx
namespace: default
spec:
externalTrafficPolicy: Local
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
run: nginx
type: LoadBalancer
Remove unschedulable nodes from the backend server group of a CLB instance
Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-remove-unscheduled-backend
Description | Default | Supported CCM version |
Specify whether to remove backend servers in the Unschedulable state from a CLB instance. Valid values:
| off | CCM 1.9.3.164-g2105d2e-aliyun and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-remove-unscheduled-backend: "on"
name: nginx
spec:
externalTrafficPolicy: Local
ports:
- name: http
port: 30080
protocol: TCP
targetPort: 80
selector:
app: nginx
type: LoadBalancer
Add pods that are assigned ENIs as the backend servers of a CLB instance
Annotation: service.beta.kubernetes.io/backend-type
Description | Default | Supported CCM version |
The type of resources that are added as the backend servers of the CLB instance. Valid values:
|
| CCM 1.9.3.164-g2105d2e-aliyun and later |
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/backend-type: "eni"
name: nginx
spec:
ports:
- name: http
port: 30080
protocol: TCP
targetPort: 80
selector:
app: nginx
type: LoadBalancer
Reuse an existing vServer group
Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-vgroup-port
This annotation can be used to reuse a vServer group. This annotation takes effect only when you use an existing CLB instance. For more information, see Use the CCM to deploy services across clusters.
Specify weights for Services to enable weighted round robin
Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-weight
When multiple Services use the same CLB instance, you can use this annotation to specify the percentage of traffic distributed to the current Service. This annotation takes effect only when an existing vServer group is reused. For more information, see Use the CCM to deploy services across clusters.