All Products
Search

This Product

    Container Service for Kubernetes:Use annotations to configure CLB instances

    Document Center

    Container Service for Kubernetes:Use annotations to configure CLB instances

    Last Updated:Aug 21, 2024

    To use various load balancing features, you can add annotations to the YAML file of a Service to configure Classic Load Balancer (CLB) instances, listeners, and backend server groups.

    Table of contents

    Category

    Hyperlink

    Annotation usage notes

    Common operations to manage CLB instances

    Common operations to manage listeners

    Common operations to manage backend server groups

    References

    Annotation usage notes

    • The values of annotations are case-sensitive.

    • Check the versions of the cloud controller manager (CCM) component supported by this feature before you use the annotations. For more information about how to update the CCM component, see Manage components. For more information about the release notes for the CCM component, see Cloud Controller Manager.

    • Starting from September 11, 2019, the keyword alicloud in annotations is changed to alibaba-cloud.

      Example:

      Before the change: service.beta.kubernetes.io/alicloud-loadbalancer-id

      After the change: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id

      The system still supports annotations that use the alicloud keyword. You do not need to modify the existing annotations.

    Common operations to manage CLB instances

    Create an Internet-facing CLB instance

    apiVersion: v1
kind: Service
metadata:
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

    Create an internal-facing CLB instance

    Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type

    Description

    Default value

    Supported CCM version

    Creates an internal-facing CLB instance. Valid values:

    • internet: uses an Internet-facing CLB instance to expose the Service. This is the default value. The address type of the CLB instance must be set to Internet.

    • intranet: uses an internal-facing CLB instance to expose the Service. The address type of the CLB instance must be set to intranet.

    internet

    CCM 1.9.3 and later

    apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type: "intranet"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

    Create a CLB instance that uses the specified specifications

    The following annotations are used.

    Annotation

    Description

    Default value

    Supported CCM version

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-charge-type

    The billing method of the CLB instance. Valid values:

    • paybytraffic

    • paybybandwidth

    paybytraffic

    CCM 1.9.3 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-spec

    The specifications of the CLB instance. For more information, see CreateLoadBalancer. You can use this annotation to create a CLB instance that uses the specified specifications or update the specifications of a CLB instance.

    Important

    If you use the CLB console to update the specifications of a CLB instance, the CCM may restore the CLB instance to the original specifications. Proceed with caution. You can use the CLB console to modify the specifications of only CLB instances that use the pay-by-specification billing method.

    slb.s1.small

    CCM 1.9.3 and later

    apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-instance-charge-type: "PayBySpec"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-spec: "slb.s1.small"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 443
  selector:
    run: nginx
  type: LoadBalancer

    Use an existing CLB instance

    Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id

    Description

    Default value

    Supported CCM version

    The ID of the CLB instance that you want to use. You can use this annotation to specify an existing CLB instance.

    • By default, if you use an existing CLB instance, the listeners of the CLB instance are not overwritten. To overwrite the existing listeners, set the service.beta.kubernetes.io/alibaba-cloud-loadbalancer-force-override-listeners to annotation to "true". This configuration only overwrites the listeners of the current Service and does not affect other listeners.

      Note

      The CCM does not overwrite the listeners of an existing CLB instance due to the following reasons:

      • If the listeners of the CLB instance are associated with applications, service interruptions may occur after the listeners are overwritten.

      • The CCM supports limited backend configurations and cannot handle complex configurations. To use complex backend configurations, you can create listeners in the console. The listeners do not overwrite the existing ones.

      Therefore, we recommend that you do not overwrite the listeners of an existing CLB instance. You can forcibly overwrite the listeners if the ports on which these listeners listen are no longer used.

    • You cannot add additional tags such as annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-additional-resource-tags if you use an existing CLB instance.

    No default value

    CCM 1.9.3.81-gca19cd4-aliyun and later

    apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id: "${YOUR_LOADBALANCER_ID}" # You can configure only one access control list (ACL). 
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 443
  selector:
    run: nginx
  type: LoadBalancer

    Use an existing CLB instance and forcefully overwrite the listeners of the CLB instance

    The following annotations are used. Use the following annotations to forcefully overwrite the listeners of a CLB instance. If a listener port conflict occurs, the original listener is deleted.

    Annotation

    Description

    Default value

    Supported CCM version

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id

    The ID of the CLB instance that you want to use. You can use this annotation to specify an existing CLB instance.

    • By default, if you use an existing CLB instance, the listeners of the CLB instance are not overwritten. To overwrite the existing listeners, set the service.beta.kubernetes.io/alibaba-cloud-loadbalancer-force-override-listeners annotation to "true". This configuration only overwrites the listeners of the current Service and does not affect other listeners.

      Note

      The CCM does not overwrite the listeners of an existing CLB instance due to the following reasons:

      • If the listeners of the CLB instance are associated with applications, service interruptions may occur after the listeners are overwritten.

      • The CCM supports limited backend configurations and cannot handle complex configurations. To use complex backend configurations, you can create listeners in the console. The listeners do not overwrite the existing ones.

      Therefore, we recommend that you do not overwrite the listeners of an existing CLB instance. You can forcibly overwrite the listeners if the ports on which these listeners listen are no longer used.

    • You cannot add additional tags such as annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-additional-resource-tags if you use an existing CLB instance.

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-force-override-listeners

    Specifies whether to overwrite the listeners of an existing CLB instance. Valid values:

    • "true": overwrites the listeners. This configuration only overwrites the listeners of the current Service and does not affect other listeners.

    • "false": does not overwrite the listeners.

    Important

    If you use an existing CLB instance and set the force-override annotation to "true", do not associate a listener of the CLB instance with multiple Services. Otherwise, a configuration conflict occurs when these Services overwrite the listener.

    "false"

    CCM 1.9.3.81-gca19cd4-aliyun and later

    apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id: "${YOUR_LOADBALANCER_ID}"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-force-override-listeners: "true"
    
    
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 443
  selector:
    run: nginx
  type: LoadBalancer

    Specify the primary and secondary zones when you create a CLB instance

    The following annotations are used. After you specify the primary and secondary zones for a CLB instance, the zones cannot be changed.

    CLB instances in some regions do not support the primary and secondary zones. The actual information on the CLB instance creation page shall prevail.

    Annotation

    Description

    Default value

    Supported CCM version

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-master-zoneid

    The ID of the zone in which the primary backend server is deployed.

    No default value

    CCM 1.9.3.10-gfb99107-aliyun and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-slave-zoneid

    The ID of the zone in which the secondary backend server is deployed.

    No default value

    CCM 1.9.3.10-gfb99107-aliyun and later

    apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-master-zoneid: "cn-hangzhou-k"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-slave-zoneid: "cn-hangzhou-j"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

    Create a pay-by-bandwidth CLB instance

    The following annotations are used. Both annotations are required.

    Annotation

    Description

    Default value

    Supported CCM version

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-charge-type

    The billing method of the CLB instance. Valid values:

    • paybytraffic

    • paybybandwidth

    paybytraffic

    CCM 1.9.3 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-bandwidth

    The maximum bandwidth of the CLB instance. This annotation applies only to Internet-facing CLB instances. For more information about the limits on the billing methods for Internet-facing CLB instances, see ModifyLoadBalancerInstanceSpec.

    50

    CCM 1.9.3.10-gfb99107-aliyun and later

    apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-charge-type: "paybybandwidth"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-bandwidth: "2"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 443
  selector:
    run: nginx
  type: LoadBalancer

    Specify a vSwitch for a CLB instance

    The following annotations are used. Both annotations are required.

    Annotation

    Description

    Default value

    Supported CCM version

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type

    To create an internal-facing CLB instance, set the value to intranet. Valid values:

    • internet: uses an Internet-facing CLB instance to expose the Service. This is the default value. The address type of the CLB instance must be set to Internet.

    • intranet: uses an internal-facing CLB instance to expose the Service. The address type of the CLB instance must be set to intranet.

    internet

    CCM 1.9.3 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-vswitch-id

    The ID of the vSwitch in which the CLB instance resides. The vSwitch and the cluster must be deployed in the same virtual private cloud (VPC).

    When you use this annotation, you must set the service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type annotation to "intranet".

    You can log on to the VPC console to query vSwitch IDs.

    No default value

    CCM 1.9.3 and later

    apiVersion: v1
kind: Service
metadata:
  annotations:
   service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type: "intranet"
   service.beta.kubernetes.io/alibaba-cloud-loadbalancer-vswitch-id: "${YOUR_VSWITCH_ID}"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 443
  selector:
    run: nginx
  type: LoadBalancer

    Specify an IP address for an internal-facing CLB instance

    The following annotations are used. These annotations are required.

    Annotation

    Description

    Default value

    Supported CCM version

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type

    To create an internal-facing CLB instance, set the value to intranet. Valid values:

    • internet: uses an Internet-facing CLB instance to expose the Service. This is the default value. The address type of the CLB instance must be set to Internet.

    • intranet: uses an internal-facing CLB instance to expose the Service. The address type of the CLB instance must be set to intranet.

    internet

    CCM 1.9.3 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-vswitch-id

    The ID of the vSwitch in which the CLB instance resides. The vSwitch and the cluster must be deployed in the same VPC.

    When you use this annotation, you must set the service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type annotation to intranet.

    You can log on to the VPC console to query vSwitch IDs.

    No default value

    CCM 1.9.3 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ip

    The IP address of the internal-facing CLB instance.

    • The IP address must fall within the CIDR block of the vSwitch. Only IPv4 addresses are supported. This annotation must be used together with the service.beta.kubernetes.io/alibaba-cloud-loadbalancer-vswitch-id annotation.

    • You cannot change the IP address after it is specified.

    No default value

    CCM 2.7.0 and later

    apiVersion: v1
kind: Service
metadata:
 annotations:
  service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type: "intranet"
  service.beta.kubernetes.io/alibaba-cloud-loadbalancer-vswitch-id: "${YOUR_VSWITCH_ID}"
  service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ip: "192.168.x.x"
 name: nginx
 namespace: default
spec:
 type: LoadBalancer
 ports:
 - port: 80
   targetPort: 80
   name: http
 selector:
   app: nginx

    Add additional tags to a CLB instance

    Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-additional-resource-tags

    Description

    Default value

    Supported CCM version

    The tags that you want to add to the CLB instance. Separate multiple tags with commas (,). Example: "k1=v1,k2=v2".

    No default value

    CCM 1.9.3 and later

    apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-additional-resource-tags: "Key1=Value1,Key2=Value2" 
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

    Create an IPv6 CLB instance

    Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ip-version

    Description

    Default value

    Supported CCM version

    The IP version of the CLB instance. You cannot change the IP version cannot be changed after it is specified. To use this annotation, make sure that the kube-proxy mode of the cluster is set to IPVS. Valid values:

    • ipv4

    • ipv6 The IPv6 address can be used only in an IPv6 network.

    ipv4

    CCM 1.9.3.220-g24b1885-aliyun and later

    apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ip-version: "ipv6"
  name: nginx
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  type: LoadBalancer

    Enable deletion protection for a CLB instance

    Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-delete-protection

    Description

    Default value

    Supported CCM version

    Specifies whether to enable deletion protection for the CLB instance. Valid values:

    • on

    • off

    Important

    If you manually enable deletion protection in the CLB console for a CLB instance that is created for a LoadBalancer Service, you can run the kubectl delete svc {your-svc-name} command to delete the CLB instance.

    on

    CCM 1.9.3.313-g748f81e-aliyun and later

    apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-delete-protection: "on"
  name: nginx
spec:
  externalTrafficPolicy: Local
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  type: LoadBalancer

    Enable the configuration read-only mode for a CLB instance

    Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-modification-protection

    Description

    Default value

    Supported CCM version

    Specifies whether to enable the configuration read-only mode for the CLB instance. Valid values:

    • ConsoleProtection

    • NonProtection

    ConsoleProtection

    CCM 1.9.3.313-g748f81e-aliyun and later

    apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-modification-protection: "ConsoleProtection"
  name: nginx
spec:
  externalTrafficPolicy: Local
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  type: LoadBalancer

    Specify the name of a CLB instance

    Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-name

    Description

    Default value

    Supported CCM version

    The name of the CLB instance. The name must be 2 to 128 characters in length, and can contain letters, digits, periods (.), underscores (_), and hyphens (-). The name must start with a letter.

    No default value

    CCM 1.9.3.313-g748f81e-aliyun and later

    apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-name: "your-svc-name"
  name: nginx
spec:
  externalTrafficPolicy: Local
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  type: LoadBalancer

    Specify a resource group to which a CLB instance belongs

    Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-resource-group-id

    Description

    Default value

    Supported CCM version

    The ID of the resource group to which the CLB instance belongs. The resource group cannot be changed after it is specified. You can log on to the Resource Management console to query resource group IDs.

    No default value

    CCM 1.9.3.313-g748f81e-aliyun and later

    apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-resource-group-id: "rg-xxxx"
  name: nginx
spec:
  externalTrafficPolicy: Local
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  type: LoadBalancer

    Specify a hostname for a Service

    The following annotations are used.

    Annotation

    Description

    Default value

    Supported CCM version

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port

    Separate multiple values with commas (,). Example: https:443,http:80.

    No default value

    CCM 1.9.3 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-hostname

    Specifies a hostname for the Service. The hostname must comply with the naming conventions of domain names.

    Take note of the following items:

    • After you add this annotation, the external IP address of the Service changes from the default CLB IP address to your_service_name. If you access the CLB IP address from within the cluster, requests are first forwarded to the corresponding CLB instance.

    • After you add this annotation, if the listener protocol is TCP or UDP, a loop occurs when requests from within the cluster are forwarded to the CLB IP address. For more information, see Why am I unable to access an SLB instance?

    • This annotation does not automatically associate a domain name with the CLB instance. You can log on to the Domain Names console, purchase a domain name, and then associate the domain name with the CLB instance. For more information about how to purchase domain names, see the Purchase a domain name section of the "What is Domain Names?" topic.

    No default value

    CCM 2.3.0 and later

    apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "http:80"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-hostname: "${your_service_hostname}"
  name: nginx-svc
  namespace: default
spec:
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  type: LoadBalancer

    Expected output:

    NAME         TYPE           CLUSTER-IP       EXTERNAL-IP            PORT(S)                      AGE
nginx-svc    loadBalancer   47.100.XX.XX     www.example.com        80:30248/TCP,443:32670/TCP   10s

    Create a pay-by-LCU CLB instance

    Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-instance-charge-type

    Description

    Default value

    Supported CCM version

    The billing method of the CLB instance. Valid values:

    • PayBySpec: The CLB instance is billed based on its specification. This is the default value.

    • PayByCLCU: The CLB instance is billed based on Loadbalancer Capacity Units (LCUs).

    Important

    You cannot change the specification of a pay-by-LCU CLB instance. You cannot specify PayByCLCU and add the service.beta.kubernetes.io/alibaba-cloud-loadbalancer-spec annotation at the same time.

    PayBySpec

    CCM 2.4.0 and later

    apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-instance-charge-type: "PayByCLCU"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

    Common operations to manage listeners

    Specify the session persistence period for a TCP-based CLB instance

    Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-persistence-timeout

    Description

    Default value

    Supported CCM version

    The session persistence period. Unit: seconds. This annotation applies only to TCP listeners. If a CLB instance has multiple TCP listeners, the configuration applies to all TCP listeners.

    Unit: seconds. Valid values: 0 to 3600. Default value: 0. By default, session persistence is disabled. For more information, see CreateLoadBalancerTCPListener.

    0

    CCM 1.9.3 and later

    apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-persistence-timeout: "1800"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 443
  selector:
    run: nginx
  type: LoadBalancer

    Enable session persistence for an HTTP or HTTPS listener by inserting a cookie

    The following annotations are used. The following annotations are required if you insert a cookie to enable session persistence.

    • The following annotations apply only to HTTP-based and HTTPS-based CLB instances.

    • If a CLB instance has multiple HTTP or HTTPS listeners, the configuration applies to all HTTP or HTTPS listeners.

    Annotation

    Description

    Default value

    Supported CCM version

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session

    Specifies whether to enable session persistence. This annotation applies only to HTTP and HTTPS listeners. Valid values:

    • on: enables session persistence.

    • off: does not enable session persistence.

    For more information, see CreateLoadBalancerHTTPListener and CreateLoadBalancerHTTPSListener.

    off

    CCM 1.9.3 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session-type

    The method that is used to process the cookie. This annotation applies only to HTTP and HTTPS listeners. This annotation is required when service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session is set to on. Valid values:

    • insert: inserts the cookie.

    • server: rewrites the cookie.

    For more information, see CreateLoadBalancerHTTPListener and CreateLoadBalancerHTTPSListener.

    No default value

    CCM 1.9.3 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cookie-timeout

    The timeout period of the cookie. This annotation is required when service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session is set to on and service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session-type is set to insert. Unit: seconds. Valid values: 1 to 86400.

    For more information, see CreateLoadBalancerHTTPListener and CreateLoadBalancerHTTPSListener.

    No default value

    CCM 1.9.3 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cookie

    The name of a cookie configured on a backend server.

    The cookie must be 1 to 200 characters in length, and can contain only ASCII letters and digits. It cannot contain commas (,), semicolons (;), or space characters. It cannot start with a dollar sign ($).

    This annotation is required when service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session is set to on and service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session-type is set to server. For more information, see CreateLoadBalancerHTTPListener and CreateLoadBalancerHTTPSListener.

    No default value

    CCM 1.9.3 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port

    Separate multiple values with commas (,). Example: https:443,http:80.

    No default value

    CCM 1.9.3 and later

    apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session-type: "insert"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cookie-timeout: "1800"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "http:80"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

    Configure an ACL for a CLB instance

    The following annotations are used. These annotations are required.

    Before you use this annotation to create a CLB instance that is configured with access control, you need to create an ACL in the CLB console and record the ID of the ACL.

    Annotation

    Description

    Default value

    Supported CCM version

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-status

    Specifies whether to enable access control. Valid values:

    • on: enables access control.

    • off: does not enable access control.

    off

    CCM 1.9.3.164-g2105d2e-aliyun and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-id

    The ID of the ACL that you want to associate with a listener. This annotation is required when the service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-status annotation is set to "on".

    No default value

    CCM 1.9.3.164-g2105d2e-aliyun and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-type

    The type of the ACL. Valid values:

    • white: a whitelist. Only requests from the IP addresses or CIDR blocks in the ACL are forwarded. You can use an IP address whitelist in scenarios in which you want to allow access from only specified IP addresses to an application. Risks may arise if a whitelist is improperly configured. After a whitelist is set, the CLB instance forwards only requests from the IP addresses in the whitelist. If you enable a whitelist but do not add an IP address to the ACL, all requests are forwarded.

    • black: a blacklist. All requests from the IP addresses or CIDR blocks in the ACL are not forwarded. You can use an IP address blacklist in scenarios in which you want to deny access from only specified IP addresses to an application. If you enable an IP address blacklist but do not add an IP address to the blacklist, all requests are forwarded. This annotation is required when the service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-status annotation is set to on.

    No default value

    CCM 1.9.3.164-g2105d2e-aliyun and later

    apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-status: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-id: "${YOUR_ACL_ID}"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-type: "white"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 443
  selector:
    run: nginx
  type: LoadBalancer

    Configure port forwarding for a CLB instance

    Port forwarding allows a CLB instance to forward requests from an HTTP port to an HTTPS port.

    The following annotations are used. These annotations are required.

    Annotation

    Description

    Default value

    Supported CCM version

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port

    Separate multiple values with commas (,). Example: https:443,http:80.

    No default value

    CCM 1.9.3 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id

    The ID of a certificate for the CLB instance.

    You can log on to the CLB console and view certificate IDs on the Certificates page.

    Note

    For more information about how to create a certificate, see Use a certificate from Alibaba Cloud SSL Certificates Service.

    No default value

    CCM 1.9.3.164-g2105d2e-aliyun and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-forward-port

    Forwards requests from an HTTP port to the specified HTTPS port. Example: 80:443.

    No default value

    CCM 1.9.3.164-g2105d2e-aliyun and later

    apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "https:443,http:80"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${YOUR_CERT_ID}"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-forward-port: "80:443"
  name: nginx
  namespace: default
spec:
  ports:
  - name: https
    port: 443
    protocol: TCP
    targetPort: 80
  - name: http
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

    Specify the scheduling algorithm for a CLB instance

    Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-scheduler

    Description

    Default value

    Supported CCM version

    The scheduling algorithm. Valid values:

    • wrr: backend servers with higher weights receive more requests than backend servers with lower weights.

    • rr: the round-robin algorithm is used. Requests are forwarded to backend servers in sequence.

    rr

    CCM 1.9.3 and later

    apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-scheduler: "wrr"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 443
  selector:
    run: nginx
  type: LoadBalancer

    Create a UDP listener

    apiVersion: v1
kind: Service
metadata:
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: UDP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

    Create an HTTP listener

    Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port

    Description

    Default value

    Supported CCM version

    Separate multiple values with commas (,). Example: https:443,http:80.

    No default value

    CCM 1.9.3 and later

    apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "http:80"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

    Create an HTTPS listener

    The following annotations are used.

    The HTTPS listeners of the CLB instance decrypt HTTPS requests into HTTP requests and forward the requests to pods on the backend servers.

    Annotation

    Description

    Default value

    Supported CCM version

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port

    Separate multiple values with commas (,). Example: https:443,http:80.

    No default value

    CCM 1.9.3 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id

    The ID of a certificate for the CLB instance.

    You can log on to the CLB console and view certificate IDs on the Certificates page.

    Note

    For more information about how to create a certificate, see Use a certificate from Alibaba Cloud SSL Certificates Service.

    No default value

    CCM 1.9.3.164-g2105d2e-aliyun and later

    apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "https:443"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${YOUR_CERT_ID}"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

    Create a listener that enables health checks

    Enable TCP health checks

    The following annotations are used. These annotations are required.

    By default, health checks are enabled for TCP ports.

    Annotation

    Description

    Default value

    Supported CCM version

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-switch

    Specifies whether to enable health checks for a TCP or UDP listener. Valid values:

    • on: enables health checks.

    • off: does not enable health checks.

    on

    CCM 2.6.0 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type

    The type of the health check. Valid values:

    • tcp

    • http

    For more information, see CreateLoadBalancerTCPListener.

    tcp

    CCM 1.9.3 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout

    The timeout period of a health check. This annotation applies to TCP health checks. If a backend server, such as an Elastic Compute Service (ECS) instance, does not return a health check response within the specified timeout period, the server fails the health check. Unit: seconds. Valid values: 1 to 300.

    If the value of the service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout annotation is smaller than that of the service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval annotation, the first annotation does not take effect. The timeout period is set to the value of the second annotation. For more information, see CreateLoadBalancerTCPListener.

    5

    CCM 1.9.3 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold

    The number of consecutive successful health checks that must occur before an unhealthy backend server is declared healthy (from fail to success).

    Valid values: 2 to 10. For more information, see CreateLoadBalancerTCPListener.

    3

    CCM 1.9.3 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold

    The number of consecutive failed health checks that must occur before a healthy backend server is declared unhealthy. Valid values: 2 to 10. For more information, see CreateLoadBalancerTCPListener.

    3

    CCM 1.9.3 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval

    The interval at which health checks are performed. Unit: seconds. Valid values: 1 to 50. For more information, see CreateLoadBalancerTCPListener.

    2

    CCM 1.9.3 and later

    apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-switch: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type: "tcp"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout: "8"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold: "4"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold: "4"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval: "3"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

    Enable UDP health checks

    The following annotations are used. These annotations are required.

    By default, health checks are enabled for UDP ports.

    Annotation

    Description

    Default value

    Supported CCM version

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-switch

    Specifies whether to enable health checks for a TCP or UDP listener. Valid values:

    • on

    • off

    on

    CCM 2.6.0 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout

    The timeout period of a health check. This annotation applies to TCP health checks. If a backend server, such as an ECS instance, does not return a health check response within the specified timeout period, the server fails the health check. Unit: seconds. Valid values: 1 to 300.

    If the value of the service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout annotation is smaller than that of the service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval annotation, the first annotation does not take effect. The timeout period is set to the value of the second annotation. For more information, see CreateLoadBalancerUDPListener.

    5

    CCM 1.9.3 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold

    The number of consecutive successful health checks that must occur before an unhealthy backend server is declared healthy.

    Valid values: 2 to 10. For more information, see CreateLoadBalancerUDPListener.

    3

    CCM 1.9.3 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold

    The number of consecutive failed health checks that must occur before a healthy backend server is declared unhealthy. Valid values: 2 to 10. For more information, see CreateLoadBalancerUDPListener.

    3

    CCM 1.9.3 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval

    The interval at which health checks are performed. Unit: seconds. Valid values: 1 to 50. For more information, see CreateLoadBalancerUDPListener.

    2

    CCM 1.9.3 and later

    apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-switch: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval: "5"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout: "10"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold: "3"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold: "3"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: UDP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

    Disable health checks for a TCP or UDP listener

    Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-switch

    Description

    Default value

    Supported CCM version

    Specifies whether to enable health checks for a TCP or UDP listener. Valid values:

    • on: enables health checks.

    • off: does not enable health checks.

    on

    CCM 2.6.0 and later

    apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-switch: "off" # Disable health checks.
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 443
  selector:
    run: nginx
  type: LoadBalancer

    Enable HTTP health checks

    The following annotations are used.

    Annotation

    Description

    Default value

    Supported CCM version

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-flag

    Valid values:

    • on: The value is set to on for TCP listeners by default and cannot be changed. This annotation is not required if you use TCP listeners.

    • off: The value is set to off for HTTP listeners by default.

    off

    CCM 1.9.3 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type

    The type of the health check. Valid values:

    • tcp

    • http

    For more information, see CreateLoadBalancerHTTPListener.

    tcp

    CCM 1.9.3 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-uri

    The Uniform Resource Identifier (URI) that you want to use for health checks. This annotation is not required when the type of the health check is set to TCP.

    No default value

    CCM 1.9.3 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-httpcode

    The HTTP status code for a successful health check. Multiple HTTP status codes are separated by commas (,). Valid values:

    • http_2xx

    • http_3xx

    • http_4xx

    • http_5xx

    For more information, see CreateLoadBalancerHTTPListener.

    http_2xx

    CCM 1.9.3 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-domain

    The domain name that is used for health checks. Valid values:

    • $_ip: The private IP address of the backend server. If an IP address is specified, or this parameter is not specified, the CLB instance uses the private IP address of each backend server as the domain name for health checks.

    • domain: The domain name must be 1 to 80 characters in length and can contain letters, digits, periods (.), and hyphens (-).

    For more information, see CreateLoadBalancerHTTPListener.

    No default value

    CCM 1.9.3 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-timeout

    The timeout period of a health check. This annotation applies to HTTP health checks. If a backend server, such as an ECS instance, does not return a health check response within the specified timeout period, the server fails the health check. Unit: seconds. Valid values: 1 to 300.

    If the value of the service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-timeout annotation is smaller than that of the service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval annotation, the first annotation does not take effect. The timeout period is set to the value of the second annotation. For more information, see CreateLoadBalancerHTTPListener.

    No default value

    CCM 1.9.3 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold

    The number of consecutive successful health checks that must occur before an unhealthy backend server is declared healthy.

    Valid values: 2 to 10. For more information, see CreateLoadBalancerHTTPListener.

    3

    CCM 1.9.3 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold

    The number of consecutive failed health checks that must occur before a healthy backend server is declared unhealthy. Valid values: 2 to 10. For more information, see CreateLoadBalancerHTTPListener.

    3

    CCM 1.9.3 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval

    The interval at which health checks are performed. Unit: seconds. Valid values: 1 to 50. For more information, see CreateLoadBalancerHTTPListener.

    2

    CCM 1.9.3 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port

    Separate multiple values with commas (,). Example: https:443,http:80.

    No default value

    CCM 1.9.3 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-method

    The health check method used by HTTP listeners. Valid values:

    • head

    • get

    No default value

    CCM 2.3.0 and later

    apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-flag: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type: "http"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-uri: "/test/index.html"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold: "4"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold: "4"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-timeout: "10"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval: "3"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "http:80"
    # Specify the HTTP status codes for health check. This annotation is optional.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-httpcode: "http_4xx"
    # Specify the domain name for health check. This annotation is optional.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-domain: "www.aliyun.com"
    # Specify the health check method. This annotation is optional.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-method: "head"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

    Configure connection draining for a listener

    The following annotations are used. These annotations are required.

    Only TCP and UDP listeners are supported.

    Annotation

    Description

    Default value

    Supported CCM version

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain

    Specifies whether to enable connection draining. Valid values:

    • on: enables connection draining.

    • off: does not enable connection draining.

    No default value

    CCM 2.0.1 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain-timeout

    The timeout period of connection draining. Unit: seconds. Valid values: 10 to 900.

    No default value

    CCM 2.0.1 and later

    apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain-timeout: "30"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

    Configure additional request headers for a listener

    The following annotations are used.

    Only HTTP and HTTPS listeners are supported.

    Annotation

    Description

    Default value

    Supported CCM version

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port

    Separate multiple values with commas (,). Example: https:443,http:80.

    No default value

    CCM 1.9.3 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-xforwardedfor-proto

    Specifies whether to use the X-Forwarded-Proto header to obtain the listener protocol of the CLB instance. Valid values:

    • on: uses the X-Forwarded-Proto header to obtain the listener protocol of the CLB instance.

    • off: does not use the X-Forwarded-Proto header to obtain the listener protocol of the CLB instance.

    off

    CCM 2.1.0 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-xforwardedfor-slbport

    Specifies whether to use the XForwardedFor_SLBPORT header to obtain the listener ports of the CLB instance. Valid values:

    • on: uses the XForwardedFor_SLBPORT header to obtain the listener ports of the CLB instance.

    • off: does not use the XForwardedFor_SLBPORT header to obtain the listener ports of the CLB instance.

    off

    CCM 2.9.1 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-xforwardedfor-clientsrcport

    Specifies whether to use the XForwardedFor_ClientSrcPort header to obtain the listener ports of the CLB instance. Valid values:

    • on: uses the XForwardedFor_ClientSrcPort header to obtain the listener ports of the CLB instance.

    • off: does not use the XForwardedFor_ClientSrcPort header to obtain the listener ports of the CLB instance.

    off

    CCM 2.9.1 and later

    apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "http:80"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-xforwardedfor-proto: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-xforwardedfor-slbport: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-xforwardedfor-clientsrcport: "on"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

    Specify the timeout period of idle connections for a listener

    The following annotations are used.

    Only HTTP and HTTPS listeners are supported.

    Annotation

    Description

    Default value

    Supported CCM version

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port

    Separate multiple values with commas (,). Example: https:443,http:80.

    No default value

    CCM 1.9.3 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-idle-timeout

    Specifies the timeout period of idle connections for a listener. Unit: seconds. Valid values: 1 to 60.

    15

    CCM 2.1.0 and later

    apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "http:80"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-idle-timeout: "30"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

    Disable HTTP/2 for a listener.

    The following annotations are used.

    Only HTTPS listeners are supported.

    Annotation

    Description

    Default value

    Supported CCM version

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port

    Separate multiple values with commas (,). Example: https:443,http:80.

    No default value

    CCM 1.9.3 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id

    The ID of a certificate for the CLB instance.

    You can log on to the CLB console and view certificate IDs on the Certificates page.

    Note

    For more information about how to create a certificate, see Use a certificate from Alibaba Cloud SSL Certificates Service.

    No default value

    CCM 1.9.3.164-g2105d2e-aliyun and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-http2-enabled

    Specifies whether to enable HTTP/2. Valid values:

    • on: enables HTTP/2.

    • off: does not enable HTTP/2.

    on

    CCM 2.1.0 and later

    apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "https:443"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${YOUR_CERT_ID}"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-http2-enabled: "off"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

    Specify the timeout period of requests for a listener.

    The following annotations are used.

    Only HTTP and HTTPS listeners are supported.

    Annotation

    Description

    Default value

    Supported CCM version

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port

    Separate multiple values with commas (,). Example: https:443,http:80.

    No default value

    CCM 1.9.3 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-request-timeout

    The timeout period of requests. Unit: seconds. Valid values: 1 to 180.

    If the backend server fails to respond within the timeout period, the CLB instance stops waiting and returns an HTTP 504 error code to the client.

    60

    CCM 2.3.0 and later

    apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "http:80"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-request-timeout: "60"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

    Specify the timeout period of connections for a listener.

    Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-established-timeout

    Only TCP listeners are supported.

    Description

    Default value

    Supported CCM version

    The timeout period of connections. Unit: seconds. Valid values: 10 to 900. For more information, see CreateLoadBalancerTCPListener.

    No default value

    CCM 2.3.0 and later

    apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-established-timeout: "60"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

    Configure security policies for a listener

    The following annotations are used.

    Only HTTPS listeners are supported.

    Annotation

    Description

    Default value

    Supported CCM version

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port

    Separate multiple values with commas (,). Example: https:443,http:80.

    No default value

    CCM 1.9.3 and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id

    The ID of a certificate for the CLB instance.

    You can log on to the CLB console and view certificate IDs on the Certificates page.

    Note

    For more information about how to create a certificate, see Use a certificate from Alibaba Cloud SSL Certificates Service.

    No default value

    CCM 1.9.3.164-g2105d2e-aliyun and later

    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-tls-cipher-policy

    The TLS security policy. Each security policy contains TLS protocol versions and cipher suites available for HTTPS. For more information, see CreateLoadBalancerHTTPSListener. Valid values:

    • tls_cipher_policy_1_0

    • tls_cipher_policy_1_1

    • tls_cipher_policy_1_2

    • tls_cipher_policy_1_2_strict

    • tls_cipher_policy_1_2_strict_with_1_3

    tls_cipher_policy_1_0

    CCM 2.4.0 and later

    apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "https:443,http:80"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${YOUR_CERT_ID}"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-tls-cipher-policy: "tls_cipher_policy_1_2"
  name: nginx
  namespace: default
spec:
  ports:
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  - name: http
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

    Configure both TCP and UDP for a listener

    Note

    Only clusters that run Kubernetes 1.24 and later support this feature. For more information about how to update a cluster, see Manually update ACK clusters. 

    apiVersion: v1
kind: Service
metadata:
  name: nginx
  namespace: default
spec:
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: udp
    port: 80
    protocol: UDP
    targetPort: 81
  selector:
    app: nginx
  sessionAffinity: None
  type: LoadBalancer

    Configure proxy protocol for TCP and UDP listeners

    Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-proxy-protocol

    Description

    Default value

    Supported CCM version

    Specifies whether to enable proxy protocol for a TCP or UDP listener. After proxy protocol is configured, client IP addresses can be passed to backend servers. Valid values:

    • on: enables Proxy Protocol.

    • off: does not enable Proxy Protocol.

    Important

    Enabling this feature may cause service interruptions. You must stop and update your applications before you enable proxy protocol. Proceed with caution.

    off

    CCM 2.6.0 and later

    apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-proxy-protocol: "on"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 443
  selector:
    run: nginx
  type: LoadBalancer

    Common operations to manage backend server groups

    Add worker nodes that have specified labels as the backend servers of a CLB instance

    Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-backend-label

    Description

    Default value

    Supported CCM version

    Specifies the labels of worker nodes that you want to add as the backend servers of the CLB instance. Separate multiple labels with commas (,). Example: "k1=v1,k2=v2". A node must have all the specified labels before you can add the node as a backend server.

    No default value

    CCM 1.9.3 and later

    apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-backend-label: "failure-domain.beta.kubernetes.io/zone=ap-southeast-5a"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 443
  selector:
    run: nginx
  type: LoadBalancer

    Add the nodes on which pods run as the backend servers of a CLB instance

    Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-scheduler

    By default, the externalTrafficPolicy parameter is set to Cluster for a Service. In Cluster mode, all nodes in the cluster are added as the backend servers of the CLB instance. In Local mode, only nodes on which the pods are deployed are added as the backend servers of the CLB instance.

    Description

    Default value

    Supported CCM version

    The scheduling algorithm. Valid values:

    • wrr: backend servers with higher weights receive more requests than backend servers with lower weights.

      In Local mode, you must set the scheduling algorithm to weighted round-robin (WRR).

      Note

      For CCM V1.9.3.164-g2105d2e-aliyun and later, node weights are calculated based on the number of pods that run on each node for Services whose externalTrafficPolicy parameter is set to Local. For more information about node weight calculation, see How does the CCM calculate node weights in Local mode? section of the "Service FAQ" topic.

    • rr: uses the round-robin algorithm. This way, requests are forwarded to backend servers in sequence.

    rr

    CCM 1.9.3 and later

    apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-scheduler: "wrr"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

    Remove unschedulable nodes from the backend server group of a CLB instance

    Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-remove-unscheduled-backend

    Description

    Default value

    Supported CCM version

    Specifies whether to remove unschedulable nodes from the backend server group of the CLB instance. Valid values:

    • on: removes unschedulable nodes from the backend server group of the CLB instance.

    • off: the kubectl cordon and kubectl drain commands are run to set nodes to the unschedulable state. In this scenario, unschedulable nodes are not removed from the backend server group of the CLB instance.

    off

    CCM 1.9.3.164-g2105d2e-aliyun and later

    apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-remove-unscheduled-backend: "on"
  name: nginx
spec:
  externalTrafficPolicy: Local
  ports:
  - name: http
    port: 30080
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  type: LoadBalancer

    Add pods that are assigned ENIs as the backend servers of a CLB instance

    Annotation: service.beta.kubernetes.io/backend-type

    Description

    Default value

    Supported CCM version

    The type of resources that are added as the backend servers of the CLB instance. Valid values:

    • eni: adds pods as the backend servers of the CLB instance. This improves the network forwarding performance. This annotation takes effect only in Terway mode.

      You can also manually set the service.beta.kubernetes.io/backend-type annotation from "eni" to "ecs" and then add ECS instances as the backend servers of the CLB instance.

    • ecs: adds ECS instances as the backend servers of the CLB instance.

    • Flannel mode: ecs.

    • Terway mode:

      • Terway clusters that are created before August 10, 2020: ecs

      • Terway clusters that are created after August 10, 2020: eni

    CCM 1.9.3.164-g2105d2e-aliyun and later

      apiVersion: v1
  kind: Service
  metadata:
    annotations:
      service.beta.kubernetes.io/backend-type: "eni"
    name: nginx
  spec:
    ports:
    - name: http
      port: 30080
      protocol: TCP
      targetPort: 80
    selector:
      app: nginx
    type: LoadBalancer

    Reuse an existing vServer group

    Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-vgroup-port This annotation can be used to reuse a vServer group. This annotation takes effect only when you use an existing CLB instance. For more information, see Use the CCM to deploy services across clusters.

    Specify weights for Services to enable weighted round robin

    Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-weight

    When multiple Services use the same CLB instance, you can use this annotation to specify the percentage of traffic distributed to the current Service. This annotation takes effect only when an existing vServer group is reused. For more information, see Use the CCM to deploy services across clusters.