Mount an independent Elastic IP Address (EIP) for pods - Container Compute Service

In an ACS cluster, the IP address of a pod is usually a private IP address. However, in some scenarios, a pod needs an independent public IP address to communicate with the Internet independently. This topic describes how to mount an independent Elastic IP Address (EIP) for a pod in an ACS cluster.

Prerequisites

An ACS cluster is created. For more information, see Create an ACS cluster.

Background information

Generally, a pod accesses the Internet data through "external SNAT+EIP". For more information, see Enable an existing ACS cluster to access the Internet. The Internet ingress data of a pod usually flows in through a LoadBalancer type Service. In special scenarios, a pod needs an independent IP address, such as:

The external mapping port of the pod is random, which is common in User Datagram Protocol (UDP) game servers or teleconferences. For example, the RTSP protocol uses different ports for different clients.
Internet data faces resource contention at the egress point.

Limits

Before using EIP, please understand the limits of EIP. For more information, see Limits.
If you use the automatic EIP allocation capability, EIP resources may be repeatedly requested and released when you recreate a pod or encounter CNI failures, which may trigger EIP limits. To avoid this situation, you can specify an EIP for the pod by configuring the pod annotation network.alibabacloud.com/allocated-eip-id.

Step 1: Configure RAM permissions required for mounting EIP

For an ACS cluster, you need to generate an AccessKey for the Resource Access Management (RAM) user. For specific operations, see Create a RAM user and Create custom policies.

Step 2: Install or upgrade the plug-in for the cluster

You need to install the ack-extend-network-controller plug-in in the ACK App Market or the ACS cluster application entry, and enable the EIP controller.

Note

ACS clusters can be viewed and managed in the ACK console. For more information, see Product introduction.

Install through the Helm entry in the ACS cluster

Log on to the ACS console. In the left-side navigation pane, click Clusters.
On the Clusters page, find the cluster that you want to manage and click its ID. In the left-side pane, choose Applications > Helm.
On the Helm page, click Deploy. Complete the Basic Information configuration with the following information.
Parameter
Example value
Application Name
ack-extend-network-controller
Namespace
kube-system
Source
Default is App Catalog.
Chart
Search box: search for ack-extend-network-controller.
Click Next.

On the Parameters page, select the Chart Version. Configure the Required parameters according to the following table, and then click OK.

Note

ACS clusters only support installing and using components with version >=v0.9.3.

The parameter descriptions are as follows:

Configuration parameter	Type	Required	Description
enableControllers	[]string	Yes	Remove the default comment of `eip` to enable the EIP feature.
vpcid	string	Yes	The VPC ID associated with the EIP.
credential.accessKey	string	Yes	The AccessKey ID of the permission account required for mounting the EIP.
credential.accessSecret	string	Yes	The AccessKey secret of the permission account required for mounting the EIP.
networkController.eipController.maxConcurrentReconciles	int	No	The concurrency count of the EIP controller.
networkController.eipController.garbageCollectionPeriodInMinutes	int	No	The period for the EIP controller to clean up fixed EIPs.
customStatefulWorkloadKinds	[]string	No	Custom Stateful Controller Kind.

Return to the Helm page, and you can see that the new ack-extend-network-controller component has been deployed.

Install through the ACK App Market

Log on to the Container Service Management Console, in the left-side navigation pane, choose Marketplace > Marketplace.
In the search bar on the Marketplace page, enterack-extend-network-controller, and then click it.
On the application details page, click One-click Deployment in the upper right corner.

In the Create pane, select the cluster and namespace, and then click Next.

On the parameter configuration page, select the version number and set the corresponding parameters, and then click OK.

Note

ACS clusters only support installing and using components with version >=v0.9.2.

The parameter descriptions are as follows:

Configuration parameter	Type	Required	Description
enableControllers	[]string	Yes	Configure `eip` to enable the EIP function.
vpcid	string	Yes	The VPC ID associated with the EIP
credential.accessKey	string	Yes	The AK of the permission account required for mounting the EIP.
credential.accessSecret	string	Yes	The SK of the permission account required for mounting the EIP.
networkController.eipController.maxConcurrentReconciles	int	No	The concurrency count of the EIP controller.
networkController.eipController.garbageCollectionPeriodInMinutes	int	No	The period for the EIP controller to clean up fixed EIPs.
customStatefulWorkloadKinds	[]string	No	Custom Stateful Controller Kind.

Parameter examples are as follows:

clusterID: "c11ba338192xxxxxxx"  
regionID: "cn-hangzhou"      
vpcID: "vpc-bp1rkq0zxxxxxx" 
enableVirtualNode: true
affinity: null
enableControllers:
  - eip              
networkController:
  eipController:
    maxConcurrentReconciles: 10   
    garbageCollectionPeriodInMinutes: 1  
  customStatefulWorkloadKinds:           
  - foo
credential:                             
  accessKey: "xxxxxxxxxxxx"
  accessSecret: "xxxxxxxxxxxxxx"

If you need to update the version and parameters of the ack-extend-network-controller plug-in, see App Marketplace.

Step 3: Enable the EIP function

ACS supports enabling the EIP function by using annotations. You can create or associate an EIP with the pod by specifying annotations in the pod. For detailed information about enabling EIP function annotations, see Annotations for enabling the EIP feature.

Note

You can choose to automatically allocate an EIP or specify an EIP as needed. These two methods differ in the annotations used and the EIP recycling policy. For the specified EIP instance method, deleting a pod does not release the EIP instance, whereas the automatic EIP allocation method releases the EIP instance by default. For detailed information on the EIP recycling policy, see Annotation for configuring the EIP release policy.

Automatically allocate an EIP

Log on to the ACS console. In the left-side navigation pane, click Clusters.
On the Clusters page, find the cluster that you want to manage and click its ID. In the left-side pane, choose Workloads > Deployments.
On the Deployments page, click Create from YAML in the upper-right corner.

Create a Deployment controller by using the following sample and name the controller "example".

Expand to view Deployment YAML

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example
  labels:
    app: example
spec:
  replicas: 1
  selector:
    matchLabels:
      app: example
  template:
    metadata:
      labels:
        app: example
      annotations:
        network.alibabacloud.com/pod-with-eip: "true"
        network.alibabacloud.com/eip-bandwidth: "5"
    spec:
      containers:
      - name: example
        image: registry.cn-hangzhou.aliyuncs.com/acs-sample/nginx:latest
        ports:
        - containerPort: 80

The annotations section indicates that an EIP instance with a bandwidth of 5 Mbps is automatically allocated for each pod.

After the creation is successful, enter the Deployment named example. Click the pod name (for example: example-78d17b7xxx-adxxx) to enter the pod Details page. You can view the created EIP information In the Annotations section on the right side of the page. You can also click Edit to view the created EIP information in the pod YAML.

Expand to view pod YAML

apiVersion: v1
kind: Pod
metadata:
  annotations:
    ...
    network.alibabacloud.com/allocated-eip-id: eip-xxxxx0y884ucrevoxxxxx
    network.alibabacloud.com/allocated-eipAddress: xxx.xxx.xxx.xxx
    network.alibabacloud.com/allocated-eni-id: eni-xxxxx563trofuhaxxxxx
    network.alibabacloud.com/eip-bandwidth: '5'
    network.alibabacloud.com/pod-with-eip: 'true'
    ...
  labels:
    alibabacloud.com/compute-class: general-purpose
    app: example
  name: example-78d17b7xxx-adxxx
  namespace: default
spec:
  ...

After completing the example, you can delete the Deployment named example. If you do not specify the pod EIP instance ID, the default recycling policy for pod EIP is to follow the pod lifecycle. Therefore, the EIP is automatically released when the pod is deleted.

Specify an EIP

Prepare the EIP instance used in the example. Please Apply for an EIP first.
Log on to the ACS console. In the left-side navigation pane, click Clusters.
On the Clusters page, find the cluster that you want to manage and click its ID. In the left-side pane, choose Workloads > Deployments.
On the Deployments page, click Create from YAML in the upper-right corner.

Create a StatefulSet controller by using the following sample and name the controller "example".

Expand to view Deployment YAML

apiVersion: apps/v1
kind: StatefulSet
metadata:
  labels:
    app: example
  name: example
spec:
  replicas: 1
  selector:
    matchLabels:
      app: example
  template:
    metadata:
      labels:
        app: example
      annotations:
        network.alibabacloud.com/pod-eip-instanceid: eip-xxxxx66wjkj16lf9xxxxx
    spec:
      containers:
      - name: example
        image: registry.cn-hangzhou.aliyuncs.com/acs-sample/nginx:latest
        ports:
        - containerPort: 80

The annotations section indicates that an EIP instance with the instance ID eip-xxxxx66wjkj16lf9xxxxx is specified for the pod. This instance ID needs to be replaced with the actual value.

After the creation is successful, enter the StatefulSet named example. Click the pod name (for example: example-0) to enter the pod Details page. You can view the bound EIP instance information in the Annotations section on the right side of the page. You can also click Edit to view the bound EIP instance information in the pod YAML.

Expand to view pod YAML

apiVersion: v1
kind: Pod
metadata:
  annotations:
    ...
    network.alibabacloud.com/allocated-eip-id: eip-xxxxx66wjkj16lf9xxxxx
    network.alibabacloud.com/allocated-eipAddress: xxx.xxx.xxx.xxx
    network.alibabacloud.com/allocated-eni-id: eni-xxxxx612ub33hunxxxxx
    network.alibabacloud.com/pod-eip-instanceid: eip-xxxxx66wjkj16lf9xxxxx
    ...
  labels:
    alibabacloud.com/compute-class: general-purpose
    app: example
  name: example-0
  namespace: default
spec:
  ...

After completing the example, you can delete the StatefulSet named example. If you specify the pod EIP instance ID, the recycling policy for pod EIP is not to release the EIP instance. Therefore, the EIP is not automatically released when the pod is deleted.

Related operations

The controller configures the EIP address for the pod after the pod IP is allocated. During this process, the pod may enter the Ready state before the EIP binding is successful. To solve this problem and ensure that the pod completes the EIP binding before entering the Ready state, which avoids potential service interruptions or connection timeouts, you can try the following methods.

Check EIP binding state using Readiness gates

The controller sets the pod conditions when readinessGates is configured in the pod, and the EIP binding is successful. The pod is not in the Ready state before the EIP is bound.

kind: Pod
...
spec:
  readinessGates:
  - conditionType: "k8s.aliyun.com/eip"
status:
  conditions:
  - lastProbeTime: "2022-12-12T03:45:48Z"
    lastTransitionTime: "2022-12-12T03:45:48Z"
    reason: Associate eip succeed
    status: "True"
    type: k8s.aliyun.com/eip
...

Check EIP binding status using initContainers

You can check whether the EIP has been allocated in the initContainers when you configure initContainers for the pod. Please refer to the following example to configure initContainers.

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    network.alibabacloud.com/pod-with-eip: "true"
spec:
  containers:
  - name: example
    image: registry-cn-hangzhou.ack.aliyuncs.com/ack-demo/busybox:1.28
    command: ['sh', '-c', 'echo The app is running! && sleep 3600']
  initContainers:
  - name: init
    image: registry-cn-hangzhou.ack.aliyuncs.com/ack-demo/busybox:1.28
    command: ['timeout', '-t' ,'60', 'sh','-c', "until grep -E '^k8s.aliyun.com\\/pod-ips=\\S?[0-9]+\\S?' /etc/podinfo/annotations; do echo waiting for annotations; sleep 2; done"]
    volumeMounts:
    - name: podinfo
      mountPath: /etc/podinfo
  volumes:
  - name: podinfo
    downwardAPI:
      items:
      - path: "labels"
        fieldRef:
          fieldPath: metadata.labels
      - path: "annotations"
        fieldRef:
          fieldPath: metadata.annotations

Parameter	Example value
Application Name	ack-extend-network-controller
Namespace	kube-system
Source	Default is App Catalog.
Chart	Search box: search for ack-extend-network-controller.