Use ACK Virtual Node for serverless computing in self-managed Kubernetes clusters - Container Service for Kubernetes

ACK Virtual Node enables Kubernetes to use serverless computing in the cloud without managing the underlying infrastructure. This topic describes how to deploy ACK Virtual Node in a Kubernetes cluster within a data center.

Introduction

With ACK Virtual Node, self-managed Kubernetes clusters can access elastic compute resources in the cloud, including both CPUs and GPUs. ACK Virtual Node enables you to create serverless pods in your self-managed Kubernetes clusters, allowing you to scale compute resources effectively in response to business growth and fluctuating traffic demands.

Data centers and Alibaba Cloud virtual private clouds (VPCs) are connected over Express Connect circuits. The following figure demonstrates how ACK Virtual Node works:

Note

For each serverless pod, ACK Virtual Node will create an elastic container instance in the cloud. No node maintenance is required.

Prerequisites

A Kubernetes cluster has been deployed. We recommend that you use a cluster with Kubernetes version 1.18 or later.
An AccessKey pair has been created for the Resource Access Management (RAM) user or role that will use the Helm chart for ACK virtual nodes, with permissions granted to call operations on the Elastic Container Instance API. The following script shows the syntax of the authorization information:

Expand to view authorization information

{
    "Version": "1",
    "Statement": [
        {
            "Action": [
                "eci:CreateContainerGroup",
                "eci:DeleteContainerGroup",
                "eci:DescribeContainerGroups",
                "eci:DescribeContainerGroupStatus",
                "eci:DescribeContainerGroupEvents",
                "eci:DescribeContainerLog",
                "eci:UpdateContainerGroup",
                "eci:UpdateContainerGroupByTemplate",
                "eci:CreateContainerGroupFromTemplate",
                "eci:RestartContainerGroup",
                "eci:ExportContainerGroupTemplate",
                "eci:DescribeContainerGroupMetric",
                "eci:DescribeMultiContainerGroupMetric",
                "eci:ExecContainerCommand",
                "eci:CreateImageCache",
                "eci:DescribeImageCaches",
                "eci:DeleteImageCache",
                "vpc:DescribeVSwitches"
            ],
            "Resource": [
                "*"
            ],
            "Effect": "Allow"
        }
    ]
}

Install the component

Use the following YAML template to create a file named my-values.yaml and save it to a local directory:

Note

In the following code, {Region} refers to the ID of the region where the elastic container instance resides. The data center needs to access this ID. For example, the ID of the China (Zhangjiakou) region is cn-zhangjiakou. For more information about regions where Elastic Container Instance is available, see Endpoints.

virtualNode:
  image:
    repository: registry-{Region}.ack.aliyuncs.com/acs/virtual-nodes-eci
    tag: v2.12.1
env:
  ECI_REGION: cn-zhangjiakou
  ECI_VSWITCH: vsw-8vb***
  ECI_SECURITY_GROUP: sg-8vb***
  ECI_ACCESS_KEY: LTAI5t***
  ECI_SECRET_KEY: 99K***

The following table describes the parameters:

Parameter	Description	Required	Example
virtualNode.image.repository	The address of the virtual-node image repository.	Yes	registry-cn-zhangjiakou.ack.aliyuncs.com/acs/virtual-nodes-eci
virtualNode.image.tag	The tag of the virtual-node image.	Yes	v2.12.1
env.ECI_REGION	The ID of the region where the elastic container instance resides.	Yes	cn-zhangjiakou
env.ECI_VSWITCH	The ID of the vSwitch that is used by the elastic container instance. Separate multiple IDs with commas (,) and ensure sufficient IP addresses for serverless pods.	Yes	vsw-8vb***
env.ECI_SECURITY_GROUP	The ID of the security group to which the elastic container instance belongs.	Yes	sg-8vb***
env.ECI_ACCESS_KEY	The AccessKey ID of the Alibaba Cloud account, with permissions to call operations on the Elastic Container Instance API.	Yes	LTAI5t***
env.ECI_SECRET_KEY	The AccessKey Secret of the Alibaba Cloud account, with permissions to call operations on the Elastic Container Instance API.	Yes	99K***
env.KUBERNETES_APISERVER_HOST	The endpoint that is used to access the API server.	No	192.168.0.1
env.KUBERNETES_APISERVER_PORT	The port that is used with the endpoint to access the API server from the internal network.	No	6443
env.ECI_VPC	The ID of the VPC where the elastic container instance resides.	No	vpc-8vb***
env.ALIYUN_RESOURCEGROUP_ID	The ID of the resource group to which the elastic container instance belongs.	No	rg-acf***

Run the following command to add the Helm repository:

helm repo add aliyunhub https://aliacs-app-catalog.oss-cn-hangzhou.aliyuncs.com/charts-incubator/

Run the following command to install ack-virtual-node:

helm install ack-virtual-node aliyunhub/ack-virtual-node --namespace kube-system --values my-values.yaml

Run the following command to check the application status:

kubectl  get no |grep virtual-kubelet

Expected output:

virtual-kubelet-cn-zhangjiakou-c   Ready    agent    10d   v1.18.8

Schedule pods to run on serverless computing in the cloud

In a self-managed Kubernetes cluster, you can use one of the following three methods to schedule pods to elastic container instances:

Method 1: Add labels to pods

Run the following command to deploy the test case on pods with the specified label:

kubectl apply -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: nginx
  name: nginx-deployment-pod-label
  namespace: default
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        alibabacloud.com/eci: 'true'   # Pods with the specified label can use serverless computing in the cloud.
        app: nginx
    spec:
      containers:
        - image: 'registry.cn-hangzhou.aliyuncs.com/eci_open/nginx:1.14.2'
          imagePullPolicy: IfNotPresent
          name: nginx
          ports:
            - containerPort: 80
              protocol: TCP
          resources:
            limits:
              cpu: 500m
EOF

Run the following command to verify the application status:

 kubectl get pod -owide | grep nginx-deployment-pod-label

Expected output:

nginx-deployment-pod-label-6575548559-7p9hf     1/1     Running   0          33s    192.168.XX.XXX    virtual-kubelet-cn-zhangjiakou-c   <none>           <none>
nginx-deployment-pod-label-6575548559-tztm6     1/1     Running   0          33s    192.168.XX.XXX    virtual-kubelet-cn-zhangjiakou-c   <none>           <none>

Method 2: Add labels to namespaces

Run the following command to create a namespace named vk:
```
kubectl create ns vk
```
Run the following command to add the label alibabacloud.com/eci=true to the namespace:
```
kubectl label namespace vk alibabacloud.com/eci=true
```

Run the following command to deploy the test case to the specified namespace:

kubectl apply -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: nginx
  name: nginx-deployment-ns-label
  namespace: vk
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
        - image: 'registry.cn-hangzhou.aliyuncs.com/eci_open/nginx:1.14.2'
          imagePullPolicy: IfNotPresent
          name: nginx
          ports:
            - containerPort: 80
              protocol: TCP
          resources:
            limits:
              cpu: 500m
EOF

Run the following command to check the application status:

kubectl get po -nvk -owide |grep  nginx-deployment-ns-label

Expected output:

nginx-deployment-ns-label-7bc784448c-5llgb   1/1     Running   0          30s   192.168.XX.XXX   virtual-kubelet-cn-zhangjiakou-c   <none>           <none>
nginx-deployment-ns-label-7bc784448c-8ns9q   1/1     Running   0          30s   192.168.XX.XXX   virtual-kubelet-cn-zhangjiakou-c   <none>           <none>

Method 3: Specify node names

Run the following command to deploy the test case to the specified node:

kubectl apply -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: nginx
  name: nginx-deployment-node-name
  namespace: default
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      nodeName: virtual-kubelet-cn-zhangjiakou-c             # Specify the node name is ack-virtual-node.
      containers:
        - image: 'registry.cn-hangzhou.aliyuncs.com/eci_open/nginx:1.14.2'
          imagePullPolicy: IfNotPresent
          name: nginx
          ports:
            - containerPort: 80
              protocol: TCP
          resources:
            limits:
              cpu: 500m
EOF

Run the following command to check the application status:

kubectl get pod -owide |grep nginx-deployment-node-name

Expect output:

nginx-deployment-node-name-864dffd59f-jq58n     1/1     Running   0          23s     192.168.XX.XXX     virtual-kubelet-cn-zhangjiakou-c   <none>           <none>
nginx-deployment-node-name-864dffd59f-r87zp     1/1     Running   0          23s     192.168.XX.XXX     virtual-kubelet-cn-zhangjiakou-c   <none>           <none>

References

For more information about the supported types and the creation of GPU-accelerated pods, see GPU-accelerated ECS instance types.
For more information about how to create a preemptible Elastic Container Instance-based pod in a Kubernetes cluster, see Create a preemptible elastic container instance.
If you run out of temporary storage and are looking to scale up, see Scale up the temporary storage space.
For more information about how to collect metrics from virtual nodes, see Collect the metrics of the specified virtual node.
For an overview and the release notes of the ACK Virtual Node component, see ACK Virtual Node.