By Xianwei
With the popularization of containers and the serverless concept, the Alibaba Cloud Container Service team and the Elastic Compute Service team worked collaboratively and released multiple Serverless container products and solutions, including Serverless Kubernetes, Elastic Container Instance (ECI), and the virtual-nodes add-on feature. These products support a variety of serverless container application scenarios.
Serverless Kubernetes, released by the Alibaba Cloud Container Service team, has been in the open beta test for a while now. Users don't need to manage servers and can normally perform various Kubernetes API operations, including creating pods, services, ingresses, and jobs. The biggest advantage of Serverless is that it not only avoids the server maintenance burden but also provides applications with powerful elastic scaling. Elasticity is no longer limited to the node scale of clusters. In addition, application billing is based on the resource usage.
We released the virtual-nodes feature to further optimize the Container Service user experience, support more application scenarios and provide extremely powerful elastic scaling for users' Kubernetes clusters.
As shown in the preceding diagram, this virtual node feature adds virtual nodes to classic Kubernetes clusters, allowing the real worker nodes and virtual nodes to exist in these clusters at the same time. Virtual nodes are used to dynamically create ECI container instances. Pods on ECI and pods on the real nodes are interconnected. They run on the same user's VPC network.
Virtual-nodes allows users to obtain "unlimited" elasticity in the original Kubernetes clusters without having to create new clusters. Users can also have "unlimited" computing capacity without creating a large number of real worker nodes. The typical application scenarios of virtual nodes include big data computing, gene computing, video rendering, and hybrid clouds. Virtual nodes are also suitable for temporary scaling of online applications.
With virtual nodes, we can implement "unlimited" scaling based on Kubernetes clusters with limited nodes.
How do we use virtual nodes? Let's first understand its implementation principle.
Elastic Container Instance (ECI) is an agile and secure Serverless container runtime service recently released by the Alibaba Cloud ECS team. ECI has the following features:
Unlike Serverless Kubernetes Container Service, ECI doesn't provide the native Kubernetes API. For example, Kubectl cannot be used to perform operations on pod/service/ingress/job resources. However, the innovative connection between Kubernetes and ECI by using virtual kubelet enables ECI instances to be created dynamically by using the Kubernetes API.
Virtual Kubelet is an open source project originally launched by Microsoft Azure with the aim to provide better product integration on public clouds (like elastic containers) with Kubernetes and implement the serverless feature. From the implementation perspective, Virtual Kubelet provides a mechanism that allows integrating products from multiple different providers. Currently, integration with Azure ACI and AWS Fargate is supported. VK is also applied in IoT Edge scenarios. Recently we submitted the ECI provider, which has been merged on https://github.com/virtual-kubelet/virtual-kubelet/tree/master/providers/alicloud
Virtual Kubelet registers a virtual node with Kubernetes APIServer, continuously listens to pod change events and dynamically creates ECI instances.
Virtual Kubelet innovatively connects Kubernetes and ECI by using virtual nodes. Virtual Kubelet and ECI are the technology infrastructure of virtual nodes.
The following section describes how to deploy virtual nodes in Alibaba Cloud Container Service.
Visit the ECI console and enable ECI.
Go to the Container Service console and create a Managed Kubernetes cluster:
https://cs.console.aliyun.com/#/k8s/cluster/create/managed
After a cluster is created, you can view vswitchid and securitygroup in the cluster management or node page.
Replace the ECI_VSWITCH, ECI_SECURITY_GROUP, ECI_ACCESS_KEY, and ECI_SECRET_KEY variables in the alicloud-virtual-kubelet.yaml file.
# cat alicloud-virtual-nodes-eci.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: alicloud-virtual-nodes-eci
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: alicloud-virtual-nodes-eci
namespace: kube-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: alicloud-virtual-nodes-eci
namespace: kube-system
---
apiVersion: apps/v1beta2
kind: Deployment
metadata:
name: alicloud-virtual-nodes-eci
namespace: kube-system
labels:
app: alicloud-virtual-nodes-eci
spec:
replicas: 1
selector:
matchLabels:
app: alicloud-virtual-nodes-eci
template:
metadata:
labels:
app: alicloud-virtual-nodes-eci
spec:
serviceAccount: alicloud-virtual-nodes-eci
containers:
- name: alicloud-virtual-kubelet
image: registry.cn-hangzhou.aliyuncs.com/ask/virtual-nodes-eci:v1.0.0.1-aliyun
imagePullPolicy: Always
args: ["--provider", "alibabacloud"]
env:
- name: KUBELET_PORT
value: "10250"
- name: VKUBELET_POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: VKUBELET_TAINT_KEY
value: "virtual-kubelet.io/provider"
- name: VKUBELET_TAINT_VALUE
value: "alibabacloud"
- name: VKUBELET_TAINT_EFFECT
value: "NoSchedule"
- name: ECI_REGION
value: "cn-hangzhou"
- name: ECI_VSWITCH
value: "xxx"
- name: ECI_SECURITY_GROUP
value: "xxx"
- name: ECI_ACCESS_KEY
value: "xxx"
- name: ECI_SECRET_KEY
value: "xxx"
Execute the yaml file and view the pod and node status. The result shows that a new virtual node has been added to the cluster.
# kubectl apply -f ./alicloud-virtual-nodes-eci.yaml
serviceaccount "alicloud-virtual-nodes-eci" created
clusterrolebinding "alicloud-virtual-nodes-eci" created
deployment "alicloud-virtual-nodes-eci" created
# kubectl -n kube-system get pod|grep virtual-nodes
alicloud-virtual-nodes-eci-54b748489c-gkcmn 1/1 Running 0 53s
# kubectl get node
NAME STATUS ROLES AGE VERSION
cn-hangzhou.i-bp19cyop0b1skne04djt Ready <none> 10m v1.11.2
cn-hangzhou.i-bp19cyop0b1skne04dju Ready <none> 10m v1.11.2
cn-hangzhou.i-bp19cyop0b1skne04djv Ready <none> 10m v1.11.2
virtual-kubelet Ready agent 55s v1.11.2
In the console, we can also see the presence of this virtual-kubelet virtual node.
Now that a virtual node is present, we need to set nodeSelector and tolerations for the pod to be run and schedule the pod onto the virtual node "virtual-kubelet".
# cat nginx.yaml
apiVersion: v1
kind: Pod
metadata:
name: nginx
spec:
containers:
- image: nginx
imagePullPolicy: Always
name: nginx
nodeSelector:
kubernetes.io/role: agent
beta.kubernetes.io/os: linux
type: virtual-kubelet
tolerations:
- key: virtual-kubelet.io/provider
operator: Exists
# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx 1/1 Running 0 1m
We can see that the nginx pod is scheduled onto the virtual node and is not running on any worker nodes in the cluster. By using virtual nodes, we implement the serverless deployment of container applications and elastic scaling. Users no longer need to worry about selecting a node scale for clusters or making capacity plans.
Users can follow these steps to quickly deploy virtual nodes. However, there is some complexity because users need to configure AK and vswitch/security groups for clusters. In the future, we will add an Alibaba Cloud Container Service console plug-in for the one-click deployment of virtual nodes. This plug-in will save our users from configuring virtual nodes and provide k8s clusters with powerful elastic scaling. Please stay updated on the latest Alibaba Cloud Container Service news.
166 posts | 30 followers
FollowAlibaba Clouder - June 9, 2020
Alibaba Cloud Native - June 9, 2022
Alibaba Clouder - July 15, 2020
Alibaba Container Service - March 12, 2024
Alibaba Container Service - July 16, 2019
Alibaba Cloud Native - March 28, 2024
166 posts | 30 followers
FollowAlibaba Cloud Container Service for Kubernetes is a fully managed cloud container management service that supports native Kubernetes and integrates with other Alibaba Cloud products.
Learn MoreProvides a control plane to allow users to manage Kubernetes clusters that run based on different infrastructure resources
Learn MoreAccelerate and secure the development, deployment, and management of containerized applications cost-effectively.
Learn MoreA secure image hosting platform providing containerized image lifecycle management
Learn MoreMore Posts by Alibaba Container Service