By Zheng Chao, Senior Development Engineer at Alibaba Cloud
With the rapid development of Internet of Things (IoT) and 5G technologies, extending cloud computing capabilities to edge devices and ensuring unified delivery and management of edge devices through a centralized Kubernetes master have become important development trends in cloud computing. To help developers keep up with the trend, Alibaba officially released the cloud-native edge computing framework OpenYurt on May 29, 2020. The framework is based on the edge cluster hosting service (ACK@Edge).
OpenYurt has attracted the attention of developers since it was released. This article introduces OpenYurt and describes how you can use the OpenYurt command line tool Yurtctl to efficiently deploy OpenYurt clusters.
OpenYurt focuses on cloud-edge integration. Based on Kubernetes' strong container application orchestration capabilities, OpenYurt provides the necessary capabilities for the distribution, delivery, and control of cloud-edge applications. Compared with other Kubernetes-based edge computing frameworks, OpenYurt adheres to the "minimum modification" principle. By installing the YurtHub component on edge nodes and deploying the Yurt-controller-manager component on the cloud, OpenYurt provides the capabilities required to manage edge computing applications with zero intrusion into Kubernetes clusters.
OpenYurt helps you perform large-scale application delivery, O&M, and control for massive volumes of edge and terminal resources. OpenYurt also provides channels for the provision of cloud computing services to edge applications. When the computing amount is large, cloud computing resources are used. When the computing amount is small, edge computing resources are used. OpenYurt is designed to conveniently extend native Kubernetes to the edge and provide a consistent user experience, without increasing the O&M workload.
To empower native Kubernetes clusters with edge computing capabilities, OpenYurt uses add-ons to provide native Kubernetes clusters with the following capabilities in an unintrusive manner:
In addition, we must find a way to seamlessly integrate enhanced edge computing capabilities with native Kubernetes clusters. Based on the online O&M experience of ACK@Edge, we released the Yurtctl command line tool for seamless conversion between native Kubernetes and OpenYurt clusters and efficient O&M for OpenYurt components.
Figure 1: Yurtctl convert process
Yurtctl is a centralized management tool. In the OpenYurt cloud-edge architecture, Yurtctl interworks with API servers. It operates and maintains each node based on Job workloads in a native Kubernetes cluster. As shown in Figure 1, Yurtctl deploys a servant pod to the specified edge node through a Job workload during the convert operation.
For more information about the operations on containers in the servant pod, see: https://github.com/alibaba/openyurt/blob/master/config/yurtctl-servant/setup_edgenode
The servant pod needs to operate on the file system of the root user on the node (such as when storing the YurtHub configuration file in the /etc/kubernetes/manifests directory) and then reset the system management program kubelet.service. Therefore, containers in the servant pod are granted privileged permissions. They can share a PID namespace with the node and use the nsenter command to enter the primary namespace of the node to perform relevant operations. After the servant job workload is executed, it is deleted automatically. If the job workload fails to be executed, it is retained to help O&M personnel troubleshoot the cause of the error. With this mechanism, Yurtctl can also update or delete YurtHub.
This demo is developed on Alibaba Cloud Kubernetes.
The OpenYurt GitHub repository contains Yurtctl source code. After you download the OpenYurt repository, you can compile the following command to obtain Yurtctl:
$ make build WHAT=cmd/yurtctl
hack/make-rules/build.sh cmd/yurtctl
Building cmd/yurtctl
After successful compilation, you can find the Yurtctl executable file in the _output/bin/
directory.
To convert a Kubernetes cluster with two nodes (node 1 and node 2) to an OpenYurt cluster with only node 2 as an autonomous edge node, run the following Yurtctl convert command:
$ yurtctl convert --cloud-nodes node1 --provider ack
I0603 14:34:33.714304 40825 convert.go:164] mark node1 as the cloud-node
I0603 14:34:33.719816 40825 convert.go:172] mark node2 as the edge-node
I0603 14:34:33.736609 40825 convert.go:198] deploy the yurt controller manager
I0603 14:34:33.742272 40825 convert.go:210] deploying the yurt-hub and resetting the kubelet service...
I0603 14:34:53.810165 40825 util.go:168] servant job(yurtctl-servant-convert-node2) has succeeded
After the nodes are configured, run the following command to mark the edge node as autonomous:
$yurtctl markautonomous# To mark only some edge nodes, use the autonomous-nodes option.
I0602 11:22:05.610222 89160 markautonomous.go:149] mark node2 as autonomous
Then, test whether node 2 is autonomous when the network is disconnected. Deploy a test pod on node 2.
$ kubectl apply -f-<<EOF
apiVersion: v1
kind: Pod
metadata:
name: bbox
spec:
nodeName: node2
containers:
- image: busybox
command:
- top
name: bbox
EOF
pod/bbox created
Log on to node 2 and set the server-addr parameter of YurtHub to an inaccessible address.
sudo sed -i 's|--server-addr=.*|--server-addr=https://1.1.1.1:1111|' /etc/kubernetes/manifests/yurt-hub.yaml
Wait for 40 seconds. The result shows that when node 2 is in the NotReady state, pod 1 is still in the Running state. This indicates that a pod deployed on an autonomous edge node will not be removed by the cloud node controller even when the autonomous edge node is offline.
$ kubectl get node
NAME STATUS ROLES AGE VERSION
node1 Ready master 14m v1.14.8
node2 NotReady <none> 12m v1.14.8
$ kubectl get pod
NAME READY STATUS RESTARTS AGE
bbox 1/1 Running 0 5m12s
If node 2 is restarted, we can run the docker ps command to verify that the bbox pod will be restarted. It is assumed that node 2 uses Docker as the container runtime.
$ docker ps --format 'table {{.ID}}\t{{.Image}}\t{{.RunningFor}}' | grep busybox
d0c8134fddc1 busybox About a minutes ago
This is because the kubelet reads the cached data from YurtHub to restore the pod status before the restart. We will describe the technical details in subsequent articles.
You can run the yurtctl revert
command to convert an OpenYurt cluster back to a Kubernetes cluster. For example, to convert the preceding two-node OpenYurt cluster back into a Kubernetes cluster, run the following command. Before running the command, reconnect YurtHub on node 2 to the API server.
$ yurtctl revert
I0603 14:38:55.522376 41016 revert.go:106] label alibabacloud.com/is-edge-worker is removed
I0603 14:38:55.527998 41016 revert.go:116] yurt controller manager is removed
I0603 14:38:55.548354 41016 revert.go:130] ServiceAccount node-controller is created
I0603 14:39:05.572686 41016 util.go:168] servant job(yurtctl-servant-revert-node2) has succeeded
I0603 14:39:05.572718 41016 revert.go:142] yurt-hub is removed, kubelet service is reset
For more information about how to use Yurtctl, see the following Yurtctl tutorial in the OpenYurt GitHub repository.
https://github.com/alibaba/openyurt/tree/master/docs/tutorial
Yurtctl is designed to be a powerful tool O&M personnel can use to manage OpenYurt clusters. We will continuously evolve Yurtctl to support new OpenYurt features and new O&M processes and scenarios. For example, Yurtctl will support OpenYurt components, such as Yurttunnel and Yurtunit that will be made open source in the near future. We also welcome you to report your needs, so we can make Yurtctl better.
The OpenYurt community always welcomes new users. You can use GitHub issues to obtain technical support, report bugs, and put forward needs and suggestions.
What Is the Difference Between KubeVela and PaaS? An In-Depth Interpretation
506 posts | 48 followers
FollowAlibaba Developer - January 21, 2021
Alibaba Developer - January 11, 2021
Alibaba Cloud Native Community - January 9, 2023
Alibaba Developer - May 31, 2021
Alibaba Developer - July 9, 2021
Alibaba Developer - January 20, 2021
506 posts | 48 followers
FollowProvides secure and reliable communication between devices and the IoT Platform which allows you to manage a large number of devices on a single IoT Platform.
Learn MoreLink IoT Edge allows for the management of millions of edge nodes by extending the capabilities of the cloud, thus providing users with services at the nearest location.
Learn MoreA cloud solution for smart technology providers to quickly build stable, cost-efficient, and reliable ubiquitous platforms
Learn MoreMigrate your Internet Data Center’s (IDC) Internet gateway to the cloud securely through Alibaba Cloud’s high-quality Internet bandwidth and premium Mainland China route.
Learn MoreMore Posts by Alibaba Cloud Native Community