By Zhongyi
As the name suggests, similar to an operating system .iso image or Docker image, the cluster image is a resource package that combines all files in a cluster with a certain format using specific technical means.
Some interesting features can be found when comparing a single machine and a cluster:
Let’s take a Kubernetes-based cluster image as an example. The image contains all files except the operating system, including:
Similarly, when a cluster image is running, it is not installed on a container or a machine. Instead, the image can be installed on multiple servers or the infrastructure of the public cloud directly.
Sealer is an implementation method of Alibaba's open-source cluster image. Click here for the project address.
Docker solves the problem of mirroring a single container, while sealer builds, shares, and runs distributed software by packaging the entire cluster!!!
Imagine that we will deliver a SaaS application that relies on databases and middleware, such as MySQL, ES, or Redis, and everything is orchestrated on Kubernetes. If there is no cluster image, the following operations will be done:
This process may not seem complicated. However, from the perspective of project delivery, the preceding operations are process-oriented and error-prone.
Now, if another method was provided to solve the problem above with only one command, would you like to use it?
sealer run your-saas-application-with-mysql-redis-es:latestWe can see that only one cluster image needs to be run, and the entire cluster is delivered. By doing so, all detailed and complex operations are blocked, and any applications can run in the same way. How did the cluster image come from?
As shown in the figure above, we only need to define a file like Dockerfile, which is called Kubefile, and then run the build command below:
sealer build -t your-saas-application-with-mysql-redis-es:latestIt is clear when comparing a single machine with a cluster.
Let's create and run a cluster image from the Kubernetes dashboard together to experience the complete process.
Write the Kubefile:
# Base image, which has been created with all Kubernetes startup-related dependencies
FROM registry.cn-qingdao.aliyuncs.com/sealer-io/cloudrootfs:v1.16.9-alpha.7
# Download the official dashboard YAML orchestration file and use the COPY command once it has been downloaded
RUN wget https://raw.githubusercontent.com/kubernetes/dashboard/v2.2.0/aio/deploy/recommended.yaml
# Specify the running methods, such as kubectl, helm, kustomiz
CMD kubectl apply -f recommended.yamlBuild a dashboard cluster image:
sealer build -t kubernetes-with-dashobard:latestRun the cluster image:
# The following command will install the Kubernetes cluster on the server and apply the dashboard. You can use passwd to specify the server ssh password, or alternatively use a key
sealer run kubernetes-with-dashobard:latest \
--master 192.168.0.2,192.168.0.3,192.168.0.4 \
--node 192.168.0.5,192.168.0.6 \
--passwd xxx
# Check the pod
kubectl get pod -A |grep dashboardPush the prepared image to the repository. The image is also compatible with the Docker registry:
sealer tag kubernetes-with-dashobard:latest docker.io/fanux/dashobard:latest
sealer push docker.io/fanux/dashobard:latestBy doing so, we can deliver the prepared image or make it available for reuse.
What specific things can Sealer help us do? The following section explains some main scenarios:
This is the simplest scenario. You can use Sealer to install whether you need to install a development test environment on a single machine or install a high availability cluster in a production environment, whether it is bare metal, docked to the public cloud, or a variety of architecture operating systems. If you only want to install Kubernetes, you can just choose a base image.
Compared with other installation tools, Sealer has the following advantages:
(1) It is very simple that Sealer only runs one command.
(2) The speed is so fast that Sealer installs six nodes in three minutes. Sealer may be installed before you have finished downloading the nodes with other tools. Furthermore, it can be optimized to within two minutes (or even one minute) with advanced technology.
(3) Sealer has compatibility and stability that is compatible with various operating systems and supports architectures, such as x86 and ARM.
(4) Sealer has a consistent design, ensuring that the cluster stays defined in the Clusterfile. For example, the upgrade can be done by simply changing the version number in the Clusterfile.
Sealer’s fast speed is because of the Golang implementation. It means that we can do concurrent processing in various detailed places, which has more advantages than Ansible. Moreover, it supports more detailed error handling. Then, the previous load method is discarded on image distribution, and file distribution will be optimized to maximize installation performance.
For compatibility, Docker Kubelet uses the binary system plus systemd to install the core components into full containerization. This eliminates the need to rely on installation tools for perceptive operating systems, such as yum or apt. ARM and x86 adopt different image supports decoupled from Sealer. Adaptation to the public cloud is also implemented by extracting a separate module. Here, we did not connect to Terraform for performance. In our scenario, the startup of the infrastructure in Terraform was about three minutes, but we optimized it using backoff retry to less than 30 seconds. In addition, in the cluster image scenario, there is no need for such complex infrastructure management capabilities since we do not want to make Sealer heavy or rely on a command-line tool.
The consistent design is worth mentioning in Sealer. Cluster images and Clusterfile determine how a cluster looks. The same image and Clusterfile can run the same cluster. The Clusterfile can be changed by adding nodes, changing node specifications, or changing images. The layer with the same hash value will not be changed when changing images since the cluster image is also a hierarchical structure, and a change in the hash will help re-apply the layer.
Sealer can create a cluster with Prometheus or install Prometheus in an existing cluster by running prometheus:latest.
What is the difference between the helm and sealer?
(1) Sealer pays more attention to packaging rather than orchestration. In the preceding example, Prometheus can use helm for orchestration. Then, Sealer packages the chart and all the container images needed in the chart. This is achieved through advanced technologies during the build process. The build process starts a temporary Kubernetes cluster like Docker build. Then, which container images the cluster depends on will be known. The final step is to package container images.
(2) Sealer may not be able to install successfully with a chart by packaging with Kubernetes. For example, Sealer uses the abandoned API version, but the image is also packaged with Kubernetes. As long as the build is fine, the run will be fine. This is similar to the way Docker packages the operating system rootfs together.
(3) Sealer is integrated, so cluster images focus on the overall packaging of the entire distributed application cluster. For example, the Prometheus ELK MySQL cluster is made into an image service and business.
Therefore, Sealer and helm are collaborative, with a clear division of work.
These common cluster images can be found in the official repository of Sealer later and can be applied to users directly.
From the perspective of distributed applications, in some situations, there can be components in the single digits, but in other situations, there can also be components in the hundreds. The entire delivery methods are process-oriented and require intervention, while Sealer can pack all these things together for one-click delivery.
Can't we add Ansible scripts to tar.gz to enable one-click delivery? The answer is yes. Just as before the Docker image appeared that everyone delivered through tar.gz, you will find that the emergence of standards and technologies has solved the problem of collaboration between users. With Cluster images, users can reuse the achievements of others and create things for others to use.
Sealer is suitable for Alibaba Cloud Apsara Stack scenarios. Many data centers of customers are offline, and cluster images put all dependencies in the image. Therefore, all local sites can carry out one-click delivery in the same way and obtain an excellent, consistent experience as long as the image is well done.
Sealer can integrate with the public cloud. In many cases, a better user experience will be given if it is connected to the public cloud. For example, when installing a cluster, users only need to specify the number and specifications of servers without caring about the IP address and then modify the numbers defined in the Clusterfile directly.
The storage of cluster images is also achieved by copy-on-write. This provides two advantages. On the one hand, users can deploy different distributed software in the same cluster at different layers to achieve reuse. On the other hand, users can push cluster images to the Docker repository directly.
During the build process, how does Sealer know which container images are in the cluster image to be built, and how can it store the container images? There are some difficult problems during the process:
(1) How do you know which container images are available in distributed software? These images need to be cached. Scanning the user's YAML file or scanning after with the helm template is always imperfect. First, there is no way to determine what user's orchestration. Since some software does not write the image address in the orchestration files but pulls it up through their own programs, there is no guarantee that a successful build run will be sufficient.
(2) A container image needs to be stored in a private repository and packaged in a cluster image. The address of the container repository is different from the container image in the orchestration file. In particular, how do you ensure users can still download the image in the private repository when users pull image.
There is a solution to the first problem. Like in a Docker build, a temporary Kubernetes cluster will be pulled up during the process of Sealer build, and the apply command defined in the Kubefile will also be executed.
As shown in the preceding figure, this ensures that all images that users depend on are packaged, regardless of the orchestration methods used.
The second problem is learning how to use this private image when packing the container image into a private repository. Let’s suppose that the private repository is named localhost:5000. This will be inconsistent with what is written in the orchestration file. Therefore, two solutions exist to solve this problem:
Such Sealer solutions solve the problem of image packaging offline perfectly.
The high availability of Sealer's cluster uses a lightweight load balancer lvscare. Compared with other load balancers, lvscare is very small, with only a few hundred lines of code. In addition, lvscare only guards the IPVS rules without loading. They monitor the apiserver on nodes. If it fails, lvscare will remove the corresponding rules and add them back automatically after starting again. In other words, it is equivalent to a dedicated load balancer It has been used in sealos for more than two years and has extensive practice.
The runtime environment supports the running of applications. For example, the Sealer runtime based on Kubernetes can be supported in a very simple and transparent way. Let’s take istio as an example, users only need to do the following:
FROM kubernetes:v1.18.3
RUN curl -L https://istio.io/downloadIstio | sh -Users can build an istio runtime for their applications by following the nodes above.
If it is not a runtime based on Kubernetes, such as k0s and k3s, users can extend the interface in the Sealer.Runtime:
FROM k3s:v1.18.3
RUN curl -L https://istio.io/downloadIstio | sh -For better extensions, such as extending the ACK runtime:
FROM aliyum.com/ACK:v1.16.9
RUN curl -L https://istio.io/downloadIstio | sh -This image helps users deploy applications in ACK. Some of the above are used in the roadmap.
Now, many users want to run their cluster images on the cloud. Sealer comes with the ability to dock to the public cloud, and it can also implement its own infrastructure manager, which benefits from a more refined back-off retry mechanism. The performance of 30 seconds to complete the infrastructure build (six nodes in Alibaba Cloud) is the best in its class, and the number of API calls is reduced significantly with a Clusterfile-compatible configuration.
There are several future visions and values of Sealer:
Finally, it can be summarized:
More information about sealer: https://github.com/alibaba/sealer
503 posts | 48 followers
FollowAlibaba Cloud Native Community - May 31, 2022
Alibaba Cloud Native - May 23, 2022
Alibaba Cloud Native Community - June 29, 2022
Alibaba Clouder - July 15, 2020
Alibaba Cloud Native - July 14, 2023
OpenAnolis - April 7, 2023
503 posts | 48 followers
Follow
Super Computing Cluster
Super Computing Service provides ultimate computing performance and parallel computing cluster services for high-performance computing through high-speed RDMA network and heterogeneous accelerators such as GPU.
Learn More
ApsaraDB for MyBase
ApsaraDB Dedicated Cluster provided by Alibaba Cloud is a dedicated service for managing databases on the cloud.
Learn More
ACK One
Provides a control plane to allow users to manage Kubernetes clusters that run based on different infrastructure resources
Learn More
Function Compute
Alibaba Cloud Function Compute is a fully-managed event-driven compute service. It allows you to focus on writing and uploading code without the need to manage infrastructure such as servers.
Learn MoreMore Posts by Alibaba Cloud Native Community