Community

Blog Events Webinars Tutorials Forum
  1. Blog
  2. Events
  3. Webinars
  4. Tutorials
  5. Forum
Create Account
×
Community Blog A Discussion on Container Runtime - Starting with Dockershim Being Deleted by Kubernetes

A Discussion on Container Runtime - Starting with Dockershim Being Deleted by Kubernetes

This article explains the evolution of OCI and Docker, explores CRI and CRI-compliant containerd and CRI-O, and describes the principle of Kubernetes log collection.

By Yemo, from Alibaba Cloud Storage Team

Background

Kubernetes 1.24 was released in May 2022. The highlight was that the dockershim component was removed from kubelet in this release. Starting from this release, users must choose containerd or CRI-O as the container runtime when using Kubernetes. If you want to continue to rely on Docker Engine as a container runtime, the cri-dockerd component is required.

Why did Kubernetes remove Docker? The main reason is that Docker does not support the CRI container runtime standard promoted by Kubernetes for a long time. Kubernetes needs to maintain the dockershim component for a long time to specifically adapt to Docker. Dockershim was a product of the early days of Kubernetes and the heyday of Docker. However, with the development of container runtime (such as containerd), Kubernetes has enough reasons to stop maintaining dockershim.

After removing dockershim, does Kubernetes not support Docker at all? The answer is no. At present, in the community, there is a shim cri-dockerd (provided by Mirantis and acquired by Docker Enterprise in 2019) that is independent of Kubernetes and supports CRI. Users can use shim to implement Kubernetes support for Docker.

In addition, since Docker Image has become the standard image format used by various container runtime, it may become a common phenomenon to use Docker in the development phase and use containerd and other container runtimes in the production environment for a long time.

Here, we briefly introduce the whole event of removing dockershim from Kubernetes. At the same time, many concepts including CRI, dockershim, containerd are mentioned. This article will take a deeper look at the container runtime.

The Evolution of OCI and Docker

When it comes to container runtime, we have to mention Open Container Initiative (OCI).

Open Container Initiative

OCI, proposed in the context of preventing the container technology from fragmenting, is a formal specification for formulating container image format and container runtime, which mainly include OCI Runtime Spec, OCI Image Spec, and OCI Distribution Spec.

The well-known runc is a reference implementation of Runtime Spec. It was first migrated from the libcontainer of Docker and later donated to OCI by Docker. Docker containers are created based on runc.

The container runtime can be divided hierarchically:

  • Low-Level Runtime (OCI Runtime): This refers to the container runtime that only focuses on running containers. It is responsible for calling the operating system and using namespaces and cgroups to isolate and restrict resources (such as runc and rkt).
  • High-Level Runtime: Compared to low-level runtimes, it is located at the upper layer of the stack and is responsible for transporting and managing container images, unpacking images, and passing off to the low-level runtime to run the container (such as containerd and cri-o).

1

Docker

Starting from Docker 1.11 (2016), Docker Daemon was split into multiple modules to adapt to the OCI standard. Among them, containerd is a container runtime that manages the lifecycle of a container (such as creation, start, stop, abort, signal processing, and deletion). Other services (such as image building, volume management, and log) are provided by other modules of Docker Daemon.

2

The container operation process in the figure above can be simplified as Docker Daemon → containerd → containerd-shim → runc. Among them, the containerd-shim is the direct operator of containers.

containerd-shim is introduced here because runc will exit after the container is created and run. As such, containerd-shim is used as the parent process of the container instead of containerd. This avoids the problem that all containers on the entire host have to exit when containerd does not work.

Container Runtime Interface (CRI)

What Is CRI?

When it comes to Kubernetes removing dockershim events, we have to mention CRI. First of all, let's understand the background of CRI. In the early days of Kubernetes (before v1.5), Docker was the only container runtime supported by Kubernetes, and Kubelet directly called Docker API through hard coding. Later, a new container runtime rkt appeared and hoped to be integrated into the Kubelet code. Then, as a growing number of container runtimes emerged, it became clear that embedding was no longer suitable. In this context, the Container Runtime Interface (CRI) standard is proposed to decouple the Kubelet code from the implementation code of specific container runtime.

3

However, when Kubernetes launched CRI, it was not as dominant as it is now. Various container runtimes did not actively implement CRI, so it is necessary to adapt various container runtimes through CRI shim. For example, Docker had no intention to support CRI for the following reasons: Docker appeared earlier than Kubernetes, and Docker was in a stable and stronger position. Docker intended to promote Swarm, which was regarded as a competitor of Kubernetes. Finally, Kubelet chose the built-in dockershim to provide support for Docker.

Below is the architecture of Kubelet after the CRI interface is introduced.

4

The API defined by CRI mainly consists of two gRPC services: ImageService and RuntimeService.

  • ImageService is responsible for pulling, viewing, and deleting images.
  • RuntimeService is responsible for managing the lifecycle of pods and containers and interaction calls among containers (such as exec, attach, and port-forward).

Thanks to CRI, kubelet can achieve unified management of container runtimes (such as Docker, containerd, and CRI-O).

5

The supporting forms of Docker:

  • Docker enjoyed a high status in the early days of CRI. Kubernetes built dockershim into kubelet to support Docker.
  • Process: Kubelet (CRI Client) calls dockershim (CRI Server) through CRI. Dockershim requests the Docker Daemon to call containerd and then use containerd-shim and runc to create a container.

The call link of Docker is too long, and there are many redundant operations. As the CRI ecosystem became more perfect, the Kubernetes community removed dockershim in July 2020. Dockershim was officially deleted in version 1.24.

The supporting forms of containerd:

  • Containerd 1.0: Adaptation to CRI is achieved through a separate CRI-Containerd process. The main reason why a separate CRI-Containerd is redundant is that CRI did not have absolute dominance in the early days and required kubelet to adapt to various container runtimes.
  • Containerd 1.1: The CRI-Containerd is removed, and the adaptation logic is directly integrated into the main process of containerd as a plug-in.

In summary, kubelet can achieve unified management of Docker, containerd, and CRI-O with the help of CRI. After Kubernetes 1.24 removes the dockershim component from kubelet, we recommend using containerd or CRI-O. Next, we will focus on containerd and CRI-O.

Containerd

Containerd is an industry-standard container runtime that emphasizes simplicity, robustness, and portability.

  • Manage the lifecycle of containers (from creation to destruction)
  • Pull / push container images
  • Storage management (managing the storage of images and container data)
  • Call runc to run the container (interact with runc and other container runtimes)
  • Manage container network interfaces and networks

Containerd was originally separated from Docker. As an independent open-source project, containerd aims to provide a more open and stable container operation infrastructure. (Please see the architecture diagram in the Docker section for details.)

6

The official architecture diagram provided by containerd shows that containerd also adopts the C/S architecture. containerd manages the complete container lifecycle of its host system: image transfer and storage, container execution and supervision, low-level storage and network attachments, etc. Runc is responsible for running containers. All containers that comply with OCI specifications can be supported.

In addition, containerd implements a richer container interface. In addition to using CRI, you can use the ctr tool to call these rich container runtime interfaces. For example:

  • crictl: A Docker-like command-line tool for operating CRI
  • critest: Verify whether the CRI interface meets expectations
  • Performance Tool: Test interface performance

CRI-O

The Kubernetes community has also developed a CRI runtime CRI-O specifically for Kubernetes, and the CRI-O is directly compatible with CRI and OCI specifications. It is a lightweight alternative to using Docker as the runtime for Kubernetes and supports any OCI-conformant runtimes. Kubelet talks directly to the CRI-O through CRI to extract an image and launch a lower-level runtime (such as runc).

An important difference between CRI-O and containerd is that CRI-O removed some unnecessary Linux features to reduce the possibility of external attacks.

CRI-O is a CRI service implemented by packaging the container interface directly on the OCI. Only specific CRI interfaces are provided to the public, and it does not have the ctr tool capability similar to containerd.

7

The overall architecture is shown below:

8

Multiple Container Runtimes

With the emergence of more container runtimes, different container runtimes have different requirement scenarios, so there is a need for multiple container runtimes, for which the Kubernetes community has introduced RuntimeClass.

9

Currently, sandboxed containers of Alibaba Cloud Container Service for Kubernetes (ACK) support multiple container runtimes. As shown in the following figure, the Pod of runc is on the left side, and the corresponding RuntimeClass is runc. The Pod of runv is on the right side, and the corresponding RuntimeClass is runv. You can configure multiple container runtimes in containerd.

  • runc: kube-apiserver → kubelet → cri-plugin (cri-plugin queries the containerd configuration file for the Handler corresponding to runc) → Shim API runtime v1 → containerd-shim (it is a plug-in that implements CRI) → create a runc container
  • runv: kube-apiserver → kubelet → cri-plugin (cri-plugin queries the containerd configuration file for the Handler corresponding to runv) → Shim API runtime v2 → containerd-shim-kata-v2 (it is a plug-in that implements CRI) → create a kata container

10

Kubernetes Log Collection Support

As an important part of observability construction, logs can record access requests and error information in detail, which is convenient for us to locate problems. Applications on Kubernetes, Kubernetes components, and hosts generate various types of log data. With the development of Kubernetes, it has gone through the phases of Docker to CRI, increasing the complexity of log collection.

iLogtail, an infrastructure for collecting observability data launched by Alibaba Cloud Log Service (SLS), has good support for Docker and containerd in Kubernetes scenarios. iLogtail generally uses two deployment modes: Sidecar and DaemonSet.

11

The prerequisite for Logtail to collect business Pods or containers is to have access to the container runtime on the host and have access to the business container's data.

  • Access the Container Runtime: The Logtail container will mount the sock of container runtime (Docker Engine/ContainerD) on the host to the container directory and the Logtail container can access the Docker Engine/ContainerD of this node.
  • Access the Data of Business Containers: The Logtail container mounts the root directory ('/' directory) of the host to the /logtail_host of the container. You can access the data of other containers through the /logtail_host directory (provided that the file system of the Docker Engine is stored on the host in the form of a common file system (such as overlays) or the log directory of the container is mounted to the host through the hostPath or emptyDir volume).

12

Please read the article entitled Comprehensive Analysis of Kubernetes Log Collection Principles for more information. iLogtail is open-source. You are welcome to follow us.

Summary

  • This topic describes the evolution of OCI and Docker.
  • This topic describes CRI and CRI-compliant containerd and CRI-O.
  • This topic describes the principle of Kubernetes log collection.

For more information, please check containerd github

Docker Container containerd Computing Kubernetes cloud native K8s Container Runtime Dockershim
0 1 0
Share on

Read previous post:

Common Causes and Troubleshooting Methods for Connection Reset

Read next post:

Alibaba Cloud's Energy Expert Helps Analyze Carbon Footprint for The First Olympic Esports Week

Alibaba Cloud Community

1,076 posts | 263 followers

You may also like

Comments

Alibaba Cloud Community

1,076 posts | 263 followers

Related Products

More Posts by Alibaba Cloud Community

See All