Community

Blog Events Webinars Tutorials Forum
  1. Blog
  2. Events
  3. Webinars
  4. Tutorials
  5. Forum
Create Account
×
Community Blog Deploy a Nydus Image Acceleration Solution on Anolis 8

Deploy a Nydus Image Acceleration Solution on Anolis 8

This article is a tutorial on Nydus, introducing how to deploy Nydus on Anolis OS to help users get familiar with the basic deployment of Nydus.

By Cloud-Native SIG

1

In the previous article, we described how Anolis OS is the first native Linux kernel that supports image acceleration. The Nydus image acceleration service has re-optimized the existing OCIv1 container image format, redefined the image file system, separated data from metadata, and loaded on demand. This article is a tutorial on Nydus, introducing how to deploy Nydus on Anolis OS to help users get familiar with the basic deployment of Nydus.

1. Deployment Environment

This tutoria uses EC2 virtual machines purchased on Alibaba Cloud. You can deploy Nydus locally or in other cloud environments.

Operating system: Anolis OS 8.4 (ANCK 64-bit)
Kernel version: Linux 4.19
CPU: 2 vCPU@3.5GHz
memory: 8GB
software dependency: Nydus currently only supports Containerd, so you need to use Containerd as the container engine.

The Method of OpenAnolis Using Containerd

dnf --enablerepo Plus install -y containerd

2. Install Nydus

OpenAnolis Community has integrated the latest stable version of Nydus, so we recommend using the OpenAnolis integrated software package to install Nydus. If you want to install the specified version of Nydus, you can download the code package of the upstream open-source version. You can use one of the following installation methods.

2.1 Software Package Integrated by OpenAnolis

dnf --enablerepo Plus install -y nydus-rs nydus-snapshotter

2.2 Use Upstream Open-Source Version

  • Download the executable file of Nydus

Get the latest compressed package from the release page and decompress it.

wget https://github.com/dragonflyoss/image-service/releases/download/v2.1.0-alpha.4/nydus-static-v2.1.0-alpha.4-linux-amd64.tgz
tar -xzvf nydus-static-v2.1.0-alpha.4-linux-amd64.tgz
  • Download the executable file of Nydus Snapshotter

Get the latest package from the release page and decompress it.

wget https://github.com/containerd/nydus-snapshotter/releases/download/v0.2.4/nydus-snapshotter-v0.2.4-x86_64.tgz
tar -xzvf nydus-snapshotter-v0.2.4-x86_64.tgz
mv nydus-snapshotter/containerd-nydus-grpc nydus-static/containerd-nydus-grpc
  • Select Nydus running mode

The Nydus acceleration framework supports three running modes to support on-demand image loading in different scenarios:

  • The most commonly used mode in Nydus is the on-demand loading capability of the container provided by FUSE.
  • Hosting the FUSE protocol through VirtioFS supports VM-based container runtime (such as Kata) to provide RootFS on-demand loading capabilities for containers in VM Guest.
  • RootFS is provided through the kernel-state EROFS read-only file system. Currently, Nydus's EROFS format support has entered the Linux 5.16 mainline, and its kernel-state cache scheme erofs over fscache has been incorporated into the Linux 5.19-rc1 mainline. The kernel-state scheme can reduce context switching and memory copy overhead. This mode can be used under extreme requirements on performance.

Since the first mode has the least environmental dependency, it is suitable for demonstration. Here, we choose the fuse mode and rename the nydusd-fusedev in the nydusd binary file to nydusd:

cd nydus-static
mv nydusd-fusedev nydusd
  • Install the executable file
sudo cp nydusd nydus-image /usr/bin  sudo cp nydusify containerd-nydus-grpc /usr/bin  sudo cp ctr-remote nydus-overlayfs /usr/  cd ..

3. Start the Nydus Snapshotter

3.1 Write the Configuration File

Nydus provides a containerized remote snapshot manager containerd-nydus-grpc to prepare container rootfs and nydus images. Save the nydusd configuration to the /etc/nydusd-config.json to begin:


sudo tee /etc/nydusd-config.json > /dev/null << EOF
{
  "device": {
    "backend": {
      "type": "registry",
      "config": {
        "scheme": "https",
        "skip_verify": false,
        "timeout": 5,
        "connect_timeout": 5,
        "retry_limit": 2
      }
    },
    "cache": {
      "type": "blobcache",
      "config": {
        "work_dir": "cache"
      }
    }
  },
  "mode": "direct",
  "digest_validate": false,
  "iostats_files": false,
  "enable_xattr": true,
  "fs_prefetch": {
    "enable": true,
    "threads_count": 4
  }
}
EOF

3.2 Start the Remote Snapshot Manager

Open a new terminal to operate containerd-nydus-grpc:

sudo /usr/bin/containerd-nydus-grpc \
    --config-path /etc/nydusd-config.json \
    --shared-daemon \
    --log-level info \
    --root /var/lib/containerd/io.containerd.snapshotter.v1.nydus \
    --cache-dir /var/lib/nydus/cache \
    --address /run/containerd/containerd-nydus-grpc.sock \
    --nydusd-path /usr/bin/nydusd \
    --nydusimg-path /usr/bin/nydus-image \
    --log-to-stdout

The cache-dir parameter indicates the local blob cache root directory. If it is not set, the default value is root + "/cache". It overrides the device.cache.config.work_dir in the nydusd-config.json.

4. Configure Containerd

4.1 Add the Following to the Containerd Configuration (Default value: /etc/containerd/config.toml):

[proxy_plugins]
  [proxy_plugins.nydus]
    type = "snapshot"
    address = "/run/containerd/containerd-nydus-grpc.sock"
[plugins."io.containerd.grpc.v1.cri".containerd]
   snapshotter = "nydus"
   disable_snapshot_annotations = false

4.2 Restart Contained

After the configuration is updated, you need to restart the Contained service.

Systemctl restart containerd

5. Start a Container in the Nydus Image Format

This is how to use crictl to start a container in the Nydus image format.

5.1 Write Sandbox yaml File nydus-sandbox.yaml and Pass Nydus Annotation to POD

metadata:
  attempt: 1
  name: nydus-sandbox
  namespace: default
log_directory: /tmp
linux:
  security_context:
    namespace_options:
      network: 2
annotations:
  "io.containerd.osfeature": "nydus.remoteimage.v1"

5.2 Write a Container yaml File nydus-container.yaml to Specify the Container Image to Use

metadata:
  name: nydus-container
image:
  image: cloud-native-sig-registry.cn-hangzhou.cr.aliyuncs.com/openanolis/anolisos:8.6-x86_64-nydus
command:
  - /bin/sleep
args:
  - 600
log_path: container.1.log

Here, we use the image of Anolis 8.6 that has been integrated with the OpenAnolis cloud-native image repository as the test image.

5.3 Pull the Image and Start the Container

date
crictl pull cloud-native-sig-registry.cn-hangzhou.cr.aliyuncs.com/openanolis/anolisos:8.6-x86_64-nydus
pod=`crictl runp nydus-sandbox.yaml`
container=`crictl create $pod nydus-container.yaml nydus-sandbox.yaml`
crictl start $container
crictl ps
date

2

It takes only two seconds to pull and start the container image using the Nydus image. Under the same conditions, we create an OCIv1 image for comparison. We still use the Anolis 8.6 image, and the content of the image is the same Nydus mentioned above. The following is the yaml file writing:

metadata:
  attempt: 1
  name: normal-sandbox
  namespace: default
log_directory: /tmp
linux:
  security_context:
    namespace_options:
      network: 2
metadata:
  name: normal-container
image:
  image: cloud-native-sig-registry.cn-hangzhou.cr.aliyuncs.com/openanolis/anolisos:8.6
command:
  - /bin/sleep
args:
  - 600
log_path: container.1.log

Use the following command:

date
crictl pull cloud-native-sig-registry.cn-hangzhou.cr.aliyuncs.com/openanolis/anolisos:8.6
pod=`crictl runp normal-sandbox.yaml`
container=`crictl create $pod normal-container.yaml normal-sandbox.yaml`
crictl start $container
crictl ps
date

3

In the same environment, it takes ten seconds to start the version of Anolis 8.6 using the OCIv1 image format, which is five times that of Nydus.

6. Convert and Start the Nydus Image

This shows how to convert the Nydus image and push it to your image repository. Log into the image repository and use the nerdctl tool for configuration.

6.1 Install Nerdctl and CNI Plugin

Nerdctl is a command line compatible with Docker. Since it can support starting Nydus images, we choose to use it here. Since the container may need to rely on some plug-ins during operation, we install the CNI plugin at the same time.

dnf update -y anolis-repos && yum install -y anolis-experimental-release && yum install -y nerdctl
dnf install -y containernetworking-plugins

Use nerdctl login to log on to the repository for authentication. You also can use a Docker login to log on to the repository.

nerdctl login --username ${your username} --password xxx

6.2 Convert the Image to the Nydus Format and Push It to the Remote Image Repository

nydusify convert --nydus-image /usr/bin/nydus-image -- source ${your image} --target ${your registry address} /${image name}: ${tag}

Follow these steps. Congratulations, you have successfully deployed the Nydus image acceleration solution on Anolis OS.

References

  1. Nydus Software Package Release Page
    https://github.com/dragonflyoss/image-service/releases/latest
  2. The Nydus Snapshotter Release Page under Containerd
    https://github.com/containerd/nydus-snapshotter/releases/tag/v0.2.4
  3. More Technical Details of Nydus (Article in Chinese)
    https://developer.aliyun.com/article/971522
  4. More Detailed Deployment Description of Nydus
    https://github.com/dragonflyoss/image-service/blob/master/docs/containerd-env-setup.md
Developers Containers Tutorials Operating System Image Acceleration Anolis Nydus Nerdctl
0 1 1
Share on

Read previous post:

Learn about the Latest Version of SysAK

Read next post:

A Paper on RunD: A Lightweight Secure Container Runtime for High-density Deployment and High-concurrency Startup in Serverless Computing

OpenAnolis

87 posts | 5 followers

You may also like

Comments

OpenAnolis

87 posts | 5 followers

Related Products

More Posts by OpenAnolis

See All