Community

Blog Events Webinars Tutorials Forum
  1. Blog
  2. Events
  3. Webinars
  4. Tutorials
  5. Forum
Create Account
×
Community Blog Inclavare Containers: The Future of Cloud-Native Confidential Computing

Inclavare Containers: The Future of Cloud-Native Confidential Computing

This article explains the development of Inclavare Containers project and interprets its core ideas and innovative technologies.

1

By SIG for Cloud Native Confidential Computing

As the first open-source container runtime in the industry for confidential computing scenarios, Inclavare Containers was open-sourced in May 2020. It has developed rapidly over a year, attracting the attention and contributions of experts and engineers in many fields. On September 15, 2021, the cloud-native computing foundation (CNCF) announced it would adopt Inclavare Containers as the CNCF official sandbox project through a vote from Technical Oversight Committee (TOC). It became the first confidential computing project in cloud-native that entered the CNCF.

However, unless you follow up on the information in the field of confidential computing, you may not know much about Inclavare Containers. This article will sort out the development of Inclavare Containers project and interpret its core ideas and innovative technologies.

What is Inclavare Containers?

In a nutshell, Inclavare Containers is the industry's first open-source container runtime for confidential computing scenarios.

What is confidential computing? What is the relationship between Inclavare Containers and confidential computing? What problems can it solve?

Data Security and Confidential Computing

Data has three states throughout its lifecycle: At-Rest, In-Transit, and In-Use.

  • At-Rest, data is stored on hard disks, flash memory, or other storage devices. There are many ways to protect data in the At-Rest state, such as encrypting files and storing them or encrypting storage devices.
  • In-Transit refers to the transmission of data from one place to other places through a public or private network. Users can encrypt files before transmission or use secure transmission protocols to ensure data security during transmissions, such as HTTPS, SSL, TLS, and FTPS.
  • In-Use refers to the data being used. Even if the data is encrypted during transmission, it can only be calculated and used after the data is decrypted. If the data is not protected when it is used, there is a risk of data breach and tampering.

We store, use, and share all kinds of sensitive data, including credit card data, medical records, firewall configurations, and geographic location data. Protecting sensitive data in all states is more important than ever. Nowadays, the widely used encryption technology can be used to guarantee data confidentiality (to prevent unauthorized access) and data integrity (to prevent or detect unauthorized modification). However, these technologies are mainly used to protect data in transmission and at rest. Currently, the technology of providing security protection for the state of In-Use is still a new frontier technology.

Confidential computing refers to the use of hardware-based Trusted Execution Environments (TEE) to protect data in use. We can protect against data In-Use by using confidential computing.

Core functions of confidential computing include:

  • Protect the Confidentiality of In-Use Data: Unauthorized entities (applications on the host, host operating systems and Hypervisor, system administrators, or any other person with physical access to the hardware) cannot view the data used in TEE. The data in the memory is encrypted. Even if the data is stolen by an attacker, the data will not be leaked.
  • Protect the Integrity of In-Use Data: This prevents unauthorized entities from tampering with the data being processed. The measurement value guarantees the integrity of the data and code. Any data or code changes in use will cause changes to the measurement value.
  • Provability: TEE can provide evidence or measures of their origin and current state so the other party can verify and decide whether to trust the code running in the TEE. Most importantly, such evidence is signed by the hardware, and the manufacturer can provide proof, so the party verifying the evidence can guarantee the evidence is reliable and not generated by malicious software or other unauthorized entities to a certain extent.

Current Situation and Difficulty of Confidential Computing

Many manufacturers in the industry have begun to pay attention to and invest in confidential computing. Major chip manufacturers and cloud service providers (CSPs) have invested R&D resources in the field of confidential computing and formed the Confidential Computing Alliance. This alliance is dedicated to cloud services and hardware ecosystems and to protecting data security during computing.

Currently, three hardware platforms support TEE: Intel® SGX, ARM TrustZone, and AMD SEV. They have different application scenarios and implementations:

  1. ARM TrustZone divides hardware resources into the safe world and the non-safe world. All operations that need to be kept secret are executed in the safe world, and the rest are executed in the non-safe world. The safe world and the non-safe world are converted through a mode called Monitor Mode. Typical application scenarios include mobile payments, digital wallets, and others.
  2. AMD leverages technologies, such as SEV (AMD Secure Encrypted Virtualization), SME (AMD Secure Memory Encryption), and SEV-ES (Secure Encrypted Virtualization-Encrypted State), to enable guest memory encryption and security isolation of virtual machines.
  3. Intel® SGX is a set of instructions provided by Intel to improve the security of the application's code and data. Intel® SGX program consists of untrusted code and trusted Enclave. Sensitive code and data are put into Enclave. Intel® SGX instruction creates and executes Enclave in a specific encrypted memory area (EPC), where developers define restricted ingress and egress functions to prevent a data breach.

Currently, confidential computing is flourishing, the market and commercialization potential is huge. However, it has shortcomings in cloud-native scenarios:

  1. The currently available technology has a high threshold for use and development. Let's use developing an Intel® SGX application as an example. Users need to learn Intel® SGX development skills and transform the business. Compared with traditional development methods, its use and development barriers are high, which makes many developers daunted.
  2. The cost and complexity of confidential computing containerization and docking Kubernetes are high. More users are accepting cloud-native. Even if users have mastered the development skills of confidential computing, many problems have to be overcome if confidential computing applications run in containers or Kubernetes, such as how to load SGX drivers in containers and how to allocate EPC memory for containers.
  3. The technical solution offered by a service provider is relatively monotonous. Currently, many service providers give their technical solutions for confidential computing, but they are monotonous and fail to fully meet the needs of users on the cloud. Google's Asylo and Azure's OpenEnclave are encapsulated based on Intel® SGX SDK to reduce the cost of confidential computing technology. However, this still requires users to master the development skills of Intel® SGX, and it still has a learning threshold. Let's use another example. LibOS technologies (such as Occlum and GraphaneSGX) are designed to allow users to run applications in Enclave without changing the code or making little changes. Users are not required to learn the complex development technology of confidential computing. However, this only solves the problem of user development. It still does not solve the problem of running confidential computing applications in containers.

In short, the existing confidential computing technology solutions have the preceding difficulties and fail to fully meet the security requirements of users in different scenarios.

Inclavare Containers provides the first open-source container runtime for confidential computing scenarios to solve these problems, which combines confidential computing technology and container technology. Its value can be summarized into three points:

  1. Lower the threshold of using confidential computing to provide users with the same sense of use as ordinary containers
  2. Based on different hardware security technologies provided by the processor, it supports different Enclave forms, providing users with more choices and flexibility between security and cost.
  3. Accelerate the construction of confidential computing cloud infrastructure based on the zero-trust model

Inclavare Containers: The Future of Cloud-Native Confidential Computing

Inclavare Containers support confidential applications to be transparently containerized by TEE based on hardware. It offers the following benefits:

  • Bring confidential applications into cloud-native
  • Run modified/unmodified applications in hardware-based TEE (depending on the Enclave Runtime)
  • Provide confidentiality, integrity, and provability for the application's data and code

As shown in the following figure, the Inclavare Containers contain multiple components that can make trusted applications run in containers using Enclave technologies based on hardware support. The included components are rune, shim-rune, and Enclave Runtime.

  • rune: rune is a command-line tool used to generate and run Enclave in containers based on OCI specifications. rune is developed based on runc code. You can run a normal runc container or an Enclave container. rune has written to the OCI run time implementation list:

https://github.com/opencontainers/runtimespec/blob/master/implementations.md

  • shim-rune: This is the shim provided by rune during container runtime, which manages the lifecycle of containers, converts normal images into images of TEE, and works with a rune to create, start, stop, and delete containers.
  • Enclave Runtime: This is responsible for loading and running trusted and protected applications within the Enclave. Enclave Runtime PAL API is the interface between the rune and the Enclave Runtime, which allows you to call the Enclave Runtime through well-defined function interfaces. Confidential computing applications interact with the cloud-native ecosystem through this interface.

A typical class of Enclave Runtime implementations is based on library operating systems. Currently, the recommended Enclave Runtime to interact with rune is Occlum, a memory-safe, multi-process Libos. Another typical type of Enclave Runtime is Intel® SGX WebAssembly Micro Runtime (WAMR), which is an independent WebAssembly (WASM) runtime with a small footprint, including a VM core, an application framework, and a WASM application for dynamic management.

In addition, you can write your Enclave Runtime in any programming language and SDK you like (such as Intel SGX SDK), as long as it implements Enclave Runtime PAL API.

2

Inclavare Containers have the following features:

  1. It can combine Intel® SGX technology and a mature container ecosystem. This allows users to deploy and run sensitive applications as Enclave containers. The goal of Inclavare Containers is to run common container images made by users seamlessly. This will allow users to create images without knowing the complexity of confidential technologies and maintain the same sense of use as common containers.
  2. Intel® SGX provides the security granularity of the application rather than the system, which provides high-security protection methods and brings some programming constraints, such as the inability to execute syscall instructions in the SGX enclave. Therefore, we have introduced LibOS technology to improve the software compatibility problems mentioned above and avoid developers from complex software adaptation in the process of porting software to Intel® SGX Enclave. Each LibOS is trying to increase the number of system calls supported, but it is difficult to achieve the compatibility of the native Linux system. Even if this goal is achieved, the shortcoming of the excessive attack surface will be exposed. Therefore, Inclavare Containers improves the applicability of Enclave containers by supporting language runtime (such as Java) instead of binding the applicability of Enclave containers only in enhancing system-call support numbers. In addition, a thriving language ecosystem like Java can also be introduced into confidential computing scenarios to enrich the variety and quantity of confidential computing applications by providing language runtime support.
  3. It can define common Enclave Runtime PAL API to access more types of Enclave Runtime. For example, LibOS is a form of Enclave Runtime. The goal of designing this layer of API is to prosper the Enclave Runtime ecosystem, allowing more Enclave Runtime to connect Inclavare Containers to cloud-native scenarios and providing users with more technical choices.

Flexible Deployment Methods for Confidential Containers

Docker Cluster

It is difficult for a common user to know how to run a confidential container. You need to master the field of confidential computing and develop and build images by the SGX application development specifications. Inclavare Containers designed and implemented a new type of OCI run time rune that conforms to the OCI run time specifications to be consistent with the existing cloud-native ecosystem and realize the confidential container form. It can help you eliminate these complicated processes and enables you to use confidential containers like ordinary containers.

Inclavare Containers can be integrated with dockerd. Specifically, you need to build the container image, install the Enclave Runtime, and add the rune configurations in docker configuration files, such as /etc/docker/daemon.json.

{
        "runtimes": {
                "rune": {
                        "path": "/usr/local/bin/rune",
                        "runtimeArgs": []
                }
        }
}

Then, restart the docker service.

3

Since the rune is developed based on the runc code, the rune can create a runc container or an Enclave container. In a Docker cluster, the process of creating a runc container and an Enclave container are the same. The difference is the image. Common runc container images cannot create an Enclave. Users must generate a special image based on the requirements of the Enclave Runtime. As shown in the preceding figure, the process of using Docker to start an Occlum confidential container is listed below:

  1. Develop confidential applications by users. Users do not need to know about confidential computing. Occlum provides tools to convert common applications into confidential applications. Note: Occlum has some usage restrictions. Please refer to this website for details.
  2. Build a image based on the file generated by Occlum and push it to the imagerepository
  3. Use the standard Docker to pull up the container image and use the rune runtime to start the Enclave container and run the Occlum and Enclave application.

Kubernetes Clusters

Enclave containers can run in Docker clusters, but there are some shortcomings:

  • It cannot dynamically manage and schedule EPC.
  • The container orchestration capability is weak, and there is no Kubernetes end-state-oriented approach to container management.

4

Therefore, Inclavare Containers provide shim-rune for creating Enclave containers in a Kubernetes confidential computing cluster on the cloud. In this scenario, shim-rune and rune can form an enclave containerized stack, and you are provided with the same experience as ordinary containers without installing Enclave Runtime when building container image.

Inclavare Containers has been added to the adopter list of containerd: https://github.com/containerd/containerd/blob/master/ADOPTERS.md

In addition, shim-rune supports the containerd shim v2 API: https://github.com/containerd/containerd/blob/master/runtime/v2/task/shim.proto

Therefore, you can add the shim-rune configuration to the containerd configuration file (such as /etc/containerd/config.toml) on the system.

   [plugins.cri.containerd]
          ...
          [plugins.cri.containerd.runtimes.rune]
            runtime_type = "io.containerd.rune.v2"

Then, restart the containerd.

As shown in the preceding figure, the workflow of creating an Occlum confidential container in a Kubernetes confidential computing cluster on the cloud is listed below:

1.  kubelet initiates a Container Runtime Interface (CRI) request to containerd, such as creating a Pod.

2.  One cri-containerd plug-in implements the CRI interface in Containerd. Containerd receives the request and forwards the request to the shim-rune.

3.  shim-rune can create a runc container or a rune container. When creating runc and rune containers, the process is different:

  1. Create a runc Container: The process is the same as creating a common runc container. For example, the pause container of a Pod is a runc container.
  2. Create a rune Container: Use the LibOS to convert a normal image to a TEE one. The rune creates an Enclave in the container and runs the application in the Enclave.

4.  Create a rune process for each container. This process is responsible for creating a container. The process of creating a runc container and an Enclave container is different:

  1. Create a runc Container: The process is the same as runc creating a runc container.
  2. Create an Enclave Container: Each Enclave Runtime provides a dynamic library, which implements the Enclave Runtime PAL API. rune loads liberpal.so on the Host, creates the container as runc, and starts the first process init-runelet in the container. Init-runelet loads and creates the Enclave after receiving the rune request. Enclave includes two parts: LibOS and App/Language Runtime. LibOS is an operating system library provided by Enclave Runtime to support the running of applications. App/Language Runtime is the application itself, and some languages also have language Runtime. For example, running OpenJDK 11 applications requires JVM.

5.  The rune process exits and sets the parent process of process 1 in the Enclave container to shim-rune.

6.  Shim-rune requests rune to start Enclave. At this point, a trusted application is running.

Kubernetes Cluster-Level Remote attestation architecture

Inclavare Containers designed a set of common and cross-platform remote attestation architecture Enclave Attestation Architecture (EAA) to further meet the security requirement that the security of sensitive data of users can no longer rely on cloud vendor on the transmission link. The EAA uses the TLS certificate that is associated with remote proof of evidence with a hardware executable environment as the root of trust to ensure the security of the communication channels of the communicating parties is based on the hardware trusted execution environment.

5

The main components of the EAA are listed below:

  • Enclave-TLS

Enclave-TLS enhances standard TLS to support trusted communication between heterogeneous hardware TEE based on confidential computing technology. Enclave-TLS is a kind of security protocol with a bidirectional transfer layer (Transport Layer Security or TLS), which supports a heterogeneous hardware environment. Non-hardware TEE platforms can communicate with hardware TEE (such as SGX Enclave) over an authenticated secure channel to transmit sensitive information using Enclave-TLS. In general, TCB's boundaries extend from the execution environment to network transport using the Enclave-TLS. In addition, Enclave-TLS has an extensible model to support various hardware TEE.

  • Confidential Container

Confidential container plays the role of certifiers. It responds to the request from the Inclavared and returns the attestation evidence (mrenclave and mrsigner values) of the confidential container.

  • Inclavared

Inclavared forwards traffic between downstream confidential containers and the upstream client verifier programs Shelter. The communication process is protected by the proven Enclave-TLS channel.

  • Shelter

As a remote attestation validator deployed on the cloud, Shelter records the runtime startup metrics of Enclave and establishes a trusted channel based on Enclave-TLS with Inclavared for communication. Finally, Shelter validates the runtime metric values of the Enclave so tenants can know if their workloads are being loaded in the environment of the TEE.

The following is the specific workflow:

If the user wants to verify that the workload is running on a trusted platform, they can start the Shelter to send a validation request to the Inclavared.

  1. After receiving the verification request from Shelter, Inclavared sends the request to the confidential container. Inclavared and the confidential container generate TLS certificates with SGX remote authentication information.
  2. Establish a proven secure channel between Inclavared and Shelter based on Enclave-TLS.
  3. Establish a secure channel between Inclavared and the confidential container based on Enclave-TLS with mutual attestation.
  4. Inclavared forwards the hardware executable environment information and sensitive information provided by the confidential container to Shelter.
  5. Shelter verifies the runtime measurement of the Enclave and returns the verification result to the user.

Five Technology Innovation

Inclavare Containers provides container runtime compatible with the OCI Runtime. Users can run Enclave containers in Docker clusters or Kubernetes clusters as needed. The Enclave Runtime erases the threshold for users to use confidential computing technologies. In Kubernetes clusters, the experience is consistent with common containers. Enclave Runtime PAL API can provide a variety of Runtime to give the user more Enclave choices between safety and costs.

The technical innovations of Inclavare Containers are reflected in the following:

1.  The standard technology of enclave is implemented.

  • Confidential computing hardware technologies (such as Intel® SGX) are compatible with OCI Runtime and OCI image standards and implement the confidential container form. User's sensitive applications are deployed and run in the form of confidential containers and maintain the same sense of use as using ordinary containers.

2.  A flexible and scalable Kubernetes cluster-level remote attestation architecture Based on Enclave-TLS, Shelter and Inclavared

  • Users can prove their sensitive workloads are running in a trusted execution environment based on hardware by building a common and cross-platform remote attestation security architecture.

3.  Develop a common PAL API, standardize the interface between enclave runtime and Inclavare Containers, and create an ecosystem of Inclavare Containers

  • Standard API specifications to connect various forms of Enclave Runtime provide users with more technical choices while simplifying the specific Enclave Runtime to connect to the cloud-native ecology.

4.  Construct a confidential computing cluster infrastructure based on the zero-trust model

  • It removes trust in cloud service providers. The Inclavare Containers security threat model assumes that users do not need to trust cloud service providers, which means the security of user workloads no longer depends on privileged components controlled by cloud service providers.

5.  Seamless integration with the Kubernetes ecosystem

  • Inclavare Containers can be deployed in any public cloud Kubernetes platform to implement a unified deployment method for secret containers.

Summary

As the industry's first open-source container runtime for confidential computing scenarios, Inclavare Containers provides best practices for the ACK-Trusted Execution Environment (ACK-TEE) to use confidential containers. ACK-TEE relies on Inclavare Containers to seamlessly run confidential container made by users and maintain the same sense of use as ordinary containers. ACK-TEE can be widely used in various privacy computing scenarios, including blockchain, secure multi-party computing, key management, genetic computing, financial security, AI, and data leasing.

In the future, Inclavare Containers will continue to provide confidential computing container technology, confidential computing cluster technology, and security architecture for cloud-native scenarios for the open-source community and become the standard in this field. We will improve the user experience of developers and users and eliminate the difference in using somatosensory with running ordinary containers while deepening the implementation of the principle of the zero-trust model. Inclavare Containers will participate in the joint construction of cloud-native communities, join hands with upstream and downstream partners to promote the progress of cloud-native security technology, and make unremitting efforts to create a safer cloud-native environment.

Cloud Computing Open Source cloud native CNCF ACK Runtime Confidential Computing Trusted Execution Environment
0 1 1
Share on

Read previous post:

Is Benchmarking a Gimmick or Strength?

Read next post:

Faster Container Image Loading Speed with Nydus, RAFS, and EROFS

OpenAnolis

85 posts | 5 followers

You may also like

Comments

OpenAnolis

85 posts | 5 followers

Related Products

More Posts by OpenAnolis

See All