By Jia Zhiguang, a Senior Development Engineer at Alibaba. He specializes in the Kubernetes sandbox and confidential computing fields and participates in the development of the Inclavare Containers community.
This article describes confidential computing, the architecture of the open-source program of Inclavare Containers, supported features, the iteration plans of the insular containers, and the development status and planning of Alibaba Cloud ACK-TEE.
According to the 2019 Annual Container Adoption Survey released by Portworx and Aqua Security, security has become the biggest challenge for users to use the container technology and migrate the business to the cloud. Data security is the most critical issue. According to the data breach report released by Risk Based Security, the number and volume of data leaks in 2019 increased by more than 50% compared with 2018.
Data has three states throughout its lifecycle: at-rest, in-transit, and in-use.
The Confidential Computing Consortium (CCC) defines confidential computing as protecting data in a Trusted Execution Environment (TEE) based on hardware.
The core functions of confidential computing are listed below:
Confidential computing was also included in Gartner's Computing Infrastructure Maturity Curve in 2019. Although it is still in the early stage, it shows how confidential computing has gradually entered the public's field of view and received attention.
In 2020 Gartner's Cloud Vendor Local Security Solution Comparison, Alibaba Cloud got an H in the TEE field after the Alibaba Cloud Container Service for Kubernetes (ACK) released a confidential computing product, ACK-TEE, in early 2020.
Confidential computing is designed to protect sensitive code and data. Business scenarios include blockchain, key management, finance, AI, multi-party computing, data leasing, and edge computing.
Taking multi-party computing as an example, different users or manufacturers share data with each other to dig out the greater economic value of data but do not want to disclose their data to each other. Confidential computing ensures that shared data is run in a TEE protected by hardware. Data is encrypted in the memory to prevent data leakage.
In addition to confidential computing, another security-related concept is the secure containers. Alibaba Cloud has dedicated time to the fields of secure containers and confidential computing. Both are related to security, but their positioning and application scenarios are different.
The positioning of secure containers is to isolate malicious applications and prevent them from destroying other applications.
There are three main application scenarios:
Confidential computing is used to prevent data theft and breaches of the application. It protects sensitive code and data.
Three hardware platforms support the TEE: Intel SGX, ARM TrustZone, and Advanced Micro Devices (AMD) Secure Encrypted Virtualization (SEV). They have different application scenarios and implementation methods:
The Alibaba Cloud ACK-TEE and the Inclavare Containers are confidential computing based on Intel SGX.
When sensitive applications are deployed normally, they depend on the operating system, Virtual Machine Monitor (VMM), hardware, and cloud vendors. If the TCB is large, it will face a large range of attacks. As long as there is an attack in TCB, the application is at risk of data leakage and destruction.
TCB only has the CPU and TEE itself when deploying sensitive applications in the TEE of Intel SGX. While reducing the attack surface, the TEE-based security mechanism makes applications more secure.
Intel SGX divides applications into trusted and untrusted zones. Users can define trusted and untrusted zones and the functions used in the Enclave Definition Language (EDL). The functions that communicate between trusted and untrusted zones are divided into ECALL and OCALL functions. ECALL functions allow you to access data in the untrusted zone, and OCALL functions allow you to access data in the untrusted zone.
The process of developing and using Intel SGX-based trusted applications is listed below:
1. Apply for secret key: Apply for SGX-related commercial signature encryption keys from Intel.
2. Install the environment:
3. Application Development:
4. Code Compilation and Building:
sgx_edger8r
to produce proxy functions for ECALL's untrusted zones and trusted proxy functions for OCALL based on edl files5. Run a container using Docker
Inclavare is the Latin etymology of the word enclave [ˈinklɑveə]. The enclave refers to a protected execution environment that provides strong security isolation based on key science algorithms for sensitive and confidential data in this environment. It prevents untrusted entities from accessing users' digital assets.
Inclavare Containers is an open-source container runtime technology for confidential computing scenarios. It is led by the Alibaba Cloud Operating System Security Team, Alibaba Cloud Cloud-Native Container Service Team, and multiple R&D teams in the Alibaba economy, including the Ant Financial Security Computing Team, Cloud Security Team, Language Runtime Team.
The current confidential computing technology in the cloud-native scenarios has many defects and shortcomings:
Due to the reasons above, it is not conducive to the popularization and application of confidential computing technology. The technology provides an open-source container runtime engine and security architecture for the industry in the field of confidential computing. Its values are listed below:
Before introducing the Inclavare Containers architecture, let's take a look at the role of each component in the architecture:
pal_init
is used to initialize the enclave, and pal_create_process
is used to create the enclave.The workflow of Inclavare Containers is listed below:
Kubelet initiates Container Runtime Interface (CRI) requests to Containerd, such as, requests to create a Pod.
A cri-containerd plug-in is provided for Containerd. After Containerd receives a request, it forwards the request to shim-rune.
Shim-rune can create both runc and rune containers. The processes for creating runc containers are different from those for rune containers:
Rune loads liberpal.so for communication between rune and LibOS.
Rune loads the Intel SGX driver into a container, creates process 1 in the container as init-runelet, and then uses init-runelet to create the enclave. The enclave is a TEE protected by Intel SGX. It includes the LibOS, language runtime, and applications. Now, a trusted application is running.
A summary of the Inclavare Containers characteristics are listed below:
The shim-rune consists of two parts: Core and Carrier. They serve the following:
The shim-rune workflow is listed below:
Inclavare Containers works in two ways, client signature and server signature. The differences are listed below:
Certificate Provider | SGX Development | Signature Location | Container Runtime | |
---|---|---|---|---|
Client Signature | User | Required | Client | Docker |
Server Signature | Service Provider | Not Required | Server | Containered |
Compared with client signature, server signature has the following advantages:
It lowers the requirements for developers to use LibOS. Developers don't need to master the technology of Intel SGX. They can build ordinary images according to LibOS requirements.
Note: Each LibOS has certain requirements for common images. For example, Occlum only supports musl libc and does not support glibc. Therefore, the glibc application needs to be transformed into the musl libc application before it can run in the Inclavare Containers.
The Inclavare Containers project is built and cooperated by multiple teams. The roles of each component and the team division are listed below:
Inclavare Containers is the first open-source container runtime technology stack in the industry to serve the cloud-native confidential computing scenarios. It has been rated as a key open-source project by the Alibaba Open-Source Committee. It is added to the official reference implementation list of confidential computing OCI runtime.
The following features are currently supported:
This project is released at the end of each month, providing the community with binary releases of Community Enterprise Operating System (CentOS) and Ubuntu. The Alibaba Cloud Linux release version is provided to the insiders.
ACK-TEE was established in September 2019. Its features are listed below:- Users with strong security requirements for digital assets (algorithms, data, and code) get a TEE based on hardware encryption technology.
ACK-TEE 1.0 launched in January 2020.
It reuses the existing capabilities of the managed Kubernetes cluster, including the integration of various cloud services and the Kubernetes cluster O&M capability. It reduces the Kubernetes cluster O&M complexity.
It supports Enclave Page Cache (EPC) encrypted memory management and scheduling and reduces the complexity of users using SGX devices.
ACK-TEE 2.0 was launched in the second half of 2020.
Q1: Does this product rely on Intel chips? Why do I need to find Intel to apply for a key?
A1: Intel chips can ensure that applications are executed in the hardware-based enclave (a type of TEE) to ensure the security of applications, but they cannot ensure that the creators are legitimate. When building the enclave, we use Intel's secret key to sign it, ensuring legitimate users.
Q2: Is Inclavare Containers essentially an implementation of container runtimes? Can it completely replace the runtime scenarios of Docker containers?
A2: Inclavare Containers is a software stack that contains multiple tools, such as rune, shim-rune, and runelet. Rune is the container runtime and is developed based on the runc code. It can run either a common runc container or a container with the enclave. Functionally, it can replace Docker container runtime (runc), but the biggest significance is to run the enclave container to ensure code and data security.
Q3: How much impact does the application performance have? Has a similar test been done?
A3: Inclavare Containers are responsible for solving data security problems. The bottom layer is based on the Intel SGX technology. Currently, the Extended Capabilities Port (ECP) of Intel SGX1 only has 128MB of memory, which makes the performance much worse than cloud-native container applications.
Q4: We only need to use it for the core needs of the in-use encryption, right?
A4: Yes, protecting in-use code and data is the greatest value of confidential computing.
Q5: Does ACK have this usage method and sample now?
A5: There is a managed version of "confidential computing" in ACK, ACK-TEE 1.0 mentioned above. However, its customers are SGX-native customers that need to transform applications and construct images based on SGX. ACK-TEE 2.0 was launched in late 2020 with the support of the Inclavare Containers capability.
Experiences and Lessons from the Management Practices of Alibaba Kubernetes
173 posts | 31 followers
FollowOpenAnolis - June 9, 2022
OpenAnolis - July 8, 2022
Alibaba Developer - April 18, 2022
Alibaba Clouder - June 28, 2020
Alibaba Clouder - October 10, 2020
OpenAnolis - September 26, 2022
173 posts | 31 followers
FollowIndustry-standard hardware security modules (HSMs) deployed on Alibaba Cloud.
Learn MoreSuper Computing Service provides ultimate computing performance and parallel computing cluster services for high-performance computing through high-speed RDMA network and heterogeneous accelerators such as GPU.
Learn MoreApsaraDB Dedicated Cluster provided by Alibaba Cloud is a dedicated service for managing databases on the cloud.
Learn MoreAlibaba Cloud is committed to safeguarding the cloud security for every business.
Learn MoreMore Posts by Alibaba Container Service