When Java Meets Confidential Computing, Another Fantastic Journey Begins!

By Wang Shaojun

Preface

Data has three states in the information world: storage state, transmission state, and computing state. The data stored in the database or disk is in the storage state, the data transmitted in the network is in the transmission state, and the data under computing and processing is in the computing state. We need to protect the system based on the three data states to ensure real data security. For the security of data in the storage state and transmission state, we can use the widely used data encryption technology for effective protection. However, the security of data in the computing state is still a field being explored.

Hardware-based confidential computing technology provides a trusted execution environment in which code and data are protected at the hardware level. Any software, including the kernel and hypervisor, has no permission to view the data information in the environment, thus realizing the protection of computing data.

Mainstream chip manufacturers worldwide have launched their confidential computing solutions, such as SGX (of Intel and) TrustZone (of ARM). TrustZone is mainly used in the terminal field, while SGX can be used in the server field. SGX can provide a very high level of confidential computing protection. However, due to the limitations in memory resources and programming models, it cannot effectively support the confidential computing business of the Java ecosystem. This is unfortunate! With the release of the seventh-generation ECS instances of Alibaba Cloud X-Dragon, the new SGX2 (of Intel) mounted on ECS instances provides conditions to build confidential computing services based on the Java ecosystem.

Demo of Java Confidential Computing

Now, let's use a specific example to demonstrate how to provide confidential computing protection for Java businesses. The instance is built based on the seven-generation ECS instance of the third-generation X-Dragon. It runs the Java SpringBoot network service in the trusted execution environment for confidential computing provided by SGX2.

Preparations

Apply for a seventh-generation ECS instance of X-Dragon that supports SGX2. The EPC memory should be no less than 24 GB.
Download the container image of LibOS Occlum: occlum/occlum:0.20.0-ubuntu18.04
Download the JDK Alibaba Dragonwell11 (Alpine), an Alpine-based Dragonwell11 image released by Alibaba
Download the source code of the SpringBoot demo, which shows a simple network service built based on the SpringBoot framework

After the source code of SpringBoot demo source code is downloaded to your local path, go to the initial directory to compile and package the source code. The package spring-boot-0.0.1-SNAPSHOT.jar will be generated in the target directory. We run the .jar package in the normal environment to verify its function first.

mvn clean package

java -jar spring-boot-0.0.1-SNAPSHOT.jar

Build an SGX Execution Environment

1. Log on to the ECS instance

2. In the ECS environment, enter the Occlum container using the Docker command.

docker run -it --rm --privileged --network host \
        -v `pwd`:`pwd` \
    -v /dev/sgx_enclave:/dev/sgx/enclave \
    -v /dev/sgx_provision:/dev/sgx/provision \
    -v /var/run/aesmd:/var/run/aesmd \
    occlum/occlum:0.20.0-ubuntu18.04

3. Create an enclave instance in the Occlum container. This instance contains a .json configuration file and an image folder.

mkdir occlum_instance
cd occlum_instance
occlum init

4. Modify the content in the Occlum.json file, including heap size, entry point, and env environment variables.

resource_limits.user_space_size = "1680MB"
resource_limits.kernel_space_heap_size="64MB"
resource_limits.max_num_of_threads = 64
process.default_heap_size = "256MB"
process.default_mmap_size = "1400MB"
entry_points = [ "/usr/lib/jvm/java-11-alibaba-dragonwell/jre/bin" ]
env.default = [ "LD_LIBRARY_PATH=/usr/lib/jvm/java-11-alibaba- \   
dragonwell/jre/lib/server:/usr/lib/jvm/java-11-alibaba-dragonwell/jre/lib:/usr/lib/jvm/java-11- \
alibaba-dragonwell/jre/../lib" ]

To run Java, you need to configure a larger memory space. The entry_points option indicates the path for placing the JDK of Occlum. This path must be in xx/xx/jre/bin mode, and you need to set the environment variable LD_LIBRARY_PATH. Currently, Occlum does not support exec system calls. Therefore, the path must meet certain conditions to avoid exec system calls when the JVM is started.

5. Enter the image folder and create a usr/lib/jvm/java-11-alibaba-dragonwell file directory in this directory to place the Dragonwell11 Alpine JDK. Note: The folder name after the JDK compressed file is decompressed is renamed as jre to ensure consistency with the .json configuration file. Create a usr/lib/spring file directory to place the spring-boot-0.0.1-SNAPSHOT.jar file prepared previously.

Note: The image file directory will be used as a root directory after SGX LibOS runs.

6. Copy the libz.so.1 file in the Occlum container environment to image/lib:

cp /usr/local/occlum/x86_64-linux-musl/lib/libz.so.1 image/lib

7. Build a confidential image:

  occlum build

8. Start the SGX confidential computing business:

occlum run /usr/lib/jvm/java-11-alibaba-dragonwell/jre/bin/java -Xmx512m -XX:-   \
UseCompressedOops -XX:MaxMetaspaceSize=64m -Dos.name=Linux -jar /usr/lib/spring/spring- \
boot-0.0.1-SNAPSHOT.jar

After SpringBoot is started, use the command curl localhost:8080 to request the SpringBoot service. The return result "Greetings from Spring Boot!" indicates that the service is started successfully. The -XX:-UseCompressedOops parameter aims to reduce the startup time consumption of Java under Occlum. The -Dos.name=Linux parameter aims to inform the JVM of the system type of the LibOS.

Figure (1): SpringBoot Startup Diagram

The entire SpringBoot network service runs in a confidential computing environment. The underlying software built in the ECS instance does not have permission to interfere with the protected services. The protection of cloud service data in the computing state is implemented.

Seventh-Generation ECS of X-Dragon and Java Confidential Computing

The seventh-generation ECS instance of X-Dragon released by Alibaba Cloud is equipped with Intel's third-generation Xeon scalable processor (codenamed Ice Lake). The next-generation Intel SGX Security Technology (SGX2) provided by this processor has a considerable improvement in the number of cores and EPC memory capacity. The specific specifications are shown in Figure (2) below. The Ice Lake processor provides a processing capacity with up to 80 physical cores (160 logical cores), while the available first-generation processors for SGX have only six physical cores at most. The EPC memory capacity has increased to 1 TB, while the first-generation SGX EPC is only 128 MB. You can select the number of cores and EPC memory capacity based on your needs.

	Pentium Silver	Xeon E3	Xeon SP Ice Lake
CPU Physical Cores	4	6	Up to 80
SGX Trusted Content Capacity	128 MB	128 MB	Up to 1 TB
SGX Dynamic Memory Management	Supported	Not supported	Supported

Figure (2): Comparison of SGX Technical Specifications

The EPC memory capacity and the number of cores provided by SGX1 are too small, so SGX1 is not suitable for Java, a relatively complex programming language. For a long time, only native languages based on C/C++ are more suitable for running in SGX1. In addition, the Host Enclave programming model defined by the SGX SDK needs to segment the business code, which is more intrusive to the code and limits the application scope of SGX1. Fortunately, SGX2 has a great improvement in the number of cores and EPC memory capacity, so we can break through the constraints of the Host Enclave programming model and meet the requirements of Java services for hardware resources. It is possible to deploy Java confidential computing services based on SGX. We expect confidential computing services in public cloud scenarios to flourish.

Confidential Computing Model

Programming Model for Confidential Computing

There are two main programming models for confidential computing, as shown in Figure (3):

Figure (3): Programming Models for Confidential Computing

The first one is the Host-Enclave programming model. This model splits the entire application into Host and Enclave parts. Host runs in a common environment and is responsible for most application logic processing. Only some business logic (such as encryption and decryption) that needs security protection runs in the Enclave environment. The switching and information transmission between the two is realized through ecall and ocall operations. Under this programming model, users need to split the service into Host and Enclave for programming and write code of ecall and ocall to realize switching and information transmission between Host and Enclave. The programming and existing business transformation are relatively difficult. However, the advantage is the Enclave TCB is small enough, and the security level is high.

// enclave.c

int function(int a, int b) {
    return a + b;
}
// host.c

int main() {
    ... ...
    enclave ec = create_enclave();
    int c = function(&enclave, 3, 5);
    destroy_enclave(ec);
    ... ...
    printf("sum is: %d\n", c);
}

The second one is the Full-Feature programming model. This model puts the complete application into the Enclave for operation. Host is only responsible for the management of Enclave and ocall and other operations. Generally, Host is provided by the toolchain of the underlying framework, and users do not need to perceive the existence of the Host. This programming model is consistent with traditional programming models, which means splitting is not needed, and programming does not become more difficult. It is also easy to transform existing business code. However, this model requires a powerful SDK or LibOS in the Enclave to support the normal operation of the complete application. In addition, the volume of the business code may be large, which may lead to large TCB in the Enclave and a lower security level.

// App.c

int function(int a, int b) {
    return a + b;
}

int main() {
    int c = function(3, 5);  
    printf("sum is: %d\n", c);
}

The two programming models for confidential computing have their advantages and disadvantages. You need to choose a suitable one by considering the ease of use and security.

Requirements for Confidential Computing Model

You need to consider both ease of use and security. For this purpose, requirements of confidential computing business can be classified according to security levels for choosing the suitable programming model, as shown in Figure (4). If only a small part of computation logic in your business requires security protection of a high level, you can select the Host-Enclave programming model. If you do not want to perform heavy modifications on your business code and can accept a relatively low-security level, you can select the Full-Feature programming model.

Figure (4): Requirements for Confidential Computing Model

Software Ecosystem of SGX Confidential Computing

The second-generation SGX technology of X-Dragon seventh-generation ECS instances has no hardware capability bottleneck. The development of the software ecosystem based on SGX technology over the next several years will determine whether SGX technology can be widely used and generate business value.

SGX SDK

When Intel released the first generation of SGX technology, it introduced Intel SGX SDK. This SDK defines the Host-Enclave programming model for SGX confidential computing for C/C++ and uses .edl files to define the interactive interface between Host and Enclave. After that, Microsoft Azure launched OpenEnclave. It is the functional extension and platform abstraction of Intel SGX SDK and can run more complex businesses in Enclave. At the same time, it supports more secure computing hardware platforms (such as SGX and TrustZone). Google Cloud has introduced the Asylo programming model, which is similar to OpenEnclave. It has the platform abstraction and functional extension feature too. The most important feature of Asylo is that it abstracts Encalve into a remote service and interacts with Host in GRPC mode. As a result, the Host and Enclave modules can be physically separated. The programming language differences between Host and Enclave are also shielded, making the Asylo programming model more flexible and worthy of referencing.

When we view the development of Intel SGX SDK, OpenEnclave, and Asylo, it is not difficult to know that OpenEnclave and Asylo have inherited from Intel SGX SDK and developed based on it. The three SDKs can be applied to some confidential computing scenarios, such as business written in C/C++, while only a small part of computing needs security protection. Due to the limitations of the Enclave SDK, complex Enclave business logic cannot be supported. The following part describes the main features:

All of them support the Host-Enclave programming model. Asylo supports the Full-Feature programming model to some extent.
It is difficult to develop. The Host-Enclave programming model needs to split the application into two parts.
Only C/C++ programming languages are supported. Advanced programming languages (such as Java/Go) are not supported.
The Full-Feature programming model is not supported, so it cannot apply to business scenarios that require high ease of use.

SGX LibOS

There are many differences between the SGX operating environment and the normal operating environment. Can an OS be introduced in Enclave to mask the differences in the SGX execution environment so the application can be unaware of the existence of SGX, just like running in the normal environment? There are many solutions in the industry. Currently, the well-known SGX LibOS projects include Occlum, Graphene, and SGX-LKL. Among them, Occlum is an open-source TEE-OS system developed by Ant Group. It uses the Rust programming language and has complete functions. It supports multiple programming languages and has advantages (such as high performance and memory security).

SGX LibOS aims to make the entire application easily run in SGX Enclave. It conforms to the Full-Feature programming model for confidential computing. Moreover, it is easy to use and supports multiple programming languages and complex applications. This solution mainly has the following disadvantages:

TCB becomes larger, and security has to be less satisfied.
It needs to consume more SGX hardware resources.
Frequent ecall and ocall operations (such as I/O) affect business performance.

Alibaba Dragonwell Confidential Computing

SGX1 has limitations on the number of cores and EPC memory size. If applications with large memory requirements and complex logic are deployed on the LibOS platform, frequent EPC memory swaps and ecall/ocall-based switches will inevitably occur, resulting in serious business performance degradation. It will be difficult to use SGX1 in the actual production environment. Therefore, the hardware conditions of SGX1 determine that it can only support simple Enclave business scenarios implemented by the C/C++ programming languages. The era of SGX2 comes with the release of the seventh-generation ECS instances of X-Dragon. The confidential computing services based on the Java programming language become practical thanks to the increase in core number and EPC memory size.

Solution to Java Confidential Computing

The Java confidential computing solution launched by the Alibaba JVM Team is shown in Figure (5). This solution uses the Full-Feature programming model. In Enclave, LibOS is introduced to support the running of Alibaba Dragonwell11, while upper-layer applications are unaware of the SGX environment.

Figure (5): Solution to Java Confidential Computing

Alibaba Dragonwell is an OpenJDK implementation opened by the Alibaba JVM Team. Currently, it has two LTS versions: 8 and 11. Alibaba Dragonwell11 has released JDK versions compatible with glibc and musl to enable Alibaba Dragonwell11 to adapt to more LibOSs. musl is lighter than glibc, and the code of musl is easy to maintain, so musl is more popular in the confidential computing field. Many LibOSs (such as Occlum) preferentially choose musl as the base library.

The musl version of Alibaba Dragonwell11 is the preferred version of the confidential computing JDK and can be used to build Alpine container images to reduce the size of the container image.

Evaluation of Java Confidential Computing Performance

Performance is always the thing that needs more attention. What is the performance of Java services running in SGX2? We conducted stress testing on the performance of Java SpringBoot services in three operating environments: SGX1, SGX2, and Linux. The concurrency was set to 400, and the stress testing duration was 40 seconds. We made a comparative analysis based on two metrics: the system throughput (MB/s) and RPS. The stress testing results are shown in Figure (6):

Figure (6): Comparison of Java Confidential Computing Performance through Stress Testing

Under the same stress, the throughput of the Linux platform is 26.91 MB/s, and RPS is 12.84 (thousand). The throughput of SGX2 is 18.53 MB/s, and RPS is 8.84 (thousand). The throughput of SGX1 is 1.26 MB/s, and RPS is 602.10 (thousand). The performance of SGX2 is significantly improved compared with SGX1, but it still lags behind Linux. The performance will be improved with the continuous optimization of Alibaba Dragonwell11.

Summary

After Alibaba Cloud released the seventh-generation ECS instance of X-Dragon, the Alibaba JVM Team proposed a programming model and solution for confidential computing for the Java language and conducted in-depth practice. Then, Alibaba Cloud released an Alibaba Dragonwell11 JDK version for Java confidential computing. According to the practical results, the Java confidential computing solution based on SGX2 has significant performance improvements and can support the stable operation of complex Java confidential computing services. The Java confidential solution based on SGX2 will effectively promote the development of Java confidential computing. We hope interested developers or developers that need Java confidential computing will try our solution. We look forward to communicating with you soon!