By Zihao Rao (Chengpu)
Java static compilation can effectively solve the problem of high cold start and runtime memory usage. The latest version of Spring Cloud Alibaba is equipped with GraalVM static compilation technology, which can help users build lightweight microservices applications.
The traditional process of developing a Java application can be roughly divided into the following steps, from writing code to starting and running the application:
The entire process is shown as follows:
Figure 1: Java program running process
The above process brings unique advantages to Java programs that are not found in other programming languages, such as cross-platform compatibility and ease of use. However, it also leads to some performance problems, including slow startup speed and high memory usage during runtime.
The detailed process of starting and running a Java program, as depicted in Figure 1, is illustrated in Figure 2 below:
Figure 2: The startup process analysis of Java programs [1]
During the startup of a Java application, the JVM (Java Virtual Machine) software program corresponding to the application needs to be loaded into memory, as shown in the red part in the figure above. Then, the JVM loads the corresponding application into memory. This process is represented by the light blue class loading (CL) part in the figure above. During the class loading process, the application begins to be interpreted and executed, as shown in the light green part in the figure above. In the interpretation and execution process, the JVM recycles garbage objects, as shown in the yellow part in the figure above. As the program runs deeper, the JVM uses just-in-time (JIT) compilation technology to compile and optimize code with higher execution frequency, improving the running speed of the application. The JIT process is represented by the white part in the figure above. The code optimized by JIT compilation is represented by the dark green part in the figure above. From this analysis, it is clear that a Java program goes through several stages, including VM initialization, App initialization, and App activation before reaching JIT dynamic compilation optimization. Compared to other compiled languages, Java's cold start problem is more severe.
Apart from the cold start problem, it is evident from the above analysis that during the execution of a Java program, the initial step involves loading a JVM, which occupies a certain amount of memory. Additionally, because the Java program first interprets and executes bytecode before performing JIT compiler optimization, it is easy to load more code than is actually required, resulting in some unnecessary memory usage compared to certain compiled languages. In conclusion, these are the main reasons why many people criticize the high memory usage of Java programs.
Considering the issues with the traditional Java program running mode of interpret-first and compile-later, is there a way for Java programs to solve these problems by compiling first and executing later, similar to other programming languages like C and C++? The answer is yes. Ahead-of-time (AOT) compilation, or static compilation, has long been discussed in the Java field. The core idea is to perform the compilation phase of the Java program earlier than the program's start, compiling and optimizing the code in this phase to achieve fast program startup, eliminate the cold start problem, and reduce runtime memory overhead. There are various implementation technologies for static compilation in the Java field, with the most representative being the GraalVM open-source high-performance multilingual runtime platform launched by Oracle. Some readers may wonder, "What is a high-performance multilingual runtime platform, and what does it have to do with static compilation?"
Figure 3: GraalVM multilingual runtime platform
As shown in Figure 3, GraalVM utilizes the Truffle interpreter implementation framework, which enables developers to quickly implement language-specific interpreters using Truffle APIs. This allows programs written in various programming languages to be compiled and executed, making GraalVM a multi-language runtime platform. The GraalVM JIT Compiler is responsible for implementing static compilation capabilities. The static compilation framework and runtime are implemented by the Substrate VM subproject and are compatible with the OpenJDK runtime implementation. They provide runtime functions for native image programs, including exception handling, synchronous scheduling, thread management, and memory management. Therefore, in addition to being a multi-language runtime platform, GraalVM can also perform static compilation on Java programs through its static compiler, GraalVM JIT Compiler.
Now that we understand the relationship between static compilation and GraalVM, you may wonder about the differences between GraalVM-based static compilation and the regular JVM interpretation and execution mode. The differences between a Java program compiled with static compilation and the widely used JVM runtime compilation, from code writing to compilation and execution, are illustrated in Figure 4.
Figure 4: Comparison between static compilation and traditional JVM running process
Compared to the JVM runtime method, static compilation involves parsing and compiling the program before execution. It then generates a native image executable file that is closely tied to the runtime environment. This file can be directly executed to start the program. Now, you might be wondering about the parsing operations performed on Java programs during the static compilation process shown in Figure 4. Additionally, how is the garbage collection problem handled for the executable programs after static compilation? Figure 5 illustrates the input and output content of the compilation process in GraalVM's static compilation technology implementation.
Figure 5: Static compilation input and output
The initial three inputs depicted on the left side of Figure 5, namely Application, Libraries, and JDK, are essential components for compiling and running a Java program. No further explanation is needed in this regard. The Substrate VM, on the other hand, serves as the core component of static compilation within GraalVM and plays a crucial role throughout the entire process.
During the static analysis phase, as illustrated in the middle section of Figure 5, the Substrate VM performs context-insensitive points-to analysis on the application program. This static analysis provides a list of all possible reachable functions based on source code analysis without the need to execute the program. It serves as an input for the subsequent compilation phase of the program. Due to the limitations of static analysis, this process cannot cover dynamic features in Java such as reflection, dynamic proxy, and JNI calls. Consequently, many Java frameworks heavily relying on these features cannot be statically analyzed using Substrate VM alone. They require additional external configuration [3] to address the limitations of static analysis.
For instance, the Spring community has developed the AOT Engine [4] depicted in Figure 6, which helps analyze and transform reflection, dynamic proxy, and other dynamic aspects of Spring projects into content recognizable by Substrate VM during the compilation phase. This ensures successful static compilation of Spring applications using Substrate VM.
Figure 6: Spring AOT Engine
Once the static analysis is completed based on the reachable function list, the GraalVM JIT Compiler, mentioned earlier, compiles the application program into platform-specific native code to finalize the compilation process. After compilation, the process enters the Native Executable File Generation phase on the right side of Figure 5. During this phase, Substrate VM saves the determined and initialized content from the static compilation phase, along with data from the Substrate VM runtime and JDK libraries, into the Image Heap of the final executable file. The Substrate VM runtime provides essential capabilities such as garbage collection and exception handling during program execution. Initially, the GraalVM community edition only offered Serial GC for garbage collection. However, the enterprise edition introduced the more powerful G1 GC. In the latest community edition, the GraalVM team has also introduced G1 GC [5] to provide developers with enhanced static compilation capabilities.
After introducing static compilation technology and its limitations in the previous section, many external community developers may wonder how a Java open source project can quickly adapt to static compilation. The key to solving this problem is to convert dynamic content that cannot be recognized and processed by GraalVM in open source frameworks into content that GraalVM can recognize. Therefore, the solutions to this problem will vary depending on the frameworks. For example, in Spring, the AOT Engine developed for its own framework can solve the following problems: the initialization process of registered classes through the @Configuration annotation provided by its framework cannot be recognized in the static compilation phase; dynamic proxy classes that can only be generated during runtime are generated in advance during the static compilation phase; static compilation proxy classes cannot be effectively generated[6]. Once these problems are solved, the static compilation adaptation of Spring applications can be implemented.
For many open source frameworks implemented based on Spring, the inability of GraalVM to recognize dynamic features is caused by the usage of the Spring standard. Since they belong to the Spring system, the static compilation process requires Spring AOT Engine. Therefore, these frameworks can have static compilation capability without providing any adaptation. For non-Spring projects or projects that use some Java native reflection or other Java dynamic features in JDK, it is necessary to provide corresponding static configuration files for the Java dynamic usage in their own code. This allows the compilers to recognize the dynamic features during the static compilation process, enabling smooth compilation and execution. In this case, GraalVM provides a tracing agent called native-image-agent to help you easily collect metadata and prepare configuration files. The agent automatically collects dynamic feature usage from applications running on the regular Java VM and converts it into a configuration file that GraalVM can recognize. Finally, the dynamic configuration files of the framework generated by the agent are stored in the META-INF/native-imag// directory of the project. This allows the dynamic features in the project package to be recognized during static compilation based on these configuration contents. All middleware clients included in Spring Cloud Alibaba 2022.0.0.0 have been adapted to GraalVM native applications. Due to the features of the project, the overall implementation of the project contains a large number of dynamic feature usages that cannot be recognized by GraalVM due to Spring syntax. This problem is directly solved by Spring AOT Engine, and the community does not need to perform additional adaptation work..
In addition to the Spring system syntax, the project also has some other Java dynamic usages, which are parsed and dynamically configured by our community with the native-image-agent.
All middleware clients included in Spring Cloud Alibaba 2022.0.0.0 have been adapted to GraalVM native applications, providing users with out-of-the-box static compilation capabilities. The experience process for related functions is as follows:
First, you need to install the GraalVM release version on your machine. You can manually download it from the Liberica Native Image Kit page, or use a download manager such as SDKMAN!. This article uses MacOS as the demo environment. For Windows users, please refer to the relevant documentation [7]. Run the following command to install the GraalVM environment:
$ sdk install java 22.3.r17-nik
$ sdk use java 22.3.r17-nikVerify whether the correct version is configured by checking the output of java -version:
$ java -version
openjdk version "17.0.5" 2022-10-18 LTS
OpenJDK Runtime Environment GraalVM 22.3.0 (build 17.0.5+8-LTS)
OpenJDK 64-Bit Server VM GraalVM 22.3.0 (build 17.0.5+8-LTS, mixed mode)To use the static compilation capability of GraalVM to build microservices, make sure that the Spring Boot version of your project is 3.0.0 or later, and the Spring Cloud version is 2022.0.0 or later. Then, introduce the required module dependencies of Spring Cloud Alibaba 2022.0.0.0 in the project. Run the following commands to generate the hints configuration files required for reflection, serialization, and dynamic proxies in your application, provided that the spring-boot-starter-parent parent module is introduced in your application:
$ mvn -Pnative spring-boot:runOnce the application starts, it will perform a pre-execution phase. It is important to thoroughly test all the application's functionalities to ensure that the majority of the code is covered by test cases. During this process, the native-image-agent of GraalVM will collect the dynamic features in the program. This ensures that all the necessary dynamic attributes during the application's runtime are generated properly. After running all the test cases, you will find the following hints files generated in the resource/META-INF/native-image directory:
• resource-config.json: resource hint files in applications
• reflect-config.json: reflection definition hint files in applications
• serialization-config.json: serialization content hint files in applications
• proxy-config.json: Java proxy-related content hint files in applications
• jni-config.json: Java native interface (JNI) content hint files in applications
Note:By default, all core modules in the official version of Spring Cloud Alibaba 2022.0.0.0 include the necessary configuration content for their own components' dynamic features in their dependencies. Therefore, the pre-execution phase mentioned earlier is primarily used to scan the dynamic features of the application's own business code and other third-party packages. This ensures a smooth static compilation process and enables the normal startup of the application.
Once all the necessary steps are completed, execute the following command to build the native image:
$ mvn -Pnative native:compileUpon successful execution, you will find the generated executable files in the /target directory.
To run the application using the generated executable files, use the command target/xxx. You will see an output similar to the following:
2023-08-01T17:21:21.006+08:00 INFO 65431 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8080 (http) with context path ''
2023-08-01T17:21:21.008+08:00 INFO 65431 --- [ main] c.a.cloud.imports.examples.Application : Started Application in 0.553 seconds (process running for 0.562)The new version of the Spring Cloud Alibaba application that utilizes the GraalVM static compilation technology has significantly improved core capabilities in terms of startup speed and memory usage, as shown in the following table.
Note: The preceding sample test code is from the examples module in the Spring Cloud Alibaba project. In the 4c16g Mac environment, each group of data is tested three times and averaged. The specific data may vary depending on the machine.
[1] The startup process analysis of Java programs https://shipilev.net/talks/j1-Oct2011-21682-benchmarking.pdf
[2] GraalVM open source high-performance multilingual runtime platform https://www.oracle.com/java/graalvm/
[3] External configuration https://www.graalvm.org/latest/reference-manual/native-image/metadata/
[4] AOT Enginehttps://spring.io/blog/2021/12/09/new-aot-engine-brings-spring-native-to-the-next-level
[5] G1 GChttps://medium.com/graalvm/a-new-graalvm-release-and-new-free-license-4aab483692f5
[6] Dynamic proxy class and other issues https://docs.spring.io/spring-boot/docs/current/reference/html/native-image.html#native-image.introducing-graalvm-native-images.understanding-aot-processing
[7] Relevant documentation https://medium.com/graalvm/using-graalvm-and-native-image-on-windows-10-9954dc071311
New Features of Grafana 10: Experience and Collaboration Enhancement
Use KubeSkoop Exporter to Monitor and Locate Container Network Jitter Issues
503 posts | 48 followers
FollowAlibaba Cloud Native Community - September 13, 2023
Alibaba Cloud Native Community - April 29, 2024
Alibaba Clouder - May 29, 2020
Alibaba Cloud Native Community - September 11, 2024
Alibaba Container Service - April 28, 2020
Alibaba Clouder - April 15, 2021
503 posts | 48 followers
Follow
Microservices Engine (MSE)
MSE provides a fully managed registration and configuration center, and gateway and microservices governance capabilities.
Learn More
Cloud-Native Applications Management Solution
Accelerate and secure the development, deployment, and management of containerized applications cost-effectively.
Learn More
Secure Content Delivery Solution
Accelerate static and dynamic web content in a fast, reliable, and safe way using Secure DCDN (Dynamic Route for CDN)
Learn More
Function Compute
Alibaba Cloud Function Compute is a fully-managed event-driven compute service. It allows you to focus on writing and uploading code without the need to manage infrastructure such as servers.
Learn MoreMore Posts by Alibaba Cloud Native Community