Performance Improvement by 10 Times, GraalVM Application Observability Practice

By Chengpu and Cengfeng

1. GraalVM Static Compilation

1.1 Background

As the cloud-native trend continues to grow, the use of cloud-native technology to provide maximum flexibility for enterprise applications has become a core requirement for digital transformation in businesses. However, as an interpretation/execution and runtime Just-In-Time (JIT) compilation language, Java has the following inherent disadvantages compared with other static compilation languages, which seriously affects its quick startup and scaling effect.

Cold Start

The detailed process of starting and running a Java program is shown in Figure 1.

Figure 1: Startup process analysis of Java programs

During the startup of a Java application, the JVM needs to be loaded into the memory, as shown in the red part in Figure 1 above. Then, the JVM re-loads the corresponding application into the memory. This process corresponds to the light blue class loading (CL) part in the above figure. During the class loading process, the application begins to be interpreted and executed, as shown in the light green part in the above figure. In the interpretation and execution process, the JVM recycles garbage objects, as shown in the yellow part in the figure above. As the program runs deeper, the JVM uses JIT compilation technology to compile and optimize the code with higher execution frequency, improving the running speed of the application. The JIT process is represented by the white part in the figure above. The code optimized by the JIT compilation is represented by the dark green part in the figure above. From this analysis, it is clear that a Java program goes through several stages, including VM initialization, App initialization, and App activation before reaching JIT dynamic compilation optimization. Compared with other compiled languages, Java's cold start problem is more severe.

High Runtime Memory Usage

Apart from the cold start problem, it is evident from Figure 1 that during the execution of a Java program, the initial step involves loading a JVM, which occupies a certain amount of memory. Additionally, JIT compilation and garbage collection (GC) will have a certain amount of memory overhead. Finally, since Java programs first interpret and execute bytecode before performing JIT compilation optimization, some unnecessary code logic may also be pre-loaded into memory for compilation due to its late compilation. So, in addition to the actual application to be executed, the unnecessary code logic is an additional overhead that cannot be ignored. In conclusion, these are the reasons why many people often criticize the high memory usage of Java programs.

1.2 Static Compilation Technology

Long cold start duration and high runtime memory usage make it difficult for Java applications to meet the requirements of cloud-native fast startup and scaling. Therefore, in the industry, the GraalVM open-source community led by Oracle Corporation launched the Java static compilation technology. Java programs can be compiled into local executable files in advance to achieve peak performance from the start. It effectively solves the problems of cold start and high runtime memory usage of Java applications, allowing Java to stay dynamic and competitive in the cloud-native era. As the only global advisory board member of the GraalVM community in China, Alibaba continues to refine GraalVM to make it more suitable for e-commerce and cloud scenarios. If you do not know the static compilation technology before, you can read the article From Local-native to Cloud-native: Practices and Challenges of Alibaba Dragonwell Static Compilation and Building Microservices Applications Based on Static Compilation for more detailed understanding. While static compilation technology offers benefits, switching to the GraalVM Native Image will cause some problems.

For example:

Many dynamic features of Java programs no longer work directly, such as dynamic class loading, reflection, and dynamic proxies. So, GraalVM should provide additional configurations to address these issues.
The platform independence that is a hallmark of the Java platform is no longer available.
Most importantly, Java Agents based on bytecode rewriting are no longer applicable because there is no concept of bytecode. As a result, various observability capabilities achieved through Java Agents, such as collecting Trace/Metrics, are no longer available.

Therefore, while reducing startup time and memory consumption, we want to ensure that applications possess out-of-the-box observability, meaning that all enhancements made by Java Agents continue to function. So how can we solve this problem?

1.3 Solutions

To address this common pain point, the Alibaba Cloud Observability Team, in collaboration with the programming language and compiler team, designed and implemented the pioneering static Java Agent instrumentation.

Before formally introducing the specific solution, it is necessary to review the working principle of the Java Agent. Key work processes of Java Agents include preMain execution, main function execution, and class loading. When an application uses a Java Agent, it will register a transformer for specific classes, such as class C in the figure. After preMain is executed, the main function is executed. During this process, various classes may be loaded. When the class loader encounters class C, it triggers the callback registered by the Java Agent, where the transformation logic for class C is executed to transform it into class C'. Finally, the class loader loads the transformed class C', effectively rewriting the bytecode of specific classes in the original application and adding additional logic.

Figure 2: How Java Agent technology works

However, in GraalVM, bytecode does not exist at runtime, making it impossible to enhance the application using a similar approach at runtime. If you want to achieve a similar capability, you need to implement it before runtime. Thus, the problem can be translated to:

How to transform target classes before runtime to obtain the transformed classes?
How to replace the original classes with the transformed ones before runtime?

To address these two issues, the overall design is shown in Figure 3 below. It consists of two phases: the pre-running phase and the static compilation phase. In the pre-running phase, the application mounts both the OTel Java Agent and the Native Image Agent for pre-running. The OTel Java Agent is responsible for transforming the class C to C' during the pre-running process. The Native Image Agent is responsible for collecting the transformed classes, such as the class C' shown in the following figure. This solves Problem A: How to transform target classes before runtime to obtain the transformed classes?

Figure 3: Static instrumentation enhancement solution for Java Agent

Next, in the static compilation phase, we use the original application, the OTel Agent, the transformed classes, and configurations as inputs and compile them. During the compilation process, we replace class C in the application with C', and generate an executable application that only contains C' for running. This addresses Problem B: How to replace the original classes with the transformed ones before runtime?

After understanding the overall scheme, you may be curious about what the Native Image Agent is and how it can be used to collect transformed classes. The Native Image Agent is actually a tool provided by GraalVM. It can scan our application to collect all the dynamic configurations required for static compilation. This helps eliminate the limitations of the GraalVM Compiler and allows developers to continue using some of the dynamic features offered by Java in GraalVM, such as reflection and dynamic proxies.

However, it does not directly help us collect transformed classes. To solve this problem, as shown in Figure 4 below, we implemented an interceptor in the Native Image Agent. This interceptor checks the bytecode of classes before and after transformation. If changes are detected, it records and saves them. Otherwise, it ignores the class.

Figure 4: Native Image Agent transformation

In fact, we found that simply recording transformed classes was not enough. Some classes are not part of the original application, such as dynamically generated classes. Therefore, we need to use the Native Image Agent to collect them. In addition, since preMain is a concept in the JVM and the Java Agent, it is not natively supported in GraalVM. We use it to generate the necessary preMain configuration to inform GraalVM about the entry point of the OTel Java Agent.

Apart from the above, we have also made some additional adaptations for special cases. For example, because the GraalVM compiler is also a Java application, we cannot directly collect transformed classes from the JDK using the Native Image Agent and replace them during compilation like non-JDK classes, as this could affect the behavior of the GraalVM compilation. Therefore, we implemented some special APIs within GraalVM. Then, we used them in the OTel Java Agent to re-instrument JDK classes, so that the GraalVM static compilation process can recognize the relevant content and the transformation logic for JDK classes is contained in the final Native Image executable file without modifying the JDK on which it depends. Lastly, there are multiple class loaders in the OTel Java Agent, while there is only one class loader in GraalVM, so we performed Shade on classes to achieve similar functionality.

2. Observe GraalVM Applications by Using ARMS

At present, ARMS Java Agent has supported the relevant capabilities based on the preceding solution. The specific steps of using ARMS to observe GraalVM applications are as follows:

2.1 Install Dependencies

Install the required dependencies in the environment.

1. Download the ARMS agent based on the region where your application resides. (The ARMS agent is available in the following five regions.

Region: China (Hangzhou)

Public endpoint

wget "http://arms-apm-cn-hangzhou.oss-cn-hangzhou.aliyuncs.com/ArmsAgentNative.zip" -O ArmsAgentNative.zip

VPC endpoint

wget "http://arms-apm-cn-hangzhou.oss-cn-hangzhou-internal.aliyuncs.com/ArmsAgentNative.zip" -O ArmsAgentNative.zip

Region: China (Shanghai)

Public endpoint

wget "http://arms-apm-cn-shanghai.oss-cn-shanghai.aliyuncs.com/ArmsAgentNative.zip" -O ArmsAgentNative.zip

VPC endpoint

wget "http://arms-apm-cn-shanghai.oss-cn-shanghai-internal.aliyuncs.com/ArmsAgentNative.zip" -O ArmsAgentNative.zip

Region: China (Beijing)

Public endpoint

wget "http://arms-apm-cn-beijing.oss-cn-beijing.aliyuncs.com/ArmsAgentNative.zip" -O ArmsAgentNative.zip

VPC endpoint

wget "http://arms-apm-cn-beijing.oss-cn-beijing-internal.aliyuncs.com/ArmsAgentNative.zip" -O ArmsAgentNative.zip

Region: China (Zhangjiakou)

Public endpoint

wget "http://arms-apm-cn-zhangjiakou.oss-cn-zhangjiakou.aliyuncs.com/ArmsAgentNative.zip" -O ArmsAgentNative.zip

VPC endpoint

wget "http://arms-apm-cn-zhangjiakou.oss-cn-zhangjiakou-internal.aliyuncs.com/ArmsAgentNative.zip" -O ArmsAgentNative.zip

Region: China (Shenzhen)

Public endpoint

wget "http://arms-apm-cn-shenzhen.oss-cn-shenzhen.aliyuncs.com/ArmsAgentNative.zip" -O ArmsAgentNative.zip

VPC endpoint

wget "http://arms-apm-cn-shenzhen.oss-cn-shenzhen-internal.aliyuncs.com/ArmsAgentNative.zip" -O ArmsAgentNative.zip

Decompress the installation package of the ARMS agent, go to the ArmsAgentNative directory, and then run the following command to install it in the environment.

sh install.sh

2. Download the GraalVM JDK: graalvm-java17-23.0.4-ali-1.2b.tar.gz, which provides observability.

3. Decompress and execute graalvm-java17-23.0.4-ali-1.2b/bin/native-image --version in the directory. The results are as follows:

4. Download Maven (apache-maven-3.8.4). If you have installed Maven, proceed to the next step.

5. Decompress the file and set the JAVA_HOME variable to the path of GraalVM and the MAVEN_HOME variable to the path of Maven. (Specify /xxx/ in the sample commands.)

export MAVEN_HOME=/xxx/apache-maven-3.8.4
export PATH=$PATH:$MAVEN_HOME/bin
export JAVA_HOME=/xxx/graalvm-java17-23.0.4-ali-1.2b
export PATH=$PATH:$JAVA_HOME/bin

2.2 Add Dependencies

After installation, add the following dependencies to the application:

<dependencies>
  <dependency>
      <groupId>com.alibaba.cloud</groupId>
      <artifactId>arms-javaagent-native</artifactId>
      <version>4.1.11</version>
      <type>pom</type>
  </dependency>
</dependencies>
<profiles>
  <profile>
    <id>native</id>
    <build>
      <plugins>
        <plugin>
          <groupId>org.graalvm.buildtools</groupId>
          <artifactId>native-maven-plugin</artifactId>
          <extensions>true</extensions>
          <executions>
            <execution>
              <id>build-native</id>
              <goals>
                <goal>compile-no-fork</goal>
              </goals>
              <phase>package</phase>
            </execution>
          </executions>
          <configuration>
            <fallback>false</fallback>
            <buildArgs>
              <arg>-H:ConfigurationFileDirectories=native-configs,/xxx/dynamic-configs</arg>
            </buildArgs>
          </configuration>
        </plugin>
      </plugins>
    </build>
  </profile>
</profiles>

Note: Replace /xxx/dynamic-configs in the code to the path of the dynamic configuration file of the application.

2.3 Pre-execute the Application

To ensure that the dynamic enhancement code injected by the ARMS agent is statically compiled into the final Native Image file, you need to mount the agent and execute the application in advance as shown in the Figure 3. When you pre-execute the application, you must ensure that all core code branches of the application are executed. A script is provided to help you with the pre-execution. Note that all RESTful interfaces of the application must be declared in the script as prompted so that they can be properly called and trigger business execution.

Open the script and supply the custom content based on the note to execute sh ArmsAgentNative/run.sh --collect --jvm --Carms. Mount the ARMS agent (run.sh in the ArmsAgentNative) to start the pre-execution, and collect configuration items of static compilation.

######## Modify the parameters
# The ARMS integration-specific parameters. Obtain a license key in the Java Application Monitor panel. Set the value of AppName to the application name. In a distributed architecture, an application can contain multiple peer application instances.
export ARMS_LICENSEKEY=
export ARMS_APPNAME=
# The list of application interfaces. Sample value: (interface1 interface2 interface3 interface4).
export PS=
# The application port. Sample value: 8080.
export PORT=
# The path where you want to store the Native Image file in the target directory of the application after static compilation. Sample value: target/graalvm-demo.
export NATIVE_IMAGE_FILE=
# The command for executing the ARMS agent. Sample command: -javaagent:./arms-native/aliyun-java-agent-native.jar -jar target/graalvm-demo-1.0.0.jar.
export JAVA_CMD=
########

2.4 Static Compilation

Perform the following steps to compile the application statically.

Execute mvn -Pnative package to start static compilation.
Execute sh ArmsAgentNative/run.sh --native --Carms to run the compiled project.

2.5 Demo

After the preceding static compilation is completed, the related executable Native Image file contains the code of ARMS Java Agent that provides observability. Execute in the normal GraalVM application deployment mode. The following shows some observable data collection effects in the ARMS console:

GraalVM application metric data collection effect

GraalVM application trace data collection effect

The following figure shows an example of initiating a scheduled task through Spring Schedule to call the RESTful interface and then using HttpClient to call it externally.

2.6 Test Results

Based on the preceding solution, we also conducted some tests to verify the startup speed and runtime memory usage of GraalVM applications. We found that after Java applications are statically compiled based on GraalVM, they can use the out-of-the-box observability capability normally, while greatly improving the runtime memory usage and startup latency. (The following tests are completed in the 32 vCPU/64 GiB/5 Mbps environment.)

Community

Performance Improvement by 10 Times, GraalVM Application Observability Practice

1. GraalVM Static Compilation

1.1 Background

Cold Start

High Runtime Memory Usage

1.2 Static Compilation Technology

1.3 Solutions

2. Observe GraalVM Applications by Using ARMS

2.1 Install Dependencies

2.2 Add Dependencies

2.3 Pre-execute the Application

2.4 Static Compilation

2.5 Demo

GraalVM application metric data collection effect

GraalVM application trace data collection effect

2.6 Test Results

Read previous post:

Read next post:

Alibaba Cloud Native Community

You may also like

Comments

Alibaba Cloud Native Community

Related Products

Application Real-Time Monitoring Service

Real-Time Livestreaming Solutions

Managed Service for Prometheus

CloudMonitor

A Free Trial That Lets You Build Big!