Mock Framework Evolution under JDK11: From PowerMockito to Mockito Only

This article discusses the necessity and practical methods for the strategy of migrating from a testing environment that uses PowerMock to one that only uses Mockito.

By Zuobing

Why Remove the PowerMock Dependency?

The impetus for this change came with the upgrade to JDK11. During research to upgrade a core application within our team to JDK11, we discovered that PowerMock, according to its documentation, supports only up to JDK9 and does not support higher JDK versions. More importantly, PowerMockito has not been maintained for a long time.

Continuing to integrate an unmaintained open-source testing framework into our projects could potentially lead to issues with future JDK versions and their new features. Therefore, taking advantage of the JDK11 upgrade as an opportunity for significant refactoring, we decided to remove the PowerMock dependency and adopt the Mockito Only testing framework.

According to the documentation about the Mockito open-source repository, Mockito continues to evolve with new versions, and the testing framework also consistently adapts to newer JDK versions.

https://github.com/powermock/powermock
https://github.com/mockito/mockito/releases/tag/v5.0.0

Additionally, we would like to share some experience with teams currently using PowerMock. When attempting to upgrade the PowerMock version, we encountered memory leak issues. Users in the PowerMock community have reported similar problems, but the corresponding issues have not been resolved.

https://github.com/powermock/powermock/issues/227

How to Remove the PowerMock Dependency?

To remove the PowerMock dependency, there are two main changes needed. One is replacing the corresponding Mock functionality with Mockito, and the other is updating Mockito to a newer version.

Replace PowerMockito with Mockito Only

JUnit Runner

When using Mockito Only, you need to use the following JUnit Runner: @RunWith(MockitoJUnitRunner.class)

In scenarios where the spring-test framework is needed, Mockito does not have a similar annotation like @PowerMockRunnerDelegate from PowerMock; however, we can achieve the same effect by configuring a Mockito JUnit Rule within the test class.

public class ExampleTestClass {

    @Rule public MockitoRule mockito = MockitoJUnit.rule();
    
    ...

    @Test
    public void test() {
        ...
    }
    ...
}

Mock Static Methods

The latest version of Mockito also supports Mock static methods, with similar usage to PowerMock.

Mock private and final Methods

Mockito does not support mocking private and final methods. When refactoring, some code restructuring may be necessary, as PowerMock makes it too easy to mock these methods, leading to lengthy and harder-to-test code.

Mock private and final Variables

Since Mockito does not support setting private and final variables, the Whitebox utility from PowerMock is no longer available. However, other third-party libraries like FieldUtils from Apache Commons can be used, but they can only set private variables and final variables still require code refactoring.

Mock Rule Reuse

Mock rule reuse simplifies unit tests and improves the efficiency of writing them. Repeated mock rules from unit tests can be extracted to achieve writing them once and reusing them multiple times.

PowerMock achieves this through the PowerMockPolicy class. Here is an example:

public class ContextMockPolicy implements PowerMockPolicy {
    @Override
    public void applyClassLoadingPolicy(MockPolicyClassLoadingSettings settings) {
        settings.addFullyQualifiedNamesOfClassesToLoadByMockClassloader(
                Xxx.class.getName());
    }

    @Override
    public void applyInterceptionPolicy(MockPolicyInterceptionSettings settings) {
        final Method getXxx = Whitebox.getMethod(Xxx.class, "getXxx");
        settings.stubMethod(getXxx, Optional.ofNullable(mockXxx());

        final Method getXxxXxx = Whitebox.getMethod(Xxx.class, "getXxxXxx");
        settings.stubMethod(getXxxXxx, Optional.ofNullable(Xxx));
    }
}

@MockPolicy({ContextMockPolicy.class})
public class ExampleTestClass {
    ...

    @Test
    public void test() {
        ...
    }
    ...
}

Using Mockito Only, you can achieve the same effect by combining the ClassRule of JUnit. Below is the corresponding implementation for Mockito Only for the example given.

public class ContextMockRule implements TestRule {
    private MockedStatic<Xxx> mockedStatic;

    @Override
    public Statement apply(Statement base, Description description) {
        return new Statement() {
            @Override
            public void evaluate() throws Throwable {
                try {
                    mockXxx();
                    base.evaluate();
                } finally {
                    if (mockedStatic != null) {
                        mockedStatic.close();
                    }
                }
            }
        };
    }

    private void mockXxx() {
        mockedStatic = Mockito.mockStatic(Xxx.class);

        mockedStatic
                .when(() -> Xxx.getXxx())
                .thenReturn(Optional.ofNullable(mockXxx()));
        mockedStatic
                .when(() -> Xxx.getXxxXxx())
                .thenReturn(Optional.ofNullable(Xxx));
    }
}

public class ExampleTestClass {

    @ClassRule public static ContextMockRule contextMockRule = new ContextMockRule();
    
    ...

    @Test
    public void test() {
        ...
    }
    ...
}

Mockito Only Limitations in Multi-threaded Scenarios

Mockito has limitations in multi-threaded testing scenarios, including those involving ExecutorService and ParallelStream, where Mock static methods may not work as expected, not being aligned with PowerMock.

For the concurrent scenario with ExecutorService thread pools, we can adopt the following method of Mock ExecutorService. However, for the concurrent scenario with Java Stream ParallelStream, we have not yet found a viable solution.

ExecutorService chatExecutor = Mockito.mock(ExecutorService.class);
doAnswer(
        (Answer<Object>)
                invocation -> {
                    Object[] args = invocation.getArguments();
                    Callable callable = (Callable) args[0];
                    Object result = callable.call();

                    FutureTask futureTask = Mockito.mock(FutureTask.class);
                    Mockito.when(futureTask.get(anyLong(), any()))
                            .thenReturn(result);
                    return futureTask;
                })
.when(chatExecutor)
.submit(any(Callable.class));

Incompatible Changes with Mockito Version Upgrade

https://groups.google.com/g/mockito/c/8_WGBB3Jbtk/m/JUUq4EpgplcJ

I'd like to give additional info on this. The origin of these methods is they come from anything i.e. anything matches, later for shortness and cast avoidance the aliases grew, but the API naming thus became inconsistent with what a human would expect. So this behavior is being changed in mockito 2 beta, to be precise here's the status on these API in the version 2.0.5-beta :

• any, anyObject, any(Class) won't check anything (at first they were just aliases for anything and for cast avoidance)
• anyX like anyString will check the arg is not null and that has the correct type
• anyList will check the argument is not null and a List instance
• anyListOf (and the likes) at the moment are just aliases to their non generic counter part like anyList
Note this is work in progress (started here in #141), these new behavior can / will change in the beta phase. I'm especially wondering if the any familly should allow null and if not do a type check. For example with these matchers :

• any, anyObject stay the same, they currently allow null and don't have to do type check anyway
• any(Class) currently allows null and doesn't do type check => allows null and if not checks for the given type
• any<Collection>Of currently doesn't allow null and does a type check of the collection, not elements => allows null, if not checks collection type, if not empty checks element type

Maybe extend the isA family that won't allow any null arguments. Thoughts ?

Referring to discussions in the Mockito Google Group about behavioral changes after upgrading Mockito versions, and based on our own experience in project refactoring, we identified the following incompatible changes:

Changes in anyXXX() Matching Behavior

Methods like anyLong(), anyString(), and anyObject() that include type checks no longer support null values. For null values, you need to modify the method to be "any()" instead.

Changes in Parameter Matcher Input Types

The anonymous class method for ArgumentMatcher now specifies the exact parameter type, rather than accepting any Object.

Changes in Obtaining Invocation Parameter Behaviors

There are changes in how invocation parameters are obtained.

How to Efficiently Refactor Large Volumes of Test Code?

It is often said that an engineer who does not know how to take shortcuts is not a good product developer. For applications that have been developed and maintained for several years, there might be hundreds or even thousands of unit tests within the project. If you were to refactor these manually, it could consume a significant amount of time and effort, potentially driving away a dedicated product developer. Here are some tools that can help improve efficiency. Feel free to share better options in the comments.

AI Agent

If your project contains a relatively small number of unit tests, you can leverage AI to lighten your workload.

In the scenario of refactoring test code, you can define a custom Prompt to automate the refactoring of test classes. However, due to the nature of large language models, the longer the input and output, the more time it takes to generate complete answers, which makes refactoring test code quite slow. Additionally, the generated content may not always meet the requirements.

Perhaps a more sophisticated Agent approach could be adopted, but this would require development and extensive debugging tailored to real-world testing scenarios.

Unit Test Refactoring Script

For projects with thousands of unit tests, the above Prompt is not feasible for speeding up refactoring.

Here is my solution. I started by manually refactoring a few unit tests and identified steps that could be automated, such as replacing annotations. Based on these observations, I created a unit test refactoring script using Python and Tree-sitter, significantly reducing repetitive tasks.

However, the automated script does not solve all incompatibility issues, such as Mockito's limitations in multi-threaded testing scenarios. These incompatibilities still require manual verification and resolution when validating unit tests.

Effects after Implementing Mockito Only

More Focused Test Classes

There are fewer PowerMock-specific annotations, and the creation and disposal of Mock resources are more traceable, eliminating concerns about memory leaks.

Significantly Improved Unit Test Performance

The duration of unit test tasks is notably reduced. On my work computer, it went from 9 minutes and 53 seconds to 3 minutes and 57 seconds, and the memory usage dropped from around 5 GB to approximately 2.5 GB.

Differences Between PowerMock and Mockito Implementations

Content sourced from GTP-4o

Both Mockito and PowerMock are commonly used Java unit testing frameworks for creating and managing mock objects, but they differ in their underlying implementation principles.

Mockito

Mockito is a relatively lightweight testing framework that primarily uses Java dynamic proxies and bytecode generation libraries (such as ByteBuddy) to create mock objects. Its implementation principles mainly include:

Dynamic Proxy: For interface-class mock objects, Mockito utilizes Java's dynamic proxy mechanism (java.lang.reflect.Proxy) to generate proxy classes. This approach relies on Java reflection APIs, allowing the creation of proxy instances at runtime to substitute for actual implementations.
Bytecode Manipulation: For concrete-class (non-interface) mock objects, Mockito employs the ByteBuddy library for bytecode manipulation. ByteBuddy enables the generation of new classes or modification of existing ones at runtime to proxy the behavior of the original class.
Method Interception: Regardless of whether a mock object is created through dynamic proxy or bytecode manipulation, Mockito uses method interceptors to capture method invocations. The interceptor checks configured behaviors, such as return values and exceptions thrown, and responds accordingly to method invocations.
Behavior Recording: Mockito also records method invocations on each mock object, enabling verification in tests that certain methods were called as expected.

PowerMock

PowerMock is a more powerful testing framework that is often used alongside other mocking frameworks like Mockito and EasyMock. It can mock scenarios that Mockito cannot handle, such as static methods, constructors, and private methods. The implementation principles of PowerMock are more complex and primarily involve the following aspects:

Bytecode Manipulation: Similar to Mockito, PowerMock uses bytecode manipulation techniques, but it mainly relies on the Java Assist and CGLib libraries to modify the bytecode. These libraries enable PowerMock to generate proxy classes and modify the bytecode of classes, including static methods and constructors.
ClassLoader Substitution: PowerMock uses a custom class loader to load the classes under test. This custom class loader can modify the bytecode of classes, allowing the tracking and control of class modifications. This is crucial for PowerMock to bypass JVM restrictions on certain classes and methods and seamlessly insert its own logic.
Proxying and Interception at the JVM Level: To mock static and private methods, PowerMock performs proxying and interception at the JVM level. This means it can redirect static method invocations by modifying the bytecode of classes and even replace constructors to insert mock behavior when an object is created.
Integration with Other Mocking Frameworks: PowerMock is typically integrated with frameworks like Mockito or EasyMock. It handles special cases that regular mocking frameworks cannot do, such as mocking static and private methods. By extending the functionality of these frameworks, PowerMock enhances the flexibility and power of testing.

Summary

• Mockito: Primarily generates mock objects through dynamic proxy and bytecode generation. It can only handle non-static and non-private methods of interfaces or concrete classes and is relatively lightweight.

• PowerMock: Uses advanced techniques such as bytecode manipulation and class loader substitution to handle complex scenarios like static methods, constructors, and private methods. It extends the functionality of frameworks like Mockito and EasyMock, offering more powerful features with a more complex implementation.

The choice of underlying technologies for these two frameworks differentiates their functionalities and use cases. While PowerMock can perform more "magic" operations, Mockito's simplicity and performance advantages make it more popular for most daily testing needs.

Disclaimer: The views expressed herein are for reference only and don't necessarily represent the official views of Alibaba Cloud.

Community