The purpose of writing unit test cases is not to pursue unit test code coverage but to use unit tests to verify regression code, trying to find BUGs hidden in the code. Therefore, we should write valid unit test cases. This article summarizes a set of methods and principles to avoid writing invalid unit test cases through daily unit test practice.
Wikipedia describes it as:
In computer programming, unit testing is a software testing method by which individual units of source code—sets of one or more computer program modules together with associated control data, usage procedures, and operating procedures—are tested to determine whether they are fit for use.
First, let's look at integration and unit testing with a simple case of service code.
Here, take the paging queryUser of UserService as an example:
@Service
public class UserService {
/** Define dependent objects. */
/** User DAO */
@Autowired
private UserDAO userDAO;
/**
* Query users
*
* @param companyId
* @param startIndex
* @param pageSize
* @return user pageData
*/
public PageDataVO<UserVO> queryUser(Long companyId, Long startIndex, Integer pageSize) {
// Query user data.
// Query user data: Total quantity
Long totalSize = userDAO.countByCompany(companyId);
// Query interface data: Data list
List<UserVO> dataList = null;
if (NumberHelper.isPositive(totalSize)) {
dataList = userDAO.queryByCompany(companyId, startIndex, pageSize);
}
// Return pageData.
return new PageDataVO<>(totalSize, dataList);
}
}Many people think all test cases that use the JUnit test framework are unit test cases, so they write the following integration test cases:
@Slf4j
@RunWith(PandoraBootRunner.class)
@DelegateTo(SpringJUnit4ClassRunner.class)
@SpringBootTest(classes = {ExampleApplication.class})
public class UserServiceTest {
/** UserServices */
@Autowired
private UserService userService;
/**
* Test: Query users
*/
@Test
public void testQueryUser() {
Long companyId = 123L;
Long startIndex = 90L;
Integer pageSize = 10;
PageDataVO<UserVO> pageData = userService.queryUser(companyId, startIndex, pageSize);
log.info("testQueryUser: pageData={}", JSON.toJSONString(pageData));
}
}Integration test cases have the following features:
@Autowired
The unit test cases written in JUnit + Mockito are listed below:
@Slf4j
@RunWith(MockitoJUnitRunner.class)
public class UserServiceTest {
/** Define static constants */
/** Resource path */
private static final String RESOURCE_PATH = "testUserService/";
/** Mock dependent objects */
/** User DAO */
@Mock
private UserDAO userDAO;
/** Define test object */
/** UserServices */
@InjectMocks
private UserService userService;
/**
* Test: Query users - no data
*/
@Test
public void testQueryUserWithoutData() {
// Mock dependent methods
// Mock dependent methods: userDAO.countByCompany
Long companyId = 123L;
Long startIndex = 90L;
Integer pageSize = 10;
Mockito.doReturn(0L).when(userDAO).countByCompany(companyId);
// Call test methods
String path = RESOURCE_PATH + "testQueryUserWithoutData/";
PageDataVO<UserVO> pageData = userService.queryUser(companyId, startIndex, pageSize);
String text = ResourceHelper.getResourceAsString(getClass(), path + "pageData.json");
Assert.assertEquals("pageData inconsistency", text, JSON.toJSONString(pageData));
// Verify dependent methods
// Verify dependent methods: userDAO.countByCompany
Mockito.verify(userDAO).countByCompany(companyId);
// Verify dependent objects.
Mockito.verifyNoMoreInteractions(userDAO);
}
/**
* Test: Query user - data available
*/
@Test
public void testQueryUserWithData() {
// Mock dependent methods.
String path = RESOURCE_PATH + "testQueryUserWithData/";
// Mock dependent methods: userDAO.countByCompany
Long companyId = 123L;
Mockito.doReturn(91L).when(userDAO).countByCompany(companyId);
// Mock dependent methods: userDAO.queryByCompany
Long startIndex = 90L;
Integer pageSize = 10;
String text = ResourceHelper.getResourceAsString(getClass(), path + "dataList.json");
List<UserVO> dataList = JSON.parseArray(text, UserVO.class);
Mockito.doReturn(dataList).when(userDAO).queryByCompany(companyId, startIndex, pageSize);
// Call test methods.
PageDataVO<UserVO> pageData = userService.queryUser(companyId, startIndex, pageSize);
text = ResourceHelper.getResourceAsString(getClass(), path + "pageData.json");
Assert.assertEquals("pageData inconsistency ", text, JSON.toJSONString(pageData));
// Verify dependent methods.
// Verify dependent methods: userDAO.countByCompany
Mockito.verify(userDAO).countByCompany(companyId);
// Verify dependent methods: userDAO.queryByCompany
Mockito.verify(userDAO).queryByCompany(companyId, startIndex, pageSize);
// Verify dependent objects.
Mockito.verifyNoMoreInteractions(userDAO);
}
}Unit test cases have the following features:
Why do integration tests not count as unit tests? We can judge from the principle of the unit test. In the industry, common unit test principles are the AIR principle and the FIRST principle.
The AIR Principle is listed below:
1. A-Automatic
Unit tests should be automated and non-interactive. Test cases are usually executed regularly, and the execution process must be automated to make sense. A unit test whose output requires manual review is not good. System.out is not allowed to be used for manual verification in unit tests, and assert must be used to verify.
2. I-Independent
Unit tests should maintain independence. Unit test cases must not call each other or rely on external resources to ensure that unit tests are stable, reliable, and easy to maintain.
3. R-Repeatable
Unit tests can be repeated and cannot be affected by the external environment. Unit tests are usually put into continuous integration and executed every time code is committed.
The FIRST Principle is listed below:
1. F-Fast
Unit tests should be run quickly. Among various test methods, unit tests are the fastest, and unit tests of large projects should be run in a few minutes.
2. I-Independent
Unit tests should be run independently. The unit test cases should have no dependency on each other and external resources.
3. R-Repeatable
Unit tests should be able to run stably and repeatedly, and the results of each run should be stable and reliable.
4. S-Self-Validating
Unit tests should be automatically verified by use cases and cannot rely on manual verification.
5. T-Timely
Unit tests must be written, updated, and maintained timely to ensure that use cases can dynamically guarantee quality as business code changes.
Xihua of Alibaba proposed an ASCII Principle:
1. A-Automatic
Unit tests should be automated and non-interactive.
2. S-Self-Validating
Assertions must be used in unit tests to verify correctness, while manual verification based on the output cannot be used.
3. C-Consistent
The parameters and results of the unit tests are determined and consistent.
4. I-Independent
Unit tests cannot call each other or on the order of execution.
5. I-Isolated
Unit tests need to be isolated and not dependent on external resources.
Based on the unit test principles in the previous section, we can compare the satisfaction of integration tests and unit tests.
We can draw the following conclusions after comparing the table:
Therefore, according to these unit test principles, it can be seen that the integration test is highly uncertain and cannot completely replace the unit test. In addition, integration tests are always integration tests, even if they are used to replace unit tests (such as testing the DAO method with the H2 memory database).
You must put yourself in their shoes to identify invalid unit tests, thinking about how to write less unit test code while ensuring unit test coverage. Then, you must start with the unit test writing process and see which stages and methods can cut corners.
Wikipedia describes it as:
Code coverage is a percentage measure of the degree to which the source code of a program is executed when a particular test suite is run..
Common metrics of unit test coverage include:
1. Line Coverage
It is used to measure whether each line of execution statement in the code under test has been tested.
2. Branch Coverage
It is used to measure whether each code branch in the code under test has been tested.
3. Condition Coverage
It is used to measure whether each subexpression (true and false) in the condition of the code under test has been tested.
4. Path Coverage
It is used to measure whether each code branch combination in the code under test has been tested.
In addition, there are Method Coverage, Class Coverage, and other unit test coverage metrics.
Here's a simple way to analyze the coverage metrics of each unit test:
public static byte combine(boolean b0, boolean b1) {
byte b = 0;
if (b0) {
b |= 0b01;
}
if (b1) {
b |= 0b10;
}
return b;
}
Unit test coverage can only represent whether the class, method, execution statement, code branch, conditional subexpression, etc. of the code under test are executed. It does not represent whether the code is executed correctly and returns the correct result. So, it doesn't make any sense to look at unit test coverage without looking at unit test validity.
First, introduce the unit test writing process summarized by the author:
The defining object phase consists of defining the objects under test, mocking dependent objects (class members), and injecting dependent objects (class members).
The mocking method phase mainly includes mocking dependent objects (parameters, return values, and exceptions) and mocking dependent methods.
The calling method phase mainly includes mocking dependent objects (parameters), calling tested methods, and verifying parameter objects (return values and exceptions).
The verifying method phase includes verifying dependent methods, verifying data objects (parameters), and verifying dependent objects.
For the phases and methods of the unit test writing process, can we cut corners without affecting unit test coverage?
From the table, it can be concluded that cutting corners mainly exists in the verification phase.
1. Calling method phase
2. Verifying method phase
Through some merging and splitting, the follow-up will be explored in the following three parts.
In unit tests, verifying a data object is to verify whether the expected parameter values are passed in, the expected return values are returned, and the expected attribute values are set.
In unit tests, the data objects that need to be verified mainly come from the following sources.
The data object comes from the return value of calling the method under test. For example:
PageDataVO<UserVO> pageData = userService.queryUser(companyId, startIndex, pageSize);The data object comes from the parameter capture of the verifying dependent methods. For example:
ArgumentCaptor<UserDO> userCreateCaptor = ArgumentCaptor.forClass(UserDO.class);
Mockito.verify(userDAO).create(userCreateCaptor.capture());
UserDO userCreate = userCreateCaptor.getValue();The data object comes from the attribute values of obtaining the objects under test. For example:
userService.loadRoleMap();
Map<Long, String> roleMap = Whitebox.getInternalState(userService, "roleMap");The data object comes from the attribute values of getting request parameters. For example:
OrderContext orderContext = new OrderContext();
orderContext.setOrderId(12345L);
orderService.supplyProducts(orderContext);
List<ProductDO> productList = orderContext.getProductList();There are other sources of data objects, but I won't go into details.
When a method under test is called, the return value and exception need to be verified. When a method call is verified, the captured parameter value also needs to be verified.
JUnit provides the Assert.assertNull and Assert.assertNotNull methods to verify whether the data object is null.
// 1. Verify that the data object is null.
Assert.assertNull( "The user ID must be null", userId);
// 2. Verify that the data object is not null.
Assert.assertNotNull( "The user ID cannot be null", userId);JUnit provides the Assert.assertTrue and Assert.assertFalse methods to verify that the Boolean value of a data object is true or false.
// 1. Verify that the data object is true.
Assert. assert True(, NumberHelper. "Return value must be true" isPositive (1) );
// 2. Verify that the data object is false.
Assert. assert False(, NumberHelper. "Return value must be false" isPositive (-1) );JUnit provides the Assert.assertSame and Assert.assertNotSame methods to verify that data object references are consistent or not.
// 1. Verify consistency of data object.
Assert.assertSame("Users must be consistent", expectedUser, actualUser);
// 2. Verify that the data object is inconsistent.
Assert.assertNotSame("Users cannot be consistent", expectedUser, actualUser);JUnit provides Assert.assertEquals, Assert.assertNotEquals, and Assert.assertArrayEquals methods that you can use to verify that data object values are equal or not.
// 1. Verify a simple data object.
Assert.assertNotEquals("User name inconsistency", "admin", userName);
Assert. assert Equals("Account amount inconsistency", 10000.0D, accountAmount, 1E-6D);
// 2. Verify a simple collection object.
Assert. "User ID list inconsistency" ArrayEquals( assert, new Long[] {1L, 2L, 3L}, userIds);
Assert. assert, Arrays. "User ID list Inconsistent" Equals( asList (1L, 2L, 3L), userIdList);
// 3. Verify complex data objects.
Assert. assert, Long. "User ID inconsistency" Equals( valueOf (1L), user.get Id() );
Assert. "User name inconsistency" Equals( assert, "admin", user. getName () );
...
// 4. Verify complex collection objects.
Assert. assert "User list length inconsistency" Equals(, expectedUserList. size (), actualUserList.size () );
UserDO[] expectedUsers = expectedUserList.toArray(new UserDO[0]);
UserDO[] actualUsers = actualUserList.toArray(new UserDO[0]);
for (int i = 0; i < actualUsers.length; i++) {
Assert.assertEquals(String.format("User (%s) ID inconsistency", i), expectedUsers[i].getId(), actualUsers[i].getId());
Assert.assertEquals(String.format("User (%s) name inconsistency ", i), expectedUsers[i].getName(), actualUsers[i].getName());
...
};
// 5. Verify data object through serialization.
String text = ResourceHelper.getResourceAsString(getClass(), "userList.json");
Assert. assert "User list consistency" Equals(, text, JSON. toJSONString (userList) );;
// 6. Verify the private attribute field of the data object.
Assert. assert "Base package inconsistency" Equals(, "com.alibaba.example", Whitebox. getInternalState (configurer, "basePackage") );There are other verification methods for data objects, which will not be illustrated here.
Here, a paging query of company users is used as an example to illustrate the problem existing in verifying data objects.
Code cases:
public PageDataVO<UserVO> queryUser(Long companyId, Long startIndex, Integer pageSize) {
// Query user data.
// Query user data: Total quantity
Long totalSize = userDAO.countByCompany(companyId);
// Query interface data: Data List
List<UserVO> dataList = null;
if (NumberHelper.isPositive(totalSize)) {
List<UserDO> userList = userDAO.queryByCompany(companyId, startIndex, pageSize);
dataList = userList.stream().map(UserService::convertUser)
.collect(Collectors.toList());
}
// Return pageData.
return new PageDataVO<>(totalSize, dataList);
}
private static UserVO convertUser(UserDO userDO) {
UserVO userVO = new UserVO();
userVO.setId(userDO.getId());
userVO.setName(userDO.getName());
userVO.setDesc(userDO.getDesc());
...
return userVO;
}Negative case:
Many people are too lazy to perform any verification on data objects.
// Call test methods.
userService.queryUser(companyId, startIndex, pageSize);Existing problems:
It is impossible to verify whether the data object is correct or not. For example, if the code under test makes the following modifications.
// Return pageData.
return null;Negative case:
Since there is a problem with not verifying the data object, I will verify that the data object is not null.
// Call test methods.
PageDataVO<UserVO> pageData = userService.queryUser(companyId, startIndex, pageSize);
Assert.assertNotNull( "pageData is not null", pageData);Existing problems:
It is impossible to verify whether the data object is correct or not. For example, if the code under test makes the following modifications:
// Return pageData.
return new PageDataVO<>();Negative case:
Since it is impossible to simply verify that the data object is not null, I will verify some attributes of the data object.
// Call test methods.
PageDataVO<UserVO> pageData = userService.queryUser(companyId, startIndex, pageSize);
Assert.assertEquals( "The total amount of data is not null.", totalSize, pageData.getTotalSize());Existing problems:
It is impossible to verify whether the data object is correct or not. For example, if the code under test makes the following modifications:
// Return pageData.
return new PageDataVO<>(totalSize, null);Negative case:
Since it is impossible to verify some attributes of the data object, I will verify all the attributes of the data object.
// Call test methods.
PageDataVO<UserVO> pageData = userService.queryUser(companyId);
Assert.assertEquals( "The total amount of data is not null.", totalSize, pageData.getTotalSize());
Assert.assertEquals( "The data list is not null", dataList, pageData.getDataList());Existing problems:
The codes look perfect. It verifies the two attribute values totalSize and dataList in PageDataVO. However, if you add startIndex and pageSize to PageDataVO, you cannot verify whether the two new attributes are assigned correctly. Sample code:
// Return pageData.
return new PageDataVO<>(startIndex, pageSize, totalSize, dataList);Note: This method only applies to data objects whose attribute fields are immutable.
Is there a perfect verification scheme for the attribute field additions of data objects? Yes! The answer is to use JSON serialization and then compare the JSON text contents. If attribute fields add data objects, you will inevitably be prompted with an inconsistent JSON string.
Perfect case:
// Call test methods.
PageDataVO<UserVO> pageData = userService.queryUser(companyId, startIndex, pageSize);
text = ResourceHelper.getResourceAsString(getClass(), path + "pageData.json");
Assert.assertEquals( "pageData inconsistency", text, JSON.toJSONString(pageData));Note: This method only applies to data objects with variable attribute fields.
Since there is no mock data object section, guidelines on mock data objects are inserted in the verification of data object section.
In the previous section, we showed how to verify data objects perfectly. Is this approach perfect? The answer is no.
For example, we assign the attribute values, name, and desc of all UserDO objects in uesrList returned by userDAO.queryByCompany method to be null and then exchange the name and desc assignments of the convertUser method. The unit test cases above cannot be verified.
private static UserVO convertUser(UserDO userDO) {
UserVO userVO = new UserVO();
userVO.setId(userDO.getId());
userVO.setName(userDO.getDesc());
userVO.setDesc(userDO.getName());
...
return userVO;
}Therefore, in unit tests, all attribute values of the mock object cannot be null except for the triggering condition branch.
In the preceding example, if the UserDO and UserVO added a new property field "age" (user age) and the following assignment statement was added:
userVO.setAge(userDO.getAge());If the unit test is still executed with the original data object, we will find that the unit test case execution passes. Since the attribute field age is null, it makes no difference whether the assignment is made or not. So, the new attribute field of the attribute class is that the attribute value of the data object must be mocked.
Note: If you use a JSON string comparison and set the output null field, it is possible to trigger execution failure of unit test cases.
In unit tests, all data objects must be verified:
Please see the Source Method of Data Object for more information.
When you use assertions to verify data objects, you must use assertions with explicit semantics.
Positive example:
Assert.assertTrue( "The return value is not true", NumberHelper.isPositive( 1 ));
Assert.assertEquals( "User inconsistency", userId, userService.createUser(userCreateVO));Negative example:
Assert.assertNotNull( "The user cannot be null", userService.getUser(userId));
Assert.assertNotEquals( "Users cannot be the same", user, userService.getUser(userId));Beware of cases that attempt to bypass this guideline and try to make ambiguous semantic judgments with explicit semantic assertions.
Assert.assertTrue( "The user cannot be null", Objects.nonNull(userService.getUser(userId)));If it is a model class, fields will be added according to business requirements. Then, for the data object corresponding to this model class, try to use the overall verification method.
Positive example:
UserVO user = userService.getUser(userId);
String text = ResourceHelper.getResourceAsString(getClass(), path + "user.json");
Assert.assertEquals( "User inconsistency", text, JSON.toJSONString(user));Negative example:
UserVO user = userService.getUser(userId);
Assert.assertEquals( "User ID inconsistency", Long.valueOf( 123L ), user.getId());
Assert.assertEquals( "User names inconsistency", "changyi", user.getName());
...This data verification method can verify whether the model class deletes the attribute field. However, if a field is added to the model class, it cannot be verified. Therefore, if this verification method is adopted, the test cases need to be sorted out and completed after attribute fields of the model class are added. Otherwise, when you use a unit test case to regress your code, it will tell you there is no problem here.
Exception, as an important feature of Java, is an important manifestation of the robustness of Java. Capturing and verifying a thrown exception is also a type of test case. Therefore, you need to verify thrown exceptions in unit tests.
Check whether the attribute field is invalid. Otherwise, an exception is thrown.
private Map<String, MessageHandler> messageHandlerMap = ...;
public void handleMessage(Message message) {
...
// Check whether the processor map is null.
if (CollectionUtils.isEmpty(messageHandlerMap)) {
throw new ExampleException("Message processor map cannot be null ");
}
...
}Check whether the input parameters are valid. Otherwise, an exception is thrown.
public void handleMessage(Message message) {
...
// Check whether the acquisition processor is null.
MessageHandler messageHandler = messageHandlerMap.get(message.getType());
if (CollectionUtils.isEmpty(messageHandler)) {
throw new ExampleException("Message acquisition processor cannot be null ");
}
...
}Note: The Assert class provided by the Spring framework used here has the same effect as the if-throw statement.
Check whether the returned value is valid. Otherwise, an exception is thrown.
public void handleMessage(Message message) {
...
// Process messages in processors.
boolean result = messageHandler.handleMessage(message);
if (!reuslt) {
throw new ExampleException("Message processing exceptions ");
}
...
}If you call a mock dependent method, the mock dependent method may throw an exception.
public void handleMessage(Message message) {
...
// Process messages in processors.
boolean result = messageHandler.handleMessage(message); // Throw an exception directly.
...
}Here, you can catch exceptions, print out logs, or continue to throw exceptions.
Sometimes, static method calls may throw exceptions.
// IOException may be thrown.
String response = HttpHelper.httpGet(url, parameterMap);In addition, there are other ways to throw exceptions, which are not described here.
There are usually four ways to verify thrown exceptions in unit tests.
try-catch Statement to Verify Thrown ExceptionsThe simplest and most direct way to catch exceptions in Java unit test cases is to use the try-catch statement.
@Test
public void testCreateUserWithException() {
// Mock dependent methods.
Mockito.doReturn(true).when(userDAO).existName(Mockito.anyString());
// Call test methods.
UserCreateVO userCreate = new UserCreateVO();
try {
userCreate.setName("changyi");
userCreate.setDescription("Java Programmer");
userService.createUser(userCreate);
} catch (ExampleException e) {
Assert.assertEquals("Exception encoding inconsistency ", ErrorCode.OBJECT_EXIST, e.getCode());
Assert.assertEquals("Exception message inconsistency", "The user already exists", e.getMessage());
}
// Verify dependent methods.
Mockito.verify(userDAO).existName(userCreate.getName());
}The @Test annotation of JUnit provides an expected attribute that specifies an expected exception type for catching and verifying exceptions.
@Test(expected = ExampleException.class)
public void testCreateUserWithException() {
// Mock dependent methods.
Mockito.doReturn(true).when(userDAO).existName(Mockito.anyString());
// Call test methods.
UserCreateVO userCreate = new UserCreateVO();
userCreate.setName("changyi");
userCreate.setDescription("Java Programmer");
userService.createUser(userCreate);
// Verify dependent methods (not executed).
Mockito.verify(userDAO).existName(userCreate.getName());
}Note: The test case will jump out of the method after being executed to userService.createUser method. As a result, subsequent verification statements cannot be executed. Therefore, this method cannot verify the exception code, message, reason, and others, nor the dependent method and its parameters.
If you want to verify the exception cause and message, you need to define the ExpectedException object with the @Rule annotation and then declare the exception type, cause, and message to be caught at the beginning of the test method.
@Rule
public ExpectedException exception = ExpectedException.none();
@Test
public void testCreateUserWithException1() {
// Mock dependent methods.
Mockito.doReturn(true).when(userDAO).existName(Mockito.anyString());
// Call test methods.
UserCreateVO userCreate = new UserCreateVO();
userCreate.setName("changyi");
userCreate.setDescription("Java Programmer");
exception.expect(ExampleException.class);
exception.expectMessage("The user already exists.");
userService.createUser(userCreate);
// Verify dependent methods (not executed).
Mockito.verify(userDAO).existName(userCreate.getName());
}Note: The test case will jump out of the method after being executed to userService.createUser method. As a result, subsequent verification statements cannot be executed. Therefore, this method cannot verify dependent methods and their parameters. Since the ExpectedException verification method only supports verifying the exception type, reason, and message, you cannot verify the custom attribute field values for exceptions. Currently, JUnit officially recommends replacing it with Assert.assertThrows.
A more concise exception verification method is provided in the latest version of JUnit: the Assert.assertThrows method.
@Test
public void testCreateUserWithException() {
// Mock dependent methods.
Mockito.doReturn(true).when(userDAO).existName(Mockito.anyString());
// Call test methods.
UserCreateVO userCreate = new UserCreateVO();
userCreate.setName("changyi");
userCreate.setDescription("Java Programmer");
ExampleException exception = Assert.assertThrows("Exception type inconsistency ", ExampleException.class, () -> userService.createUser(userCreate));
Assert.assertEquals("Exception encoding inconsistency ", ErrorCode.OBJECT_EXIST, exception.getCode());
Assert.assertEquals("Exception message inconsistency ", " The user already exists ", exception.getMessage());
// Verify dependent methods.
Mockito.verify(userDAO).existName(userCreate.getName());
}Compare the four verification methods based on the different verification exception features. The results are listed below:
In summary, it is best to use the Assert.assertThrows method to verify thrown exceptions, which is officially recommended by JUnit.
Here, take the thrown exception when creating a user as an example to illustrate the problems when verifying thrown exceptions.
Code cases:
private UserDAO userDAO;
public void createUser(@Valid UserCreateVO userCreateVO) {
try {
UserDO userCreateDO = new UserDO();
userCreateDO.setName(userCreateVO.getName());
userCreateDO.setDesc(userCreateVO.getDesc());
userDAO.create(userCreateDO);
} catch (RuntimeException e) {
log.error("User creation exception: userName={}", userName, e)
throw new ExampleException(ErrorCode.DATABASE_ERROR, " User creation exception ", e);
}
}Negative case:
When verifying thrown exceptions, many people use the expected attribute of @ Test annotation and specify the value as Exception.class. The main reasons are listed below:
@Test(expected = Exception.class)
public void testCreateUserWithException() {
// Mock dependent methods.
Throwable e = new RuntimeException();
Mockito.doThrow(e).when(userDAO).create(Mockito.any(UserCreateVO.class));
// Call test methods.
UserCreateVO userCreateVO = ...;
userService.createUser(userCreate);
}Existing problems:
The use case specifies a generic exception type and does not verify the type of thrown exceptions. Therefore, if the ExampleException exception is changed to the RuntimeException exception, the unit test case cannot be verified.
throw new RuntimeException("User creation exception", e);Negative case:
Since you need to verify the exception type, simply specify the expected attribute of the @ Test annotation as ExampleException.class.
@Test(expected = ExampleException.class)
public void testCreateUserWithException() {
// Mock dependent methods.
Throwable e = new RuntimeException();
Mockito.doThrow(e).when(userDAO).create(Mockito.any(UserCreateVO.class));
// Call test methods.
UserCreateVO userCreateVO = ...;
userService.createUser(userCreate);
}Existing problems:
The preceding use case only verifies the exception types but does not verify the attribute fields that throw exceptions (exception message, exception reason, and error code). Therefore, if the error code DATABASE_ERROR is changed to PARAMETER_ERROR, the unit test case cannot be verified.
throw newExampleException(ErrorCode.PARAMETER_ERROR, " User creation exception", e);Negative case:
If you want to verify exception attributes, you must use the Assert.assertThrows method to catch the exception and verify the common attributes of the exception. However, some people are lazy and only verify part of the attributes of thrown exceptions.
// Mock dependent methods.
Throwable e = new RuntimeException();
Mockito.doThrow(e).when(userDAO).create(Mockito.any(UserCreateVO.class));
// Call test methods.
UserCreateVO userCreateVO = ...;
ExampleException, "Exception type inconsistency" ExampleException exception=Assert.assertThrows. class, () -> userService. createUser (userCreateVO ));
Assert.assertEquals( "Exception encoding inconsistency", ErrorCode.DATABASE_ERROR, exception.getCode());Existing problems:
The preceding use case only verifies the exception type and error code. If the error message User creation exception is changed to User creation error, the unit test case cannot be verified.
throw new ExampleException(ErrorCode.DATABASE_ERROR, "User creation error", e);Negative case:
It looks perfect to catch the thrown exception first and then verify the exception encoding and exception message.
// Mock dependent methods.
Throwable e = new RuntimeException();
Mockito.doThrow(e).when(userDAO).create(Mockito.any(UserCreateVO.class));
// Call test methods.
UserCreateVO userCreateVO = ...;
ExampleException, "Exception type inconsistency" ExampleException exception=Assert.assertThrows. class, () -> userService. createUser (userCreateVO ));
Assert.assertEquals( "Exception encoding inconsistency", ErrorCode.OBJECT_EXIST, exception.getCode());
Assert.assertEquals( "Exception message inconsistency", "User creation exception", exception.getMessage());Existing problems:
It can be seen from the code that when an ExampleException exception is thrown, the last parameter e is the RuntimeException exception thrown by our mock userService.createUser method. However, we do not verify the cause of throwing the exception. If you modify the code and remove the last parameter e, the unit test above case cannot be verified.
throw new ExampleException(ErrorCode.DATABASE_ERROR, "User creation exception");Negative case:
Many people think that verifying a thrown exception is enough, and verifying a dependent method call is unnecessary.
// Mock dependent methods.
Throwable e = new RuntimeException();
Mockito.doThrow(e).when(userDAO).create(Mockito.any(UserCreateVO.class));
// Call test methods.
UserCreateVO userCreateVO = ...;
ExampleException, "Exception type inconsistency" ExampleException exception=Assert.assertThrows. class, () -> userService. createUser (userCreateVO ));
Assert.assertEquals( "Exception encoding inconsistency", ErrorCode.OBJECT_EXIST, exception.getCode());
Assert.assertEquals( "Exception message inconsistency", "User creation exception", exception.getMessage());
Assert.assertEquals( "Exception cause inconsistency", e, exception.getCause());Existing problems:
How can you prove the code has gone through this branch without verifying the relevant method calls? For example, before we create a user, we check that the username is invalid and throw an exception.
// Check that the username is valid.
String userName = userCreateVO.getName();
if (StringUtils.length(userName) < USER_NAME_LENGTH) {
throw new ExampleException(ErrorCode.INVALID_USERNAME, " Invalid user name ");
}A perfect exception validation should not only verify the exception type, exception properties, and exception causes, but also verify the dependent method calls before throwing the exception.
The perfect case:
// Mock dependent methods.
Throwable e = new RuntimeException();
Mockito.doThrow(e).when(userDAO).create(Mockito.any(UserCreateVO.class));
// Call test methods.
String text = ResourceHelper.getResourceAsString(getClass(), path + "userCreateVO.json");
UserCreateVO userCreateVO = JSON.parseObject(text, UserCreateVO.class);
ExampleException, "Exception type inconsistency" ExampleException exception=Assert.assertThrows. class, () -> userService. createUser (userCreateVO ));
Assert.assertEquals( "Exception encoding inconsistency", ErrorCode.OBJECT_EXIST, exception.getCode());
Assert.assertEquals( "Exception message inconsistency", "User creation exception", exception.getMessage());
Assert.assertEquals( "Exception cause inconsistency", e, exception.getCause());
// Verify dependent methods.
ArgumentCaptor<UserDO> userCreateCaptor = ArgumentCaptor.forClass(UserDO.class);
Mockito.verify(userDAO).create(userCreateCaptor.capture());
text = ResourceHelper.getResourceAsString(getClass(), path + "userCreateDO.json");
Assert.assertEquals( "User creation inconsistency", text, JSON.toJSONString(userCreateCaptor.getValue()));You must verify all thrown exceptions in unit tests:
Please see the Source Method of Thrown Exception for more information.
When verifying a thrown exception, you must verify the exception type, exception attribute, exception cause, etc.
Positive example:
ExampleException, "Exception type inconsistency" ExampleException exception=Assert.assertThrows. class, () -> userService. createUser (userCreateVO ));
Assert.assertEquals( "Exception encoding inconsistency", ErrorCode.OBJECT_EXIST, exception.getCode());
Assert.assertEquals( "Exception message inconsistency", "The user already exists", exception.getMessage());
Assert.assertEquals( "Exception cause inconsistency", e, exception.getCause());Negative example:
@Test(expected = ExampleException.class)
public void testCreateUserWithException() {
...
userService.createUser(userCreateVO);
}After verifying the thrown exception, the relevant method call must be verified to ensure the unit test case takes the desired branch.
Positive example:
// Call test methods.
...
// Verify dependent methods.
ArgumentCaptor<UserDO> userCreateCaptor = ArgumentCaptor.forClass(UserDO.class);
Mockito.verify(userDAO).create(userCreateCaptor.capture());
text = ResourceHelper.getResourceAsString(getClass(), path + "userCreateDO.json");
Assert.assertEquals( "User creation inconsistency", text, JSON.toJSONString(userCreateCaptor.getValue()));In unit tests, verification method calls verify the number and order of calls of dependent methods and whether the expected parameter values are passed in.
The most common method call is injecting a dependent object:
private UserDAO userDAO;
public UserVO getUser(Long userId) {
UserDO user = userDAO.get(userId); // Method call
return convertUser(user);
}Sometimes, you can pass in the dependent object through input parameters and then call the dependent object.
public <T> List<T> executeQuery(String sql, DataParser<T> dataParser) {
List<T> dataList = new ArrayList<>();
List<Record> recordList = SQLTask.getResult(sql);
for (Record record : recordList) {
T data = dataParser.parse(record); // Method call
if (Objects.nonNull(data)) {
dataList.add(data);
}
}
return dataList;
}private UserHsfService userHsfService;
public User getUser(Long userId) {
Result<User> result = userHsfService.getUser(userId);
if (!result.isSuccess()) { // Method call 1
throw new ExampleException("User acquisition exception ");
}
return result.getData(); // Method call 2
}In Java, a static method is a member method that is modified by static and can be called without an object instance. In daily code, a static method calls the account for a certain proportion.
String text=JSON.toJSONString (user); // Method callIn unit tests, verifying a dependent method call is the process of confirming whether the dependent method of the mock object is called as expected.
// 1. Verify that the dependent method calls without parameters.
Mockito.verify(userDAO).deleteAll();
// 2. Verify dependent method call of specified parameters .
Mockito.verify(userDAO).delete(userId);
// 3. Verify dependent method calls of any parameters.
Mockito.verify(userDAO).delete(Mockito.anyLong());
// 4. Verify dependent method calls of nullable parameters.
Mockito.verify(userDAO).queryCompany(Mockito.anyLong(), Mockito.nullable(Long.class));
// 5. Verify dependent method calls of null parameters.
Mockito.verify(userDAO).queryCompany(Mockito.anyLong(), Mockito.isNull());
// 6. Verify dependent method calls of variable parameters.
Mockito.verify(userService).delete(1L, 2L, 3L);
Mockito.verify(userService).delete(Mockito.any(Long. class )) ;) // Single matching
Mockito.verify(userService).delete(Mockito.<Long>any()); // Multiple matching// 1. Verify that the dependent method is called once by default.
Mockito.verify(userDAO).delete(userId);
// 2. Verify that the dependent method is never called.
Mockito.verify(userDAO, Mockito.never()).delete(userId);
// 3. Verify that the dependent method is called n times.
Mockito.verify(userDAO, Mockito.times(n)).delete(userId);
// 4. Verify that the dependent method is called at least once.
Mockito.verify(userDAO, Mockito.atLeastOnce()).delete(userId);
// 5. Verify that the dependent method is called at least n times.
Mockito.verify(userDAO, Mockito.atLeast(n)).delete(userId);
// 6. Verify that the dependent method can be called up to once.
Mockito.verify(userDAO, Mockito.atMostOnce()).delete(userId);
// 7. Verify that the dependent method is called up to n times.
Mockito.verify(userDAO, Mockito.atMost(n)).delete(userId);
// 8. Verify that the dependent method is called the specified n times.
Mockito.verify(userDAO, Mockito.call(n)).delete(userId); // Not marked as verified.
// 9. Verify that the dependent object and its methods are called only once.
Mockito.verify(userDAO, Mockito.only()).delete(userId);// 1. Use the ArgumentCaptor.forClass method to define an ArgumentCaptor.
ArgumentCaptor<UserDO> userCaptor = ArgumentCaptor.forClass(UserDO.class);
Mockito.verify(userDAO).modify(userCaptor.capture());
UserDO user = userCaptor.getValue();
// 2. Use the @Captor annotation to define an ArgumentCaptor.
@Captor
private ArgumentCaptor<UserDO> userCaptor;
// 3. Capture a list of parameter values for multiple method calls.
ArgumentCaptor<UserDO> userCaptor = ArgumentCaptor.forClass(UserDO.class);
Mockito.verify(userDAO, Mockito.atLeastOnce()).modify(userCaptor.capture());
List<UserDO> userList = userCaptor.getAllValues();// 1. Verify the final method call.
The verification of the final method is similar to that of the normal method.
// 2. Verify private method call.
PowerMockito.verifyPrivate(mockClass, times(1)).invoke("unload", any(List.class));
// 3. Verify construction method call.
PowerMockito.verifyNew(MockClass.class).withNoArguments();
PowerMockito.verifyNew(MockClass.class).withArguments(someArgs);
// 4. Verify static method call.
PowerMockito.verifyStatic(StringUtils.class);
StringUtils.isEmpty(string);// 1. Verify that the mock object does not have any method calls.
Mockito.verifyNoInteractions(idGenerator, userDAO);
// 2. Verify that the mock object has no more method calls.
Mockito.verifyNoMoreInteractions(idGenerator, userDAO);Here, take cacheUser as an example to illustrate the problems in verifying dependent methods.
Code case:
private UserCache userCache;
public boolean cacheUser(List<User> userList) {
boolean result = true;
for (User user : userList) {
result = result && userCache.set(user.getId(), user);
}
return result;
}Negative case:
Some people think that since the dependent method has been mocked and the method under test has returned the value as expected, it is unnecessary to verify the dependent method.
// Mock dependent methods.
Mockito.doReturn(true).when(userCache).set(Mockito.anyLong(), Mockito.any(User.class));
// Call test methods.
List<User> userList = ...;
Assert.assertTrue( "The processing result is not true", userService.cacheUser(userList));
// Do not verify the dependent method.Existing problems:
A dependent method is mocked, and the method under test has returned a value as expected, which does not mean that the dependent method was called or called correctly.
For example, the unit test case cannot be verified by setting userList to a null list before the for the loop.
// Clear user list.
userList = Collections.emptyList();Negative case:
Some people like to use Mockito.verify to verify at least once and any combination of parameters because it can be applied to any verification of dependent method calls.
// Verify dependent methods.
Mockito.verify(userCache, Mockito.atLeastOnce()).set(Mockito.anyLong(), Mockito.any(User.class));Existing problems:
Although this method is suitable for the verification of any dependent method call, it has no real effect.
For example, if we accidentally wrote the cache statement twice, this unit test case cannot be verified.
// The cache is written twice.
result = result && userCache.set(user.getId(), user);
result = result && userCache.set(user.getId(), user);Negative case:
Since it is said that there is a problem with verification at least once, I will specify the number of verification times.
// Verify dependent methods.
Mockito.verify(userCache, Mockito.times(userList.size())).set(Mockito.anyLong(), Mockito.any(User.class));Existing problems:
Although the problem of the number of verification methods has been solved, the problem of verification method parameters still exists.
For example, if we accidentally wrote each user of the loop cache as the first user of the loop cache, the unit test case cannot be verified.
User user = userList.get(0);
for (int i = 0; i < userList.size(); i++) {
result = result && userCache.set(user.getId(), user);
}Negative case:
You cannot use the verification method of any parameter, so you have to use the verification method of the actual parameter. However, there is too much code to verify all dependent method calls, so verifying one or two dependent method calls is enough.
Mockito.verify(userCache).set(user1.getId(), user1);
Mockito.verify(userCache).set(user2.getId(), user2);Existing problems:
If only one or two method calls are verified, you can only guarantee that these one or two method calls are not problematic.
For example, we accidentally perform a user cache after the for the loop.
// Cache the last user.
User user = userList.get(userList.size() - 1);
userCache.set(user.getId(), user);Negative case:
Since there is a problem with not verifying all method calls, I will verify all method calls.
for (User user : userList) {
Mockito.verify(userCache).set(user.getId(), user);
}Existing problems:
It seems there should be no problem if all method calls have been verified. However, if there are other method calls in the cache user method, this is still problematic.
For example, if clearing all user caches is added before we enter the cache user method, this unit test cannot be verified.
// Delete all user caches.
userCache.clearAll();Verifying all method calls can only guarantee there is no problem with the current logic. If new method calls are involved, this unit test case cannot be verified. Therefore, we need to verify that all dependent objects have no more method calls.
The perfect case:
// Verify dependent methods.
ArgumentCaptor<Long> userIdCaptor = ArgumentCaptor.forClass(Long.class);
ArgumentCaptor<User> userCaptor = ArgumentCaptor.forClass(User.class);
Mockito.verify(userCache, Mockito.atLeastOnce()).set(userIdCaptor.capture(), userCaptor.capture());
Assert.assertEquals( "User ID list inconsistency", userIdList, userIdCaptor.getAllValues());
Assert.assertEquals( "User information list inconsistency", userList, userCaptor.getAllValues());
// Verify dependent objects.
Mockito.verifyNoMoreInteractions(userCache);Note: You can verify the parameters and the number and order of calls using ArgumentCaptor.
In unit tests, all mock methods involved are verified:
Please see the Source Method of Method Call for more information.
In unit tests, it is necessary to verify that all mock objects have no more method calls to prevent the existence or addition of other method calls to the method under test.
Positive example:
// Verify dependent objects.
Mockito.verifyNoMoreInteractions(userDAO, userCache);Remarks:
The author likes to verify all mock objects in the @ After method, so it is unnecessary to verify mock objects in each unit test case.
@After
public void afterTest() {
Mockito.verifyNoMoreInteractions(userDAO, userCache);
}Unfortunately, Mockito.verifyNoMoreInteractions does not support the ability to verify all mock objects without parameters, otherwise, this code would be more concise.
When verifying dependent methods, you must use parameter values or matchers with explicit semantics. You cannot use matchers with ambiguous semantics (such as any series parameter matchers).
Positive example:
Mockito.verify(userDAO).get(userId);
Mockito.verify(userDAO).query(Mockito.eq(companyId), Mockito.isNull());Negative example:
Mockito.verify(userDAO).get(Mockito.anyLong());
Mockito.verify(userDAO).query(Mockito.anyLong(), Mockito.isNotNull());Java Programming Skills: The Simplified Methods of Unit Test Cases
1,044 posts | 257 followers
FollowAlibaba Cloud Community - May 6, 2023
FlyingFox - January 20, 2021
Alibaba Cloud Community - February 14, 2022
Alibaba Cloud Community - March 22, 2023
Yee - September 9, 2020
Alibaba Clouder - November 2, 2020
1,044 posts | 257 followers
Follow
Web Hosting Solution
Explore Web Hosting solutions that can power your personal website or empower your online business.
Learn More
YiDA Low-code Development Platform
A low-code development platform to make work easier
Learn More
Web Hosting
Explore how our Web Hosting solutions help small and medium sized companies power their websites and online businesses.
Learn More
mPaaS
Help enterprises build high-quality, stable mobile apps
Learn MoreMore Posts by Alibaba Cloud Community