Core elements for online stress testing

Rules for stress testing

To accurately measure service capabilities, you must perform stress testing based on the following rules:

• The same environment.

• The same user size.

• The same business scenario.

• The same business workloads.

• The same source of traffic.

Stress testing based on the preceding rules helps you accurately measure the actual processing capacity of business models. You can formulate performance improvement solutions and throttling and downgrade solutions based on the stress testing result.

Core elements for stress testing

The following core elements are required for stress testing:

• Test environment

• Basic test data

• Test traffic (model and data)

• Traffic initiation and control

• Problem identification

• The test traffic (model and data) can be orchestrated by using easy and efficient interactions in Performance Testing (PTS). For more information, see Create a PTS-based stress testing scenario.

• PTS provides industry-leading features such as stress configuration and real-time adjustment for you to initiate traffic of different volumes. For more information, see Configure the stress testing model and level and Adjust the number of concurrent users.

• For problem identification, PTS integrates Application Real-Time Monitoring Service (ARMS) to facilitate end-to-end monitoring and troubleshooting.

Test environments and basic test data

• New production environment

  If you have just migrated business to the cloud or built new data centers, you can perform end-to-end stress testing on the new services and use the services in production. In this case, the verification effect is good because the environment that will be used for production is used for testing. In most cases, you can mask the production environment data and import the masked data for stress testing. The data includes transaction data, revenue data, and various core business records. Make sure that complete business data, including cross-system business data, of more than half a year is used for stress testing. A core business traffic model is built based on the basic data. The traffic includes logon, shopping cart behavior, and transaction behavior. You must cleanse data and initialize the basic data before the services are used in production.

• Equal-performance environment

  A separate stress testing environment is split from the production environment to provide equal capacity and shared access layers. This solution is characterized by simplicity, easy risk control, and accurate capacity planning, but has the disadvantage of high costs.

  Note

  • Shared access layers, such as Alibaba Cloud CDN for dynamic acceleration, Border Gateway Protocol (BGP), Web Application Firewall (WAF), Server Load Balancer (SLB), and Layer-4 and Layer-7 load balancers, must be provided to ensure that the most important network access layers are the same and issues can be detected.

  • The backend capacity is at least 25% of the production environment. A test capacity that is close to the actual capacity of the production environment ensures high accuracy. We recommend that you use the same database configuration as in the production environment.

  • The basic data can be prepared in the same manner that basic data is prepared in the new production environment.

• Production environment

  Basic data in the production environment is divided into two types. The first type of data consists of complete data in databases, which can be directly used by the test account without the need for modification. You can obtain the test account in the basic table. After stress testing is complete, the data generated in the test is cleared. The clearing operation can be written into the SQL script or integrated into the system. The second type of data consists of tagged data for stress testing, such as data that is tagged by using a defined header. The data is identified based on tags, passed during the business process (including asynchronous messages and middleware), and finally stored in the shadow tables or shadow databases. We recommend that you perform stress testing in the production environment during off-peak hours to prevent impacts on the production business. You can also deploy dedicated clusters for stress testing to further prevent impacts on the production business.

  

Other factors of stress testing

In most cases, stress testing in the production environment involves interactions, such as short messages, payment, and channel connections, with third parties. To select the most suitable solutions, you must consider actual conditions, such as human resources, machine resources, time costs, business complexity, business requirements, and maintenance costs.