Solutions to common errors in stress testing - Performance Testing

This topic describes the possible causes of errors that may occur when you use Performance Testing Service (PTS) and provides solutions.

Common error messages

class java.net.ConnectException:null
The request has failed or been rejected by the tested end when attempting to establish a TCP connection. Check the health status of the backend server and assess whether the network connection is experiencing any bottlenecks.
org.apache.http.ConnectionClosedException:Connection closed
The server abnormally terminates the connection.
org.apache.hc.core5.http.ConnectionClosedException:Connection is closed
The request is initiated on a connection that has been terminated by the server. Check whether the bandwidth at the gateway layer and the network connection is experiencing any bottlenecks.
java.io.IOException:Connection reset by peer
The backend server resets the connection. If Server Load Balancer (SLB) is used, check the settings of SLB.
org.apache.http.ConnectionClosedException:Connection closed unexpectedly
The connection has been terminated before the data is received. The server may have failed to respond within the expected timeframe, or the debugging or stress testing may have been stopped earlier than expected.
java.lang.RuntimeException:java.net.UnknownHostException
The domain name cannot be resolved. Check whether the domain name has been properly registered and can be resolved, and whether the unregistered domain name has been bound.
org.apache.http.client.CircularRedirectException
The request is redirected in a loop (A -> B -> C -> A) or redirected more than 10 times (A1 -> A2 -> A3... -> A10 -> A11). In this case, we recommend that you disable 302 redirect, perform a stress test again, and then view the original request information. You can also use the timing waterfall flow to view the specific redirect path.
- Disable 302 redirect: On the Scenario Settings page, turn off the Allow 302 Redirect switch.
- View the specific redirect path in the timing waterfall flow: You can view the timing waterfall flow in the sampling log details of the stress testing report. For more information, see Analyze stress testing results.
org.apache.hc.core5.http.ProtocolException:Header 'key: value' is illegal for HTTP/2 messages
Headers that are not supported by the HTTP/2 protocol are configured in scenarios where the server preferentially uses the HTTP/2 protocol. Remove the corresponding headers and try again. HTTP/2 does not support the following headers: Connection, Keep-Alive, Proxy-Connection, Transfer-Encoding, Host, and Upgrade.
java.nio.channels.CancelledKeyException:null
The backend server terminates the connection under the HTTP/2 protocol. Troubleshoot the backend server.
java.util.concurrent.TimeoutException:null
The connection times out. The request has failed or been rejected by the tested end when attempting to establish a TCP connection. You can use the timing waterfall flow to check whether the connection phase takes a long time. Check the health status of the backend server and assess whether the network connection is experiencing any bottlenecks.
org.apache.hc.core5.http2.H2StreamResetException:Timeout due to inactivity (5000 MILLISECONDS) * class
The response from the backend server times out. The default request timeout period is five seconds. You can extend the request timeout period in the Advanced Settings section of the Create Scenario page.
java.net.SocketTimeoutException:null
The request times out while waiting for a response or while reading data (idle). Check whether the server is healthy, whether the timeout period of the API for stress testing is appropriate, and whether the server has a performance bottleneck.
java.lang.RuntimeException: Could not find the TestPlan class!
The JMeter script is not compatible with the JMeter version supported by PTS. Use JMeter V5.0 to edit the script.
java.lang.SecurityException: class "xxx"'s signer information does not match signer information of other classes in the same package
The ApacheJMeter_core or ApacheJMeter_java version of the Java sampler dependency in the JMeter script is not compatible with JMeter V5.0 supported by PTS. Use JMeter V5.0 to repackage the dependency into a JAR package and try again.
Attempt to resolve method: xxx() on undefined variable or class name:
The class on which the BeanShell sampler depends in the JMeter script is not uploaded. Upload the relevant JAR package and try again.
class java.lang.IllegalArgumentException:forbidden uri, uri host must match vpc cidr pattern 10.0.0.0/8, 172.16.0.0/12 or 192.168.0.0/16
You are using a VPC for stress testing, but the domain name in the test URL is resolved to a non-internal IP address. In this case, you must use an internal IP address for stress testing, or log on to the PTS console to bind the domain name to the internal IP address.

Common error codes

403 (Forbidden)

The 403 error code is returned because you do not have access to the API and the server receives the request but refuses to provide the service.

Possible cause 1

The request fails to be accepted due to the backend authentication settings of your server. In this case, the error is caused by the backend service.

Solution: Check the authentication mechanism of the service.

Possible cause 2

The server gateway performs strong verification on the User-Agent (UA) in the request header. If the UA is invalid, the server gateway returns an error message indicating that you do not have the required permissions. The default UA in the request initiated by PTS contains a special character so that some services can distinguish between statistical traffic and throttling rules.

Solution

Log on to the PTS console and choose Performance Test > Scenarios.

Select your stress testing scenario and click Edit in the Actions column. On the Header Definition tab of the Scenario Settings page, add a common UA, as shown in the following figure.
Key: User-Agent
Value: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.109 Safari/537.36
Click Debug. On the Request Details page, check whether the request is normal. If the request is normal after the UA is modified, the error is caused by the UA verification. In this case, you can continue the stress testing by modifying the UA.

Possible cause 3
The request is blocked by Web Application Firewall (WAF). A 403 error is returned due to this reason only in rare cases.
Solution
If a WAF whitelist is required, you must configure a rule that allows PTS traffic. For more information, see What do I do if the stress testing traffic cannot access my web application due to security policies?

Possible cause 4

The domain name for stress testing is not filed, or the domain name is resolved to another domain name that is not filed.

Solution: Check whether the domain name is filed based on the returned result. If a 403 error occurs and the following HTML information is returned, the domain name is not filed. In this case, you must apply for an ICP filing for the domain name first.

<html>
<head>
<meta http-equiv="Content-Type" content="textml;charset=UTF-8" />
   <style>body{background-color:#FFFFFF}</style>
<title>TestPage184</title>
  <script language="javascript" type="text/javascript">
         window.onload = function () {
           document.getElementById("mainFrame").src= "http://****.aliyun.com/alww.html";
            }
</script>
</head>
  <body>
    <iframe style="width:860px; height:500px;position:absolute;margin-left:-430px;margin-top:-250px;top:50%;left:50%;" id="mainFrame" src="" frameborder="0" scrolling="no"></iframe>
    </body>
</html>

405 (Method Not Allowed)

The 405 error may occur due to the following reasons:

A 302 redirect occurs for POST requests. The request method is changed when a 302 redirect occurs. In this case, the server may not be able to recognize the request method. Therefore, a 405 error is returned.
The request server directly verifies the method, and the corresponding response header contains the Allow=GET information.
When you use an SLB instance or web server for forwarding, you modify the request method but the backend server cannot recognize the modification.

406 (Not Acceptable)

A client error occurs. A response entity cannot be generated because the content of the requested resource cannot meet the conditions in the request header.

Possible cause

A 406 error occurs during debugging because the Accept field configured on the Header Definition tab of the request is invalid.

Accept: the type of data that the sender expects to accept. In this example, the sender is PTS.
Content-Type: the type of the data sent by the sender (PTS). The Content-Type that you configure on the Body Definition tab of the Scenario Settings page is automatically synchronized to the Header Definition tab. If the Accept field configured on the Header Definition tab does not match the agreed content, a 406 error is returned.

Solution

Make sure that the server can verify the Accept type. You can confirm the Accept types that the server is able to validate by setting a different Value for each Accept type. The following table describes the format types of the Accept header and the matching order.

Table 1. Format types of the Accept header

Format type	Format type
`text/html`	HTML format
`text/plain`	Plain text format
`text/xml`	XML format
`image/gif`	GIF image format
`image/jpeg`	JPG image format
`image/png`	PNG image format
`application/xhtml+xml`	XHTML format
`application/xml`	XML data format
`application/atom+xml`	Atom XML aggregation format
`application/json`	JSON data format
`application/pdf`	PDF format
`application/msword`	Word document format
`application/octet-stream`	Binary stream data, such as common file downloads.
`application/x-www-form-urlencoded`	The default encType in `<form encType="">`. By default, the data submitted in forms is encoded as key-value pairs and sent to the server.
When the following Accept headers are configured: `application/xml`, `text/html`, and `application/json`	The produces are matched in the following order: `application/xml` > `text/html` > `application/json`
When the following Accept headers are configured: `application/xml;q=0.3`, `application/json;q=0.8`, and `text/html`	The produces are matched in the following order: `text/html` > `application/json` > `application/xml` Note The application/xml parameter is the quality factor of the media type. A larger number indicates a higher priority (from 0 to 1).
When the following Accept headers are configured: `/`, `text/*`, and `text/html`	The produces are matched in the following order: `text/html` > `text/` > `/*`

503 (Service Unavailable)

Possible cause 1
The backend server refuses to receive the request because it is overloaded with traffic.
Solution: Check whether relevant error logs exist on the backend server.
Possible cause 2
A large number of 503 errors appear in the sampling log details of PTS, but no relevant information is found on the backend server. If the following conditions are met in your stress testing scenario, the error is returned by SLB.
- An HTTP or HTTPS API is used.
- The entry point of the stress testing environment is an SLB instance (Internet-facing SLB instance or internal-facing SLB instance).
- The backend service does not return a 503 error and no relevant records can be found.
- The 503 error message is the same as the following content:
```
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body bgcolor="white">
<h1>503 Service Temporarily Unavailable</h1>
<p>The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.</body>
</html>
```
The source IP addresses that initiate stress testing may be limited. A single IP address triggers the single-proxy throttling of the SLB cluster. Alternatively, the connection of the requesting client is long-connected by default. When the number of IP addresses is small, SLB cannot fully balance loads.
Note
By default, a long connection is established between PTS and your server.
Solution
1. Use the IP extension feature provided by PTS. For more information, see Start a scenario.
2. Set the maximum number of virtual users or the requests per second (RPS) to a greater value. For more information, see Configure load models and levels.
3. On the Scenario Settings page, configure Connection:close for the request on the Header Definition tab. After the configuration is complete, newly-added APIs use the configuration by default. You can modify the configuration based on your business requirements.

504 (BadGateway Timeout)

Possible cause

The response from the server attached to the gateway times out.

Solution

Check whether the backend server is providing services as expected. Then, you can extend the timeout period at the gateway or extend the request timeout period of the load generator in the Advanced Settings section of the Create Scenario page.

Common error messages

class java.net.ConnectException:null

org.apache.http.ConnectionClosedException:Connection closed

org.apache.hc.core5.http.ConnectionClosedException:Connection is closed

java.io.IOException:Connection reset by peer

org.apache.http.ConnectionClosedException:Connection closed unexpectedly

java.lang.RuntimeException:java.net.UnknownHostException

org.apache.http.client.CircularRedirectException

org.apache.hc.core5.http.ProtocolException:Header 'key: value' is illegal for HTTP/2 messages

java.nio.channels.CancelledKeyException:null

java.util.concurrent.TimeoutException:null

org.apache.hc.core5.http2.H2StreamResetException:Timeout due to inactivity (5000 MILLISECONDS) * class

java.net.SocketTimeoutException:null

java.lang.RuntimeException: Could not find the TestPlan class!

java.lang.SecurityException: class "xxx"'s signer information does not match signer information of other classes in the same package

Attempt to resolve method: xxx() on undefined variable or class name:

class java.lang.IllegalArgumentException:forbidden uri, uri host must match vpc cidr pattern 10.0.0.0/8, 172.16.0.0/12 or 192.168.0.0/16

Common error codes

403 (Forbidden)

405 (Method Not Allowed)

406 (Not Acceptable)

503 (Service Unavailable)

504 (BadGateway Timeout)