1. Overview
Clients invoke API Gateway, and API Gateway invokes backend services all based on TCP connections. For TCP connections, improper timeout configurations can cause issues or even failures. This topic advises on how to configure timeout periods for TCP connections between clients and API Gateway and between API Gateway and backend services. Properly configured timeout periods help you better utilize API Gateway and prevent known issues.
The following timeout parameters are commonly configured for TCP connections:
ConnectionTimeout: the timeout period for establishing a TCP connection
WriteTimeout: the timeout period for sending a request
ReadTimeout: the timeout period for reading a response from a backend service
You can configure different values for ConnectionTimeout and for WriteTimeout based on your network type. Typically, the Internet requires larger values, such as 10 seconds for both ConnectionTimeout and WriteTimeout, and internal networks require smaller values.
We recommend that you configure the ReadTimeout parameter based on the processing time of the backend service. In particular, if you want to configure ReadTimeout for different applications, you must follow the rules described in this topic.
2. Configuration rules
The following figure shows the overall process of an API call. The client sends a request to API Gateway, which forwards the request to the backend service. After receiving and processing the request, the backend service returns a response to API Gateway, which forwards the response to the client.
Two timeout configurations are crucial in this process:
1. ClientReadTimeout: the timeout period for the client to receive the response from API Gateway. This period is configured on the client.
2. APIGatewayBackendTimeout: the timeout period for API Gateway to receive the response from the backend service. This period is configured in API Gateway.
From the preceding figure, we can see that:
ClientReadTimeout = T2 + T3 + T4 + T5
APIGatewayBackendTimeout = T2 + T3 + T4
Therefore, when configuring ClientReadTimeout and APIGatewayBackendTimeout, you must comply with the following rules:
The value of APIGatewayReaderTimeout must be larger than the business processing time that is required by the backend service.
The value of ClientReadTimeout must be larger than that of APIGatewayBackendTimeout.
For example, if you estimate that your backend service processes an API request within 10 seconds in most cases, you can set APIGatewayBackendTimeout to 10 seconds and ClientReadTimeout to 15 seconds to ensure that the TCP connection is not closed when the backend service returns a response.
If you do not follow these rules and configure short timeout periods, API Gateway fails to find an available TCP connection because the connection is already closed in cases of lengthy request processing by the backend service. In this case, API Gateway reports an N502RE error. In extreme cases, an avalanche occurs. Therefore, we recommend that you pay special attention to these rules.
3. Examples
The following figure shows an example about the ClientReadTimeout configuration. This parameter is configured in the client initialization code of the HTTP client connection pool. In this example, an API Gateway SDK is used.
The following figure shows an example about the APIGatewayBackendTimeout configuration. This parameter is configured in the backend service settings of an API definition in the API Gateway console.
