:The endpoint of the SLB instance timed out

Disclaimer: This article may contain information about third-party products. Such information is for reference only. Alibaba Cloud does not make any guarantee, express or implied, with respect to the performance and reliability of third-party products, as well as potential impacts of operations on the products.

Question

The access to the IP address of the SLB instance timed out.

Cause

The possible causes of the problem are as follows:

• The endpoint of the SLB instance is protected by security risks, such as blackholing and traffic scrubbing. For more information, see traffic blackholing and traffic scrubbing and WAF protection.
• Insufficient client ports Insufficient client ports can lead to connection failures especially during stress tests. SLB will erase the timestamp attribute of TCP connections by default. As a result, tw_reuse of Linux protocol stack (reuse of ports in time_wait state) does not work, and connections in time_wait state accumulate, resulting in insufficient client ports and eventually access timeout.
• If the backend server accept queue is full, the backend server does not return the syn_ack packet and the client times out.
• Some applications may not properly handle connection-timeout RST packets. After an SLB TCP connection is established, if the connection is inactive for 900 seconds, the SLB sends RST packets to the client and the server bidirectionally to terminate the connection. For applications that do not properly handle RST exceptions, the system may send data to closed connections again, causing the application to time out.
• The connection fails when the backend server accesses the IP address of the layer -4 SLB instance. A common scenario is that the backend application uses URL concatenation to redirect the access.

Answer

Alibaba Cloud reminds you that:

• Before you perform operations that may cause risks, such as modifying instance configurations or data, we recommend that you check the disaster recovery and fault tolerance capabilities of the instances to ensure data security.
• You can modify the configurations and data of instances including but not limited to Elastic Compute Service (ECS) and Relational Database Service (RDS) instances. Before the modification, we recommend that you create snapshots or enable RDS log backup.
• If you have authorized or submitted sensitive information such as the logon account and password in the Alibaba Cloud Management Console, we recommend that you modify such information in a timely manner.

Use the following solutions to solve the problem:

• The IP address of the SLB instance is under security protection.
  Log on to the Anti-DDoS console to check the status of the SLB instance. For more information, see asset Center.
  
• Insufficient client ports
  The client uses a persistent connection instead of a short-lived connection. It disconnects the connection by using the RST packet, and sets the SO_LINGER attribute for the socket, instead of using a FIN packet.
  
• The backend server accept queue is full.
  If the back-end server in the default net.core.somaxconn has a value of 128, you can refer to the following command to change the parameter value, and restart on the back-end server applications.

  sysctl -w net.core.somaxconn=1024

  Note: This document is only recommended. You can modify the document based on your business requirements. We recommend that you create snapshots or back up related data files in advance.

  
  
• In some applications, the connection-timeout RST packet is not properly handled.
  We recommend that you optimize the client program and promptly close the used connections.
  
• The connection fails when backend servers access the IP address of the layer -4 SLB instance.
  We recommend that you use layer -7 HTTP listeners or HTTPS listeners instead.
  

Application scope

• SLB