Server Load Balancer:Use connection draining to implement graceful undeployment of services

Scenario 1: Removing backend servers

Remove ECS01, enable connection draining, and specify a connection draining timeout period. The following figure shows the status changes of ECS01.

Scenario 2: Unhealthy backend servers

Connection draining for WebSocket sessions

Step 1: Enable connection draining

In this example, a server group is already prepared. Connection draining is configured by modifying the server group. If you do not have a server group, you can enable connection draining when you create a server group.

1. Log on to the ALB console.

2. In the top navigation bar, select the region where the server group is deployed.

3. In the left-side navigation pane, choose ALB > Server Groups.

4. On the Server Groups page, click the ID of the server group that you want to manage.

5. On the Details tab, click Modify Basic Information in the Basic Information section.

6. In the Modify Basic Information dialog box, click Advanced Settings and turn on Connection Draining.

7. Set Connection Draining Timeout to 300 seconds and click Save.

Step 2: Verify the result

Configure connection draining on the backend server

1. Remotely log on to ECS01. For more information, see Connection method overview.

2. Run the following commands to create a WebSocket file and open the WebSocket directory:

   mkdir WebSocket
   cd WebSocket

3. Run the following commands to install dependency packages:

   pip install tornado
   pip install websocket-client

4. Run the following command to modify the server.py configuration file:

   vim server.py

   1. Press the I key to open the editor and configure the following parameters to enable a WebSocket service:

      #!/usr/bin/env python3
      # encoding=utf-8
      
      import tornado.websocket
      import tornado.ioloop
      import tornado.web
      from datetime import datetime
      
      
      # The WebSocket handler
      class WebSocketHandler(tornado.websocket.WebSocketHandler):
          def open(self):
              current_time = datetime.now()
              formatted_time = current_time.strftime("%Y-%m-%d %H:%M:%S")
              print("Time:", formatted_time, "The WebSocket connection is established")
      
          def on_message(self, message):
              current_time = datetime.now()
              formatted_time = current_time.strftime("%Y-%m-%d %H:%M:%S")
              print("Time:", formatted_time, "Received a message:", message)
              self.write_message("The server received your message:" + message)
      
          def on_close(self):
              current_time = datetime.now()
              formatted_time = current_time.strftime("%Y-%m-%d %H:%M:%S")
              print("Time:", formatted_time, "The WebSocket connection is closed")
      
      # Routes
      application = tornado.web.Application([
          (r"/websocket", WebSocketHandler),
      ])
      
      if __name__ == "__main__":
          print("WebSocket Server Start on 8080 ...")
          application.listen(8080)
          tornado.ioloop.IOLoop.current().start()
      

   2. After you complete the modifications, press the Esc key, enter :wq, and then press the Enter key to save and close the configuration file.

5. In the directory of the server.py file, run the following command to enable the WebSocket service:

   python3 server.py

   The following response message indicates that the WebSocket service is enabled:

   Websocket Server Start on 8080 ...

Add ECS01 to the server group

1. Log on to the ALB console.

2. In the top navigation bar, select the region where the server group is deployed.

3. In the left-side navigation pane, choose ALB > Server Groups.

4. On the Server Groups page, find the server group that you want to manage and click Modify Backend Server in the Actions column.

5. On the Backend Servers tab, click Add Backend Server, select ECS01 in the Select Servers step, and then click Next.

6. In the Ports/Weights step, select ECS01, set the port to 8080, and then click OK.

Configure connection draining on the client

1. Log on to the client and open the command-line interface (CLI). Run the following commands to create a WebSocket file and open the WebSocket directory:

   mkdir WebSocket
   cd WebSocket

2. Run the following command to install dependency packages:

   pip install websocket-client

3. Run the following command to modify the client.py file:

   vim client.py

   1. Press the I key to open the editor and configure the following parameters to enable an access service for the WebSocket client:

      #!/usr/bin/env python3
      # encoding=utf-8
      
      import websocket
      import time
      from datetime import datetime
      
      def on_message(ws, message):
          print("Received a server message:", message)
      
      if __name__ == "__main__":
          ws = websocket.WebSocket()
          ws.connect("ws://<ALB domain name>:80/websocket") # Enter the domain name of your ALB instance
          print("The WebSocket connection is established")
          try:
              while True:
                  current_time = datetime.now()
                  formatted_time = current_time.strftime("%Y-%m-%d %H:%M:%S")
                  print("Packet time:", formatted_time)
                  ws.send("Hello, Server!")
                  result = ws.recv()
                  on_message(ws, result)
                  time.sleep(1)
          except Exception:
              print("The WebSocket connection is closed")
      

   2. After you complete the modifications, press the Esc key, enter :wq, and then press the Enter key to save and close the configuration file.

4. In the directory of the client.py file, run the following command to access ECS01:

   python3 client.py

   The following response message indicates that the client can access ECS01:

   The WebSocket connection is established
   Packet time: 2024-04-28 17:00:53
   Received server message: The server received your message: Hello, Server!
   Packet time: 2024-04-28 17:00:54
   Received server message: The server received your message: Hello, Server!

   ECS01 returns the following response message:

   WebSocket Server Start on 8080 ...
   Time: 2024-04-28 17:00:53 The WebSocket connection is established
   Time: 2024-04-28 17:00:53 Received message: Hello, Server!
   Time: 2024-04-28 17:00:54 Received message: Hello, Server!

Remove backend servers

1. Log on to the ALB console.

2. In the top navigation bar, select the region where the server group is deployed.

3. In the left-side navigation pane, choose ALB > Server Groups.

4. Click the ID of the server group from which you want to remove backend servers.

5. On the Backend Servers tab, find ECS01 and click Remove in the Actions column.

6. In the Remove Backend Server message, click OK.

Wait for the connection draining timeout period to end

In this example, the connection draining timeout period is set to 300 seconds. As a result, the connections on ECS01 are closed 300 seconds after you remove ECS01.

• The client returns the following response message:

  Packet time: 2024-04-28 17:06:23
  Received server message: The server received your message: Hello, Server!
  Packet time: 2024-04-28 17:06:24
  The WebSocket connection is closed

• ECS01 returns the following response message:

  Time: 2024-04-28 17:06:22 Received message: Hello, Server!
  Time: 2024-04-28 17:06:23 Received message: Hello, Server!
  Time: 2024-04-28 17:06:23 The WebSocket connection is closed

Connection draining for HTTP sessions

In HTTPS scenarios, the response that clients receive varies based on the connection draining timeout period, connection request timeout period, and backend server processing time.

• If the connection draining timeout period is shorter than the backend server processing time, responses from ECS01 are interrupted. As a result, clients receive the HTTP 500 status code.

• If the backend server processing time is longer than the connection request timeout period, responses from ECS01 time out. As a result, clients receive the HTTP 504 status code.

In this example, the connection draining timeout period is set to 15 seconds and the backend server processing time is set to 30 seconds. In this example, responses from ECS01 are interrupted and clients receive the HTTP 500 status code.

Step 1: Enable connection draining

In this example, a server group is already prepared. Connection draining is configured by modifying the server group. If you do not have a server group, you can enable connection draining when you create a server group.

1. Log on to the ALB console.

2. In the top navigation bar, select the region where the server group is deployed.

3. In the left-side navigation pane, choose ALB > Server Groups.

4. On the Server Groups page, click the ID of the server group that you want to manage.

5. On the Details tab, click Modify Basic Information in the Basic Information section.

6. In the Modify Basic Information dialog box, click Advanced Settings and turn on Connection Draining.

7. Set Timeout Period to 15 seconds and click Save.

Step 2: Specify a connection request timeout period

1. Log on to the ALB console.

2. In the top navigation bar, select the region where the ALB instance is deployed.

3. On the Instances page, click the ID of the ALB instance that you want to manage.

4. On the Listener tab, click the ID of the HTTP listener that you want to manage.

5. In the Basic Information section, click Modify Listener.

6. In the Modify Listener dialog box, click Modify on the right side of Advanced Settings.

7. Set Connection Request Timeout to 60 seconds, which is the default timeout period, and click Save.

Step 3: Verify the result

Configure connection draining on the backend server

1. Remotely log on to ECS01. For more information, see Connection method overview.

2. Run the following commands to create an HTTP folder and open the HTTP directory:

   mkdir http
   cd http

3. Run the following command to modify the http_server.py configuration file:

   vim http_server.py

   1. Press the I key to open the editor and configure the following parameters to enable the HTTP Server service:

      #!/usr/bin/env python3
      # encoding=utf-8
      
      from http.server import SimpleHTTPRequestHandler, HTTPServer
      from datetime import datetime
      import time
      
      class DelayedHTTPRequestHandler(SimpleHTTPRequestHandler):
          def do_GET(self):
              current_time = datetime.now()
              formatted_time = current_time.strftime("%Y-%m-%d %H:%M:%S")
              print("Time:", formatted_time, "Received the GET request and will respond in 30 seconds....")
              time.sleep(30) # Configure the time.sleep function to simulate the backend server processing time
              SimpleHTTPRequestHandler.do_GET(self)
      
      PORT = 8080
      server = HTTPServer(("", PORT), DelayedHTTPRequestHandler)
      print(f"Serving HTTP on 0.0.0.0 port {PORT} (http://0.0.0.0:{PORT}/) ...")
      server.serve_forever()
      

   2. After you complete the modifications, press the Esc key, enter :wq, and then press the Enter key to save and close the configuration file.

4. Enter the directory of http_server.py and run the following command to start the HTTP Server service:

   python3 http_server.py

   The following response message indicates that the HTTP Server service is running:

   Serving HTTP on 0.0.0.0 port 8080 (http://0.0.0.0:8080/) ...

Add ECS01 to a server group

1. Log on to the ALB console.

2. In the top navigation bar, select the region where the server group is deployed.

3. On the Server Groups page, find the server group that you want to manage and click Modify Backend Server in the Actions column.

4. On the Backend Servers tab, click Add Backend Server, select ECS01 in the Select Servers step, and then click Next.

5. In the Ports/Weights step, select ECS01, set the port to 8080, and then click OK.

Configure connection draining on the client

1. Log on to the client and open the command-line interface (CLI). Run the following command to access ECS01:

   curl http://<ALB domain name>:80/ -v

   The following response message indicates that the ALB instance can access the backend service:

   * About to connect() to alb-nssnq5a********.cn-guangzhou.alb.aliyuncs.com port 80 (#0)
   *   Trying 10.X.X.225...
   * Connected to alb-nssnq5a********.cn-guangzhou.alb.aliyuncs.com (10.X.X.225) port 80 (#0)
   > GET / HTTP/1.1
   > User-Agent: curl/7.29.0
   > Host: alb-nssnq5a********.cn-guangzhou.alb.aliyuncs.com
   > Accept: */*

   The server receives the following response message:

   Serving HTTP on 0.0.0.0 port 8080 (http://0.0.0.0:8080/) ...
   Time: 2024-02-07 13:57:33 Received the Get request and will respond in 30 seconds....

Remove backend servers

1. Log on to the ALB console.

2. In the top navigation bar, select the region where the server group is deployed.

3. In the left-side navigation pane, choose ALB > Server Groups.

4. Click the ID of the server group from which you want to remove backend servers.

5. On the Backend Servers tab, find ECS01 and click Remove in the Actions column.

6. In the Remove Backend Server message, click OK.

Wait for the connection draining timeout period to end

The result shows that when the connection draining timeout period is shorter than the backend server processing time, clients receive the HTTP 500 status code.

* About to connect() to alb-nssnq5a********.cn-guangzhou.alb.aliyuncs.com port 80 (#0)
*   Trying 10.X.X.224...
* Connected to alb-nssnq5a********.cn-guangzhou.alb.aliyuncs.com (10.1.0.224) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: alb-nssnq5a********.cn-guangzhou.alb.aliyuncs.com
> Accept: */*
> 
< HTTP/1.1 500 Internal Server Error
< Date: Wed, 07 Feb 2024 06:02:24 GMT
< Content-Type: text/html
< Content-Length: 186
< Connection: close
< Via: HTTP/1.1 SLB.87
< 
<html>
<head><title>500 Internal Server Error</title></head>
<body bgcolor="white">
<center><h1>500 Internal Server Error</h1></center>
<hr><center>nginx</center>
</body>
</html>
* Closing connection 0