Use vtoa to get the real client address after address translation - Alibaba Cloud Linux

In FullNAT scenarios, such as with Anti-DDoS Proxy, the client address is translated to the address of the FullNAT node. A backend server with vtoa installed on Alibaba Cloud Linux 3 (kernel version 5.10.134-15 or later) can use the TCP Option Address (TOA) to obtain the real client address by calling getsockopt or getpeername. This supports both IPv4 and IPv6.

Limits

System requirements: Alibaba Cloud Linux 3, kernel version 5.10.134-15 or later.

Note

Run the uname -r command to check the kernel version of the image.

Scenarios

Anti-DDoS Proxy: User requests undergo address translation (FullNAT) at a forwarding node, which is the Anti-DDoS node in the diagram. Anti-DDoS enables the TOA mechanism. It places the real client IP address and port information into the TCP Option and passes it to the backend origin server. The backend origin server then uses vtoa to obtain the real client IP address.

CDN acceleration: Service requests are forwarded by CDN acceleration nodes to the origin server. The origin server can use vtoa to obtain the real client address.

Installation and configuration

Warning

vtoa changes the result returned by the getpeername system call. This introduces the following threats. Confirm that you need this feature before you install it:

If other network components, such as Cilium, modify the getpeername result using methods like eBPF, they may conflict with vtoa and cause abnormal behavior.
If an application depends on getpeername, its behavior may be affected.

Installation and uninstallation

Install
```
sudo yum install vtoa -y
```
Uninstall
```
sudo yum remove vtoa -y
```

Note

After installation, vtoa takes effect immediately and is enabled to start automatically on system boot by default. After uninstallation, the vtoa feature is disabled.

Configure vtoa

After installation, vtoa takes effect immediately and is enabled to start automatically on system boot by default. You do not usually need to configure vtoa manually.

Start vtoa
```
sudo systemctl start vtoa
```
Stop vtoa
```
sudo systemctl stop vtoa
```
Enable vtoa to start on system boot
```
sudo systemctl enable vtoa
```
Disable vtoa from starting on system boot
```
sudo systemctl disable vtoa
```
Check the status of vtoa
```
systemctl status vtoa
```

Get the real client address

After you enable vtoa, you can obtain the real client address using the getsockopt or getpeername system call.

(Recommended) Use the getsockopt call

Important

This requires kernel version 5.10.134-17 or later. You can run the uname -r command to check the kernel version.

vtoa provides a new optname to obtain the real client address. Its static field is 1348. The following C code provides an example.

struct sockaddr caddr;
int optlen = sizeof(caddr);
int optname = 1348;

getsockopt(fd, IPPROTO_IP, optname, &caddr, &optlen);
// caddr.sa_family can be AF_INET or AF_INET6, which correspond to IPv4 and IPv6 addresses.

Use the getpeername call

Important

This feature is supported on Alibaba Cloud Linux 3 (kernel version 5.10.134-15 or later). It may be deprecated in the future.

After you enable vtoa, it returns the real client address information. The following C code provides an example.

struct sockaddr caddr;
int caddr_len = sizeof(caddr);

getpeername(fd, &caddr, &caddr_len);
// caddr.sa_family can be AF_INET or AF_INET6, which correspond to IPv4 and IPv6 addresses.
// accept() can also be used to get the client address in a similar way.

FAQ

Supported TOA option formats

The address information that vtoa parses is carried in the TCP option. This option must meet specific format requirements. If the TCP option does not meet the format requirements, vtoa ignores it. This has no effect on the application and is equivalent to vtoa being disabled.

TOA (opcode = 254)

opsize = 8, ip/port is in network byte order

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |     opcode    |    opsize     |              port             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                              ip                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

TOA_V6 (opcode = 253)

opsize = 20, ip/port is in network byte order

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |     opcode    |    opsize     |              port             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   +                                                               +
   |                                                               |
   +                              ip                               +
   |                                                               |
   +                                                               +
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Enable vtoa in container scenarios

vtoa does not support container-level fencing.
In typical container scenarios, such as an ACK containerd runtime, install vtoa on the host. This makes it effective for all containers on the node. Therefore, install vtoa on the host, not inside a container.