To use elastic Remote Direct Memory Access (eRDMA) to accelerate network communication for TCP applications that require high communication performance, low latency, and high throughput, you can use Network Accelerator (NetACC) to adapt eRDMA and the TCP applications without the need to modify application code. This topic describes NetACC and how to use NetACC.
NetACC is in public preview.
Introduction to NetACC
NetACC is a user-mode network acceleration library that leverages the benefits of eRDMA, such as low latency and high throughput, and uses compatible socket interfaces to accelerate existing TCP applications.
Scenarios
NetACC is suitable for scenarios that involve high network overheads.
Scenarios in which the packets per second (PPS) rate is high, especially scenarios in which a large number of small packets are sent and received. You can use NetACC to reduce CPU overheads and improve the system throughput in specific scenarios, such as when Redis processes requests.
Network latency-sensitive scenarios: eRDMA provides lower network latency than TCP to accelerate network responses.
Repeated creation of short-lived connections: NetACC can accelerate the process of establishing secondary connections to reduce the connection creation time and improve system performance.
Install NetACC
Installation methods
Use the eRDMA driver to install NetACC
When you install the eRDMA driver, NetACC is automatically installed. For information about how to install the eRDMA driver, see the Configure eRDMA on an existing ECS instance section of the "Configure eRDMA on an enterprise-level instance" topic.
Separately install NetACC
You can use a specific version of NetACC or temporarily use NetACC on an Elastic Compute Service (ECS) instance. Run the following command on the instance to separately install NetACC:
sudo curl -fsSL https://netacc-release.oss-cn-hangzhou.aliyuncs.com/release/netacc_download_install.sh | sudo sh
Configuration file
After you install NetACC, the
/etc/netacc.conf
configuration file is automatically generated. You can configure the parameters of NetACC, such as NACC_SOR_IO_THREADS and NACC_LOG_PATH, in the configuration file. The following sample code provides an example on how to configure the parameters in the configuration file:
Use NetACC
You can use NetACC in applications by running the netacc_run
command or configuring the LD_PRELOAD
environment variable.
Run the netacc_run
command
netacc_run
is a tool that loads NetACC on application startup. You can add netacc_run
before the COMMAND command to start an application and load NetACC at the same time. The COMMAND command specifies the command that is used to start an application.
netacc_run
provides multiple parameters to improve the performance of NetACC. For example, -t specifies the number of I/O threads, and -p specifies the number of QPs to be reused. The parameters that you configure by running the netacc_run
command overwrite the parameters in the configuration file.
Examples:
In the following examples, Redis applications are used. Add
netacc_run
before a Redis command to start a Redis application and load NetACC at the same time.Run the following command to start the Redis service and load NetACC at the same time:
netacc_run redis-server
Run the following command to start the redis-benchmark utility and load NetACC at the same time:
netacc_run redis-benchmark
Configure the LD_PRELOAD
environment variable
The LD_PRELOAD
environment variable specifies the shared libraries that are preloaded when a program starts. To automate the loading of NetACC, specify NetACC in the value of the LD_PRELOAD
environment variable in the relevant script.
Run the following command to query the location of the NetACC dynamic library:
ldconfig -p | grep netacc
The following command output is returned.
Run the following command to configure the
LD_PRELOAD
environment variable to specify the preloaded shared libraries:LD_PRELOAD=/lib64/libnetacc-preload.so your_application
Replace
your_application
with the application that you want to accelerate.Examples: In the following examples, Redis applications are used.
Run the following command to start the Redis service and load NetACC at the same time:
LD_PRELOAD=/lib64/libnetacc-preload.so redis-server
Run the following command to start the redis-benchmark utility and load NetACC at the same time:
LD_PRELOAD=/lib64/libnetacc-preload.so redis-benchmark
Monitor NetACC
netacc_ss
is a monitoring tool provided by NetACC. You can run the netacc_ss
command to monitor the status of data sent and received by a NetACC-accelerated process. You can run the command on a server and a client to monitor NetACC.
Run the following command to monitor the status of data sent and received by a NetACC-accelerated process:
netacc_ss -s all -p <Process ID>
To query the ID of a process, run the ps -ef | grep <Process name>
command.
Use NetACC in Redis applications
Benefits of NetACC for Redis applications
Improved system throughput
NetACC is suitable for scenarios in which Redis processes a large number of requests per second. This reduces CPU overheads and improves system throughput.
Accelerated network responses
NetACC leverages the low latency benefit of eRDMA to significantly accelerate network responses to Redis applications.
NetACC used in Redis performance benchmarks
Redis-benchmark
is a built-in benchmark utility of Redis, which is designed to measure the performance of the Redis server under various workloads by simulating a number of clients to concurrently send requests to the Redis server.
Test scenario
Use NetACC in the redis-benchmark
utility to simulate 100 clients and 4 threads to make 5 million SET requests.
Preparations
Create two eRDMA-capable ECS instances on the instance buy page in the ECS console. Select Auto-install eRDMA Driver and then select eRDMA Interface to enable the eRDMA Interface (ERI) feature for the primary elastic network interface (ENI). Use one ECS instance as the Redis server and the other ECS instance as a Redis client.
The ECS instances have the following configurations:
Image: Alibaba Cloud Linux 3
Instance type: ecs.g8ae.4xlarge
Private IP address of the primary ENI: 172.17.0.90 for the server and 172.17.0.91 for the client. In the following benchmark, replace the IP addresses with actual values based on your business requirements.
NoteIn this topic, the ERI feature is enabled for the primary ENIs of the ECS instances to perform the benchmark. 172.17.0.90 is the private IP address of the primary ENI of the ECS instance that serves as the Redis server.
If you enable the ERI feature for the secondary ENIs of the ECS instances, replace the preceding IP addresses with the private IP addresses of the secondary ENIs. For more information, see the Configure eRDMA when you create an ECS instance section of the "Configure eRDMA on an enterprise-level instance" topic.
Procedure
Connect to the ECS instance that serves as the Redis server and the ECS instance that serves as a Redis client.
For more information, see Connect to a Linux instance by using a password or key.
Check whether the eRDMA driver is installed on the ECS instances.
After the ECS instances start, run the
ibv_devinfo
command to check whether the eRDMA driver is installed.The following command output indicates that the eRDMA driver is installed.
The following command output indicates that the eRDMA driver is being installed. The eRDMA driver requires a few minutes to install. Try again later.
Run the following command on the ECS instances to install Redis:
sudo yum install -y redis
The following command output indicates that Redis is installed.
Use the
redis-benchmark
utility to benchmark the performance of Redis.Perform a benchmark with NetACC
Run the following command on the ECS instance that serves as the Redis server to start Redis and accelerate Redis by using NetACC:
netacc_run redis-server --port 6379 --protected-mode no
NoteReplace 6379 with the number of the actual port on which you want to start Redis. For more information, see the Common parameters used with the redis-server command section of this topic.
In this example, the
netacc_run
command is run to use NetACC. For other methods of using NetACC, see the Use NetACC section of this topic.
The following command output indicates that Redis is started as expected.
Run the following command on the ECS instance that serves as a Redis client to start redis-benchmark:
netacc_run redis-benchmark -h 172.17.0.90 -p 6379 -c 100 -n 5000000 -r 10000 --threads 4 -d 512 -t set
NoteReplace 172.17.0.90 with the actual IP address of the Redis server and 6379 with the number of the actual port on which Redis is started. For more information, see the Common command parameters used with redis-benchmark section of this topic.
The benchmark results may vary based on the network conditions. The benchmark data provided in this topic is only for reference.
The Summary section at the end of the preceding benchmark result indicates that approximately 770,000 requests can be processed per second. For information about the metrics in Redis benchmark results, see the Common metrics in redis-benchmark benchmark results section of this topic.
Perform a benchmark without NetACC
Run the following command on the ECS instance that serves as the Redis server to start Redis:
redis-server --port 6379 --protected-mode no --save
NoteReplace 6379 with the number of the actual port on which you want to start Redis. For more information, see the Common parameters used with the redis-server command section of this topic.
The following command output indicates that Redis is started as expected.
Run the following command on the ECS instance that serves as a Redis client to start redis-benchmark:
redis-benchmark -h 172.17.0.90 -c 100 -n 5000000 -r 10000 --threads 4 -d 512 -t set
NoteReplace 172.17.0.90 with the actual IP address of the Redis server and 6379 with the number of the actual port on which Redis is started. For more information, see the Common command parameters used with redis-benchmark section of this topic.
The benchmark results may vary based on the network conditions. The benchmark data provided in this topic is only for reference.
The Summary section at the end of the preceding benchmark result indicates that approximately 330,000 requests can be processed per second. For information about the metrics in Redis benchmark results, see the Common metrics in redis-benchmark benchmark results section of this topic.