All Products
Search

This Product

    :Use Pktgen to test the network performance of ECS instances

    Document Center

    :Use Pktgen to test the network performance of ECS instances

    Last Updated:Aug 05, 2024

    Note

    Disclaimer: This article may contain information about third-party products. Such information is for reference only. Alibaba Cloud does not make any guarantee, express or implied, with respect to the performance and reliability of third-party products, as well as potential impacts of operations on the products.

    Overview

    This topic describes how to use Pktgen to test the network performance of Elastic Compute Service (ECS) instances.

    Background information

    Note

    Take note of the following items:

    • Before you perform risky operations such as modifying instance configurations or data, we recommend that you check the disaster recovery and fault tolerance capabilities of the instances to ensure data security.
    • You can modify the configurations and data of instances including ECS and ApsaraDB RDS instances. We recommend that before you modify instance configurations or data, you create snapshots or enable RDS log backup.
    • If you have authorized or submitted sensitive information such as the logon account and password in the Alibaba Cloud Management Console, we recommend that you modify these information in a timely manner.

    The following steps describe how to use Pktgen to test the network performance of ECS instances:

    Step 1: Prepare the environment

    Prepare two ECS instances, one for the generator and the other for the receiver. Prepare two network interface controllers (NICs), one for logon and the other for testing. Make sure that the two NICs can communicate with each other. Then, run the ifconfig command and record the IP addresses and media access control (MAC) addresses of the two NICs, which are required when you configure traffic.

    Step 2: Install software

    Perform the following operations on both the generator and the receiver:

    1. Log on to a Linux instance. For more information about how to log on to the Linux instance, see Connect to a Linux instance by using a username and password.
    2. Run the following command to create the pktgen directory: 
      mkdir pktgen
    3. Run the following command to go to the pktgen directory: 
      cd pktgen
    4. Run the following command to download the installation packages of Pktgen, Lua, and Data Plane Development Kit (DPDK): 
      wget https://git.dpdk.org/apps/pktgen-dpdk/snapshot/pktgen-dpdk-pktgen-3.6.5.zip https://www.lua.org/ftp/lua-5.4.3.tar.gz https://fast.dpdk.org/rel/dpdk-18.11.11.tar.xz
      Note

      Note: The versions of Pktgen, Lua, and DPDK must be compatible with each other. Otherwise, compilation errors may occur. In this topic, the versions in the preceding command are used. Before you use the installation packages of other versions, make sure that the versions are compatible with each other.

    5. Run the following commands in sequence to decompress the installation packages of Pktgen, Lua, and DPDK: 
      tar -Jxvf dpdk-18.11.11.tar.xz

tar -zxvf lua-5.4.3.tar.gz

unzip pktgen-dpdk-pktgen-3.6.5.zip
    6. If some dependencies are missing during the compilation process, run the following commands to install the corresponding installation packages:
      • CentOS 7 
        yum -y install kernel-devel.x86_64
yum -y install numactl-devel.x86_64
yum -y install elfutils-libelf-devel
yum -y install libpcap-devel
      • CentOS 8
        Note

        Note: By default, libpcap-devel is not installed on the CentOS 8 operating system. Run the following commands to install libpcap-devel: 

        dnf config-manager --add-repo /etc/yum.repos.d/CentOS-PowerTools.repo 
dnf config-manager --set-enabled PowerTools 
dnf install epel-release 
dnf install libpcap-devel
    7. Run the following commands in sequence to compile and install Pktgen: 
      cd dpdk-stable-18.11.11
make config T=x86_64-native-linuxapp-gcc 
make all -j32
    8. Run the following commands in sequence to compile and install Lua: 
      cd lua-5.4.3
make
make install
    9. Run the following commands in sequence to compile and install DPDK: 
      cd pktgen-dpdk-pktgen-3.6.5/
 export RTE_SDK=/root/pktgen/dpdk-stable-18.11.11/
export RTE_TARGET=build 
make -j32

    Step 3: Bind drivers to VFIO

    Perform the following operations on both the generator and the receiver:

    1. View the /proc/cmdline file and check whether the line starting with BOOT_IMAGE contains the intel_iommu=on parameter. If the line does not contain the intel_iommu=on parameter, edit the /etc/default/grub file, add the intel_iommu=on parameter to the line starting with GRUB_CMDLINE_LINUX, and then restart the ECS instances.
    2. Run the following commands in sequence to install the Virtual Function I/O (VFIO) and VFIO-PCI drivers: 
      modprobe vfio

modprobe vfio-pci
    3. Run the following command to configure noniommu_mode: 
      echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode
    4. Run the following command to check and record the bus-info value of the NIC: 
      ethtool -i eth1
      Note

      Note: In this topic, the eth1 NIC is used. In actual operations, replace eth1 with your NIC.

      A command output similar to the following one is returned: 
      driver: virtio_net


version: 1.0.0


firmware-version:


expansion-rom-version:

bus-info: 0000:00:08.0

supports-statistics: no

supports-test: no

supports-eeprom-access: no

supports-register-dump: no

supports-priv-flags: no
    5. Run the following command to unbind the virtio driver from eth1. 
      echo "[$Bus-info]" > /sys/bus/pci/drivers/virtio-pci/unbind
      Note

      Note: [$Bus-info] is the bus-info value of the NIC obtained in the previous step.

    6. Run the following command to bind bus-info to the VFIO-PCI driver: 
      /root/pktgen/dpdk-stable-18.11.11/usertools/dpdk-devbind.py -b vfio-pci [$Bus-info]

    Step 4: Configure Hugepages

    Perform the following operations on both the generator and the receiver. Check the number of your nodes and run the corresponding commands:

    • Dual nodes 
      echo 8000 > /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages
echo 8000 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
    • Single node 
      echo 8000 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages

    Step 5: Start Pktgen

    Perform the following operations on both the generator and the receiver:

    1. Run the following command in the codepktgen-dpdk-pktgen-3.6.5code directory to start Pktgen: 
      ./app/build/pktgen -c 0x1ffff -n 4 --socket-mem 16000 -w [$Bus-info] -- -P -m "[1-16:1-16].0"
      Note

      Note:

      • 0x1ffff: the mask that indicates the CPUs used by threads. In this example, the lowest set bit of the mask is bit0, which indicates that CPU0 is used to control shell and CPU1 to CPU16 are used to generate and receive packets.
      • "[1-16:1-16].0" indicates the CPU number. For more information, see the following descriptions. The CPU number cannot exceed the CPU range within which to generate and receive packets in the mask.
        • 1-16 (before the colon): CPU number. One of 16 CPUs is used to receive packets.
        • 1-16 (after the colon): CPU number. One of 16 CPUs is used to generate packets
        • .0 indicates that the port number is 0.
      • You can modify the CPU range to improve the generation performance. Note that the more CPUs required for generating packets, the more Hugepages are needed. The Pktgen process may stop due to insufficient Hugepages.

      A command output similar to the following one indicates that Pktgen is started: 
      Copyright (c) <2010-2019>, Intel Corporation. All rights reserved. Powered by DPDK
EAL: Detected 64 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: No free hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: PCI device 0000:00:08.0 on NUMA socket -1
EAL:   Invalid NUMA socket, default to 0
EAL:   probe driver: 1af4:1000 net_virtio
EAL:   using IOMMU type 8 (No-IOMMU)
Lua 5.4.3  Copyright (C) 1994-2021 Lua.org, PUC-Rio

*** Copyright (c) <2010-2019>, Intel Corporation. All rights reserved.
*** Pktgen created by: Keith Wiles -- >>> Powered by DPDK <<<

Initialize Port 0 -- TxQ 16, RxQ 16,  Src MAC 00:16:3e:24:9b:a4

Port  0: Link Up - speed 10000 Mbps - full-duplex <Enable promiscuous mode>

  RX/TX processing lcore:   1 rx:  1 tx:  1
  RX/TX processing lcore:   2 rx:  1 tx:  1
  RX/TX processing lcore:   3 rx:  1 tx:  1
  RX/TX processing lcore:   4 rx:  1 tx:  1
- Ports 0-0 of 1   <Main Page>  Copyright (c) <2010-2019>, Intel Corporation
  Flags:Port        :   P-----R--------:0
Link State          :       <UP-10000-FD>     ----TotalRate----
Pkts/s Max/Rx       :                 1/0                   1/0
       Max/Tx       :                 0/0                   0/0
MBits/s Rx/Tx       :                 0/0                   0/0
Broadcast           :                   0
Multicast           :                   0
Sizes 64            :                   0
      65-127        :                   0
      128-255       :                   0
      256-511       :                   0
      512-1023      :                   0
      1024-1518     :                   0
Runts/Jumbos        :                 2/0
ARP/ICMP Pkts       :                 2/0
Errors Rx/Tx        :                 0/0
Total Rx Pkts       :                   2
      Tx Pkts       :                   0
      Rx MBs        :                   0
      Tx MBs        :                   0
                    :
Pattern Type        :             abcd...
Tx Count/% Rate     :       Forever /100%
Pkt Size/Tx Burst   :           64 /   64
Port Src/Dest       :         1234 / 5678
Pkt Type:VLAN ID    :     IPv4 / UDP:0001
802.1p CoS/DSCP/IPP :           0/  0/  0
VxLAN Flg/Grp/vid   :    0000/    0/    0
IP  Destination     :         192.168.X.X
    Source          :      192.168.X.X/24
MAC Destination     :   00:00:00:00:00:00
    Source          :   00:XX:XX:24:9b:a4
PCI Vendor/Addr     :   1af4:1000/00:08.0

-- Pktgen Ver: 3.6.5 (DPDK 18.11.11)  Powered by DPDK  (pid:4873) -------------
Pktgen:/>
    2. Run the following commands in sequence in Pktgen to configure traffic for the generator: 
      set 0 proto udp
range 0 src mac start [$MAC]
range 0 src mac min [$MAC]
range 0 src mac inc 00:00:00:00:00:00
range 0 dst mac start ee:ff:ff:ff:ff:ff
range 0 dst mac min ee:ff:ff:ff:ff:ff
range 0 dst mac inc 00:00:00:00:00:00
range 0 src port start 10000
range 0 src port min 10000
range 0 src port max 10000
range 0 src port inc 0
range 0 dst port start 2000
range 0 dst port min 2000
range 0 dst port max 2255
range 0 dst port inc 1
range 0 proto udp
range 0 src ip start [$Local_IP]
range 0 src ip min [$Local_IP]
range 0 src ip inc 0.0.0.0
range 0 size start 64
range 0 size min 64
range 0 size max 64
range 0 size inc 0
range 0 dst ip start [$Receive_IP]
range 0 dst ip min [$Receive_IP]
range 0 dst ip inc 0.0.0.0
enable all range
      Note

      Note:

      • [$MAC] is the MAC address of the generator.
      • [$Local_IP] is the IP address of the generator.
      • [$Receive_IP] is the IP address of the receiver.

    3. Run the following command in Pktgen to start the sending process in the generator: 
      start 0
      View the traffic data:
      • Generator 
        -iPorts 0-0 of 1   <Main Page>  Copyright (c) <2010-2019>, Intel Corporation
  Flags:Port        :   P-----R--------:0                   1/0
Link State          :       <UP-10000-FD>     ----TotalRate----
Pkts/s Max/Rx       :                 1/0                   1/0
       Max/Tx       :   14906304/14904704     14906304/14904704
MBits/s Rx/Tx       :             0/10015               0/10015
Broadcast           :                   0
Multicast           :                   0
Sizes 64            :                   0
      65-127        :                   0
      128-255       :                   0
      256-511       :                   0
      512-1023      :                   0
      1024-1518     :                   0
Runts/Jumbos        :                16/0
ARP/ICMP Pkts       :                16/0
Errors Rx/Tx        :                 0/0
Total Rx Pkts       :                  16
      Tx Pkts       :          1655962864
      • Receiver 
        - Ports 0-0 of 1   <Main Page>  Copyright (c) <2010-2019>, Intel Corporation
  Flags:Port        :   P--------------:0
Link State          :       <UP-10000-FD>     ----TotalRate----
Pkts/s Max/Rx       :   14911585/14910795     14911585/14910795
       Max/Tx       :                 0/0                   0/0
MBits/s Rx/Tx       :             10020/0               10020/0
Broadcast           :                   0
Multicast           :                   0
Sizes 64            :          1862740038
      65-127        :                   0
      128-255       :                   0
      256-511       :                   0
      512-1023      :                   0
      1024-1518     :                   0
Runts/Jumbos        :                 6/0
ARP/ICMP Pkts       :                 6/0
Errors Rx/Tx        :                 0/0
Total Rx Pkts       :          2298743466
      Tx Pkts       :                   0
      Rx MBs        :             1544755
      Tx MBs        :                   0
                    :
Pattern Type        :             abcd...
Tx Count/% Rate     :       Forever /100%
Pkt Size/Tx Burst   :           64 /   64
Port Src/Dest       :         1234 / 5678
Pkt Type:VLAN ID    :     IPv4 / TCP:0001
802.1p CoS/DSCP/IPP :           0/  0/  0
VxLAN Flg/Grp/vid   :    0000/    0/    0
IP  Destination     :         192.168.X.1
    Source          :      192.168.X.X/24
MAC Destination     :   00:00:00:00:00:00
    Source          :   00:XX:XX:24:2d:55
PCI Vendor/Addr     :   1af4:1000/00:08.0

-- Pktgen Ver: 3.6.5 (DPDK 18.11.11)  Powered by DPDK  (pid:4072) -------------
    4. Run the following command in the generator to stop the sending process: 
      stop 0

    Application scope

    • ECS